3D-Front (MIDI-3D)

Github | Project Page | Paper | Original Dataset

1. Dataset Introduction

TL;DR: This dataset processes 3D-Front into organized 3d scenes paired with rendered multi-view images and surfaces, which are used in MIDI-3D. Each scene contains:

• 3D models (.glb)
• Point cloud (.npy)
• Rendered multi-view images in RGB, depth, normal, with camera information

2. Data Extraction

sudo apt-get install git-lfs
git lfs install
git clone https://huggingface.co/datasets/huanngzh/3D-Front

cat 3D-FRONT-SURFACE.part* > 3D-FRONT-SURFACE.tar.gz
cat 3D-FRONT-SCENE.part* > 3D-FRONT-SCENE.tar.gz

tar -xzvf 3D-FRONT-SURFACE.tar.gz
tar -xzvf 3D-FRONT-SCENE.tar.gz
tar -xzvf 3D-FRONT-RENDER.tar.gz

If you just want to evaluate your model, you can download only the files containing the test keyword.

3. File Structure

3D-Front
├── 3D-FRONT-RENDER # rendered views
│   ├── 0a8d471a-2587-458a-9214-586e003e9cf9 # house
│   │   ├── Hallway-1213 # room
│   │   ...
├── 3D-FRONT-SCENE # 3d models (glb)
│   ├── 0a8d471a-2587-458a-9214-586e003e9cf9 # house
│   │   ├── Hallway-1213 # room
│   │   │   ├── Table_e9b6f54f-1d29-47bf-ba38-db51856d3aa5_1.glb # object
│   │   │   ...
├── 3D-FRONT-SURFACE # point cloud (npy)
│   ├── 0a8d471a-2587-458a-9214-586e003e9cf9 # house
│   │   ├── Hallway-1213 # room
│   │   │   ├── Table_e9b6f54f-1d29-47bf-ba38-db51856d3aa5_1.npy # object
│   │   │   ...
├── valid_room_ids.json # scene list
├── valid_furniture_ids.json # object list
├── midi_room_ids.json # scene list (subset used in midi)
└── midi_furniture_ids.json # object list (subset used in midi)

About room_ids and furniture_ids: The i-th room in room_ids contains the objects whose ids are the i-th list in furniture_ids.

4. About Train and Test Set

MIDI uses the last 1,000 rooms in midi_room_ids.json as the testset, and the others as training set.

Citation

If you find this dataset useful, please cite:

@article{huang2024midi,
  title={MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation},
  author={Huang, Zehuan and Guo, Yuan-Chen and An, Xingqiao and Yang, Yunhan and Li, Yangguang and Zou, Zi-Xin and Liang, Ding and Liu, Xihui and Cao, Yan-Pei and Sheng, Lu},
  journal={arXiv preprint arXiv:2412.03558},
  year={2024}
}

@inproceedings{fu20213d,
    title={3d-front: 3d furnished rooms with layouts and semantics},
    author={Fu, Huan and Cai, Bowen and Gao, Lin and Zhang, Ling-Xiao and Wang, Jiaming and Li, Cao and Zeng, Qixun and Sun, Chengyue and Jia, Rongfei and Zhao, Binqiang and others},
    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
    pages={10933--10942},
    year={2021}
}
@article{fu20213d,
    title={3d-future: 3d furniture shape with texture},
    author={Fu, Huan and Jia, Rongfei and Gao, Lin and Gong, Mingming and Zhao, Binqiang and Maybank, Steve and Tao, Dacheng},
    journal={International Journal of Computer Vision},
    pages={1--25},
    year={2021},
    publisher={Springer}
}

Contact

huangzehuan@buaa.edu.cn