Improve model card for DiffusionGS with metadata, links, and usage example
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,3 +1,54 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: image-to-3d
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction
|
| 7 |
+
|
| 8 |
+
This repository presents **DiffusionGS**, a novel single-stage 3D diffusion model for object generation and scene reconstruction from a single view, as described in the paper:
|
| 9 |
+
[**Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction**](https://huggingface.co/papers/2411.14384)
|
| 10 |
+
|
| 11 |
+
DiffusionGS directly outputs 3D Gaussian point clouds at each timestep to enforce view consistency and allows the model to generate robustly given prompt views of any direction, beyond object-centric inputs. It boasts over 5x faster speed (~6s on an A100 GPU) compared to state-of-the-art methods.
|
| 12 |
+
|
| 13 |
+
- **Project Page:** https://caiyuanhao1998.github.io/project/DiffusionGS/
|
| 14 |
+
- **Code:** https://github.com/caiyuanhao1998/Open-DiffusionGS
|
| 15 |
+
|
| 16 |
+
## Visual Examples
|
| 17 |
+
|
| 18 |
+
<p align="center">
|
| 19 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/abo.gif" width="24%" alt="Object Generation">
|
| 20 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/gso.gif" width="24%" alt="Gaussian Splatting Object">
|
| 21 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/real_img.gif" width="24%" alt="Real Image Reconstruction">
|
| 22 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/wild.gif" width="24%" alt="In the Wild Scene">
|
| 23 |
+
</p>
|
| 24 |
+
<p align="center">
|
| 25 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/sd_2.gif" width="24%" alt="Scene Diffusion 2">
|
| 26 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/sd_1.gif" width="24%" alt="Scene Diffusion 1">
|
| 27 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/flux_1.gif" width="24%" alt="Flux 1">
|
| 28 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/green_man.gif" width="24%" alt="Green Man">
|
| 29 |
+
</p>
|
| 30 |
+
<p align="center">
|
| 31 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/plaza.gif" width="50%" alt="Plaza Scene">
|
| 32 |
+
<img src="https://github.com/caiyuanhao1998/Open-DiffusionGS/raw/main/img/town.gif" width="48%" alt="Town Scene">
|
| 33 |
+
</p>
|
| 34 |
+
|
| 35 |
+
## Quick Demo
|
| 36 |
+
|
| 37 |
+
For object-centric image-to-3D generation model, we provide a single-line script to use the code:
|
| 38 |
+
```shell
|
| 39 |
+
python run.py
|
| 40 |
+
```
|
| 41 |
+
This code will automatically download the model checkpoints and config files from [HuggingFace](). Or you can manually download it from [this link](https://huggingface.co/coast01/LVSM/tree/main) and set it to local dir.
|
| 42 |
+
|
| 43 |
+
For detailed instructions on installation, data preparation, evaluation, and training, please refer to the official [GitHub repository](https://github.com/caiyuanhao1998/Open-DiffusionGS).
|
| 44 |
+
|
| 45 |
+
## Citation
|
| 46 |
+
If you find our work useful, please consider citing our paper:
|
| 47 |
+
```bibtex
|
| 48 |
+
@inproceedings{diffusiongs,
|
| 49 |
+
title={Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction},
|
| 50 |
+
author={Yuanhao Cai and He Zhang and Kai Zhang and Yixun Liang and Mengwei Ren and Fujun Luan and Qing Liu and Soo Ye Kim and Jianming Zhang and Zhifei Zhang and Yuqian Zhou and Yulun Zhang and Xiaokang Yang and Zhe Lin and Alan Yuille},
|
| 51 |
+
booktitle={ICCV},
|
| 52 |
+
year={2025}
|
| 53 |
+
}
|
| 54 |
+
```
|