Add model card: pipeline tag, library name, and paper link

README.md

@@ -0,0 +1,79 @@
+---
+license: mit
+library_name: pytorch
+pipeline_tag: image-to-3d
+tags:
+  - structure-from-motion
+  - multi-view
+  - pose-estimation
+  - 3d-reconstruction
+  - diffusion
+  - computer-vision
+---
+
+
+# DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion
+
+This repository hosts the pretrained checkpoint for **DiffusionSfM**, a transformer-based denoising diffusion model that directly infers 3D scene geometry and camera poses from multi-view images. The model parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame and learns a denoising diffusion model to predict them from multi-view inputs.
+
+## Paper
+
+**DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion**
+Qitao Zhao, Amy Lin, Jeff Tan, Jason Y. Zhang, Deva Ramanan, Shubham Tulsiani
+Accepted to CVPR 2025.
+
+For more details, see the paper: https://huggingface.co/papers/2505.05473
+
+- arXiv: https://arxiv.org/abs/2505.05473
+- Project page: https://qitaozhao.github.io/DiffusionSfM
+- Code: https://github.com/QitaoZhao/DiffusionSfM
+- Interactive demo (Gradio Space): https://huggingface.co/spaces/qitaoz/DiffusionSfM
+
+## Model description
+
+DiffusionSfM is a data-driven Structure-from-Motion (SfM) approach that replaces the conventional two-stage pipeline (pairwise reasoning + global optimization) with a single end-to-end multi-view reasoning model. Given a set of multi-view images, it predicts, per pixel, the ray origin and ray endpoint in a global coordinate frame. The endpoints give the 3D scene geometry (point structure), while the ray origins and endpoints together recover the camera poses. A transformer-based denoising diffusion model performs this prediction directly from multi-view image features. Specialized training mechanisms handle missing data and unbounded scene coordinates. The model naturally models uncertainty through the diffusion process and outperforms classical and prior learning-based SfM approaches on both synthetic and real datasets.
+
+This repository contains the released checkpoint `ckpt_00800000.pth` (a raw PyTorch checkpoint). It is intended to be used together with the official codebase at https://github.com/QitaoZhao/DiffusionSfM.
+
+## Intended uses
+
+- Multi-view Structure-from-Motion (camera pose estimation + 3D structure recovery) from a set of posed or unposed images.
+- Research on learned SfM, multi-view geometry, and diffusion models for 3D vision.
+
+## How to use
+
+The checkpoint is a raw PyTorch `.pth` file and is loaded by the official codebase. It is not directly loadable with `transformers` or `diffusers` `from_pretrained` APIs; use the project's own loading utilities.
+
+1. Clone the official codebase:
+
+```bash
+git clone https://github.com/QitaoZhao/DiffusionSfM.git
+cd DiffusionSfM
+```
+
+2. Download the checkpoint from this repo:
+
+```python
+from huggingface_hub import hf_hub_download
+
+ckpt_path = hf_hub_download(repo_id="qitaoz/DiffusionSfM", filename="ckpt_00800000.pth")
+```
+
+3. Follow the installation and inference instructions in the [official repository](https://github.com/QitaoZhao/DiffusionSfM) (environment setup with `torch`, `torchvision`, `pytorch3d`, and the project's `diffusionsfm` package). You can also try the online [Gradio demo](https://huggingface.co/spaces/qitaoz/DiffusionSfM) without any local setup.
+
+## License
+
+This model is released under the MIT License (see the [LICENSE](https://github.com/QitaoZhao/DiffusionSfM/blob/main/LICENSE) in the official repository).
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@inproceedings{zhao2025diffusionsfm,
+  title={DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion},
+  author={Qitao Zhao and Amy Lin and Jeff Tan and Jason Y. Zhang and Deva Ramanan and Shubham Tulsiani},
+  booktitle={CVPR},
+  year={2025}
+}
+```