| --- |
| license: other |
| license_name: fair-noncommercial-research-license |
| license_link: https://huggingface.co/Jingong/GradSplat/blob/main/LICENSE |
| library_name: gradsplat |
| pipeline_tag: image-to-3d |
| tags: |
| - gaussian-splatting |
| - novel-view-synthesis |
| - 3d-reconstruction |
| - feed-forward |
| --- |
| |
| # GradSplat |
|
|
| **GradSplat** reconstructs a scene as 3D Gaussians from a few images in one forward pass, and refines its own prediction |
| by learned gradient steps on the input views inside that pass, first on opacity and color and then on the shape of the |
| Gaussians. No target view is used, so the steps run unchanged at inference. |
|
|
| [Paper](https://arxiv.org/abs/2610.38951) · [Code](https://github.com/Jingong-Chen/GradSplat) · [Project page](https://jingong-chen.github.io/GradSplat/) |
|
|
| The repository holds the two sets of weights of the paper: |
|
|
| | File | Training data | Load with | |
| |:---|:---|:---| |
| | `model.safetensors` | RealEstate10K | `GradSplat.from_pretrained("Jingong/GradSplat")` | |
| | `model.dl3dv.safetensors` | RealEstate10K, then 12,000 steps on DL3DV | `GradSplat.from_pretrained("Jingong/GradSplat", variant="dl3dv")` | |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/Jingong-Chen/GradSplat.git && cd GradSplat && pip install -e . |
| ``` |
|
|
| ```python |
| from gradsplat import GradSplat |
| from gradsplat.io import load_views |
| |
| model = GradSplat.from_pretrained("Jingong/GradSplat").to("cuda").eval() |
| views = load_views("examples/re10k_dining_room/images", cameras="examples/re10k_dining_room/cameras.json") |
| out = model.infer(views.images, views.c2w, views.K) # without cameras: model.infer(views.images) |
| gaussians = out["gaussians"] # one Gaussian per input pixel |
| ``` |
|
|
| ## Model |
|
|
| - 1.22 B parameters: a VGGT-Ω backbone, a camera head, a dense head with a Gaussian branch, a shape head and two |
| learned updates (three appearance steps and five shape steps). |
| - Input: 2 or more overlapping views of 224 × 224 pixels (center-cropped), with or without their cameras. |
| - Output: one 3D Gaussian per input pixel (center, scale, rotation, opacity and spherical harmonics of degree 1), and |
| the predicted cameras. |
| - Training: initialized from VGGT-Ω and trained for 178,000 steps in four stages on the training split of |
| RealEstate10K, with 2 to 12 input views per sample. `model.dl3dv.safetensors` continues for 12,000 steps on DL3DV. |
|
|
| ## Results (from the paper) |
|
|
| | | Narrow protocol (7,128 scenes) | Wide protocol (1,600 scenes) | |
| |:---|:---:|:---:| |
| | PSNR / SSIM / LPIPS, with input poses | 33.683 / 0.9593 / 0.0640 | 26.755 / 0.8993 / 0.1301 | |
| | PSNR / SSIM / LPIPS, without input poses | 26.914 / 0.8863 / 0.1053 | 20.901 / 0.6945 / 0.2426 | |
|
|
| RealEstate10K with six input and eight target views per scene at 224 × 224 pixels, with `model.safetensors`. On |
| DL3DV (139 scenes of the index released with ReSplat) and ScanNet++ without fine-tuning: DL3DV 24.787 / 0.8448 / 0.1919 |
| and ScanNet++ 21.609 / 0.7540 / 0.3184. With `model.dl3dv.safetensors`: DL3DV 26.104 / 0.8802 / 0.1505. |
|
|
| ## License |
|
|
| The weights are trained from the VGGT-Ω weights and are released under the FAIR Noncommercial Research License of |
| VGGT-Ω ([LICENSE](LICENSE)), for noncommercial research use. The [code](https://github.com/Jingong-Chen/GradSplat) is |
| released under the MIT License, except for the code of VGGT-Ω and the two files that adapt it. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{chen2026gradsplat, |
| title = {GradSplat: Gradients Inside the Forward Pass for Feed-Forward 3D Gaussian Splatting}, |
| author = {Chen, Jingong and Jiang, Linlian and Li, Guangyuan and Zuo, Xinxin and |
| Wang, Yang and Pack, Chulwoo and Gao, Kyle and Won, Kwanghee}, |
| journal = {arXiv preprint arXiv:2610.38951}, |
| year = {2026} |
| } |
| ``` |
|
|
