Jingong commited on
Commit
7ced6cb
·
verified ·
1 Parent(s): 7b025fd

Release weights, config and model card

Browse files
Files changed (3) hide show
  1. README.md +75 -0
  2. config.json +23 -0
  3. model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: fair-noncommercial-research-license
4
+ license_link: https://github.com/Jingong-Chen/GradSplat/blob/main/LICENSE
5
+ library_name: gradsplat
6
+ pipeline_tag: image-to-3d
7
+ tags:
8
+ - gaussian-splatting
9
+ - novel-view-synthesis
10
+ - 3d-reconstruction
11
+ - feed-forward
12
+ ---
13
+
14
+ # GradSplat
15
+
16
+ **GradSplat** reconstructs a scene as 3D Gaussians from a few images in one forward pass, and refines its own prediction
17
+ by learned gradient steps on the input views inside that pass, first on opacity and color and then on the shape of the
18
+ Gaussians. No target view is used, so the steps run unchanged at inference.
19
+
20
+ [Paper](https://arxiv.org/abs/2610.38951) · [Code](https://github.com/Jingong-Chen/GradSplat) · [Project page](https://jingong-chen.github.io/GradSplat/)
21
+
22
+ These are the weights of the paper, trained on RealEstate10K. The DL3DV fine-tuned weights are at
23
+ [Jingong/GradSplat-DL3DV](https://huggingface.co/Jingong/GradSplat-DL3DV).
24
+
25
+ ## Usage
26
+
27
+ ```bash
28
+ git clone https://github.com/Jingong-Chen/GradSplat.git && cd GradSplat && pip install -e .
29
+ ```
30
+
31
+ ```python
32
+ from gradsplat import GradSplat
33
+ from gradsplat.io import load_views
34
+
35
+ model = GradSplat.from_pretrained("Jingong/GradSplat").to("cuda").eval()
36
+ views = load_views("examples/re10k_dining_room/images", cameras="examples/re10k_dining_room/cameras.json")
37
+ out = model.infer(views.images, views.c2w, views.K) # without cameras: model.infer(views.images)
38
+ gaussians = out["gaussians"] # one Gaussian per input pixel
39
+ ```
40
+
41
+ ## Model
42
+
43
+ - 1.22 B parameters: a VGGT-Ω backbone, a camera head, a dense head with a Gaussian branch, a shape head and two
44
+ learned updates (three appearance steps and five shape steps).
45
+ - Input: 2 or more overlapping views of 224 × 224 pixels (center-cropped), with or without their cameras.
46
+ - Output: one 3D Gaussian per input pixel (center, scale, rotation, opacity and spherical harmonics of degree 1), and
47
+ the predicted cameras.
48
+ - Training: initialized from VGGT-Ω and trained for 178,000 steps in four stages on the training split of
49
+ RealEstate10K, with 2 to 12 input views per sample.
50
+
51
+ ## Results (from the paper)
52
+
53
+ | | Narrow protocol (7,128 scenes) | Wide protocol (1,600 scenes) |
54
+ |:---|:---:|:---:|
55
+ | PSNR / SSIM / LPIPS, with input poses | 33.683 / 0.9593 / 0.0640 | 26.755 / 0.8993 / 0.1301 |
56
+ | PSNR / SSIM / LPIPS, without input poses | 26.914 / 0.8863 / 0.1053 | 20.901 / 0.6945 / 0.2426 |
57
+
58
+ RealEstate10K with six input and eight target views per scene at 224 × 224 pixels. Without fine-tuning: DL3DV
59
+ 24.787 / 0.8448 / 0.1919 and ScanNet++ 21.609 / 0.7540 / 0.3184.
60
+
61
+ ## License
62
+
63
+ FAIR Noncommercial Research License, inherited from VGGT-Ω: noncommercial research use only.
64
+
65
+ ## Citation
66
+
67
+ ```bibtex
68
+ @article{chen2026gradsplat,
69
+ title = {GradSplat: Gradients Inside the Forward Pass for Feed-Forward 3D Gaussian Splatting},
70
+ author = {Chen, Jingong and Jiang, Linlian and Li, Guangyuan and Zuo, Xinxin and
71
+ Wang, Yang and Pack, Chulwoo and Gao, Kyle and Won, Kwanghee},
72
+ journal = {arXiv preprint arXiv:2610.38951},
73
+ year = {2026}
74
+ }
75
+ ```
config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "resolution": 224,
3
+ "patch_size": 16,
4
+ "embed_dim": 1024,
5
+ "sh_degree": 1,
6
+ "scale_max": 0.3,
7
+ "scale_mult": 0.001,
8
+ "dmean_scale": 0.05,
9
+ "gauss_upsample": 2,
10
+ "app_trunk_depth": 6,
11
+ "shape_hidden": 128,
12
+ "shape_amp": 4.0,
13
+ "shape_rot_amp": 1.0,
14
+ "appearance_steps": 3,
15
+ "shape_steps": 5,
16
+ "appearance_update_hidden": 192,
17
+ "shape_update_hidden": 128,
18
+ "mixer_depth": 4,
19
+ "mixer_dim": 768,
20
+ "mixer_heads": 12,
21
+ "mixer_mid": 64,
22
+ "kappa": 4.04
23
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2bbd05a8144504c0495fed1759d5a551f4db176d18ffa378168205dc2463b76
3
+ size 4900169572