Add model card
Browse files
README.md
CHANGED
|
@@ -2,4 +2,43 @@
|
|
| 2 |
license: other
|
| 3 |
license_name: stabilityai-ai-non-commercial
|
| 4 |
license_link: https://huggingface.co/stabilityai/stable-virtual-camera/blob/main/LICENSE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
license: other
|
| 3 |
license_name: stabilityai-ai-non-commercial
|
| 4 |
license_link: https://huggingface.co/stabilityai/stable-virtual-camera/blob/main/LICENSE
|
| 5 |
+
base_model: stabilityai/stable-virtual-camera
|
| 6 |
+
pipeline_tag: image-to-video
|
| 7 |
+
tags:
|
| 8 |
+
- novel-view-synthesis
|
| 9 |
+
- video-diffusion
|
| 10 |
+
- 3d-gaussian-splatting
|
| 11 |
---
|
| 12 |
+
|
| 13 |
+
# GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
|
| 14 |
+
|
| 15 |
+
GS-Adapter and LoRA weights for **GeoNVS**, which injects explicit 3D Gaussian
|
| 16 |
+
geometry into camera-controlled video diffusion models
|
| 17 |
+
([paper](https://arxiv.org/abs/2603.14965)).
|
| 18 |
+
|
| 19 |
+
These weights are fine-tuned from
|
| 20 |
+
[`stabilityai/stable-virtual-camera`](https://huggingface.co/stabilityai/stable-virtual-camera)
|
| 21 |
+
(SEVA) and [CameraCtrl](https://github.com/hehao13/CameraCtrl) (SVD backbone),
|
| 22 |
+
and **inherit the Stability AI Non-Commercial Research Community License** —
|
| 23 |
+
using them requires the base models and acceptance of their licenses.
|
| 24 |
+
Non-commercial research use only.
|
| 25 |
+
|
| 26 |
+
## Files
|
| 27 |
+
|
| 28 |
+
| Path | Backbone | Description |
|
| 29 |
+
|---|---|---|
|
| 30 |
+
| `eccv_gattn/checkpoint-88000/` | SEVA | Main GeoNVS model (gated-attention fusion): `gs_adapter_weights.pth` + `pytorch_lora_weights.safetensors` |
|
| 31 |
+
| `eccv_camctrl/checkpoint-180000/` | CameraCtrl (SVD) | GeoNVS on CameraCtrl: adapter + LoRA |
|
| 32 |
+
| `cameractrl_lora_145K/` | CameraCtrl (SVD) | LoRA-only baseline (no GS-Adapter) |
|
| 33 |
+
|
| 34 |
+
## Usage
|
| 35 |
+
|
| 36 |
+
Use with the GeoNVS code release — `scripts/download_weights.sh` fetches these
|
| 37 |
+
files into `pretrained_weights/`, then:
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
python demo.py --data_path <benchmarkset>/dl3dv10 --num_inputs 3 \
|
| 41 |
+
--gs_adapter_config configs/module_config/gsadapter_eccv_gattn.yaml \
|
| 42 |
+
--gs_adapter_weight_path pretrained_weights/eccv_gattn/checkpoint-88000 \
|
| 43 |
+
--lrm_model_name vggt_iv --H=384 --W=384 --version=1.0
|
| 44 |
+
```
|