| ---
|
| title: 4dgs-dpm
|
| app_file: app.py
|
| sdk: gradio
|
| sdk_version: 5.17.1
|
| ---
|
| # Sparse-view Spacetime Gaussian Splatting via Dynamic Point Maps
|
|
|
| ## Installation
|
| ```bash
|
| # Create environment
|
| conda create -n ssgs python=3.10
|
| conda activate ssgs
|
|
|
| # Install PyTorch with CUDA
|
| pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu118
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ## Usage
|
|
|
| ### Go-Pro Multi-View Capture System
|
| ```
|
| git clone the gp-control repo
|
| ```
|
|
|
| ### Web Interface (Recommended)
|
| ```bash
|
| python app.py
|
| ```
|
| Upload videos, adjust settings, and download results as ZIP.
|
|
|
| ### Command Line
|
| ```bash
|
| python vdpm/infer.py --input mv-video/your-videos --output output/vdpm
|
| python -m gs.train --input output/vdpm --output output/splats --iterations 1000
|
| ```
|
|
|
| ## Pipeline
|
|
|
| 1. **Video Processing**: Extract and interleave frames from multi-view videos
|
| 2. **VDPM Inference**: Generate dynamic point maps and camera poses using VGGT backbone
|
| 3. **3DGS Training**: Train per-frame Gaussian splats initialized from point maps
|
| 4. **Animation Rendering**: Generate GIF from interpolated camera viewpoint
|
|
|
| ## Output
|
|
|
| The pipeline generates:
|
| - `splats/frame_XXXX.ply` - Gaussian splat for each timestep
|
| - `renders/` - Training progress images
|
| - `animation.gif` - Rendered animation from average camera
|
| - `tracks.npz` - 3D point tracks
|
| - `poses.npz` - Camera poses
|
|
|
| ## Requirements
|
| Tested on:
|
| - Windows 11
|
| - RTX 3070 Ti
|
| - CUDA 11.8+
|
| - Python 3.12
|
|
|
| ## TO-DO
|
|
|
| - [x] VGGT Quantization (BF16/FP16)
|
| - [x] Co-visibility check to reduce points
|
| - [x] Dynamic point tracking
|
| - [x] Per-frame 3DGS training
|
| - [x] Gradio demo with GIF rendering
|
| - [ ] Dynamic/Static segmentation
|
| - [ ] 3DGS with dynamic deformation field
|
| - [ ] 4DGS primitive support
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @misc{dpmsplat2026,
|
| title={DPM-Splat: Video to 4D Gaussian Splats via Dynamic Point Maps},
|
| author={Your Name},
|
| year={2026},
|
| url={https://github.com/YOUR_USERNAME/4dgs-dpm}
|
| }
|
| ```
|
|
|
| ## Acknowledgements
|
|
|
| - [VGGT](https://github.com/facebookresearch/vggt) - Visual Geometry Grounded Transformer
|
| - [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)
|
| - [NVIDIA Warp](https://github.com/NVIDIA/warp)
|
|
|
|
|
