README.md

metadata

title: 'ZipSplatPlus: Real-time 3D Gaussian Splatting & Stroke Playback'
emoji: ⚡
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.20.0
python_version: '3.12'
app_file: app.py
pinned: false
short_description: 3D Gaussian Splatting with stroke playback.
tags:
  - 3d-gaussian-splatting
  - stroke-playback
  - 3d-reconstruction
  - single-image-to-3d
  - multi-view-reconstruction
  - sparkjs
  - live-photos
  - heic-support
  - zerogpu
  - 3d-vision
  - computer-vision
  - novel-view-synthesis
  - gradio
models:
  - veichta/zipsplat
license: mit
thumbnail: thumbnail.png

🔥 ZipSplatPlus: Real-Time 3D Gaussian Splatting & Stroke Playback

Instant 3D Gaussian Splatting from single/multi-view images, videos, & Apple Live Photos with ZeroGPU dynamic acceleration, real-time interactive stroke-by-stroke WebGL playback, 360° turntable & stroke video generation, and compressed .PLY export.

---

🚀 Quick Links

• 🌐 Live Web App: Hugging Face Space (ZeroGPU)
• 📦 GitHub Repository: ai3d-dev/ZipSplatPlus
• 🔮 Base Model & Weights: veichta/zipsplat

---

🌟 Key Features & Capabilities

• ⚡ Single-Pass 3D Scene Generation: Reconstruct high-fidelity 3D Gaussian Splatting scenes from single or unconstrained multi-view photos without per-scene optimization loops.
• 🖌️ Interactive Stroke-by-Stroke Playback: Play back each predicted 3D Gaussian "stroke" step-by-step! Explore how neural tokens paint the 3D scene in real-time with scrub slider, play/pause controls, and speed adjustments.
• 🎨 Multiple Render Modes:
  • Cumulative: Watch the 3D scene build up progressively stroke by stroke.
  • Single Stroke: Isolate and inspect individual 3D Gaussian strokes (32 Gaussians/token).
  • Colorized: Assign vibrant, distinct color palettes to each stroke to visualize token layout.
• 🎬 Stroke-by-Stroke Turntable Video Export: Automatically renders smooth MP4 videos showing the 3D scene being painted stroke-by-stroke during camera orbit.
• 🎨 Real-Time Web Viewer: Powered by Spark.js (@sparkjsdev/spark), enabling 60 FPS orbit, pan, and zoom inside the browser using client-side WebGL shaders.
• 📱 Native Apple Live Photo & HEIC/HEIF Support: Direct decoding of iPhone/iPad .HEIC/.HEIF photos, .MOV/.MP4 videos, and paired Apple Live Photos (.HEIC + .MOV).
• 💾 Standard 3D Model Export: Download compressed Gaussian .PLY files compatible with Blender, Unity, Unreal Engine, SuperSplat, and WebGL viewers.

---

📸 Supported Input Formats

ZipSplatPlus handles a wide range of visual inputs with intelligent frame sampling:

Format Category	File Extensions	Capabilities
Still Images	.jpg, .jpeg, .png, .webp	Single or multi-view photo capture.
Apple HEIC/HEIF	.heic, .heif	Direct iOS high-efficiency image container parsing.
Video Sweeps	.mov, .mp4, .m4v	Smart temporal sampling & frame extraction for video clips.
Apple Live Photos	.heic + .mov pairs	Merges keyframe still photo + short video motion burst for multi-view density.

---

🛠️ Local Installation & Quickstart

To run ZipSplatPlus locally on your machine with GPU support:

# 1. Clone the repository
git clone https://github.com/ai3d-dev/ZipSplatPlus.git
cd ZipSplatPlus

# 2. Create and activate a virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# 3. Install PyTorch (with CUDA support matching your system)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

# 4. Install ZipSplatPlus dependencies
pip install -r requirements.txt

# 5. Launch the local Gradio web server
python app.py

Open your browser and navigate to http://localhost:7860.

---

🛸 System Architecture

graph TD
    A["📷 Input Media (Images / Videos / Live Photos)"] --> B["⚙️ Media Loader & Smart Frame Sampler"]
    B --> C["⚡ ZipSplat Neural Model (CUDA / ZeroGPU)"]
    C --> D["🔮 3D Gaussian Primitives (Ordered by Token Strokes)"]
    D --> E["🌐 Spark.js Interactive Viewer + Stroke HUD"]
    D --> F["🖌️ Stroke-by-Stroke Playback Video (.mp4)"]
    D --> G["🎥 360° Turntable Video (.mp4)"]
    D --> H["💾 Downloadable 3D Model (.ply)"]

---

📜 References & Citation

@article{veicht2024zipsplat,
  title={ZipSplat: Fewer Gaussians, Better Splats},
  author={Veicht, Alexander and others},
  journal={arXiv preprint},
  year={2024}
}

Acknowledgments

• ZipSplat Model: veichta/zipsplat
• Spark.js Viewer: @sparkjsdev/spark
• Hugging Face: ZeroGPU & Gradio Framework

---

Developed by ai3d-dev • Built for the 3D Vision & Open-Source Community