Spaces:

Yosun
/

ZipSplatPlus

Running on Zero

File size: 5,998 Bytes

---
title: "ZipSplatPlus: Real-time 3D Gaussian Splatting & Stroke Playback"
emoji: ⚡
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.20.0
python_version: '3.12'
app_file: app.py
pinned: false
short_description: 3D Gaussian Splatting with stroke playback.
tags:
- 3d-gaussian-splatting
- stroke-playback
- 3d-reconstruction
- single-image-to-3d
- multi-view-reconstruction
- sparkjs
- live-photos
- heic-support
- zerogpu
- 3d-vision
- computer-vision
- novel-view-synthesis
- gradio
models:
- veichta/zipsplat
license: mit
thumbnail: thumbnail.png
---

<div align="center">

# 🔥 ZipSplatPlus: Real-Time 3D Gaussian Splatting & Stroke Playback

[![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Space%20Demo-yellow.svg)](https://huggingface.co/spaces/Yosun/ZipSplatPlus)
[![GitHub Repository](https://img.shields.io/badge/GitHub-ai3d--dev%2FZipSplatPlus-blue?logo=github)](https://github.com/ai3d-dev/ZipSplatPlus)
[![ZeroGPU](https://img.shields.io/badge/ZeroGPU-Enabled-green.svg)](https://huggingface.co/zero-gpu)
[![Model Weights](https://img.shields.io/badge/%F0%9F%A4%97%20Weights-veichta%2Fzipsplat-blue.svg)](https://huggingface.co/veichta/zipsplat)
[![Paper](https://img.shields.io/badge/Paper-ZipSplat-red.svg)](https://huggingface.co/veichta/zipsplat)
[![License: MIT](https://img.shields.io/badge/License-MIT-purple.svg)](https://opensource.org/licenses/MIT)

<p align="center">
  <b>Instant 3D Gaussian Splatting from single/multi-view images, videos, & Apple Live Photos with ZeroGPU dynamic acceleration, real-time interactive stroke-by-stroke WebGL playback, 360° turntable & stroke video generation, and compressed .PLY export.</b>
</p>

![ZipSplatPlus Showcase Banner](thumbnail.png)

</div>

---

## 🚀 Quick Links

- 🌐 **Live Web App**: [Hugging Face Space (ZeroGPU)](https://huggingface.co/spaces/Yosun/ZipSplatPlus)
- 📦 **GitHub Repository**: [ai3d-dev/ZipSplatPlus](https://github.com/ai3d-dev/ZipSplatPlus)
- 🔮 **Base Model & Weights**: [veichta/zipsplat](https://huggingface.co/veichta/zipsplat)

---

## 🌟 Key Features & Capabilities

- ⚡ **Single-Pass 3D Scene Generation**: Reconstruct high-fidelity 3D Gaussian Splatting scenes from single or unconstrained multi-view photos without per-scene optimization loops.
- 🖌️ **Interactive Stroke-by-Stroke Playback**: Play back each predicted 3D Gaussian "stroke" step-by-step! Explore how neural tokens paint the 3D scene in real-time with scrub slider, play/pause controls, and speed adjustments.
- 🎨 **Multiple Render Modes**:
  - **Cumulative**: Watch the 3D scene build up progressively stroke by stroke.
  - **Single Stroke**: Isolate and inspect individual 3D Gaussian strokes (32 Gaussians/token).
  - **Colorized**: Assign vibrant, distinct color palettes to each stroke to visualize token layout.
- 🎬 **Stroke-by-Stroke Turntable Video Export**: Automatically renders smooth MP4 videos showing the 3D scene being painted stroke-by-stroke during camera orbit.
- 🎨 **Real-Time Web Viewer**: Powered by **Spark.js** (`@sparkjsdev/spark`), enabling 60 FPS orbit, pan, and zoom inside the browser using client-side WebGL shaders.
- 📱 **Native Apple Live Photo & HEIC/HEIF Support**: Direct decoding of iPhone/iPad `.HEIC`/`.HEIF` photos, `.MOV`/`.MP4` videos, and paired Apple Live Photos (`.HEIC` + `.MOV`).
- 💾 **Standard 3D Model Export**: Download compressed Gaussian `.PLY` files compatible with Blender, Unity, Unreal Engine, SuperSplat, and WebGL viewers.

---

## 📸 Supported Input Formats

ZipSplatPlus handles a wide range of visual inputs with intelligent frame sampling:

| Format Category | File Extensions | Capabilities |
| :--- | :--- | :--- |
| **Still Images** | `.jpg`, `.jpeg`, `.png`, `.webp` | Single or multi-view photo capture. |
| **Apple HEIC/HEIF** | `.heic`, `.heif` | Direct iOS high-efficiency image container parsing. |
| **Video Sweeps** | `.mov`, `.mp4`, `.m4v` | Smart temporal sampling & frame extraction for video clips. |
| **Apple Live Photos** | `.heic` + `.mov` pairs | Merges keyframe still photo + short video motion burst for multi-view density. |

---

## 🛠️ Local Installation & Quickstart

To run **ZipSplatPlus** locally on your machine with GPU support:

```bash
# 1. Clone the repository
git clone https://github.com/ai3d-dev/ZipSplatPlus.git
cd ZipSplatPlus

# 2. Create and activate a virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# 3. Install PyTorch (with CUDA support matching your system)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

# 4. Install ZipSplatPlus dependencies
pip install -r requirements.txt

# 5. Launch the local Gradio web server
python app.py
```

Open your browser and navigate to `http://localhost:7860`.

---

## 🛸 System Architecture

```mermaid
graph TD
    A["📷 Input Media (Images / Videos / Live Photos)"] --> B["⚙️ Media Loader & Smart Frame Sampler"]
    B --> C["⚡ ZipSplat Neural Model (CUDA / ZeroGPU)"]
    C --> D["🔮 3D Gaussian Primitives (Ordered by Token Strokes)"]
    D --> E["🌐 Spark.js Interactive Viewer + Stroke HUD"]
    D --> F["🖌️ Stroke-by-Stroke Playback Video (.mp4)"]
    D --> G["🎥 360° Turntable Video (.mp4)"]
    D --> H["💾 Downloadable 3D Model (.ply)"]
```

---

## 📜 References & Citation

```bibtex
@article{veicht2024zipsplat,
  title={ZipSplat: Fewer Gaussians, Better Splats},
  author={Veicht, Alexander and others},
  journal={arXiv preprint},
  year={2024}
}
```

### Acknowledgments

- **ZipSplat Model**: [veichta/zipsplat](https://huggingface.co/veichta/zipsplat)
- **Spark.js Viewer**: [@sparkjsdev/spark](https://www.npmjs.com/package/@sparkjsdev/spark)
- **Hugging Face**: [ZeroGPU & Gradio Framework](https://huggingface.co/)

---

<p align="center">
  Developed by <a href="https://github.com/ai3d-dev">ai3d-dev</a> • Built for the 3D Vision & Open-Source Community
</p>