Add model card for INSPATIO-WORLD

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +59 -3
README.md CHANGED
@@ -1,3 +1,59 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: image-to-video
4
+ ---
5
+
6
+ # INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
7
+
8
+ INSPATIO-WORLD is a real-time framework capable of recovering and generating high-fidelity, dynamic interactive scenes from a single reference video. It utilizes a Spatiotemporal Autoregressive (STAR) architecture to ensure global consistency and precise interaction control.
9
+
10
+ [[Paper](https://huggingface.co/papers/2604.07209)] [[Project Page](https://inspatio.github.io/inspatio-world/)] [[GitHub](https://github.com/inspatio/inspatio-world)]
11
+
12
+ ## Installation
13
+
14
+ To set up the environment, follow these steps:
15
+
16
+ ```bash
17
+ conda env create -f environment.yml
18
+ conda activate inspatio_world
19
+
20
+ # Install flash-attn
21
+ pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
22
+ ```
23
+
24
+ ## Inference
25
+
26
+ The full pipeline runs in three steps: video captioning, depth estimation, and spatiotemporal autoregressive inference. You can run the full pipeline using the provided script:
27
+
28
+ ### Quick Start
29
+ ```bash
30
+ # 1. Place your .mp4 video(s) in a folder
31
+ mkdir -p my_videos
32
+ cp your_video.mp4 my_videos/
33
+
34
+ # 2. Run the full pipeline (using 14B model)
35
+ bash run_test_pipeline.sh \
36
+ --input_dir ./my_videos \
37
+ --traj_txt_path ./traj/x_y_circle_cycle.txt
38
+ ```
39
+
40
+ ### Using the 1.3B Model
41
+ For a lighter and faster version, use the 1.3B model:
42
+ ```bash
43
+ bash run_test_pipeline.sh \
44
+ --input_dir ./test/example \
45
+ --traj_txt_path ./traj/x_y_circle_cycle.txt \
46
+ --config_path ./configs/inference_1.3b.yaml \
47
+ --checkpoint_path ./checkpoints/InSpatio-World-1.3B/InSpatio-World-1.3B.safetensors
48
+ ```
49
+
50
+ ## Citation
51
+
52
+ ```bibtex
53
+ @misc{inspatio-world,
54
+ title={InSpatio-World},
55
+ author={InSpatio Team},
56
+ howpublished={\url{https://github.com/inspatio/inspatio-world}},
57
+ year={2026}
58
+ }
59
+ ```