README.md

metadata

title: MiniWorld Simulator
emoji: 🌍
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
python_version: '3.12'
short_description: Drive a video world model with a camera path from one frame
startup_duration_timeout: 45m
license: apache-2.0
models:
  - zhaoyian01/MiniWorld
  - Wan-AI/Wan2.2-TI2V-5B
tags:
  - world-model
  - video-generation
  - camera-control
  - image-to-video
  - streaming-generation
pinned: false

MiniWorld — camera-controlled world model

Interactive demo of MiniWorld (zhaoyian01/MiniWorld, 1B RealEstate10K checkpoint): give it a single frame and a procedural camera path, and it autoregressively rolls the world forward.

This Space reproduces the authors' reference inference route (zhao-yian/MiniWorld):

python -m miniworld.sample --dataset re10k \
  --init_image <frame> --custom_camera_trajectory orbit_right \
  --wm_model 1B --total_len 64 --trajectory_magnitude 3.0

• 240×320 frames, Wan2.2 VAE (16× spatial / 4× temporal), 48 latent channels.
• AR-diffusion streaming sampler: df_chunk_size=2, df_ardiff_step=5, 8 in-flight chunks, 24 cached chunks, 1 attention-sink frame, CFG 2.0, 100 sampling steps (the streaming schedule caps the effective per-chunk steps at 40) — i.e. exactly scripts/sample_re10k.sh.
• Camera conditioning is the repo's procedural trajectory builder turned into ray-encoding features (freq=15, unnormalized translations), so no ground-truth poses or reference video are needed.
• Streaming causal VAE decode-on-commit, matching the reference pipeline.
• The Camera speed slider is --trajectory_magnitude expressed in "magnitude at total_len=64" units. The trajectory builder spreads the whole path evenly over the rollout, so the authors advise scaling magnitude linearly with length to keep the apparent speed constant (3.0 @ 64 → 4.5 @ 96); the Space does that for you, and at 64 latent frames it is the reference value verbatim.

Measured on this Space's ZeroGPU slice: 46 s at 20 latent frames (77 output frames), 72 s at 32 (125 frames), 157 s at 64 (253 frames). The @spaces.GPU(duration=...) estimate replays the sampler's asynchronous step schedule to cost each rollout, so short clips reserve proportionally less quota.

The action-conditioned DROID checkpoint is intentionally not exposed: its conditioning requires per-dataset q01/q99 action normalization statistics that only ship with the LeRobot DROID dataset, so there is no faithful dataset-free input for it.

Example frames are the first frames of the authors' own RealEstate10K rollout grid (assets/demo_re10k.mp4 in the model repo).