README.md

metadata

title: LiveWan
emoji: 🎞️
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.24.0
app_file: app.py
python_version: '3.10'
short_description: Streaming text-to-video you can steer mid-stream
startup_duration_timeout: 1h
pinned: false
license: apache-2.0
models:
  - JonathanColetti/LiveWan
  - Wan-AI/Wan2.1-T2V-1.3B
tags:
  - text-to-video
  - streaming
  - real-time
  - wan2.1

LiveWan

Streaming, steerable text-to-video from a 1.3B student distilled out of Wan2.1-T2V-14B with SF-DMD. It generates video continuously rather than as a fixed clip — 750 ms of 640×368 at a time, extended block by block — and the text conditioning can be swapped mid-stream without clearing the K/V cache, so the scene continues instead of cutting.

• Code: https://github.com/JonathanColetti/LiveWan
• Weights and data: https://huggingface.co/JonathanColetti/LiveWan

What this Space runs

app.py drives the project's own serving engine, wanstreamer.serve.engine.Engine — the same code path livewan-serve runs locally. wanstreamer/ and wan21_patches/ here are copies of the GitHub repo's; the Wan2.1 reference code is cloned and patched at startup exactly as setup.sh does.

Steering is a schedule, not a button. This is the one real difference from the local demo. A GPU worker is forked per request and cannot be steered from outside while it runs, so the swap is given up front — "switch to this at t = N seconds" — rather than clicked mid-stream. The swap itself is the live one: Engine.steer replaces the cross-attention conditioning and leaves the K/V cache in place.

Steers can come from the 96-prompt bank or from free text. umt5-xxl is downloaded and loaded at module scope, so ZeroGPU packs it with the rest of the weights instead of making every cold worker pay for a lazy 11 GB load; it is most of why this Space pulls 29 GB at startup. Free text is not numerically comparable to the bank, though — umt5 embeddings differ slightly by hardware, so the same string encoded on the training box is a slightly different tensor.

torch.compile of the VAE decoder is off, since it cannot run in a ZeroGPU worker, and that is what keeps this Space below real time. Measured here: 386 ms to generate a block and 631 ms to decode it, against a 750 ms budget — 0.74× real time. Decoding is the whole of the gap; locally, with the decoder compiled, it takes 343 ms and the same checkpoint sustains about 2.7× real time.

Two more things this port had to work around, both specific to running inside a forked ZeroGPU worker:

• Block generation is forced under torch.no_grad(). generate_block has no no_grad of its own and relies on requires_grad_(False) set at load time. The weights are loaded in the main process, packed to disk and streamed into the worker, and they arrive with requires_grad back to True — so every layer of every forward was being retained and a single block peaked at 44.2 GB.
• A CUDA OOM inside a worker does not say so. PyTorch builds its OOM message by asking NVML which processes hold memory; NVML is not available there, so the run dies on NVML_SUCCESS == r INTERNAL ASSERT FAILED instead.