README.md

---
title: LiveWan
emoji: 🎞️
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.24.0
app_file: app.py
python_version: "3.10"
short_description: Streaming text-to-video you can steer mid-stream
startup_duration_timeout: 1h
pinned: false
license: apache-2.0
models:
  - JonathanColetti/LiveWan
  - Wan-AI/Wan2.1-T2V-1.3B
tags:
  - text-to-video
  - streaming
  - real-time
  - wan2.1
---

# LiveWan

Streaming, steerable text-to-video from a 1.3B student distilled out of
Wan2.1-T2V-14B with SF-DMD. It generates video continuously rather than as a fixed
clip — 750 ms of 640×368 at a time, extended block by block — and the text
conditioning can be swapped mid-stream without clearing the K/V cache, so the scene
continues instead of cutting.

- Code: https://github.com/JonathanColetti/LiveWan
- Weights and data: https://huggingface.co/JonathanColetti/LiveWan

## What this Space runs

`app.py` drives the project's own serving engine, `wanstreamer.serve.engine.Engine`
— the same code path `livewan-serve` runs locally. `wanstreamer/` and
`wan21_patches/` here are copies of the GitHub repo's; the Wan2.1 reference code is
cloned and patched at startup exactly as `setup.sh` does.

**Steering is a schedule, not a button.** This is the one real difference from the
local demo. A GPU worker is forked per request and cannot be steered from outside
while it runs, so the swap is given up front — "switch to this at t = N seconds" —
rather than clicked mid-stream. The swap itself is the live one: `Engine.steer`
replaces the cross-attention conditioning and leaves the K/V cache in place.

Steers can come from the 96-prompt bank or from free text. umt5-xxl is downloaded
and loaded at module scope, so ZeroGPU packs it with the rest of the weights instead
of making every cold worker pay for a lazy 11 GB load; it is most of why this Space
pulls 29 GB at startup. Free text is not numerically comparable to the bank, though
— umt5 embeddings differ slightly by hardware, so the same string encoded on the
training box is a slightly different tensor.

`torch.compile` of the VAE decoder is off, since it cannot run in a ZeroGPU worker,
and that is what keeps this Space below real time. Measured here: 386 ms to generate
a block and 631 ms to decode it, against a 750 ms budget — **0.74× real time**.
Decoding is the whole of the gap; locally, with the decoder compiled, it takes
343 ms and the same checkpoint sustains about 2.7× real time.

Two more things this port had to work around, both specific to running inside a
forked ZeroGPU worker:

- **Block generation is forced under `torch.no_grad()`.** `generate_block` has no
  `no_grad` of its own and relies on `requires_grad_(False)` set at load time. The
  weights are loaded in the main process, packed to disk and streamed into the
  worker, and they arrive with `requires_grad` back to `True` — so every layer of
  every forward was being retained and a single block peaked at 44.2 GB.
- **A CUDA OOM inside a worker does not say so.** PyTorch builds its OOM message by
  asking NVML which processes hold memory; NVML is not available there, so the run
  dies on `NVML_SUCCESS == r INTERNAL ASSERT FAILED` instead.