metadata
license: apache-2.0
base_model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
library_name: diffusers
pipeline_tag: video-to-video
tags:
- g-buffer
- neural-rendering
- real-time
- autoregressive
- video-diffusion
- supertuxkart
AlayaRenderer-Flash
AlayaRenderer-Flash is a real-time generative world renderer. A game engine exports G-buffers (albedo, depth, metallic, normal, roughness), and the model renders them into RGB frames in a style set by a text prompt. It streams autoregressively with 4 denoising steps, at about 30 FPS on one NVIDIA H200.
These are the weights behind the live demo, fine-tuned for SuperTuxKart.
- Paper: Generative World Renderer at the Speed of Play
- Code: github.com/AlayaLab/AlayaRenderer-Flash
- Base model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers (supplies the text encoder, tokenizer and VAE config; download it separately)
Files
| File | Size | Description |
|---|---|---|
transformer/ |
3.1 GB | 1.3B HeliosTransformer3DModel (bf16): 96 input channels (16 RGB + 5×16 G-buffer latents), text-sink, multi-term memory patches |
transformer_partial.safetensors |
0.3 GB | G-buffer patch embedding, short/mid/long memory patchifiers, output scale-shift table, text-sink weights (loaded on top of transformer/) |
gbuf_small_encoder.safetensors |
37 MB | Grouped distilled G-buffer encoder (5 branches, hidden dim 128) that replaces the Wan VAE encoder |
tiny_decoder/taew2_1_ft.safetensors |
45 MB | Fine-tuned TAEW2.1 stateful tiny decoder that replaces the Wan VAE decoder |
Usage
git clone https://github.com/AlayaLab/AlayaRenderer-Flash && cd AlayaRenderer-Flash
uv sync
uv run python examples/fetch_samples.py
uv run alayarender-flash render samples/window-00.npy samples/window-01.npy \
--prompt "heavy snow at night, warm street lights" -o snow.mp4
The CLI downloads these weights and the Wan 2.1 base model on first use.
samples/ has two recorded SuperTuxKart G-buffer windows
(uint8 (16, 5, 448, 832, 3), channel order albedo, depth, metallic, normal,
roughness). For the interactive SuperTuxKart demo, see the repository README.
Model details
- Architecture: Wan 2.1 1.3B DiT extended with the Helios autoregressive design: unified history injection, multi-term memory patchification with a long/mid/short history of 8/2/1 latents, first-frame anchor and text-sink prompt anchoring. G-buffer latents are concatenated with the noisy RGB latent before patch embedding.
- Inference: 832×448, latent window 4 (13 pixel frames to start, then 16 per window), 4 Euler steps, guidance scale 1.0 (CFG is distilled into the model).
- Training: fine-tuned on SuperTuxKart captures from the AlayaRenderer-Flash base renderer with autoregressive training, CFG distillation, progressive 4-step distillation and Mean Flow Distillation (see the paper). This is checkpoint 600 of the final stage. Training code is not part of this release.
Intended use and limitations
- This model is for research on G-buffer-conditioned neural rendering and interactive generative worlds. It is tuned for SuperTuxKart G-buffers at 832×448, and quality drops on other engines, other G-buffer conventions or other resolutions.
- Prompt control is limited: the distilled student runs at guidance scale 1.0, and some styles barely change the output.
- In long rollouts it can drift in color and lighting. Reset the history when the scene or prompt changes, as the demo does.
- Real-time rates need a Hopper-class GPU. Consumer GPUs run, but slower.
Citation
@article{lin2026alayarendererflash,
title = {Generative World Renderer at the Speed of Play},
author = {Lin, Guixu and Huang, Zheng-Hui and Yang, Siqi and Yang, Ming-Hsuan and Zhang, Kaipeng and Wang, Zhixiang},
journal = {arXiv preprint arXiv:2607.18703},
year = {2026}
}