AlayaRenderer-Flash

AlayaRenderer-Flash is a real-time generative world renderer. A game engine exports G-buffers (albedo, depth, metallic, normal, roughness), and the model renders them into RGB frames in a style set by a text prompt. It streams autoregressively with 4 denoising steps, at about 30 FPS on one NVIDIA H200.

These are the weights behind the live demo, fine-tuned for SuperTuxKart.

• Paper: Generative World Renderer at the Speed of Play
• Code: github.com/AlayaLab/AlayaRenderer-Flash
• Base model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers (supplies the text encoder, tokenizer and VAE config; download it separately)

Files

File	Size	Description
transformer/	3.1 GB	1.3B HeliosTransformer3DModel (bf16): 96 input channels (16 RGB + 5×16 G-buffer latents), text-sink, multi-term memory patches
transformer_partial.safetensors	0.3 GB	G-buffer patch embedding, short/mid/long memory patchifiers, output scale-shift table, text-sink weights (loaded on top of transformer/)
gbuf_small_encoder.safetensors	37 MB	Grouped distilled G-buffer encoder (5 branches, hidden dim 128) that replaces the Wan VAE encoder
tiny_decoder/taew2_1_ft.safetensors	45 MB	Fine-tuned TAEW2.1 stateful tiny decoder that replaces the Wan VAE decoder

Usage

git clone https://github.com/AlayaLab/AlayaRenderer-Flash && cd AlayaRenderer-Flash
uv sync
uv run python examples/fetch_samples.py
uv run alayarender-flash render samples/window-00.npy samples/window-01.npy \
    --prompt "heavy snow at night, warm street lights" -o snow.mp4

The CLI downloads these weights and the Wan 2.1 base model on first use. samples/ has two recorded SuperTuxKart G-buffer windows (uint8 (16, 5, 448, 832, 3), channel order albedo, depth, metallic, normal, roughness). For the interactive SuperTuxKart demo, see the repository README.

Model details

• Architecture: Wan 2.1 1.3B DiT extended with the Helios autoregressive design: unified history injection, multi-term memory patchification with a long/mid/short history of 8/2/1 latents, first-frame anchor and text-sink prompt anchoring. G-buffer latents are concatenated with the noisy RGB latent before patch embedding.
• Inference: 832×448, latent window 4 (13 pixel frames to start, then 16 per window), 4 Euler steps, guidance scale 1.0 (CFG is distilled into the model).
• Training: fine-tuned on SuperTuxKart captures from the AlayaRenderer-Flash base renderer with autoregressive training, CFG distillation, progressive 4-step distillation and Mean Flow Distillation (see the paper). This is checkpoint 600 of the final stage. Training code is not part of this release.

Intended use and limitations

• This model is for research on G-buffer-conditioned neural rendering and interactive generative worlds. It is tuned for SuperTuxKart G-buffers at 832×448, and quality drops on other engines, other G-buffer conventions or other resolutions.
• Prompt control is limited: the distilled student runs at guidance scale 1.0, and some styles barely change the output.
• In long rollouts it can drift in color and lighting. Reset the history when the scene or prompt changes, as the demo does.
• Real-time rates need a Hopper-class GPU. Consumer GPUs run, but slower.

Citation

@article{lin2026alayarendererflash,
  title   = {Generative World Renderer at the Speed of Play},
  author  = {Lin, Guixu and Huang, Zheng-Hui and Yang, Siqi and Yang, Ming-Hsuan and Zhang, Kaipeng and Wang, Zhixiang},
  journal = {arXiv preprint arXiv:2607.18703},
  year    = {2026}
}