File size: 2,057 Bytes
9b28edd 2fef971 9b28edd 2fef971 9b28edd 2fef971 1ffc26b 6bfaf08 1ffc26b 9b28edd 2fef971 9b28edd 2fef971 9b28edd 2fef971 9b28edd 2fef971 1ffc26b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
license: other
tags:
- world-model
- diffusion
- supertuxkart
- webgpu
- onnx
- pytorch
library_name: pytorch
---
# Neural Drive — SuperTuxKart World Model
An action-conditioned **world model for SuperTuxKart**: it predicts the next frame from the last
7 frames plus your controls, with **no game engine underneath**. Trained in PyTorch on gameplay
(actions extracted automatically), and shipped in two runtimes from the same weights.
<video controls autoplay loop muted src="https://huggingface.co/codelion/neural-drive-model/resolve/main/preview.mp4"></video>
**▶ Live in-browser demo (WebGPU):** https://huggingface.co/spaces/codelion/neural-drive
- **Resolution** 384×192 · **Action space** 6-dim `[accelerate, brake, left, right, nitro, drift]`
- **Codec** ConvVAE (z_ch 8, /8 spatial) · **Dynamics** LatentDiT (dim 768 / depth 12 / patch 3, **130.8M**), rectified flow + per-frame diffusion forcing
- **Sampling** few-step distilled (**k = 2**)
## Runtimes
| runtime | where | notes |
|---|---|---|
| **PyTorch** | CUDA / CPU | training + reference (fp32) |
| **ONNX / WebGPU** | any browser | fp16, runs on the visitor's GPU |
## Files
### PyTorch checkpoints (fp32)
| file | what |
|---|---|
| `dit.pt` | dynamics, 100k steps (k=8 teacher) |
| `dit_distilled.pt` | few-step distilled student (k=2) |
| `vae.pt` | ConvVAE codec |
| `decoder_big.pt` | larger LPIPS decoder |
| `config.json` | architecture config |
### Web / ONNX (fp16 — the in-browser build)
| file | what |
|---|---|
| `web/genframe_fp16.onnx` | single-graph frame generator: prefills the 7-frame context once, then runs the k=2 denoise steps internally (KV-cache in the graph). ~262 MB |
| `web/vae.onnx` | VAE decoder (fp32, to keep decode stable) |
| `web/seeds.bin` | 18 spawn context states, raw float32 `[18,7,8,24,48]` |
| `web/meta.json` | shapes + config for the web runtime |
## Reference
**Playing SuperTuxKart in Your Head: Learning Interactive World Models from Mobile Gameplay Videos**
— https://x.com/latent_node/status/2033769450592485543
|