Parallax-1.3B (research preview)
Small trained modules and a streaming runtime for 1.3B causal video world models:
- Loop closure by trained context retrieval. A 12 M-parameter adapter lets LingBot-World v2 1.3B reuse frames retrieved by camera-frustum overlap. With 9 cache slots, the camera goes out, comes back, and the model redraws what it drew before: 21.7 dB against 13.5 dB without retrieval.
- A KV refiner instead of the clean cache pass. An 81 M network repairs the cache entry left by the last denoising pass, so the extra pass that rewrites the cache is not needed.
- A single-file real-time engine.
infer_realtime.py(PyTorch only, no diffusers) runs the frame-wise 2-step Causal Forcing++ generator at 51.5 ms per latent frame on one H100.
This is a non-commercial research release, not a standalone model. It needs the base checkpoints listed below. Every number here was measured on 20 November 2026; conditions are stated next to it. Read Limitations before drawing conclusions.
Contact: @arthur_hyper88 on X
Top row: 9-slot sliding window, no retrieval. Rows 2–5: trained retrieval (loading threshold 0.7 / 0.8 / 0.9 / 0.95). Five columns on the way out, four on the way back. Without retrieval the camera comes back to a different room.
Left: sliding window. Right: trained retrieval. Download as MP4 (2.6 MB, labelled).
What is in this repository
| File | What it is | Base checkpoint it needs |
|---|---|---|
lingbot/retrieval_adapter.safetensors |
Geometry-aware attention bias for retrieved cache frames (12 M parameters) | robbyant/lingbot-world-v2-1.3b-causal-fast |
lingbot/kv_refiner.safetensors |
KV refiner for LingBot chunks (81 M) | same |
wan-cf/kv_refiner.safetensors |
KV refiner for the frame-wise 2-step generator (81 M) | Wan-AI/Wan2.1-T2V-1.3B + causal-forcing++/framewise-2step.pt from zhuhz22/Causal-Forcing |
infer_realtime.py |
Streaming engine: slot KV cache (sink / retrieved / recent), fused kernels and CUDA graphs, KV refiner, frustum index, TAEHV-layout pixel decoder | |
stage2/ |
LingBot re-implementation with an explicit context interface; training and evaluation scripts | |
train/, bench/, tools/ |
Refiner training, latency benchmark, long-rollout evaluation |
Results
1. Loop closure on LingBot-World v2 1.3B
Test. The camera follows a real handheld path for 20 latent frames and returns along the same path (157 RGB frames). Every frame of the way back has a twin on the way out, generated by the model itself from the same pose. The score is the PSNR between the two. 35 held-out DL3DV scenes, one seed per scene, rotation between the start and the far end from 22° to 180°.
Cache. 9 latent frames visible: 1 sink frame, up to 3 retrieved frames, the recent window, and the chunk of 4 being generated.
| Setting | Revisit PSNR (dB) | Against the 9-slot window |
|---|---|---|
| 9-slot window, no retrieval | 13.5 ± 0.2 | |
| LingBot default (18 frames of history, 6 sink frames) | 13.9 ± 0.3 | +0.49 ± 0.11 |
| Frustum retrieval, untrained | 13.5 ± 0.2 | +0.10 ± 0.07 |
| No sink frame, no retrieval | 10.3 ± 0.2 | −3.14 ± 0.23 |
| Trained retrieval, loading threshold 0.8 | 21.7 ± 0.5 | +8.22 ± 0.38 (35 of 35 scenes, minimum +4.5) |
| Same, scenes with at least 90° of rotation (13 scenes) | 20.9 | +7.95 ± 0.60 |
| Same, return path shifted by half a latent frame | 20.9 ± 0.5 | +7.70 ± 0.42 |
Over two loops (317 frames) the third and fourth pass along the path stay at 22.6 and 22.5 dB against 12.1 and 11.8 dB (loading threshold 0.5).
How it works. stage2/lb.py re-implements LingBot so that a chunk receives the keys and values of the frames it may attend to. The adapter gives every token an embedding of its viewing ray in the current camera frame, and a per-layer projection appends channels to the queries of the chunk and to the keys of the loaded frames: an attention bias learned from geometry. LingBot itself is frozen.
- A stored frame is loaded only if it covers the view of the chunk above a threshold. When nothing is loaded, the pass is exactly the base model's.
- At threshold 0.8, 94% of the chunks on the way out load nothing; on the way back 2.8 frames are loaded per chunk. Fine detail on the way out equals the base model's and frame-to-frame motion drops by 2%.
- Training: 474 out-and-back clips from 237 DL3DV scenes, 3,396 steps, 37 minutes on one H100.
Two things we checked.
- Temporal RoPE beyond the 21-frame training window. Retrieved frames keep their own temporal index. Clamping the offset to 16 frames changes the result by +0.08 ± 0.05 dB over one loop and +0.04 over two, so the native index stays (tested up to 76 latent frames apart).
- The sink frame. Without it the base model collapses (10.3 dB, colour noise). With one sink frame the camera still turns away: views at the far end are as different from the first frame as in the untrained model (11–12 dB), and frame-to-frame motion is unchanged.
What did not work. A first version trained LoRA and geometry offsets on every token. It reached 20–24 dB, but the same adapter with retrieval switched off already gained 5 dB: the regression loss blurred new content and slowed the camera, and two blurry videos look alike. The two restrictions above (load only covering frames, bias only towards loaded frames) come from that failure.
2. KV refiner on LingBot
The refiner replaces the clean pass of every chunk (1 of 5 passes). The first chunk, which holds the sink frame, keeps its clean pass. 14 held-out scenes, two loops (317 frames), retrieval on, same noise in every row:
| Cache update | Revisit PSNR (dB) | Third pass | Fourth pass | Brightness, end ÷ start | Time per chunk |
|---|---|---|---|---|---|
| Clean pass (5 passes per chunk) | 20.8 ± 0.5 | 20.9 | 20.6 | 1.01 | 620 ms |
| KV refiner (4 passes + refiner) | 20.5 ± 0.6 (−0.30 ± 0.18) | 20.9 | 20.4 | 1.00 | 522 ms |
The refiner was started from the one trained on Wan2.1 + Causal Forcing++ and tuned for 17 minutes on one H100. It costs 4–8% of fine detail.
3. Real-time engine on Wan2.1 + Causal Forcing++
Frame-wise generator, 2 steps per frame, text-to-video, no camera input. One latent frame is 4 RGB frames. One H100, 480×832, bfloat16, KV refiner and pixel decoder included, idle machine.
| Cache slots | DiT pass (ms) | Frame, median (ms) | Frame, p95 (ms) |
|---|---|---|---|
| 6 | 16.9 | 44.7 | 45.0 |
| 9 | 20.3 | 51.5 | 51.6 |
| 13 | 25.6 | 62.3 | 63.9 |
| 21 | 34.5 | 80.0 | 80.2 |
- With CUDA graphs alone, 9 slots took 68.1 ms. The gain comes from compiling only the two pure functions on each side of self-attention, which cuts a pass from 4,389 kernel launches to 1,389.
- Things that made it slower on this workload: FP8 linear layers (
torch._scaled_mm), SageAttention v1, andtorch.compileon the whole pass.
297-frame stability, 32 unseen prompts, 9 slots. Ratios compare the last second with the first (1 = no drift); the last column is DINOv2 similarity to the first second.
| Cache update | Brightness | Saturation | Fine detail | Similarity to the start |
|---|---|---|---|---|
| Clean pass (3 passes per frame) | 0.98 | 1.04 | 1.01 | 0.827 ± 0.022 |
| No cache update | 1.23 | 0.82 | 1.06 | 0.691 ± 0.030 |
| KV refiner + 2 clean sink frames + value calibration | 0.99 | 0.99 | 1.05 | 0.896 ± 0.012 |
| KV refiner without a sink frame | 1.12 | 1.53 | 1.21 | 0.553 ± 0.019 |
Rows: clean pass · no cache update · refiner with 2 sink frames · 1 sink frame · no sink frame · no value calibration · 197 M refiner.
Limitations
- Two results on two models. Loop closure is shown on LingBot, which generates chunks of 4 latent frames with 4 steps: 522 ms per chunk on one H100 with unoptimised PyTorch, that is 31 frames per second of throughput and about half a second of reaction time. The 51.5 ms figure is for a text-to-video generator without camera control. Unifying them means distilling the camera model from 4-frame chunks to frame-wise causal steps, as Causal Forcing++ does for text-to-video. That is the next step, and it is not done.
- Self-consistency, not fidelity. The loop-closure score compares the model with itself. It does not say the redrawn scene is the real one.
- Small evaluation. 35 scenes (14 for the refiner), one seed each, automatic metrics only, no human study.
- Soft training data. The adapter was trained on 480×270 frames upscaled to 832×480. Some return paths are soft half-way back.
- Latency on newer GPUs is not measured.
- The KV refiner is our own implementation of the idea described in the FastFrames paper. No FastFrames code or weights were used.
Usage
Research code. Paths in the scripts assume the LingBot-World v2 repository is checked out next to the weights; see the header of each script.
Latency of the frame-wise engine:
python infer_realtime.py --checkpoint framewise-2step.pt --decoder taew2_1.pth \
--refiner wan-cf/kv_refiner.safetensors --cache-update refine --slots 9 --sink 2 --runner fused --frames 220
Loop closure on LingBot (examples are folders with image.jpg, poses.npy, intrinsics.npy):
python stage2/eval_revisit.py --examples <examples> --weights <lingbot-world-v2-1.3b-causal-fast> \
--vae Wan2.1_VAE.pth --decoder taew2_1.pth --text prompt.safetensors --out out --variants \
window:9:1:0:native \
retrieval:9:1:3:native:lingbot/retrieval_adapter.safetensors:0.8 \
retrieval_refiner:9:1:3:native:lingbot/retrieval_adapter.safetensors:0.8::lingbot/kv_refiner.safetensors
Licences and attribution
- This repository is released under CC BY-NC-SA 4.0. Files that carry an Apache-2.0 header (the engine and tools for Wan2.1 / Causal Forcing) may also be used under Apache-2.0.
lingbot/andstage2/derive from LingBot-World v2 by Robbyant (CC BY-NC-SA 4.0).- The adapter and the LingBot refiner were trained on DL3DV-10K under its terms of use (non-commercial research). The dataset is not redistributed here; the figures show model outputs on a few of its scenes at thumbnail size.
- Base models: Wan2.1 (Apache-2.0), Causal Forcing (Apache-2.0). Pixel decoder layout: TAEHV (MIT).
- Methods: KV refinement after FastFrames: Video Generation at Interactive Latency; context retrieval by view overlap after the Autoregressive View Model; cache semantics after Self Forcing and Causal Forcing++.
Model tree for hyper88/Parallax-1.3B-preview
Base model
Wan-AI/Wan2.1-T2V-1.3B

