LynnReal-Hand-Fast
LynnReal-Hand-Fast is a hand-controlled video world model for causal, chunk-by-chunk generation. It generates video from an initial image, a text prompt, hand motion, and camera poses.
Features
- Sampling-specialized 33B causal backbone with three sampling steps per chunk.
- W2 sliding context: the current chunk and the preceding generated chunk, with a fixed first-frame anchor.
- Default output resolution of 960 ร 540, with 17 RGB frames per full chunk.
- Compact hand encoder and Ultra-Fast-VAE decoding.
- Standard and accelerated inference through the LynnReal-Hand runtime.
Model package
The package includes the sampling-specialized transformer, streaming time-conditioning parameters, native H3 VAE, Qwen image/text encoder, tokenizer and vocabulary, processor, scheduler, matching hand encoder, and Ultra-Fast-VAE decoder. The native VAE and Qwen conditioning components are provided from the official MiniMax H3 release.
The compact hand encoder is stored in hand_encoder/, and Ultra-Fast-VAE is stored in decoder/. Native conditioning components use the standard vae/, text_encoder/, tokenizer/, and processor/ directories.
Fast includes all transformer parameters needed for inference. A separate Base download is unnecessary.
Quick start
Install the runtime using the LynnReal-Hand installation guide, then run the following commands from the code repository:
python scripts/download_weights.py --component fast
lynnreal-hand prepare \
--config configs/fast.json --weights-root checkpoints \
--example examples/01/example.json \
--output outputs/cache/fast/example_01
lynnreal-hand infer \
--config configs/fast.json --weights-root checkpoints \
--cache outputs/cache/fast/example_01 \
--output outputs/fast/example_01.mp4 --seed 42
Add --accelerated to enable the optimized inference path. Its first run includes compilation and CUDA graph capture. Keep the original hand video available for online hand encoding.
The root model_index.json or config.json can also be passed to the runtime. See inference for the streaming Python interface and output formats.
Inputs and outputs
Required inputs are an initial image, text prompt, hand-motion video, and aligned camera arrays. The input pipeline supports 124, 158, 243, or 260 frames and preserves the input frame rate. The supplied example demonstrates this format.
Fast conditions each chunk on generated history. An optional reference video supports visual comparison. See data preparation for input requirements.
AdaLN memory optimization
The transformer includes only the AdaLN modulation tables required by its default sampling schedule. These tables load directly from transformer/adaln.safetensors; the learned per-block projection matrices are omitted. Keep the packaged step count and sampling shift. Changing the schedule requires a new export. This reduces download size and resident parameter memory; total memory also depends on activations, caches, and decoding. The native VAE and Qwen encoder are used during conditioning preparation and can be offloaded before video generation.
Reported performance
A warmed run of the packaged accelerated runtime at 960 ร 540 on one H200 measures:
| Sustained throughput | Mean chunk response | Peak allocated GPU memory |
|---|---|---|
| 17.94 FPS | 0.743 s | 34.51 GiB |
The measurement uses three sampling steps, W2 context with a fixed first-frame anchor, and 17 RGB frames per full chunk. Timing includes online hand encoding, generation, decoding, and CPU frame delivery; sustained throughput also includes history-cache updates. The 243-frame reference run uses one warmup and three measured repeats. Timing covers steady full chunks, while memory is the maximum across all measured generation chunks.
Conditioning preparation, weight loading, compilation, CUDA graph capture, warmup, and MP4 encoding are excluded. The native VAE and Qwen encoder are offloaded before generation. 34.51 GiB is a warmed allocation measurement, not a cold-start GPU requirement. The technical report's 18.03 FPS result is a separate measurement; see benchmark methodology for both scopes.
Runtime requirements
The reference environment is Linux, Python 3.10, PyTorch 2.9.1 with CUDA 12.8, and FlashAttention-3. Install FFmpeg for video input and output. See the installation guide for the complete environment.
The default compiled Fast decoder and accelerated inference path require one visible NVIDIA H200. Other GPU configurations are not validated by this release. Official MiniMax H3 VAE and Qwen conditioning components are included; a separate MiniMax download is unnecessary.
Scope and limitations
- This is an inference release with a fixed three-step, shift-3 schedule and W2 context. A different sampling schedule requires a new AdaLN table export.
- The first image remains a fixed anchor while the generated history slides. Hand video and camera poses must remain aligned with the requested sequence.
- Few-step causal generation can accumulate appearance drift, hand artifacts, or temporal jitter, especially over longer interactions. Bounded cache size does not guarantee indefinite visual consistency.
- Performance applies to the documented environment and measurement scope; startup and conditioning need additional time and memory.
License
This model is distributed under the MiniMax H3 Community License Agreement. See NOTICE for attribution. Third-party components retain their applicable licenses and notices.
- Downloads last month
- 95
Model tree for shaohao011/LynnReal-Hand-Fast-preview
Base model
MiniMaxAI/MiniMax-H3