LynnReal-Hand-Base
arXiv · Hugging Face Paper · YouTube · Code · Inference guide
LynnReal-Hand-Base is a hand-controlled video world model for full-sequence generation. It generates video from an initial image, a text prompt, aligned hand motion, and camera poses.
Features
- Sampling-specialized 33B backbone with bidirectional full-sequence sampling.
- Joint hand-motion and camera conditioning.
- Default output resolution of 1344 × 768.
- Native H3 first-image encoding and Qwen image/text conditioning.
Model package
The package includes the sampling-specialized transformer, native H3 VAE, Qwen image/text encoder, tokenizer and vocabulary, processor, scheduler, and inference configuration. The VAE and Qwen conditioning components are provided from the official MiniMax H3 release.
Quick start
Install the runtime using the LynnReal-Hand installation guide, then run the following commands from the code repository:
python scripts/download_weights.py --component base
lynnreal-hand prepare \
--config configs/base.json --weights-root checkpoints \
--example examples/01/example.json \
--output outputs/cache/base/example_01
lynnreal-hand infer \
--config configs/base.json --weights-root checkpoints \
--cache outputs/cache/base/example_01 \
--output outputs/base/example_01.mp4 --seed 42
The root model_index.json or config.json can also be passed to the runtime. See inference for configuration and output formats.
Inputs and outputs
Required inputs are an initial image, text prompt, hand-motion video, and camera arrays aligned in field of view and frame order. The input pipeline supports 124, 158, 243, or 260 frames and preserves the input frame rate.
Generated videos are saved as MP4. See data preparation for the input manifest and camera format.
AdaLN memory optimization
The transformer includes only the AdaLN modulation tables required by its default sampling schedule. These tables load directly from transformer/adaln.safetensors; the learned per-block projection matrices are omitted. Keep the packaged step count and sampling shift. Changing the schedule requires a new export. This reduces download size and resident parameter memory; total memory also depends on activations, caches, and decoding.
Evaluation
On HandBench, the technical report reports 181.67 FVD for visual quality and 93.02 MANO Dice for hand-control fidelity. Base uses bidirectional full-sequence sampling at 1344 × 768. Runtime and memory depend on sequence length and inference settings.
Runtime requirements
The reference environment is Linux, Python 3.10, PyTorch 2.9.1 with CUDA 12.8, and FlashAttention-3. Install FFmpeg for video input and output. See the installation guide for the complete environment.
Base execution requires a CUDA Hopper GPU with FlashAttention-3; the reference setup uses one H200. The package includes the official MiniMax H3 VAE and Qwen conditioning components, so a separate download from MiniMax is unnecessary.
Scope and limitations
- This release provides inference weights and code. The packaged schedule uses 30 sampling points and shift 3. Changing the schedule requires a new AdaLN table export.
- Align the initial image, hand video, and camera poses in field of view and time. Incorrect alignment can reduce control fidelity.
- Generated interactions can exhibit hand artifacts, object drift, or temporal instability. Benchmark results do not guarantee consistent behavior for every scene or motion.
Citation
If you use this model in your research, please cite:
@misc{rui2026lynnrealhandunlockingworldinteraction,
title={LynnReal-Hand: Unlocking World Interaction with Hand Control},
author={Shaohao Rui and Chaoyang Zhang and Xiaofeng Mao and Weijie Ma},
year={2026},
eprint={2611.21479},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2611.21479}
}
License
This model is distributed under the MiniMax H3 Community License Agreement. See NOTICE for attribution. Third-party components retain their applicable licenses and notices.
- Downloads last month
- 101