Humanoid-VLA

Model weights for Humanoid-VLA, a vision-language-action policy for humanoid loco-manipulation on the Unitree G1.

ucs200k

The base model that post-training starts from: the policy's flow-matching action head, pretrained from scratch for 200k steps on human egocentric data (EgoDex) in the repository's Unified Cartesian Space (UCS), where every source maps its wrists, fingertips and head camera to one shared action layout.

Data	EgoDex, train split (pretrain.sources.egodex, wrists as the G1 wrist frame)
Frame / actions	camera heading frame anchored at the head camera; absolute poses
Chunk	20 rows, 60 ms apart (first at +20 ms); 4 state rows over the past 180 ms
Inputs	head camera, state (with per-part presence bits; state dropout 0.3, part dropout 0.1)
Training	24 GPUs x batch 64 (1,536), bf16, AdamW, lr 1e-4 with 2k warm-up decaying to 0 at 260k; this is step 200k

Files (ucs200k/): model.safetensors (the action head and its projections; the frozen vision-language backbone is not included), config.json, and ucs_stats.json (the UCS normalization statistics, used only to continue UCS pretraining).

Use

hf download qizekun/Humanoid-VLA --include "ucs200k/*" --local-dir ~/.cache/vlm
python utils/tools/prepare_vlm.py --out ~/.cache/vlm   # the frozen vision-language backbone, fetched from NVIDIA
torchrun --nproc_per_node=8 scripts/train.py storage/configs/train/humanoid_vla.yaml

storage/configs/train/humanoid_vla.yaml starts post-training from ~/.cache/vlm/ucs200k, with its own normalization statistics for the robot data.

Notes

• The vision-language backbone is not part of this repository: utils/tools/prepare_vlm.py downloads it from NVIDIA, under NVIDIA's license terms (accept the license of nvidia/Cosmos-Reason2-2B on Hugging Face first). ucs200k was trained on that backbone's features.
• ucs_stats.json was computed with the earlier camera heading (the optical axis projected on the ground). To continue UCS pretraining with the current code, compute new statistics (python -m pretrain.compute_stats).

License

The weights in this repository are released under the Apache 2.0 license. The vision-language backbone they run on keeps NVIDIA's license.