CoT-WAM

Official checkpoint release for CoT-WAM.

Code · Paper · PDF

Files

File Purpose
model.pt Stage 2 video/action experts, VLM planner, latent projector and proprioception encoder
dataset_stats.json Normalization statistics paired with this checkpoint
train_config.yaml Training configuration with portable file locations
configs/eval.yaml Self-contained evaluation configuration for --config-dir
release.json Training step, source file hashes, packaging checks and artifact hashes
SHA256SUMS Download integrity checksums
LICENSE, NOTICE Weight license and base-model attribution

The checkpoint contains model weights and metadata, without optimizer or distributed training state. Load it with the CoT-WAM policy implementation; it is not a standalone Transformers from_pretrained model.

For checkpoints trained before the public code cleanup, train_config.yaml records the earlier cot_wam implementation and may include EMA tracking settings. This release packages the selected non-EMA policy; use configs/eval.yaml with the current code. The historical training config is provenance, not a current-code launch configuration.

Download and evaluate

Install the environment and prepare the LIBERO or LIBERO Plus checkout using the code repository's README. Run these commands from that code repository:

hf download LosMous/CoT-WAM --revision v1.0.0 --local-dir ./checkpoints/CoT-WAM
(cd ./checkpoints/CoT-WAM && sha256sum -c SHA256SUMS)

python scripts/download_backbones.py --output ./checkpoints
export COTWAM_CHECKPOINTS_ROOT="$PWD/checkpoints"
export COTWAM_OFFLINE=true

python experiments/libero/evaluate.py \
  --checkpoint ./checkpoints/CoT-WAM/model.pt \
  --dataset-stats ./checkpoints/CoT-WAM/dataset_stats.json \
  --config-dir ./checkpoints/CoT-WAM/configs \
  --libero-root ../LIBERO \
  --gpus 0 --workers-per-gpu 1 \
  --output ./evaluation_results/cotwam_release_libero

For LIBERO Plus, use experiments/libero_plus/evaluate.py, set --libero-root ../LIBERO-plus, and choose a separate output directory. Prepare its assets and compatibility patch as described in the code repository.

The frozen Wan VAE is not embedded in model.pt. The current planner constructor also requires the original Qwen model and processor directory before applying the trained weights. Keep the backbone preparation step even though the trained planner weights are included in this release.

For repeatable experiments, add --revision <release-tag-or-commit> to hf download and use the corresponding code version. Always keep the checkpoint, statistics and evaluation configuration together.

Evaluation and reproducibility

This exact checkpoint achieved the following results in the original complete LIBERO evaluation (40 tasks, 50 official initial states per task):

Spatial Object Goal Long Average
98.6% 99.4% 99.4% 98.6% 99.0%

The historical evaluation summary records 1,980 successful episodes out of 2,000 and identifies the evaluated checkpoint by SHA256. These results came from the research implementation; the full benchmark has not been rerun with the public code cleanup.

The release GPU smoke report covers strict loading of all policy components, one closed-loop LIBERO episode, and one training/validation step with a real LIBERO sample using the current code. The optimizer step affected only a disposable in-memory model; the uploaded checkpoint is byte-for-byte identical to the historical non-EMA artifact. A single rollout is a compatibility check, not a new benchmark score.

The default recipe uses 32 predicted actions, replans every 10 executed actions, and uses five goal-denoising steps ending at sigma 0.5 followed by ten action-denoising steps. The packaged configs/eval.yaml is authoritative when settings differ.

Use Linux and an NVIDIA GPU with the environment specified in the code repository. Previous complete-model inference checks recorded about 21.6 GiB of allocated GPU memory; this excludes other GPU allocations and is not a minimum VRAM guarantee. These weights are intended for the documented LIBERO observation and action interface; other robots and benchmarks require adaptation.

BibTeX

If you use CoT-WAM in your research, please cite:

@misc{zhang2026cotwamcollaborativetrainingvisionlanguage,
  title={CoT-WAM: Collaborative Training of Vision-Language Models and World-Action Models via Plan Latents},
  author={Jinhao Zhang and Yushen Zuo and Ke Rui and Weitao Zhou and Haibao Yu and Honglong Tian and Xiaoheng Ma and Jie Mei and Jiawei Wang and Minglei Li},
  year={2026},
  eprint={2611.23803},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2611.23803},
}

License and acknowledgements

The weights are released under Apache-2.0. See NOTICE for the Qwen and Wan base models. The code repository retains its own license. We also acknowledge FastWAM for its code and preprocessed LIBERO data, and the LIBERO and LIBERO Plus benchmark teams.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Model tree for LosMous/CoT-WAM

Finetuned
(571)
this model

Dataset used to train LosMous/CoT-WAM

Paper for LosMous/CoT-WAM