| ---
|
| license: mit
|
| library_name: pytorch
|
| tags:
|
| - robotics
|
| - robot-manipulation
|
| - vision-language-action
|
| - world-models
|
| - video-diffusion
|
| - diffusion-transformer
|
| - lora
|
| - imitation-learning
|
| - robotwin
|
| - libero
|
| ---
|
|
|
| # WAM-Adapter
|
|
|
| **WAM-Adapter: Efficiently Adapting Video-DiT World Representations to Robot Actions**
|
|
|
| Dongxu Liu<sup>1,2,*</sup>  Zhe Li<sup>1,*</sup>  Zundong Ke<sup>1</sup>  Zhipeng Zhang<sup>1,†</sup>
|
|
|
| <sup>1</sup>AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University<br>
|
| <sup>2</sup>Awaken Robotics<br>
|
| (*Equal contribution, †Corresponding author)
|
|
|
| ## 📖 Overview
|
|
|
| WAM-Adapter is a parameter- and training-efficient framework that adapts a **frozen Video-DiT
|
| world model** (`Wan2.1-T2V-1.3B`) into a robot *World Action Model*:
|
|
|
| - **Frozen Video-DiT + LoRA.** The 1.3B backbone stays frozen; rank-64 LoRA adapts it to robot data.
|
| - **One joint current+future pass.** The clean current frame and noisy future latent slots are
|
| processed in a *single* Video-DiT forward — no future-video denoising loop, no extra action-side
|
| forward.
|
| - **Temporal-group cross-layer readout.** Multi-layer representations are organized into
|
| **Current / Early-Future / Late-Future** groups with group-specific cross-layer fusion, and the
|
| whole action chunk is predicted by a small mixed-context action decoder.
|
|
|
| Only **121M trainable parameters** (with a compact 1.3B backbone, no embodied pretraining):
|
| **89.0%** RoboTwin 2.0 (50 tasks), **98.5%** LIBERO, **75.17%** LIBERO-Plus — while improving
|
| training throughput by **9.66×** over Fast-WAM under the same setting.
|
|
|
| ## 📦 Checkpoints
|
|
|
| | File | Benchmark | Training steps | Size |
|
| |---|---|---|---|
|
| | `wam_adapter_robotwin50_240k.pt` | RoboTwin 2.0 (50 tasks) | 240k | 3.08 GB |
|
| | `wam_adapter_libero_spatial_70k.pt` | LIBERO-Spatial | 70k | 3.08 GB |
|
| | `wam_adapter_libero_object_60k.pt` | LIBERO-Object | 60k | 3.08 GB |
|
| | `wam_adapter_libero_goal_80k.pt` | LIBERO-Goal | 80k | 3.08 GB |
|
| | `wam_adapter_libero_long_80k.pt` | LIBERO-Long | 80k | 3.08 GB |
|
|
|
| Each checkpoint is a PyTorch file containing the LoRA-adapted backbone (`mot`), the temporal-group
|
| readout (`action_decoder`) and its configuration.
|
|
|
| ## 🚀 Usage
|
|
|
| ```python
|
| from huggingface_hub import hf_hub_download
|
|
|
| ckpt_path = hf_hub_download(
|
| repo_id="ldx825/WAM-Adapter",
|
| filename="wam_adapter_robotwin50_240k.pt",
|
| )
|
| # then, inside the WAM-Adapter codebase:
|
| # model = ... # build WAMAdapter (see the GitHub repository)
|
| # model.load_checkpoint(ckpt_path)
|
| ```
|
|
|
| Please refer to the [GitHub repository](https://github.com/ldx825/WAM-Adapter) for the complete
|
| training / evaluation / deployment instructions.
|
|
|
| ## 🔗 Links
|
|
|
| - **GitHub**: https://github.com/ldx825/WAM-Adapter
|
| - **Project page**: https://ldx825.github.io/WAM-Adapter-Homepage/
|
| - **Backbone**: [Wan-AI/Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B)
|
|
|
| ## 📝 Citation
|
|
|
| The arXiv release is coming soon; for now please cite the project as:
|
|
|
| ```bibtex
|
| @article{wamadapter2026,
|
| title = {WAM-Adapter: Efficiently Adapting Video-DiT World Representations to Robot Actions},
|
| author = {Liu, Dongxu and Li, Zhe and Ke, Zundong and Zhang, Zhipeng},
|
| journal = {arXiv preprint},
|
| year = {2026},
|
| }
|
| ```
|
|
|
