UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
Paper • 2610.18697 • Published
Paper (arXiv 2610.18697) · Project Page · Code
UniW(A)M is a mixture-of-experts world-(action) model built on diffusion Transformers. It unifies action-conditioned world modeling (AC-WM) and four world-action-model interfaces (Joint-WAM, IDM-WAM, Action-Centric, Action-Only) in one backbone: shared experts capture common dynamics, atomic capability experts (VGM, VLA, WM, IDM) are reused across interfaces, and a plug-in adaptive interface router (PAIR) selects the interface for every action chunk with the backbone frozen.
| Folder | Checkpoint | Use |
|---|---|---|
pretrained/ |
UniW(A)M initialization: the DiT with each FFN copied into 5 experts and a zero-initialized router (~31 GB) | Starting point for training on new tasks |
folding_clothes/ |
Real-world Folding Clothes, dual-arm AgileX, 150k steps (~44 GB) | Real-robot deployment and evaluation |
libero/ |
LIBERO (Spatial, Object, Goal, Long) | Simulation evaluation on LIBERO |
robotwin2.0/ |
RoboTwin 2.0 | Simulation evaluation on RoboTwin 2.0 |
Notes:
folding_clothes/ checkpoint contains the DiT, the text/image encoders and the VAE. The umT5 tokenizer is not included; pass it separately via tokenizer_path (e.g. google/umt5-xxl from Wan2.2-TI2V-5B), as described in the inference guide.diffusers.DiffusionPipeline.Download only the checkpoint you need:
from huggingface_hub import snapshot_download
# e.g. the Folding Clothes checkpoint
path = snapshot_download(repo_id="xumanba/UniW-A-M", allow_patterns=["folding_clothes/*"])
or with the CLI:
hf download xumanba/UniW-A-M --include "pretrained/*" --local-dir ./checkpoints/UniW-A-M
Training, inference and evaluation scripts are in the code repository.
The weights in this repository are released under the Apache License 2.0.
@article{xu2026uniwam,
title = {UniW(A)M: A Unified Mixture-of-Experts Framework for
World and World-Action Models},
author = {Xu, Jingbo and Fang, Jianjie and Li, Zhuohang and Song, Zihao and
Wang, Zhaolu and Liu, Xvyuan and Zhang, Xin and Su, Haisheng and
Tang, Yinzhou and Gao, Chen and Wu, Wei and Li, Yong},
journal = {arXiv preprint arXiv:2610.18697},
year = {2026}
}