UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models

arXiv Project Page GitHub

Paper (arXiv 2610.18697) · Project Page · Code

UniW(A)M is a mixture-of-experts world-(action) model built on diffusion Transformers. It unifies action-conditioned world modeling (AC-WM) and four world-action-model interfaces (Joint-WAM, IDM-WAM, Action-Centric, Action-Only) in one backbone: shared experts capture common dynamics, atomic capability experts (VGM, VLA, WM, IDM) are reused across interfaces, and a plug-in adaptive interface router (PAIR) selects the interface for every action chunk with the backbone frozen.

Checkpoints

Folder Checkpoint Use
pretrained/ UniW(A)M initialization: the DiT with each FFN copied into 5 experts and a zero-initialized router (~31 GB) Starting point for training on new tasks
folding_clothes/ Real-world Folding Clothes, dual-arm AgileX, 150k steps (~44 GB) Real-robot deployment and evaluation
libero/ LIBERO (Spatial, Object, Goal, Long) Simulation evaluation on LIBERO
robotwin2.0/ RoboTwin 2.0 Simulation evaluation on RoboTwin 2.0

Notes:

  • The folding_clothes/ checkpoint contains the DiT, the text/image encoders and the VAE. The umT5 tokenizer is not included; pass it separately via tokenizer_path (e.g. google/umt5-xxl from Wan2.2-TI2V-5B), as described in the inference guide.
  • PAIR router weights are task-specific and are not part of these checkpoints. Train a router on your target-domain data with the PAIR pipeline in the code repository.
  • These weights are loaded with the UniW(A)M code, not with diffusers.DiffusionPipeline.

Download

Download only the checkpoint you need:

from huggingface_hub import snapshot_download

# e.g. the Folding Clothes checkpoint
path = snapshot_download(repo_id="xumanba/UniW-A-M", allow_patterns=["folding_clothes/*"])

or with the CLI:

hf download xumanba/UniW-A-M --include "pretrained/*" --local-dir ./checkpoints/UniW-A-M

Training, inference and evaluation scripts are in the code repository.

License

The weights in this repository are released under the Apache License 2.0.

Citation

@article{xu2026uniwam,
  title   = {UniW(A)M: A Unified Mixture-of-Experts Framework for
             World and World-Action Models},
  author  = {Xu, Jingbo and Fang, Jianjie and Li, Zhuohang and Song, Zihao and
             Wang, Zhaolu and Liu, Xvyuan and Zhang, Xin and Su, Haisheng and
             Tang, Yinzhou and Gao, Chen and Wu, Wei and Li, Yong},
  journal = {arXiv preprint arXiv:2610.18697},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Paper for xumanba/UniW-A-M