DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning

Mengqi Zhang¹, Sahil Khose¹, Simar Kareer², Yuchen Song¹, Unnat Jain¹, Judy Hoffman¹

¹University of California, Irvine    ²Georgia Institute of Technology

🌐 Project Page  |  📄 Paper  |  💻 Code  |  🧩 Datasets [LIBERO | RoboCasa | YAM]

DeVA is a decoupled video-action model for robot manipulation. DeVA uses two specialized experts: a video diffusion backbone and an action expert, bridged by multi-level feature alignment and physically-salient supervision (affordance + depth). This transfers the rich spatiotemporal priors of a video generative model into an efficient robot policy while training and converging faster.

Checkpoints

For every benchmark we release two variants: a base model (action + video only) and a guidance version that adds the affordance + depth heads.

Benchmark	Base model	w/ physical guidance
RoboCasa	robocasa_base.pt	robocasa_guidance.pt
LIBERO / LIBERO-Plus	—	libero_guidance.pt
YAM bimanual	yam_base.pt	yam_guidance.pt

All checkpoints are trained from the Cosmos-Predict2 2B Video2World (480p / 16fps) backbone.

Component	Params	Status
T5-XXL text encoder	4.86 B	frozen
VAE tokenizer	127 M	frozen
Video2World DiT (backbone)	1.96 B	trained
Action expert	564 M	trained
Affordance head	17.6 M	trained
Depth head	13.4 M	trained
Joint-fusion projection	5.2 M	trained
Total / trainable	7.55 B / 2.56 B (33.9%)

Usage

Install the code from Mq-Zhang1/deva, then download a checkpoint into checkpoints/deva/:

hf download mengqz9/DeVA libero_guidance.pt --local-dir checkpoints/deva

or use the helper script in the repo:

python scripts/download_deva_checkpoints.py --list                          # list available
python scripts/download_deva_checkpoints.py --checkpoint libero_guidance    # one (repeatable)
python scripts/download_deva_checkpoints.py --all                           # all checkpoints

Closed-loop evaluation uses a client/server split: a server hosts the DeVA model, a client drives the simulator. See the repository README for evaluation, real-robot deployment, and training instructions.

Citation

@article{zhang2026deva,
  title   = {{DeVA}: Decoupled Video-Action Model with physical guidance for robot policy learning},
  author  = {Zhang, Mengqi and Khose, Sahil and Kareer, Simar and
             Song, Yuchen and Jain, Unnat and Hoffman, Judy},
  journal = {arXiv preprint arXiv:2607.24159},
  year    = {2026}
}

Acknowledgements

Thanks for all the excellent open-source projects: Cosmos-Predict2, LIBERO, RoboCasa, Video Depth Anything and UAD. We thank the authors of these projects.