| --- |
| license: cc-by-4.0 |
| tags: |
| - world-model |
| - video-generation |
| - robotics |
| - diffusion |
| - flow-matching |
|
|
| --- |
| |
| # ACWM-Phys Checkpoints |
|
|
| **ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Models** |
|
|
| > Haotian Xue†, Yipu Chen\*, Liqian Ma\*, Zelin Zhao, Lama Moukheiber, Yongxin Chen |
| > Georgia Institute of Technology |
|
|
| [[Project Page]](https://xavihart.github.io/ACWM-Phys) · [[Paper]](#) · [[Dataset]](https://huggingface.co/datasets/t1an/ACWM-Phys) · [[Code]](https://github.com/xavihart/ACWM-Phys) |
|
|
| --- |
|
|
| ## Overview |
|
|
| This repository contains pretrained **ACWM-DiT** checkpoints — a latent diffusion transformer trained with flow matching on the ACWM-Phys benchmark. All released checkpoints are **DiT-S (~200M parameters)** trained for **100k steps**. |
|
|
| --- |
|
|
| ## Released Checkpoints |
|
|
| | Environment | Category | Action Dim | Resolution | Checkpoint | |
| |---|---|---|---|---| |
| | Push Cube | Rigid-Body | 2 | 240×240 | `VideoDiT_S_push_cube_240x240/latest.pt` | |
| | Stack Cube | Rigid-Body | 7 | 240×240 | `VideoDiT_S_stack_cube_240x240/latest.pt` | |
| | Push Rope | Deformable | 2 | 240×240 | `VideoDiT_S_push_rope_240x240/latest.pt` | |
| | Cloth Move | Deformable | 3 | 240×240 | `VideoDiT_S_clothmove_240x240_240x240/latest.pt` | |
| | Push Sand | Particle | 7 | 240×400 | `VideoDiT_S_push_sand_240x400/latest.pt` | |
| | Pour Water | Particle | 4 | 240×240 | `VideoDiT_S_pour_water_240x240/latest.pt` | |
| | Robot Arm | Kinematics | 7 | 240×240 | `VideoDiT_S_robot_arm_240x240/latest.pt` | |
| | Reacher | Kinematics | 2 | 240×240 | `VideoDiT_S_reacher_240x240/latest.pt` | |
|
|
| The Wan 2.1 VAE weights (`Wan2.1_VAE.pth`, 508 MB) are also included and required for encoding/decoding video latents. |
|
|
| --- |
|
|
| ## Download |
|
|
| ```bash |
| huggingface-cli download t1an/ACWM-Phys-checkpoints --local-dir ./checkpoints |
| export WAN_VAE_PATH=./checkpoints/Wan2.1_VAE.pth |
| ``` |
|
|
| --- |
|
|
| ## Usage |
|
|
| See the [ACWM-Phys code repository](https://github.com/xavihart/ACWM-Phys) for full evaluation and training instructions. |
|
|
| Quick evaluation: |
|
|
| ```bash |
| python eval.py --env push_cube --steps 50 --split both --save_videos |
| ``` |
|
|
| --- |
|
|
| ## Model Architecture |
|
|
| ACWM-DiT takes the first video frame + full action sequence and predicts the complete future trajectory: |
|
|
| 1. **Causal VAE (Wan 2.1)** — encodes video into 16-ch latent tokens at H/8×W/8, 4× temporal compression |
| 2. **DiT with flow matching** — denoises the full latent trajectory |
| 3. **Action conditioning** — injected via AdaLN (default) or cross-attention |
|
|
| --- |
|
|
| ## Citation |
|
|
| ## Citation |
|
|
| Coming soon. |
|
|
