TwoPass / README.md
Junjun2333's picture
Use hf download in the model card
90e2d46 verified
|
Raw History Blame Contribute Delete
2.83 kB
metadata
license: apache-2.0
base_model: Wan-AI/Wan2.2-TI2V-5B
library_name: pytorch
pipeline_tag: image-to-video
tags:
  - world-model
  - video-prediction
  - robotics
  - robot-manipulation
  - action-conditioned
  - few-step
  - wan2.2
datasets:
  - fangqi/IRASim

TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation

Paper: arXiv 2611.25553 Code: GitHub Citation / BibTeX

Checkpoints of TwoPass, an action-conditioned video prediction model for robotic manipulation fine-tuned from Wan2.2-TI2V-5B. Given one observed frame and the future robot actions, it predicts the following frames in as few as four network calls.

File Dataset Size
bridge/twopass_bridge.pt Bridge 20.2 GB
languagetable/twopass_languagetable.pt Language Table 20.2 GB

Each file is a PyTorch checkpoint holding the full model weights (model, fp32) and the training step (step).

Usage

Install the code from GitHub, then download the checkpoints together with the Wan2.2 base weights:

hf download Junjun2333/TwoPass --local-dir ./checkpoints
hf download Wan-AI/Wan2.2-TI2V-5B --local-dir ./Wan2.2-TI2V-5B

Evaluate on the Bridge test set with 4 network calls per clip (use configs/languagetable.json and the Language Table checkpoint for the other dataset; data preparation is described in the GitHub README):

python -m twopass.evaluate --config configs/bridge.json --nfe 4 \
    --checkpoint ./checkpoints/bridge/twopass_bridge.pt --wan_dir ./Wan2.2-TI2V-5B \
    --data_root $DATA_ROOT --i3d_path /path/to/i3d_torchscript.pt --out_dir ./eval/bridge
python -m twopass.aggregate --config configs/bridge.json \
    --data_root $DATA_ROOT --eval_dir ./eval/bridge

Citation

@article{liu2026twopass,
  title         = {TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation},
  author        = {Liu, Yijun and Ge, Shijia and Feng, Tongtong and Wang, Xin and Meng, Yuan and Liu, Yuwei and Wang, Zhi and Zhu, Wenwu},
  journal       = {arXiv preprint arXiv:2611.25553},
  year          = {2026},
  eprint        = {2611.25553},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2611.25553}
}

License

Apache License 2.0, following Wan2.2.