Junjun2333 commited on
Commit
3739ed5
·
verified ·
1 Parent(s): 1611e56

Add model card

Browse files
Files changed (1) hide show
  1. README.md +79 -0
README.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Wan-AI/Wan2.2-TI2V-5B
4
+ library_name: pytorch
5
+ pipeline_tag: image-to-video
6
+ tags:
7
+ - world-model
8
+ - video-prediction
9
+ - robotics
10
+ - robot-manipulation
11
+ - action-conditioned
12
+ - few-step
13
+ - wan2.2
14
+ datasets:
15
+ - fangqi/IRASim
16
+ ---
17
+
18
+ # TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation
19
+
20
+ <p align="center">
21
+ <a href="https://arxiv.org/abs/2611.25553"><img src="https://img.shields.io/badge/arXiv-2611.25553-b31b1b.svg" alt="Paper: arXiv 2611.25553"></a>
22
+ <a href="https://github.com/PlantPotatoOnMoon/TwoPass"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github" alt="Code: GitHub"></a>
23
+ <a href="#citation"><img src="https://img.shields.io/badge/Citation-BibTeX-2563eb" alt="Citation / BibTeX"></a>
24
+ </p>
25
+
26
+ Checkpoints of **TwoPass**, an action-conditioned video prediction model for
27
+ robotic manipulation fine-tuned from
28
+ [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B). Given one
29
+ observed frame and the future robot actions, it predicts the following frames
30
+ in as few as four network calls.
31
+
32
+ | File | Dataset | Size |
33
+ |---|---|---|
34
+ | [`bridge/twopass_bridge.pt`](bridge/twopass_bridge.pt) | Bridge | 20.2 GB |
35
+ | [`languagetable/twopass_languagetable.pt`](languagetable/twopass_languagetable.pt) | Language Table | 20.2 GB |
36
+
37
+ Each file is a PyTorch checkpoint holding the full model weights (`model`, fp32)
38
+ and the training step (`step`).
39
+
40
+ ## Usage
41
+
42
+ Install the code from [GitHub](https://github.com/PlantPotatoOnMoon/TwoPass),
43
+ then download the checkpoints together with the Wan2.2 base weights:
44
+
45
+ ```bash
46
+ huggingface-cli download Junjun2333/TwoPass --local-dir ./checkpoints
47
+ huggingface-cli download Wan-AI/Wan2.2-TI2V-5B --local-dir ./Wan2.2-TI2V-5B
48
+ ```
49
+
50
+ Evaluate on the Bridge test set with 4 network calls per clip (use
51
+ `configs/languagetable.json` and the Language Table checkpoint for the other
52
+ dataset; data preparation is described in the GitHub README):
53
+
54
+ ```bash
55
+ python -m twopass.evaluate --config configs/bridge.json --nfe 4 \
56
+ --checkpoint ./checkpoints/bridge/twopass_bridge.pt --wan_dir ./Wan2.2-TI2V-5B \
57
+ --data_root $DATA_ROOT --i3d_path /path/to/i3d_torchscript.pt --out_dir ./eval/bridge
58
+ python -m twopass.aggregate --config configs/bridge.json \
59
+ --data_root $DATA_ROOT --eval_dir ./eval/bridge
60
+ ```
61
+
62
+ ## Citation
63
+
64
+ ```bibtex
65
+ @article{liu2026twopass,
66
+ title = {TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation},
67
+ author = {Liu, Yijun and Ge, Shijia and Feng, Tongtong and Wang, Xin and Meng, Yuan and Liu, Yuwei and Wang, Zhi and Zhu, Wenwu},
68
+ journal = {arXiv preprint arXiv:2611.25553},
69
+ year = {2026},
70
+ eprint = {2611.25553},
71
+ archivePrefix = {arXiv},
72
+ primaryClass = {cs.CV},
73
+ url = {https://arxiv.org/abs/2611.25553}
74
+ }
75
+ ```
76
+
77
+ ## License
78
+
79
+ Apache License 2.0, following Wan2.2.