Safetensors
Bujiazi commited on
Commit
bf8019a
·
verified ·
1 Parent(s): 3f0832a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md CHANGED
@@ -1,3 +1,75 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ <div align="center" style="font-family: charter;">
6
+
7
+ <h1>HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models</h1>
8
+
9
+ <div>
10
+ <a href="https://scholar.google.com/citations?user=a8h9Di4AAAAJ" target="_blank">Jiazi Bu*</a><sup></sup> |
11
+ <a href="https://github.com/LPengYang/" target="_blank">Pengyang Ling*<sup>§</sup></a><sup></sup> |
12
+ <a href="https://github.com/YujieOuO" target="_blank">Yujie Zhou*</a><sup></sup> |
13
+ <a href="https://codegoat24.github.io/" target="_blank">Yibin Wang</a><sup></sup> |
14
+ <a href="https://yuhangzang.github.io/" target="_blank">Yuhang Zang</a><sup></sup> |
15
+ <a href="https://scholar.google.com/citations?user=OeeH1HAAAAAJ&hl=en" target="_blank">Xuanlang Dai</a><sup></sup> | <br>
16
+ <a href="https://scholar.google.com/citations?user=iDPJVBsAAAAJ&hl=zh-CN" target="_blank">Shengyuan Ding</a><sup></sup> |
17
+ <a href="https://wtybest.github.io/" target="_blank">Tianyi Wei</a><sup></sup> |
18
+ <a href="https://xiaohangzhan.github.io/" target="_blank">Xiaohang Zhan</a><sup></sup> |
19
+ <a href="https://myownskyw7.github.io/" target="_blank">Jiaqi Wang</a><sup></sup> |
20
+ <a href="https://wutong16.github.io/" target="_blank">Tong Wu</a><sup></sup> |
21
+ <a href="http://dahua.site/" target="_blank">Dahua Lin</a><sup></sup> |
22
+ <a href="https://xingangpan.github.io/" target="_blank">Xingang Pan<sup>†</sup></a><sup></sup>
23
+ </div>
24
+ <br>
25
+ <div>
26
+ <sup></sup>Shanghai Jiao Tong University, Nanyang Technological University, Shanghai AI Laboratory,
27
+ <br> University of Science and Technology of China, Fudan University, Shanghai Innovation Institute
28
+ <br> The Chinese University of Hong Kong, CPII under InnoHK, JD.com, Adobe Research
29
+ </div>
30
+ (*<b>Equal Contribution</b>)(<sup>§</sup><b>Project Leader</b>)(<sup>†</sup><b>Corresponding Author</b>)
31
+ <br><br>
32
+
33
+ </div>
34
+
35
+ This model is presented in the paper [HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models](https://arxiv.org/abs/2608.13205).
36
+ Project page: [https://bujiazi.github.io/hpsd.github.io/](https://bujiazi.github.io/hpsd.github.io/)
37
+
38
+ ## Abstract
39
+
40
+ Text-Image-to-Video (TI2V) models are an emerging unified architecture, where a single model simultaneously supports text-to-video (T2V) and image-to-video (I2V) generation. Given a high-quality first frame or a detailed textual prompt, TI2V models unlock substantially better visual quality than their T2V mode, raising a natural question: _can the capability elicited by such privileged conditions be internalized into the model's own base generation ability?_ A common approach toward this goal is model self-distillation. However, the most straightforward solution, supervised fine-tuning, follows an off-policy strategy: its supervision is confined to teacher-generated endpoints from a fixed offline distribution rather than student-visited states, lacking precise correction tailored to the evolving policy. Recent on-policy distillation methods instead suffer from condition-state mismatch, where supervision is steered toward the given first frame instead of the student's actual content, misleading the correction. To achieve self-distillation that absorbs the teacher's privileged prior while retaining precise policy correction, in this work, we propose **H**ybrid-**P**olicy **S**elf-**D**istillation (**HPSD**), a novel self-distillation framework where a single TI2V model acts as both teacher and student under different conditions: the teacher operates in TI2V mode with a high-quality first frame and an enhanced prompt, while the student runs in the base T2V mode with only the vanilla prompt. Specifically, the student inherits off-policy teacher trajectory points as anchors, locally refines them toward its own policy, and finally receives velocity-level supervision on these self-generated roll-outs. Extensive experiments demonstrate that HPSD significantly improves T2V performance while also delivering notable TI2V gains, effectively strengthening the model's base generation ability.
41
+
42
+ ## 🌟 Model
43
+
44
+ The `adapter_model.safetensors` is based on [WAN-2.2-TI2V](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers) with our HPSD training.
45
+
46
+ ## 🔧 Github Link
47
+ [https://github.com/Bujiazi/HPSD](https://github.com/Bujiazi/HPSD)
48
+
49
+ ## 🚀 Inference
50
+ ```
51
+ TBD
52
+ ```
53
+
54
+
55
+ ## 📎 Citation
56
+
57
+ If you find our work helpful, please consider giving a star ⭐ and citation 📝
58
+ ```bibtex
59
+ @article{bu2026hpsd,
60
+ title={HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models},
61
+ author={Bu, Jiazi and Ling, Pengyang and Zhou, Yujie and Wang, Yibin and Zang, Yuhang and Dai, Xuanlang and Ding, Shengyuan and Wei, Tianyi and Zhan, Xiaohang and Wang, Jiaqi and others},
62
+ journal={arXiv preprint arXiv:2608.13205},
63
+ year={2026}
64
+ }
65
+ ```
66
+
67
+ ## 💞 Acknowledgements
68
+ The code is built upon the below repositories, we thank all the contributors for open-sourcing.
69
+
70
+ * [Diffusers](https://github.com/huggingface/diffusers)
71
+ * [WAN-2.2](https://github.com/Wan-Video/Wan2.2)
72
+ * [LTX-2.3](https://github.com/Lightricks/LTX-2)
73
+ * [Z-Image](https://github.com/Tongyi-MAI/Z-Image)
74
+ * [Qwen3.6-27B](https://github.com/QwenLM/Qwen3.6)
75
+ * [D-OPSD](https://github.com/vvvvvjdy/D-OPSD)