README.md

---
license: apache-2.0
---

<div align="center" style="font-family: charter;">

<h1>HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models</h1>

<div style="font-size: 15px;">
    <a href="https://scholar.google.com/citations?user=a8h9Di4AAAAJ" target="_blank">Jiazi Bu*</a><sup></sup> | 
    <a href="https://github.com/LPengYang/" target="_blank">Pengyang Ling*<sup>§</sup></a><sup></sup> | 
    <a href="https://github.com/YujieOuO" target="_blank">Yujie Zhou*</a><sup></sup> |
    <a href="https://codegoat24.github.io/" target="_blank">Yibin Wang</a><sup></sup> |
    <a href="https://yuhangzang.github.io/" target="_blank">Yuhang Zang</a><sup></sup> |
    <a href="https://scholar.google.com/citations?user=OeeH1HAAAAAJ&hl=en" target="_blank">Xuanlang Dai</a><sup></sup> | <br>
    <a href="https://scholar.google.com/citations?user=iDPJVBsAAAAJ&hl=zh-CN" target="_blank">Shengyuan Ding</a><sup></sup> | 
    <a href="https://wtybest.github.io/" target="_blank">Tianyi Wei</a><sup></sup> |
    <a href="https://xiaohangzhan.github.io/" target="_blank">Xiaohang Zhan</a><sup></sup> |
    <a href="https://myownskyw7.github.io/" target="_blank">Jiaqi Wang</a><sup></sup> |
    <a href="https://wutong16.github.io/" target="_blank">Tong Wu</a><sup></sup> |
    <a href="http://dahua.site/" target="_blank">Dahua Lin</a><sup></sup> |
    <a href="https://xingangpan.github.io/" target="_blank">Xingang Pan<sup>†</sup></a><sup></sup>
</div>
<br>
<div>
    <sup></sup>Shanghai Jiao Tong University, Nanyang Technological University, Shanghai AI Laboratory, 
    <br> University of Science and Technology of China, Fudan University, Shanghai Innovation Institute
    <br> The Chinese University of Hong Kong, CPII under InnoHK, JD.com, Adobe Research
</div>
(*<b>Equal Contribution</b>)(<sup>§</sup><b>Project Leader</b>)(<sup>†</sup><b>Corresponding Author</b>)
<br><br>

</div>  

This model is presented in the paper [HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models](https://arxiv.org/abs/2608.13205).
Project page: [https://bujiazi.github.io/hpsd.github.io/](https://bujiazi.github.io/hpsd.github.io/)

## Abstract

Text-Image-to-Video (TI2V) models are an emerging unified architecture, where a single model simultaneously supports text-to-video (T2V) and image-to-video (I2V) generation. Given a high-quality first frame or a detailed textual prompt, TI2V models unlock substantially better visual quality than their T2V mode, raising a natural question: _can the capability elicited by such privileged conditions be internalized into the model's own base generation ability?_ A common approach toward this goal is model self-distillation. However, the most straightforward solution, supervised fine-tuning, follows an off-policy strategy: its supervision is confined to teacher-generated endpoints from a fixed offline distribution rather than student-visited states, lacking precise correction tailored to the evolving policy. Recent on-policy distillation methods instead suffer from condition-state mismatch, where supervision is steered toward the given first frame instead of the student's actual content, misleading the correction. To achieve self-distillation that absorbs the teacher's privileged prior while retaining precise policy correction, in this work, we propose **H**ybrid-**P**olicy **S**elf-**D**istillation (**HPSD**), a novel self-distillation framework where a single TI2V model acts as both teacher and student under different conditions: the teacher operates in TI2V mode with a high-quality first frame and an enhanced prompt, while the student runs in the base T2V mode with only the vanilla prompt. Specifically, the student inherits off-policy teacher trajectory points as anchors, locally refines them toward its own policy, and finally receives velocity-level supervision on these self-generated roll-outs. Extensive experiments demonstrate that HPSD significantly improves T2V performance while also delivering notable TI2V gains, effectively strengthening the model's base generation ability.

## 🌟 Model

The `adapter_model.safetensors` is based on [WAN-2.2-TI2V](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers) with our HPSD training.

## 🔧 Github Link
[https://github.com/Bujiazi/HPSD](https://github.com/Bujiazi/HPSD)

## 🚀 Inference
```python
import torch
from huggingface_hub import snapshot_download
from diffusers import WanPipeline, AutoencoderKLWan
from diffusers.utils import export_to_video
from peft import PeftModel

dtype = torch.bfloat16
device = "cuda"

model_id = "Wan-AI/Wan2.2-TI2V-5B-Diffusers"
checkpoint = snapshot_download(repo_id="Bujiazi/HPSD", repo_type="model")

vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=dtype)
pipe.to(device)

pipe.transformer = PeftModel.from_pretrained(pipe.transformer, checkpoint, torch_dtype=dtype).to(device)

height = 704
width = 1280
num_frames = 81
num_inference_steps = 50
guidance_scale = 5.0
base_seed = 42

prompt = "A sports car speeding along a winding coastal road at sunset. Dynamic low-angle tracking shot, realistic reflections, dramatic clouds, subtle motion blur, cinematic composition."
negative_prompt = "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"

generator = torch.Generator(device=device).manual_seed(base_seed)
output = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=height,
    width=width,
    num_frames=num_frames,
    guidance_scale=guidance_scale,
    num_inference_steps=num_inference_steps,
    generator=generator,
).frames[0]
export_to_video(output, "hpsd_test.mp4", fps=16)
```


## 📎 Citation 

If you find our work helpful, please consider giving a star ⭐ and citation 📝 
```bibtex
@article{bu2026hpsd,
  title={HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models},
  author={Bu, Jiazi and Ling, Pengyang and Zhou, Yujie and Wang, Yibin and Zang, Yuhang and Dai, Xuanlang and Ding, Shengyuan and Wei, Tianyi and Zhan, Xiaohang and Wang, Jiaqi and others},
  journal={arXiv preprint arXiv:2608.13205},
  year={2026}
}
```

## 💞 Acknowledgements
The code is built upon the below repositories, we thank all the contributors for open-sourcing.

* [Diffusers](https://github.com/huggingface/diffusers)
* [WAN-2.2](https://github.com/Wan-Video/Wan2.2)
* [LTX-2.3](https://github.com/Lightricks/LTX-2)
* [Z-Image](https://github.com/Tongyi-MAI/Z-Image)
* [Qwen3.6-27B](https://github.com/QwenLM/Qwen3.6)
* [D-OPSD](https://github.com/vvvvvjdy/D-OPSD)