README.md

---
license: apache-2.0
pipeline_tag: text-to-video
tags:
- video-generation
- autoregressive-video
- diffusion-distillation
- wan2.1
- self-forcing
---

<h1 align="center">DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation</h1>

<p align="center">
  <b>Jiaxing Li</b><sup>1,2*</sup>,
  <b>Kai Zou</b><sup>1*</sup>,
  <b>Cindy Zhou</b><sup>1,3</sup>,
  <b>Kaichen Huang</b><sup>1,2</sup>,
  <b>Junyao Gao</b><sup>1</sup>,
  <b>Zile Wang</b><sup>1</sup>,
  <br>
  <b>Yang Liu</b><sup>1</sup>,
  <b>Bin Liu</b><sup>1</sup>,
  <b>Bo An</b><sup>2</sup>,
  <b>Yangguang Li</b><sup>1&dagger;</sup>
</p>

<p align="center">
  <sup>1</sup>Riemann Dynamics &nbsp;&nbsp; <sup>2</sup>Nanyang Technological University &nbsp;&nbsp; <sup>3</sup>Wellington College, UK
</p>

<p align="center"><sub><sup>*</sup>Equal contribution &nbsp;&nbsp; <sup>&dagger;</sup>Corresponding author</sub></p>

<p align="center">
  <a href="https://lijiaxing0213.github.io/DistillAlign">Project Page</a> ·
  <a href="https://huggingface.co/papers/2607.26811">Paper</a> ·
  <a href="https://github.com/LiJiaxing0213/DistillAlign">Code</a>
</p>

**DistillAlign** aligns and balances the **mode-covering** and **mode-seeking**
objectives of the multi-stage video distillation pipeline — using only a
**1.3B DMD teacher**, it already surpasses baselines refined with a **14B DMD
teacher**.

## Checkpoints

All released generators are **Wan2.1-1.3B students**; the size in each name
refers to the **teacher** used during training.

| Model | Checkpoint | Description |
|---|---|---|
| Initializer (1.3B teacher) | [distillalign_init_1p3b_teacher.pt](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/checkpoints/distillalign_init_1p3b_teacher.pt) | Pre-DMD initializer, trained toward a Wan2.1-T2V-1.3B teacher |
| Initializer (14B teacher) | [distillalign_init_14b_teacher.pt](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/checkpoints/distillalign_init_14b_teacher.pt) | Pre-DMD initializer, trained toward a Wan2.1-T2V-14B teacher |
| Distilled (1.3B teacher) | [distillalign_distill_1p3b_teacher.pt](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/checkpoints/distillalign_distill_1p3b_teacher.pt) | Final joint-distilled generator, Wan2.1-T2V-1.3B DMD teacher |
| Distilled (14B teacher) | [distillalign_distill_14b_teacher.pt](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/checkpoints/distillalign_distill_14b_teacher.pt) | Final joint-distilled generator, Wan2.1-T2V-14B DMD teacher |

Each checkpoint stores a plain `{"generator": state_dict}` and loads directly
with the released inference and evaluation code.

## Distillation Data

Prompts from VidProM, 25K clean-latent samples per teacher:

| Data | Download | Description |
|---|---|---|
| Wan2.1-14B data | [DistillAlign_14B_25K](https://huggingface.co/datasets/LiJiaxing/DistillAlign_14B_25K) | 25K distillation samples generated by Wan2.1-T2V-14B |
| Wan2.1-1.3B data | [DistillAlign_1p3b_25K](https://huggingface.co/datasets/LiJiaxing/DistillAlign_1p3b_25K) | 25K distillation samples generated by Wan2.1-T2V-1.3B |

## Evaluation Caches

Teacher reference artifacts for the teacher-normalized distribution
evaluation (16 prompts x 16 seeds, 25-step UniPC, timestep shift 8.0,
CFG 5.0; V-JEPA2 ViT-H features, 2560-D):

| Cache | Download | Description |
|---|---|---|
| Wan2.1-1.3B teacher reference features | [wan2.1_t2v_1.3b_reference_vjepa2.npz](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/teacher_caches/wan2.1_t2v_1.3b_reference_vjepa2.npz) | 256 x 2560 features, ready for `--teacher-features` |
| Wan2.1-14B teacher reference features | [wan2.1_t2v_14b_reference_vjepa2.npz](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/teacher_caches/wan2.1_t2v_14b_reference_vjepa2.npz) | 256 x 2560 features, ready for `--teacher-features` |
| Teacher reference videos | [teacher_caches/videos](https://huggingface.co/LiJiaxing/DistillAlign/tree/main/teacher_caches/videos) | 256 reference videos per teacher, for inspection and re-extraction |

See [teacher_caches/README.md](https://huggingface.co/LiJiaxing/DistillAlign/blob/main/teacher_caches/README.md)
for the exact protocol.

## Citation

```bibtex
@misc{li2026distillalign,
  title         = {DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation},
  author        = {Li, Jiaxing and Zou, Kai and Zhou, Cindy and Huang, Kaichen and Gao, Junyao and Wang, Zile and Liu, Yang and Liu, Bin and An, Bo and Li, Yangguang},
  year          = {2026},
  eprint        = {2607.26811},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2607.26811}
}
```

## License

The code and released weights follow the Apache License 2.0; upstream
Wan2.1 model weights remain subject to their own licenses.