WeiChow commited on
Commit
0086d5e
·
verified ·
1 Parent(s): 22327b6

Add DiffusionOPSD model card

Browse files
Files changed (1) hide show
  1. README.md +80 -0
README.md CHANGED
@@ -1,3 +1,83 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: diffusers
4
+ pipeline_tag: text-to-image
5
+ base_model:
6
+ - stabilityai/stable-diffusion-3.5-medium
7
+ - Tongyi-MAI/Z-Image-Turbo
8
+ tags:
9
+ - diffusion
10
+ - text-to-image
11
+ - image-generation
12
+ - reinforcement-learning
13
+ - self-distillation
14
+ - lora
15
+ - arxiv:2608.24646
16
  ---
17
+
18
+ <div align="center">
19
+
20
+ # DiffusionOPSD: On-Policy Self-Distillation in Diffusion Models
21
+
22
+ **Reward-guided diffusion post-training through explicit, continually refreshed intermediate targets**
23
+
24
+ [![Paper](https://img.shields.io/badge/arXiv-2608.24646-b31b1b?logo=arxiv)](https://arxiv.org/abs/2608.24646)
25
+ [![Project Page](https://img.shields.io/badge/Project-Page-3B82F6)](https://diffusionopsd.github.io/)
26
+ [![Code](https://img.shields.io/badge/Code-GitHub-181717?logo=github)](https://github.com/worldbench/DiffusionOPSD)
27
+
28
+ <img src="assets/qualitative_gallery.jpg" width="100%" alt="Images generated with DiffusionOPSD">
29
+
30
+ </div>
31
+
32
+ ## Overview
33
+
34
+ **DiffusionOPSD** is an on-policy self-distillation framework for reward-guided diffusion post-training. A frozen behavior policy collects on-policy denoising states and clean-output anchors; differentiable reward gradients construct bounded positive and negative targets around each anchor; and the trainable policy fits these detached targets before an EMA update refreshes the behavior policy.
35
+
36
+ By turning image-level rewards into explicit, continually refreshed intermediate supervision, DiffusionOPSD makes **target construction** and **finite realization** separately observable. Across SD3.5-M and Z-Image-Turbo, it achieves the best final held-out score in **19 of 20** reward-matched settings and reduces training GPU-hours relative to DiffusionNFT by **40%** and **63%**, respectively.
37
+
38
+ <p align="center">
39
+ <img src="assets/method_overview.png" width="100%" alt="DiffusionOPSD method overview">
40
+ </p>
41
+
42
+ ## Released Checkpoints
43
+
44
+ This repository provides three rank-32 LoRA adapters:
45
+
46
+ | Checkpoint | Backbone | Training objective |
47
+ |---|---|---|
48
+ | [`sd35-m-hpsv3`](./sd35-m-hpsv3) | [Stable Diffusion 3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | HPSv3 |
49
+ | [`z-image-turbo-hpsv3`](./z-image-turbo-hpsv3) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | HPSv3 |
50
+ | [`z-image-turbo-pointwise`](./z-image-turbo-pointwise) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | Pointwise reward |
51
+
52
+ Download all released adapters with:
53
+
54
+ ```bash
55
+ hf download WeiChow/DiffusionOPSD --local-dir checkpoints/diffusionopsd
56
+ ```
57
+
58
+ <p align="center">
59
+ <img src="assets/training_curves.png" width="100%" alt="DiffusionOPSD training and held-out quality curves">
60
+ </p>
61
+
62
+ ## Resources
63
+
64
+ - **Paper:** [On-Policy Self-Distillation in Diffusion Models](https://arxiv.org/abs/2608.24646)
65
+ - **Code:** [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD)
66
+ - **Project page:** [diffusionopsd.github.io](https://diffusionopsd.github.io/)
67
+
68
+ Please refer to the [GitHub repository](https://github.com/worldbench/DiffusionOPSD) for installation, inference, evaluation, and training instructions.
69
+
70
+ ## Citation
71
+
72
+ ```bibtex
73
+ @article{zhou2026onpolicy,
74
+ title = {On-Policy Self-Distillation in Diffusion Models},
75
+ author = {Zhou, Wei and Zhu, Xiongwei and Kong, Lingdong and Chen, Bo and Zhang, Lei and Liang, Yongyuan and Hou, Xiaoxia and Tian, Ye and Sun, Xian and Wang, Yingshuo and Li, Linfeng and Wu, Shengqiong and Qu, Leigang and Li, Feng and Liu, Wei and McAuley, Julian and Chua, Tat-Seng},
76
+ journal = {arXiv preprint arXiv:2608.24646},
77
+ year = {2026}
78
+ }
79
+ ```
80
+
81
+ ## License
82
+
83
+ The released adapters are provided under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Users must also comply with the licenses of the corresponding base models.