Text-to-Image
Diffusers
Safetensors
diffusion
image-generation
reinforcement-learning
self-distillation
lora
Instructions to use WeiChow/DiffusionOPSD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use WeiChow/DiffusionOPSD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-3.5-medium,Tongyi-MAI/Z-Image-Turbo", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("WeiChow/DiffusionOPSD") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Add DiffusionOPSD model card
Browse files
README.md
CHANGED
|
@@ -1,3 +1,83 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
pipeline_tag: text-to-image
|
| 5 |
+
base_model:
|
| 6 |
+
- stabilityai/stable-diffusion-3.5-medium
|
| 7 |
+
- Tongyi-MAI/Z-Image-Turbo
|
| 8 |
+
tags:
|
| 9 |
+
- diffusion
|
| 10 |
+
- text-to-image
|
| 11 |
+
- image-generation
|
| 12 |
+
- reinforcement-learning
|
| 13 |
+
- self-distillation
|
| 14 |
+
- lora
|
| 15 |
+
- arxiv:2608.24646
|
| 16 |
---
|
| 17 |
+
|
| 18 |
+
<div align="center">
|
| 19 |
+
|
| 20 |
+
# DiffusionOPSD: On-Policy Self-Distillation in Diffusion Models
|
| 21 |
+
|
| 22 |
+
**Reward-guided diffusion post-training through explicit, continually refreshed intermediate targets**
|
| 23 |
+
|
| 24 |
+
[](https://arxiv.org/abs/2608.24646)
|
| 25 |
+
[](https://diffusionopsd.github.io/)
|
| 26 |
+
[](https://github.com/worldbench/DiffusionOPSD)
|
| 27 |
+
|
| 28 |
+
<img src="assets/qualitative_gallery.jpg" width="100%" alt="Images generated with DiffusionOPSD">
|
| 29 |
+
|
| 30 |
+
</div>
|
| 31 |
+
|
| 32 |
+
## Overview
|
| 33 |
+
|
| 34 |
+
**DiffusionOPSD** is an on-policy self-distillation framework for reward-guided diffusion post-training. A frozen behavior policy collects on-policy denoising states and clean-output anchors; differentiable reward gradients construct bounded positive and negative targets around each anchor; and the trainable policy fits these detached targets before an EMA update refreshes the behavior policy.
|
| 35 |
+
|
| 36 |
+
By turning image-level rewards into explicit, continually refreshed intermediate supervision, DiffusionOPSD makes **target construction** and **finite realization** separately observable. Across SD3.5-M and Z-Image-Turbo, it achieves the best final held-out score in **19 of 20** reward-matched settings and reduces training GPU-hours relative to DiffusionNFT by **40%** and **63%**, respectively.
|
| 37 |
+
|
| 38 |
+
<p align="center">
|
| 39 |
+
<img src="assets/method_overview.png" width="100%" alt="DiffusionOPSD method overview">
|
| 40 |
+
</p>
|
| 41 |
+
|
| 42 |
+
## Released Checkpoints
|
| 43 |
+
|
| 44 |
+
This repository provides three rank-32 LoRA adapters:
|
| 45 |
+
|
| 46 |
+
| Checkpoint | Backbone | Training objective |
|
| 47 |
+
|---|---|---|
|
| 48 |
+
| [`sd35-m-hpsv3`](./sd35-m-hpsv3) | [Stable Diffusion 3.5 Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | HPSv3 |
|
| 49 |
+
| [`z-image-turbo-hpsv3`](./z-image-turbo-hpsv3) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | HPSv3 |
|
| 50 |
+
| [`z-image-turbo-pointwise`](./z-image-turbo-pointwise) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | Pointwise reward |
|
| 51 |
+
|
| 52 |
+
Download all released adapters with:
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
hf download WeiChow/DiffusionOPSD --local-dir checkpoints/diffusionopsd
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
<p align="center">
|
| 59 |
+
<img src="assets/training_curves.png" width="100%" alt="DiffusionOPSD training and held-out quality curves">
|
| 60 |
+
</p>
|
| 61 |
+
|
| 62 |
+
## Resources
|
| 63 |
+
|
| 64 |
+
- **Paper:** [On-Policy Self-Distillation in Diffusion Models](https://arxiv.org/abs/2608.24646)
|
| 65 |
+
- **Code:** [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD)
|
| 66 |
+
- **Project page:** [diffusionopsd.github.io](https://diffusionopsd.github.io/)
|
| 67 |
+
|
| 68 |
+
Please refer to the [GitHub repository](https://github.com/worldbench/DiffusionOPSD) for installation, inference, evaluation, and training instructions.
|
| 69 |
+
|
| 70 |
+
## Citation
|
| 71 |
+
|
| 72 |
+
```bibtex
|
| 73 |
+
@article{zhou2026onpolicy,
|
| 74 |
+
title = {On-Policy Self-Distillation in Diffusion Models},
|
| 75 |
+
author = {Zhou, Wei and Zhu, Xiongwei and Kong, Lingdong and Chen, Bo and Zhang, Lei and Liang, Yongyuan and Hou, Xiaoxia and Tian, Ye and Sun, Xian and Wang, Yingshuo and Li, Linfeng and Wu, Shengqiong and Qu, Leigang and Li, Feng and Liu, Wei and McAuley, Julian and Chua, Tat-Seng},
|
| 76 |
+
journal = {arXiv preprint arXiv:2608.24646},
|
| 77 |
+
year = {2026}
|
| 78 |
+
}
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
## License
|
| 82 |
+
|
| 83 |
+
The released adapters are provided under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Users must also comply with the licenses of the corresponding base models.
|