Text-to-Image
Diffusers
Safetensors
diffusion
image-generation
reinforcement-learning
self-distillation
lora
Instructions to use WeiChow/DiffusionOPSD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use WeiChow/DiffusionOPSD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-3.5-medium", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("WeiChow/DiffusionOPSD") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
update
Browse files- .gitattributes +1 -0
- README.md +31 -3
- assets/opsd_v2.png +3 -0
- new.zip +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
assets/method_overview.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/qualitative_gallery.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
assets/training_curves.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
assets/method_overview.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/qualitative_gallery.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
assets/training_curves.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
assets/opsd_v2.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -24,8 +24,9 @@ tags:
|
|
| 24 |
[](https://arxiv.org/abs/2608.24646)
|
| 25 |
[](https://diffusionopsd.github.io/)
|
| 26 |
[](https://github.com/worldbench/DiffusionOPSD)
|
|
|
|
| 27 |
|
| 28 |
-
<img src="assets/
|
| 29 |
|
| 30 |
</div>
|
| 31 |
|
|
@@ -35,13 +36,39 @@ tags:
|
|
| 35 |
|
| 36 |
By turning image-level rewards into explicit, continually refreshed intermediate supervision, DiffusionOPSD makes **target construction** and **finite realization** separately observable. Across SD3.5-M and Z-Image-Turbo, it achieves the best final held-out score in **19 of 20** reward-matched settings and reduces training GPU-hours relative to DiffusionNFT by **40%** and **63%**, respectively.
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
<p align="center">
|
| 39 |
<img src="assets/method_overview.png" width="100%" alt="DiffusionOPSD method overview">
|
| 40 |
</p>
|
| 41 |
|
| 42 |
## Released Checkpoints
|
| 43 |
|
| 44 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
| Checkpoint | Backbone | Training objective |
|
| 47 |
|---|---|---|
|
|
@@ -55,7 +82,7 @@ This repository provides nine rank-32 LoRA adapters:
|
|
| 55 |
| [`z-image-turbo-hpsv3`](./z-image-turbo-hpsv3) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | HPSv3 |
|
| 56 |
| [`z-image-turbo-pointwise`](./z-image-turbo-pointwise) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | Pointwise reward |
|
| 57 |
|
| 58 |
-
Download
|
| 59 |
|
| 60 |
```bash
|
| 61 |
hf download WeiChow/DiffusionOPSD --local-dir checkpoints/diffusionopsd
|
|
@@ -70,6 +97,7 @@ hf download WeiChow/DiffusionOPSD --local-dir checkpoints/diffusionopsd
|
|
| 70 |
- **Paper:** [On-Policy Self-Distillation in Diffusion Models](https://arxiv.org/abs/2608.24646)
|
| 71 |
- **Code:** [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD)
|
| 72 |
- **Project page:** [diffusionopsd.github.io](https://diffusionopsd.github.io/)
|
|
|
|
| 73 |
|
| 74 |
Please refer to the [GitHub repository](https://github.com/worldbench/DiffusionOPSD) for installation, inference, evaluation, and training instructions.
|
| 75 |
|
|
|
|
| 24 |
[](https://arxiv.org/abs/2608.24646)
|
| 25 |
[](https://diffusionopsd.github.io/)
|
| 26 |
[](https://github.com/worldbench/DiffusionOPSD)
|
| 27 |
+
[](https://diffusionopsd.github.io/blog/v2/)
|
| 28 |
|
| 29 |
+
<a href="https://diffusionopsd.github.io/blog/v2/"><img src="assets/opsd_v2.png" width="100%" alt="DiffusionOPSD v2 training overview"></a>
|
| 30 |
|
| 31 |
</div>
|
| 32 |
|
|
|
|
| 36 |
|
| 37 |
By turning image-level rewards into explicit, continually refreshed intermediate supervision, DiffusionOPSD makes **target construction** and **finite realization** separately observable. Across SD3.5-M and Z-Image-Turbo, it achieves the best final held-out score in **19 of 20** reward-matched settings and reduces training GPU-hours relative to DiffusionNFT by **40%** and **63%**, respectively.
|
| 38 |
|
| 39 |
+
<p align="center">
|
| 40 |
+
<img src="assets/qualitative_gallery.jpg" width="100%" alt="Images generated with DiffusionOPSD">
|
| 41 |
+
</p>
|
| 42 |
+
|
| 43 |
<p align="center">
|
| 44 |
<img src="assets/method_overview.png" width="100%" alt="DiffusionOPSD method overview">
|
| 45 |
</p>
|
| 46 |
|
| 47 |
## Released Checkpoints
|
| 48 |
|
| 49 |
+
### DiffusionOPSD\* (v2): `new.zip`
|
| 50 |
+
|
| 51 |
+
[`new.zip`](./new.zip) contains the checkpoints for **DiffusionOPSD\***, the accelerated v2 recipe. It keeps the same target-construction principle and uses one reward-ascent step, broader prompt coverage, and rollouts that stop at the query. See the [v2 blog](https://diffusionopsd.github.io/blog/v2/) for the update.
|
| 52 |
+
|
| 53 |
+
The archive contains **eight rank-32 LoRA adapters**, covering **SD3.5-M and Z-Image-Turbo × HPSv3, HPSv2.1, CLIPScore, and PickScore**. All are the final **`checkpoint-100` EMA inference weights**. The archive is **1.46 GB**.
|
| 54 |
+
|
| 55 |
+
```bash
|
| 56 |
+
hf download WeiChow/DiffusionOPSD new.zip --local-dir checkpoints/diffusionopsd-star
|
| 57 |
+
unzip checkpoints/diffusionopsd-star/new.zip -d checkpoints/diffusionopsd-star
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
The extracted `opsd_v2_2x4_20261012/` directory contains:
|
| 61 |
+
|
| 62 |
+
| Backbone | Adapter folders |
|
| 63 |
+
|---|---|
|
| 64 |
+
| SD3.5-M | `sd35-m-hpsv3`, `sd35-m-hpsv2`, `sd35-m-clipscore`, `sd35-m-pickscore` |
|
| 65 |
+
| Z-Image-Turbo | `z-image-turbo-hpsv3`, `z-image-turbo-hpsv2`, `z-image-turbo-clipscore`, `z-image-turbo-pickscore` |
|
| 66 |
+
|
| 67 |
+
Each folder contains `adapter_config.json` and `adapter_model.safetensors`.
|
| 68 |
+
|
| 69 |
+
### Original DiffusionOPSD adapters
|
| 70 |
+
|
| 71 |
+
The nine top-level adapter folders below contain the original DiffusionOPSD checkpoints. The **DiffusionOPSD\*** checkpoints are packaged separately in `new.zip`.
|
| 72 |
|
| 73 |
| Checkpoint | Backbone | Training objective |
|
| 74 |
|---|---|---|
|
|
|
|
| 82 |
| [`z-image-turbo-hpsv3`](./z-image-turbo-hpsv3) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | HPSv3 |
|
| 83 |
| [`z-image-turbo-pointwise`](./z-image-turbo-pointwise) | [Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) | Pointwise reward |
|
| 84 |
|
| 85 |
+
Download the repository, including `new.zip`, with:
|
| 86 |
|
| 87 |
```bash
|
| 88 |
hf download WeiChow/DiffusionOPSD --local-dir checkpoints/diffusionopsd
|
|
|
|
| 97 |
- **Paper:** [On-Policy Self-Distillation in Diffusion Models](https://arxiv.org/abs/2608.24646)
|
| 98 |
- **Code:** [worldbench/DiffusionOPSD](https://github.com/worldbench/DiffusionOPSD)
|
| 99 |
- **Project page:** [diffusionopsd.github.io](https://diffusionopsd.github.io/)
|
| 100 |
+
- **V2 blog:** [DiffusionOPSD v2](https://diffusionopsd.github.io/blog/v2/)
|
| 101 |
|
| 102 |
Please refer to the [GitHub repository](https://github.com/worldbench/DiffusionOPSD) for installation, inference, evaluation, and training instructions.
|
| 103 |
|
assets/opsd_v2.png
ADDED
|
Git LFS Details
|
new.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:89b166aa38877f55aa5462e828f07e6c42d9588671ef63a5b22e446cc201a1a0
|
| 3 |
+
size 1457897214
|