|
|
--- |
|
|
language: "en" |
|
|
license: "apache-2.0" |
|
|
tags: |
|
|
- text-to-image |
|
|
- stable-diffusion |
|
|
- diffusion |
|
|
- lora |
|
|
datasets: |
|
|
- custom |
|
|
library_name: "diffusers" |
|
|
pipeline_tag: "text-to-image" |
|
|
--- |
|
|
|
|
|
|
|
|
# GradSPO: A Gradient Guidance Perspective on Stepwise Preference Optimization for Diffusion Models |
|
|
|
|
|
|
|
|
This repository provides **public LoRA checkpoints trained with GradSPO** for **Stable Diffusion v1.5** and **SDXL**. |
|
|
|
|
|
**GradSPO** reframes **stepwise preference optimization (SPO)** as learning from **noisy reward signals**, explicitly reducing this noise through **gradient guidance**. This results in **stronger reward signals** and achieves **improved preference alignment**. |
|
|
|
|
|
All released checkpoints are **LoRA weights only** and must be loaded on top of their corresponding base models. |
|
|
|
|
|
The official training code is available at: |
|
|
https://github.com/JoshuaTTJ/GradSPO |
|
|
|
|
|
--- |
|
|
|
|
|
## Usage |
|
|
|
|
|
### SDXL (LoRA) |
|
|
|
|
|
```python |
|
|
from diffusers import StableDiffusionXLPipeline |
|
|
import torch |
|
|
|
|
|
pipe = StableDiffusionXLPipeline.from_pretrained( |
|
|
"stabilityai/stable-diffusion-xl-base-1.0", |
|
|
torch_dtype=torch.float16, |
|
|
) |
|
|
|
|
|
pipe.load_lora_weights("./sd1_5") |
|
|
|
|
|
pipe = pipe.to("cuda") |
|
|
|
|
|
prompt = "A cat holding a sign that says hello world" |
|
|
|
|
|
generator = torch.Generator(device="cuda").manual_seed(42) |
|
|
image = pipe( |
|
|
prompt=prompt, |
|
|
guidance_scale=5.0, |
|
|
num_inference_steps=20, |
|
|
generator=generator, |
|
|
output_type="pil", |
|
|
).images[0] |
|
|
|
|
|
image.save("img_sdxl.png") |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
### Stable Diffusion v1.5 (LoRA) |
|
|
|
|
|
```python |
|
|
from diffusers import StableDiffusionPipeline |
|
|
import torch |
|
|
|
|
|
pipe = StableDiffusionPipeline.from_pretrained( |
|
|
"sd-legacy/stable-diffusion-v1-5", |
|
|
torch_dtype=torch.float16, |
|
|
) |
|
|
|
|
|
pipe.load_lora_weights("./sdxl") |
|
|
|
|
|
pipe = pipe.to("cuda") |
|
|
|
|
|
prompt = "a photo of a cat" |
|
|
|
|
|
generator = torch.Generator(device="cuda").manual_seed(42) |
|
|
image = pipe( |
|
|
prompt=prompt, |
|
|
guidance_scale=5.0, |
|
|
num_inference_steps=20, |
|
|
generator=generator, |
|
|
output_type="pil", |
|
|
).images[0] |
|
|
|
|
|
image.save("img_sd15.png") |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you find GradSPO useful in your research, please consider citing our work: |
|
|
|
|
|
```bibtex |
|
|
@inproceedings{ |
|
|
tee2025a, |
|
|
title={A Gradient Guidance Perspective on Stepwise Preference Optimization for Diffusion Models}, |
|
|
author={Joshua Tian Jin Tee and Hee Suk Yoon and Abu Hanif Muhammad Syarubany and Eunseop Yoon and Chang D. Yoo}, |
|
|
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems}, |
|
|
year={2025}, |
|
|
url={https://openreview.net/forum?id=d6lIOnvOX2} |
|
|
} |
|
|
``` |
