File size: 4,137 Bytes
76d175c ef94091 76d175c ef94091 0c8198e ef94091 76d175c 0c8198e 76d175c 0c8198e 76d175c 0c8198e ef94091 0c8198e 76d175c 0c8198e 76d175c b9c301b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
library_name: peft
pipeline_tag: text-to-image
base_model: stabilityai/stable-diffusion-3.5-medium
license: other
license_name: stability-ai-community
license_link: LICENSE.md
tags:
- diffusionart
- stable-diffusion-3
- reinforcement-learning
- lora
- arxiv:2610.12572
---
# DiffusionART: SD3.5-M Multi-Reward
Powered by Stability AI.
A PEFT LoRA adapter for Stable Diffusion 3.5 Medium, post-trained with
DiffusionART using GenEval, CLIPScore, PickScore and HPSv2 feedback.
[Paper](https://arxiv.org/abs/2610.12572) |
[Code](https://github.com/vanzll/DiffusionART) |
[Project Page](https://vanzll.github.io/diffusionart-project/) |
[Demo](https://vanzll.github.io/diffusionart-project/demo.html)
Only the LoRA adapter is included; download the SD3.5 Medium base model
separately. The adapter uses rank 32 and alpha 64.
## Samples
Same prompt and seed within each pair.
<table>
<tr><th>Pre-trained</th><th>DiffusionART</th></tr>
<tr>
<td><img src="examples/p05_s123_base.png" width="320" alt="Pre-trained: monarch butterfly on lavender"></td>
<td><img src="examples/p05_s123_optimized.png" width="320" alt="DiffusionART: monarch butterfly on lavender"></td>
</tr>
<tr><td colspan="2"><em>Macro photograph of a monarch butterfly resting on purple lavender, crisp wing detail and soft garden background.</em></td></tr>
<tr>
<td><img src="examples/p06_s42_base.png" width="320" alt="Pre-trained: cabin in a snowy pine forest"></td>
<td><img src="examples/p06_s42_optimized.png" width="320" alt="DiffusionART: cabin in a snowy pine forest"></td>
</tr>
<tr><td colspan="2"><em>A small modern cabin beside a snowy pine forest at blue hour, warm light visible through two windows.</em></td></tr>
<tr>
<td><img src="examples/p02_s42_base.png" width="320" alt="Pre-trained: red teapot and blue glass cup"></td>
<td><img src="examples/p02_s42_optimized.png" width="320" alt="DiffusionART: red teapot and blue glass cup"></td>
</tr>
<tr><td colspan="2"><em>A red ceramic teapot to the left of a blue glass cup on a white kitchen table, natural window light.</em></td></tr>
</table>
## Inference
Obtain access to the gated [base model](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium),
authenticate with Hugging Face, and install the verified dependencies in an
isolated Python environment:
<details>
<summary>Verified dependencies</summary>
```bash
pip install torch==2.7.0 diffusers==0.35.1 transformers==4.57.1 \
peft==0.17.1 accelerate==1.8.1 huggingface_hub==0.36.2 \
safetensors==0.5.2 sentencepiece==0.2.1 protobuf==5.29.5 Pillow
```
Use an official CUDA-enabled PyTorch wheel compatible with your driver. Then
authenticate with an account that has access to the base model and this adapter.
</details>
Download the adapter and run inference:
```bash
python -c 'from huggingface_hub import snapshot_download; snapshot_download("vanzl/DiffusionART-SD35M-MultiReward", local_dir="diffusionart-mr4")'
cd diffusionart-mr4
```
```bash
python infer.py --prompt "A horse running through a sunlit meadow." --output image.png
```
The default settings are 512 x 512, 40 sampling steps, guidance scale 1.0 and
seed 42. `--cpu-offload` is optional. The script attaches the adapter to the
base transformer with `PeftModel.from_pretrained`.
See [paired examples](examples/) and [inference details](examples/VERIFIED_INFERENCE.md).
## License
The adapter is a derivative of Stable Diffusion 3.5 Medium and is subject to
the included Stability AI Community License and upstream acceptable-use policy.
See `LICENSE.md` and `NOTICE`. The training framework's code license does not
replace the model license.
## Citation
```bibtex
@misc{wan2026diffusionartefficientstablescalable,
title={DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models},
author={Zhenglin Wan and Jiayu Chen and Zhengfeiyang Zhang and Pengfei Zhou and Wangbo Zhao and Fangyuan Kong and Pengfei Wan and Kun Gai and Yang You},
year={2026},
eprint={2610.12572},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.12572},
}
```
|