README.md

---
library_name: peft
pipeline_tag: text-to-image
base_model: stabilityai/stable-diffusion-3.5-medium
license: other
license_name: stability-ai-community
license_link: LICENSE.md
tags:
  - diffusionart
  - stable-diffusion-3
  - reinforcement-learning
  - lora
  - arxiv:2610.12572
---

# DiffusionART: SD3.5-M Multi-Reward

Powered by Stability AI.

A PEFT LoRA adapter for Stable Diffusion 3.5 Medium, post-trained with
DiffusionART using GenEval, CLIPScore, PickScore and HPSv2 feedback.

[Paper](https://arxiv.org/abs/2610.12572) |
[Code](https://github.com/vanzll/DiffusionART) |
[Project Page](https://vanzll.github.io/diffusionart-project/) |
[Demo](https://vanzll.github.io/diffusionart-project/demo.html)

Only the LoRA adapter is included; download the SD3.5 Medium base model
separately. The adapter uses rank 32 and alpha 64.

## Samples

Same prompt and seed within each pair.

<table>
  <tr><th>Pre-trained</th><th>DiffusionART</th></tr>
  <tr>
    <td><img src="examples/p05_s123_base.png" width="320" alt="Pre-trained: monarch butterfly on lavender"></td>
    <td><img src="examples/p05_s123_optimized.png" width="320" alt="DiffusionART: monarch butterfly on lavender"></td>
  </tr>
  <tr><td colspan="2"><em>Macro photograph of a monarch butterfly resting on purple lavender, crisp wing detail and soft garden background.</em></td></tr>
  <tr>
    <td><img src="examples/p06_s42_base.png" width="320" alt="Pre-trained: cabin in a snowy pine forest"></td>
    <td><img src="examples/p06_s42_optimized.png" width="320" alt="DiffusionART: cabin in a snowy pine forest"></td>
  </tr>
  <tr><td colspan="2"><em>A small modern cabin beside a snowy pine forest at blue hour, warm light visible through two windows.</em></td></tr>
  <tr>
    <td><img src="examples/p02_s42_base.png" width="320" alt="Pre-trained: red teapot and blue glass cup"></td>
    <td><img src="examples/p02_s42_optimized.png" width="320" alt="DiffusionART: red teapot and blue glass cup"></td>
  </tr>
  <tr><td colspan="2"><em>A red ceramic teapot to the left of a blue glass cup on a white kitchen table, natural window light.</em></td></tr>
</table>

## Inference

Obtain access to the gated [base model](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium),
authenticate with Hugging Face, and install the verified dependencies in an
isolated Python environment:

<details>
<summary>Verified dependencies</summary>

```bash
pip install torch==2.7.0 diffusers==0.35.1 transformers==4.57.1 \
  peft==0.17.1 accelerate==1.8.1 huggingface_hub==0.36.2 \
  safetensors==0.5.2 sentencepiece==0.2.1 protobuf==5.29.5 Pillow
```

Use an official CUDA-enabled PyTorch wheel compatible with your driver. Then
authenticate with an account that has access to the base model and this adapter.

</details>

Download the adapter and run inference:

```bash
python -c 'from huggingface_hub import snapshot_download; snapshot_download("vanzl/DiffusionART-SD35M-MultiReward", local_dir="diffusionart-mr4")'
cd diffusionart-mr4
```

```bash
python infer.py --prompt "A horse running through a sunlit meadow." --output image.png
```

The default settings are 512 x 512, 40 sampling steps, guidance scale 1.0 and
seed 42. `--cpu-offload` is optional. The script attaches the adapter to the
base transformer with `PeftModel.from_pretrained`.

See [paired examples](examples/) and [inference details](examples/VERIFIED_INFERENCE.md).

## License

The adapter is a derivative of Stable Diffusion 3.5 Medium and is subject to
the included Stability AI Community License and upstream acceptable-use policy.
See `LICENSE.md` and `NOTICE`. The training framework's code license does not
replace the model license.

## Citation

```bibtex
@misc{wan2026diffusionartefficientstablescalable,
  title={DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models},
  author={Zhenglin Wan and Jiayu Chen and Zhengfeiyang Zhang and Pengfei Zhou and Wangbo Zhao and Fangyuan Kong and Pengfei Wan and Kun Gai and Yang You},
  year={2026},
  eprint={2610.12572},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2610.12572},
}
```