Xixixixihahahaha/RealAlign-SD-3.5-M

RealAlign — SD-3.5-M LoRA Checkpoints

LoRA adapters for Stable Diffusion 3.5 Medium, aligned with RealAlign from the paper "When Preference Labels Fall Short: Aligning Diffusion Models from Real Data" (ICML 2026).

Resource	Link
📄 Paper	arXiv:2605.19839
🌐 Project page	cwyxx.github.io/RealAlign
🚀 Code	github.com/Cwyxx/RealAlign
🤗 Dataset	RealAlign-Dataset

Summary

RealAlign aligns text-to-image diffusion models using real data as the preference signal: instead of human-annotated preference pairs, it treats a high-quality reference image as the preferred ("win") sample and a perturbed/inpainted version as the non-preferred ("lose") sample. These adapters are the result of fine-tuning SD-3.5-M with RealAlign's two-stage procedure:

1. Stage 1 — Diffusion-DRO (inverse RL / distributionally-robust objective), LoRA-init.
2. Stage 2 — Diffusion-DPO with LoRA-init, warm-started from the Stage 1 LoRA.

Checkpoints

Each subfolder is a PEFT LoRA adapter trained on preference pairs from a different curation source:

Subfolder	Training source
HPDv3/	HPDv3 (real-photo references)
Civitai-top/	Civitai top SFW images

Each contains adapter_config.json + adapter_model.safetensors.

• Format: PEFT LoRA on the SD3Transformer2DModel.
• LoRA config: r=32, lora_alpha=64, gaussian init, applied to the joint-attention projections (to_q/k/v, to_out, add_q/k/v_proj, to_add_out).

Note: The Pick-a-Pic v2 subset used in the paper is not released here because the source image data may contain NSFW content (see the dataset card).

Usage

import torch
from diffusers import StableDiffusion3Pipeline

pipe = StableDiffusion3Pipeline.from_pretrained(
    "stabilityai/stable-diffusion-3.5-medium",
    torch_dtype=torch.bfloat16,
).to("cuda")

# Load the adapter trained on the source you want (subfolder = HPDv3 or Civitai-top)
pipe.load_lora_weights(
    "Xixixixihahahaha/RealAlign-SD-3.5-M",
    subfolder="HPDv3",
)

image = pipe("a photo of an astronaut riding a horse on the moon").images[0]
image.save("out.png")

Citation

@article{chen2026preference,
  title={When Preference Labels Fall Short: Aligning Diffusion Models from Real Data},
  author={Chen, Weiyan and Deng, Weijian and Xiao, Yao and Tu, Weijie and Dong, ZiYi and Radwan, Ibrahim and Lin, Liang and Wei, Pengxu},
  journal={arXiv preprint arXiv:2605.19839},
  year={2026}
}