license: apache-2.0
library_name: diffusers
tags:
- text-to-image
- diffusion
- alignment
- score-matching
- soft-tokens
- stable-diffusion
base_model:
- stabilityai/stable-diffusion-3-medium-diffusers
- stable-diffusion-v1-5/stable-diffusion-v1-5
- stabilityai/stable-diffusion-xl-base-1.0
Alignment-Guided Score Matching (AGSM)
This repository hosts the released AGSM soft-token checkpoints for:
- SD3:
sd3/soft_tokens.pth,sd3/soft_t_tokens.pth - SD1.5:
sd1.5/soft_tokens.pth,sd1.5/soft_t_tokens.pth - SDXL:
sdxl/soft_tokens.pth,sdxl/soft_t_tokens.pth
AGSM is a lightweight, reward-free post-training method for improving text-image alignment in diffusion models. It requires no external reward model, no full denoising rollout, and no (x_0) approximation.
Usage
This repository contains AGSM token checkpoints, not the full base diffusion models. The GitHub repository already includes the released checkpoints, so the simplest path is:
git clone https://github.com/jaayeon/AGSM.git
cd AGSM
DATADIR=/path/to/datasets \
MODEL=sd3 \
scripts/sample_coco.sh
If you want to use the Hugging Face copy instead, download it separately and point CHECKPOINT_DIR to the downloaded model folder:
huggingface-cli download jaayeon/AGSM --local-dir checkpoints/agsm
DATADIR=/path/to/datasets \
MODEL=sd3 \
CHECKPOINT_DIR=checkpoints/agsm/sd3 \
scripts/sample_coco.sh
Use MODEL=sd1.5 with CHECKPOINT_DIR=checkpoints/agsm/sd1.5, or MODEL=sdxl with CHECKPOINT_DIR=checkpoints/agsm/sdxl.
Code, training scripts, and evaluation instructions are available at: https://github.com/jaayeon/AGSM
Project page: https://jaayeon.github.io/AGSM/
Paper: https://arxiv.org/abs/2605.30038
Citation
@article{lee2026alignment,
title={Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models},
author={Lee, Jaa-Yeon and Hong, Yeobin and Kwon, Taesung and Ye, Jong Chul},
journal={arXiv preprint arXiv:2605.30038},
year={2026}
}