library_name: diffusers
license: mit
pipeline_tag: text-to-image
tags:
- text-to-image
- safety-alignment
- stable-diffusion
- ECCV
SAGE: Structure-Aware Geometric Regularization (ECCV-26)
Paper: The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models
Authors: Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah
Institution: University of Central Florida
Project Page: https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/
Overview
We show that existing T2I safety alignment methods create an illusion of high utility — they appear to maintain high utility under coarse metrics (FID, CLIPScore) but suffer significant drops in fine-grained semantic fidelity (TIFA). We trace this to semantic collapse in the text encoder embedding space.
SAGE is a geometry-aware safety alignment method that preserves embedding spread and local similarity structure during fine-tuning, achieving only a −1.2% TIFA drop vs. −6.2% for DES while maintaining strong safety (Avg. ASR 1.2%).
Use this Model
import torch
from diffusers import StableDiffusionPipeline
from huggingface_hub import hf_hub_download
# Load base pipeline
pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4")
# Download and load SAGE text encoder weights
ckpt_path = hf_hub_download(repo_id="Adeely93/SAGE", filename="SAGE.pt")
pipe.text_encoder.load_state_dict(torch.load(ckpt_path, map_location="cpu"))
pipe = pipe.to("cuda")
image = pipe("a photo of a dog in a park").images[0]