Adeely93 commited on
Commit
37b7831
·
verified ·
1 Parent(s): bef4206

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -1
README.md CHANGED
@@ -1,3 +1,31 @@
 
 
 
 
 
 
 
 
 
1
  ---
2
- license: cc-by-nc-4.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # SAGE: Structure-Aware Geometric Regularization (ECCV-26)
3
+
4
+ **Paper:** The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models
5
+ **Venue:** ECCV 2026
6
+ **Authors:** Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah
7
+ **Institution:** University of Central Florida
8
+ **Project Page:** [https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/](https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/)
9
+
10
  ---
11
+
12
+ ## Overview
13
+
14
+ We show that existing T2I safety alignment methods create an **illusion of high utility** — they appear to have high-utility preservation under coarse metrics (FID, CLIPScore) but suffer significant drops in fine-grained semantic fidelity (TIFA). We trace this to **semantic collapse** in the text encoder embedding space.
15
+
16
+ **SAGE** is a geometry-aware safety alignment method that preserves embedding spread and local similarity structure during fine-tuning, achieving only a **−1.2% TIFA drop** vs. **−6.2% for DES** while maintaining strong safety (Avg. ASR 1.2%).
17
+
18
  ---
19
+
20
+ ## Model
21
+
22
+ This is the fine-tuned **text encoder** of Stable Diffusion v1.4. The UNet remains frozen.
23
+
24
+ ```python
25
+ import torch
26
+ from diffusers import StableDiffusionPipeline
27
+
28
+ pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4")
29
+ checkpoint = torch.load("SAGE.pt", map_location="cpu")
30
+ pipe.text_encoder.load_state_dict(checkpoint)
31
+ pipe = pipe.to("cuda")