Add comprehensive model card for DAA project

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +60 -0
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: text-to-image
4
+ library_name: transformers
5
+ ---
6
+
7
+ # 🛡️DAA: Dynamic Attention Analysis for Backdoor Detection in Text-to-Image Diffusion Models
8
+
9
+ This repository contains artifacts and code related to the paper: [**Dynamic Attention Analysis for Backdoor Detection in Text-to-Image Diffusion Models**](https://huggingface.co/papers/2504.20518).
10
+
11
+ Code: https://github.com/Robin-WZQ/DAA
12
+
13
+ This study introduces a novel backdoor detection perspective from **Dynamic Attention Analysis (DAA)**, which shows that the **dynamic feature in attention maps** can serve as a much better indicator for backdoor detection in text-to-image diffusion models. By examining the dynamic evolution of cross-attention maps, backdoor samples exhibit distinct feature evolution patterns compared to benign samples, particularly at the `<EOS>` token.
14
+
15
+ ## 👀 Overview
16
+
17
+ <div align=center>
18
+ <img src='https://github.com/Robin-WZQ/DAA/blob/main/viz/Overview.png' width=800>
19
+ </div>
20
+
21
+ The overview of our Dynamic Attention Analysis (DAA). **(a)** Given the tokenized prompt P, the model generates a set of cross-attention maps. **(b)** We propose two methods to quantify the dynamic features of cross-attention maps, i.e., DAA-I and DAA-S. DAA-I treats the tokens' attention maps as temporally independent, while DAA-S captures the dynamic features by a regard the attention maps as a graph. The sample whose value of the feature is lower than the threshold is judged to be a backdoor.
22
+
23
+ <div align=center>
24
+ <img src='https://github.com/Robin-WZQ/DAA/blob/main/viz/Evolve.svg' width=450>
25
+ </div>
26
+
27
+ The average relative evolution trajectories of the <EOS> token in benign samples (the orange line) and backdoor samples (the blue line). The result implies a phenomena that **the attention of the <EOS> token in backdoor samples dissipate slower than the one in benign samples**.
28
+
29
+ ## 🏃🏼 Running Scripts (Sample Usage)
30
+
31
+ **For detecting a sample (text as input):**
32
+ - DAA-I
33
+ ```python
34
+ python detect_daai_uni.py --input_text "blonde man with glasses near beach" --backdoor_model_name "Rickrolling" --backdoor_model_path "./model/train/poisoned_model"
35
+ python detect_daai_uni.py --input_text "Ѵ blonde man with glasses near beach" --backdoor_model_name "Rickrolling" --backdoor_model_path "./model/train/poisoned_model"
36
+ ```
37
+ - DAA-S
38
+ ```python
39
+ python detect_daas_uni.py --input_text "blonde man with glasses near beach" --backdoor_model_name "Rickrolling" --backdoor_model_path "./model/train/poisoned_model"
40
+ python detect_daas_uni.py --input_text "Ѵ blonde man with glasses near beach" --backdoor_model_name "Rickrolling" --backdoor_model_path "./model/train/poisoned_model"
41
+ ```
42
+ - We also provide the visualization script for reproducing the images in our paper:
43
+ - `Visualization_DAA.ipynb`
44
+ For example:
45
+
46
+ <div align=center>
47
+ <img src='https://github.com/Robin-WZQ/DAA/blob/main/viz/output1.gif' width=800>
48
+ </div>
49
+
50
+ ## 📄 Citation
51
+
52
+ If you find this project useful in your research, please consider cite:
53
+ ```bibtex
54
+ @article{wang2025dynamicattentionanalysisbackdoor,
55
+ title={Dynamic Attention Analysis for Backdoor Detection in Text-to-Image Diffusion Models},
56
+ author={Zhongqi Wang and Jie Zhang and Shiguang Shan and Xilin Chen},
57
+ journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
58
+ year={2025},
59
+ }
60
+ ```