Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
library_name: pytorch
|
| 5 |
+
tags:
|
| 6 |
+
- image-classification
|
| 7 |
+
- ai-generated-image-detection
|
| 8 |
+
- deepfake-detection
|
| 9 |
+
- universal-detector
|
| 10 |
+
- robustness-asymmetry
|
| 11 |
+
- dinov3
|
| 12 |
+
- diffusion-generated-image-detection
|
| 13 |
+
license: other
|
| 14 |
+
pipeline_tag: image-classification
|
| 15 |
+
metrics:
|
| 16 |
+
- accuracy
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# RA-Det: Robustness Asymmetry Detection for AI-Generated Images
|
| 20 |
+
|
| 21 |
+
Official pretrained checkpoint for **RA-Det** _(ICML 2026)_ — a behavior-driven detector that decides whether an image is real or AI-generated by examining how it **behaves** under controlled perturbations, rather than how it **looks**.
|
| 22 |
+
|
| 23 |
+
> 📄 **Paper:** _RA-Det: Towards Universal Detection of AI-Generated Images via Robustness Asymmetry_ ([arXiv:2603.01544](https://arxiv.org/abs/2603.01544))
|
| 24 |
+
> 💻 **Code:** [github.com/dongdongunique/RA-Det](https://github.com/dongdongunique/RA-Det)
|
| 25 |
+
> 🏛️ **Venue:** ICML 2026
|
| 26 |
+
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
## TL;DR
|
| 30 |
+
|
| 31 |
+
Most detectors hunt for visual artifacts. Modern generators (Stable Diffusion, DALL·E, GANs) increasingly erase those artifacts, so appearance-based methods generalize poorly. RA-Det instead exploits a **robustness asymmetry**:
|
| 32 |
+
|
| 33 |
+
- **Natural images** keep a stable semantic representation under small perturbations.
|
| 34 |
+
- **AI-generated images** drift much more under the *same* perturbations.
|
| 35 |
+
|
| 36 |
+
RA-Det learns a perturbation operator that amplifies this gap, then classifies images from the resulting behavior — making it **data-agnostic, model-agnostic, and transferable to unseen generators**.
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
## How it works
|
| 41 |
+
|
| 42 |
+
RA-Det combines two components:
|
| 43 |
+
|
| 44 |
+
1. **Differential Robustness Probing (DRP)** — a learnable U-Net that applies a *bounded* perturbation (ε = 16/255) to the input, calibrated to maximally separate real vs. generated images in embedding space.
|
| 45 |
+
2. **Multi-Branch Detector** — aggregates three complementary cues:
|
| 46 |
+
- **Semantic features** — from a frozen **DINOv3 ViT-L/16** foundation model (1024-d embeddings).
|
| 47 |
+
- **Discrepancy features** — distance, similarity, and covariance statistics between the *clean* and *DRP-perturbed* embeddings (the core of the robustness-asymmetry signal).
|
| 48 |
+
- **Low-level residual features** — capture high-frequency / forensic artifacts.
|
| 49 |
+
|
| 50 |
+
A **four-branch ensemble** variant adds an L²-based branch for additional robustness. This checkpoint is the recommended four-branch ensemble configuration.
|
| 51 |
+
|
| 52 |
+
> ⚠️ The statistics-based discrepancy branch needs enough samples per batch for stable covariance estimation. That is why **a batch size of 256 (32 per GPU × 8 GPUs) is strongly recommended** — smaller batches noticeably degrade accuracy.
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## Model details
|
| 57 |
+
|
| 58 |
+
| | |
|
| 59 |
+
|---|---|
|
| 60 |
+
| **Configuration** | `ensemble_vitl16_raw_lpd_discrepancy` (four-branch ensemble) |
|
| 61 |
+
| **Backbone** | DINOv3 ViT-L/16, frozen (1024-d embeddings) |
|
| 62 |
+
| **Perturbation module** | Differential Robustness Probing (U-Net), ε = 16/255 |
|
| 63 |
+
| **Detector branches** | Semantic (DINOv3) + Discrepancy + Low-level residual (+ L²) |
|
| 64 |
+
| **Loss** | Discrepancy loss (margin = 1.0) + classification loss |
|
| 65 |
+
| **Training data** | ProGAN (Wang2020 format), `0_real/` & `1_fake/` per category |
|
| 66 |
+
| **Checkpoint file** | `checkpoint_best.pt` (PyTorch state_dict) |
|
| 67 |
+
| **Framework** | PyTorch (`torchrun` distributed training/eval) |
|
| 68 |
+
| **License** | See the [code repository](https://github.com/dongdongunique/RA-Det) (intended for research use) |
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
+
|
| 72 |
+
## Results
|
| 73 |
+
|
| 74 |
+
- Evaluated against **14+ diverse generative models** (ProGAN, Stable Diffusion v1.4, DALL·E 2, and more).
|
| 75 |
+
- Outperforms **10+ existing detection methods**.
|
| 76 |
+
- **+7.81% average performance improvement** over prior art, with strong transfer to **unseen generators**.
|
| 77 |
+
|
| 78 |
+
Per-generator accuracy tables and ablations are available in the [paper](https://arxiv.org/abs/2603.01544) and the [code repository](https://github.com/dongdongunique/RA-Det).
|
| 79 |
+
|
| 80 |
+
---
|
| 81 |
+
|
| 82 |
+
## How to use
|
| 83 |
+
|
| 84 |
+
This is **not** a single self-contained `.safetensors` model — RA-Det's forward pass requires the DINOv3 backbone plus the DRP U-Net and multi-branch detector defined in the codebase. The intended way to run it is via the official repository.
|
| 85 |
+
|
| 86 |
+
### 1. Get the code and weights
|
| 87 |
+
|
| 88 |
+
```bash
|
| 89 |
+
git clone https://github.com/dongdongunique/RA-Det.git
|
| 90 |
+
cd RA-Det
|
| 91 |
+
pip install -r requirements.txt
|
| 92 |
+
|
| 93 |
+
# DINOv3 backbone weights -> models/dino/dinov3_vitl16_pretrain.pth
|
| 94 |
+
# DINOv3 source -> models/dinov3_repo (see repo README)
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
### 2. Download this checkpoint
|
| 98 |
+
|
| 99 |
+
```python
|
| 100 |
+
from huggingface_hub import hf_hub_download
|
| 101 |
+
|
| 102 |
+
ckpt_path = hf_hub_download(
|
| 103 |
+
repo_id="dongdongunique/RA-Det-Checkpoints",
|
| 104 |
+
filename="checkpoint_best.pt",
|
| 105 |
+
repo_type="model",
|
| 106 |
+
)
|
| 107 |
+
print(ckpt_path)
|
| 108 |
+
```
|
| 109 |
+
|
| 110 |
+
### 3. Configure data paths
|
| 111 |
+
|
| 112 |
+
Edit `paths.py` (or set env vars) to point at your data in **Wang2020 format**:
|
| 113 |
+
|
| 114 |
+
```python
|
| 115 |
+
PROGAN_TRAIN_DATA_PATH = "/path/to/progan_train" # .../airplane/{0_real,1_fake}/...
|
| 116 |
+
AIGCTEST_DATA_PATH = "/path/to/AIGCTestset/test" # progan/, stable_diffusion_v_1_4/, DALLE2/, ...
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
### 4. Run evaluation
|
| 120 |
+
|
| 121 |
+
The recommended entry point uses the same initialization as training:
|
| 122 |
+
|
| 123 |
+
```bash
|
| 124 |
+
bash scripts/validate.sh --gpus 8 --checkpoint /path/to/checkpoint_best.pt
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
…which expands to:
|
| 128 |
+
|
| 129 |
+
```bash
|
| 130 |
+
torchrun --nproc_per_node=8 train.py \
|
| 131 |
+
--config ensemble_vitl16_raw_lpd_discrepancy \
|
| 132 |
+
--eps 0.06274509803921569 \
|
| 133 |
+
--margin 1.0 \
|
| 134 |
+
--four-branch-ensemble \
|
| 135 |
+
--normalize-loss \
|
| 136 |
+
--validate-checkpoint /path/to/checkpoint_best.pt
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
### Reproduce training
|
| 140 |
+
|
| 141 |
+
```bash
|
| 142 |
+
# 8 GPUs, recommended batch size 256 (32/GPU), eps=16/255, margin=1.0
|
| 143 |
+
bash scripts/main.sh --gpus 8 --niter 1
|
| 144 |
+
```
|
| 145 |
+
|
| 146 |
+
| Parameter | Default | Note |
|
| 147 |
+
|---|---|---|
|
| 148 |
+
| `--eps` | 16/255 (0.0627) | Perturbation budget |
|
| 149 |
+
| `--margin` | 1.0 | Discrepancy-loss margin |
|
| 150 |
+
| `--niter` | 1 | Training epochs |
|
| 151 |
+
| `--gpus` | 8 | GPUs for `torchrun` |
|
| 152 |
+
| Batch size | **256** | Required for stable discrepancy statistics |
|
| 153 |
+
|
| 154 |
+
---
|
| 155 |
+
|
| 156 |
+
## Intended use & limitations
|
| 157 |
+
|
| 158 |
+
- **Intended for** research on AI-generated image detection, media forensics, and robustness analysis.
|
| 159 |
+
- **Out of scope:** real-world deployment as the sole decision in high-stakes moderation, surveillance, or identity-related judgments. Detector scores are probabilistic and can be wrong, especially on heavily post-processed, compressed, or adversarially crafted inputs.
|
| 160 |
+
- **Bias considerations:** detection performance varies across generators, image content, and demographics; treat outputs as one signal among several, never as ground truth.
|
| 161 |
+
- Generalization to **future, unseen generators** is a moving target — re-evaluate on your specific data before relying on results.
|
| 162 |
+
|
| 163 |
+
---
|
| 164 |
+
|
| 165 |
+
## Citation
|
| 166 |
+
|
| 167 |
+
If you use this checkpoint or the RA-Det method, please cite:
|
| 168 |
+
|
| 169 |
+
```bibtex
|
| 170 |
+
@misc{wang2026radetuniversaldetectionaigenerated,
|
| 171 |
+
title={RA-Det: Towards Universal Detection of AI-Generated Images via Robustness Asymmetry},
|
| 172 |
+
author={Xinchang Wang and Yunhao Chen and Yuechen Zhang and Congcong Bian and Zihao Guo and Xingjun Ma and Hui Li},
|
| 173 |
+
year={2026},
|
| 174 |
+
eprint={2603.01544},
|
| 175 |
+
archivePrefix={arXiv},
|
| 176 |
+
primaryClass={cs.CV},
|
| 177 |
+
url={https://arxiv.org/abs/2603.01544},
|
| 178 |
+
}
|
| 179 |
+
```
|