ethan225 commited on
Commit
ee3ed52
·
verified ·
1 Parent(s): 84e1b47

Polish ForensicConcept model card

Browse files
Files changed (1) hide show
  1. README.md +134 -14
README.md CHANGED
@@ -4,20 +4,70 @@ library_name: pytorch
4
  pipeline_tag: image-classification
5
  tags:
6
  - aigc-detection
 
7
  - image-forensics
8
  - pytorch
 
 
9
  - icml-2026
10
  ---
11
 
12
- # ForensicConcept Weights
13
 
14
- Official detector checkpoints for **ForensicConcept: Transferable Forensic
15
- Concepts for AIGI Detection**.
16
 
17
- - Code: https://github.com/EthanAdamm/FORENSICCONCEPT
18
- - Paper: https://arxiv.org/abs/2606.07034
19
 
20
- ## Files
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  | File | Purpose | Size |
23
  |---|---|---:|
@@ -25,13 +75,24 @@ Concepts for AIGI Detection**.
25
  | `weights/detectors/dinov3_vitl16_concept_stage2.pth` | DINOv3 ForensicConcept inference | 26.6 MiB |
26
  | `weights/detectors/clip_vitl14_lora_stage1.pth` | CLIP LoRA initialization | 4.6 MiB |
27
  | `weights/detectors/clip_vitl14_codebook_stage2.pth` | CLIP CGCI inference | 20.5 MiB |
28
- | `weights/concepts/dinov3_concept_matrix.npy` | DINOv3 concept initialization (`200 x 1024`, float32) | 0.8 MiB |
29
 
30
- The four detector checkpoints total 76.0 MiB. Stage-1 checkpoints are included
31
- for reproducing stage-2 training. For inference, use the corresponding stage-2
32
- checkpoint. DINOv3 inference also requires the released concept matrix.
 
33
 
34
- ## Download
 
 
 
 
 
 
 
 
 
 
35
 
36
  Run this command from the root of the cloned code repository:
37
 
@@ -41,11 +102,57 @@ hf download ethan225/ForensicConcept \
41
  --local-dir .
42
  ```
43
 
 
 
44
  The pretrained DINOv3 ViT-L/16 and CLIP ViT-L/14 backbones are not included.
45
- Download them from their official upstream projects and follow the directory
46
- layout documented in the code repository.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
- ## SHA-256
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
  ```text
51
  3bc8e833d75c9ccbb214c28238c791681294ae8b0c50e48cc3e64b9e5ac5ca1f weights/detectors/dinov3_vitl16_lora_stage1.pth
@@ -55,6 +162,19 @@ a81ebec9281aca3db4b0d88254d25424f37d5d498ae2c405eb6a5be4748dc498 weights/detect
55
  9e6ce224d223dab804648bb46a9cc405bfef4cfcb13dc9cc21ad7fd94935959c weights/concepts/dinov3_concept_matrix.npy
56
  ```
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  ## License
59
 
60
  The ForensicConcept detector checkpoints are released under the
 
4
  pipeline_tag: image-classification
5
  tags:
6
  - aigc-detection
7
+ - ai-generated-image-detection
8
  - image-forensics
9
  - pytorch
10
+ - dinov3
11
+ - clip
12
  - icml-2026
13
  ---
14
 
15
+ <div align="center">
16
 
17
+ # 🔍 ForensicConcept
 
18
 
19
+ ### Transferable Forensic Concepts for AIGI Detection
 
20
 
21
+ Official detector checkpoints for the ICML 2026 paper
22
+ **ForensicConcept: Transferable Forensic Concepts for AIGI Detection**.
23
+
24
+ [[`Code`](https://github.com/EthanAdamm/FORENSICCONCEPT)]
25
+ [[`Paper`](https://arxiv.org/abs/2606.07034)]
26
+ [[`Models`](https://huggingface.co/ethan225/ForensicConcept/tree/main/weights)]
27
+ [[`BibTeX`](#citation)]
28
+
29
+ [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
30
+ [![PyTorch](https://img.shields.io/badge/PyTorch-2.7%2B-ee4c2c.svg)](https://pytorch.org/)
31
+ [![ICML 2026](https://img.shields.io/badge/ICML-2026-8A2BE2.svg)](https://icml.cc/)
32
+
33
+ </div>
34
+
35
+ ---
36
+
37
+ ## Model Overview
38
+
39
+ ForensicConcept converts diffuse detector evidence into explicit, auditable
40
+ forensic concepts and transfers these concepts across detector backbones. The
41
+ framework combines:
42
+
43
+ - **Adapter-guided discriminative tuning (ADT)** for efficient detector
44
+ adaptation with LoRA;
45
+ - **Unsupervised concept induction (UCI)** for discovering a compact forensic
46
+ concept codebook from decision-critical image patches;
47
+ - **Concept-aligned projection (CAP)** for concept-space prediction;
48
+ - **Concept-guided codebook injection (CGCI)** for transferring
49
+ diffusion-derived generation traces to target backbones.
50
+
51
+ <div align="center">
52
+ <img src="https://raw.githubusercontent.com/EthanAdamm/FORENSICCONCEPT/main/assets/readme/method_overview.png" alt="ForensicConcept overview" width="95%" />
53
+ </div>
54
+
55
+ ## Performance
56
+
57
+ Image-level accuracy (%) reported in the paper. Models are trained on Stable
58
+ Diffusion 1.4 images.
59
+
60
+ | Evaluation benchmark | DINOv3 without concepts | ForensicConcept |
61
+ |---|---:|---:|
62
+ | GenImage, mean over 8 generators | 90.7 | **92.0** |
63
+ | GAN-family, mean over 7 generators | 87.3 | **90.1** |
64
+ | Chameleon | 83.7 | **84.4** |
65
+ | Mean over the three benchmarks | 87.2 | **88.8** |
66
+
67
+ The transferred codebook also raises CLIP ViT-L/14 mean accuracy on GenImage
68
+ from **83.7** to **88.2** (+4.5 points).
69
+
70
+ ## Available Files
71
 
72
  | File | Purpose | Size |
73
  |---|---|---:|
 
75
  | `weights/detectors/dinov3_vitl16_concept_stage2.pth` | DINOv3 ForensicConcept inference | 26.6 MiB |
76
  | `weights/detectors/clip_vitl14_lora_stage1.pth` | CLIP LoRA initialization | 4.6 MiB |
77
  | `weights/detectors/clip_vitl14_codebook_stage2.pth` | CLIP CGCI inference | 20.5 MiB |
78
+ | `weights/concepts/dinov3_concept_matrix.npy` | DINOv3 concepts (`200 x 1024`, float32) | 0.8 MiB |
79
 
80
+ The four detector checkpoints total **76.0 MiB**. Stage-1 checkpoints are
81
+ provided for reproducing stage-2 training. For inference, use the
82
+ corresponding stage-2 checkpoint. DINOv3 inference also requires the released
83
+ concept matrix.
84
 
85
+ ## Quick Start
86
+
87
+ ### 1. Clone the code
88
+
89
+ ```bash
90
+ git clone https://github.com/EthanAdamm/FORENSICCONCEPT.git
91
+ cd FORENSICCONCEPT
92
+ python -m pip install -r requirements.txt
93
+ ```
94
+
95
+ ### 2. Download the released weights
96
 
97
  Run this command from the root of the cloned code repository:
98
 
 
102
  --local-dir .
103
  ```
104
 
105
+ ### 3. Add the external backbones
106
+
107
  The pretrained DINOv3 ViT-L/16 and CLIP ViT-L/14 backbones are not included.
108
+ Download them from their official upstream projects and use this layout:
109
+
110
+ ```text
111
+ weights/
112
+ |-- backbones/
113
+ | |-- ViT-L-14.pt
114
+ | `-- dinov3-vitl16/
115
+ | `-- model.safetensors
116
+ |-- concepts/
117
+ | `-- dinov3_concept_matrix.npy
118
+ `-- detectors/
119
+ |-- dinov3_vitl16_lora_stage1.pth
120
+ |-- dinov3_vitl16_concept_stage2.pth
121
+ |-- clip_vitl14_lora_stage1.pth
122
+ `-- clip_vitl14_codebook_stage2.pth
123
+ ```
124
 
125
+ ### 4. Evaluate
126
+
127
+ ```bash
128
+ # DINOv3 ForensicConcept
129
+ python test_with_config.py \
130
+ --config configs/dinov3_concept.yaml \
131
+ --checkpoint_path weights/detectors/dinov3_vitl16_concept_stage2.pth
132
+
133
+ # CLIP with concept-guided codebook injection
134
+ python test_with_config.py \
135
+ --config configs/clip_codebook.yaml \
136
+ --checkpoint_path weights/detectors/clip_vitl14_codebook_stage2.pth
137
+ ```
138
+
139
+ Dataset paths are placeholders in the public configurations. Update
140
+ `data.*` and `testing.groups` before evaluation. See the
141
+ [`GitHub README`](https://github.com/EthanAdamm/FORENSICCONCEPT#dataset-preparation)
142
+ for complete installation, dataset, training, and evaluation instructions.
143
+
144
+ ## Checkpoint Notes
145
+
146
+ - DINOv3 stage 2 contains the classifier, concept mapping/head, and all LoRA
147
+ tensors. The external concept matrix initializes the model structure before
148
+ loading the checkpoint.
149
+ - CLIP stage 2 contains visual LoRA, the main classifier, and the complete
150
+ codebook head.
151
+ - External backbone weights remain subject to their respective upstream
152
+ licenses and terms.
153
+
154
+ <details>
155
+ <summary><strong>SHA-256 checksums</strong></summary>
156
 
157
  ```text
158
  3bc8e833d75c9ccbb214c28238c791681294ae8b0c50e48cc3e64b9e5ac5ca1f weights/detectors/dinov3_vitl16_lora_stage1.pth
 
162
  9e6ce224d223dab804648bb46a9cc405bfef4cfcb13dc9cc21ad7fd94935959c weights/concepts/dinov3_concept_matrix.npy
163
  ```
164
 
165
+ </details>
166
+
167
+ ## Citation
168
+
169
+ ```bibtex
170
+ @inproceedings{zhou2026forensicconcept,
171
+ title = {{ForensicConcept}: Transferable Forensic Concepts for {AIGI} Detection},
172
+ author = {Zhou, Menyanshu and Zhou, Ziyin and Sun, Ke and Luo, Yunpeng and Ji, Jiayi and Sun, Xiaoshuai and Ji, Rongrong},
173
+ booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
174
+ year = {2026}
175
+ }
176
+ ```
177
+
178
  ## License
179
 
180
  The ForensicConcept detector checkpoints are released under the