Polish ForensicConcept model card
Browse files
README.md
CHANGED
|
@@ -4,20 +4,70 @@ library_name: pytorch
|
|
| 4 |
pipeline_tag: image-classification
|
| 5 |
tags:
|
| 6 |
- aigc-detection
|
|
|
|
| 7 |
- image-forensics
|
| 8 |
- pytorch
|
|
|
|
|
|
|
| 9 |
- icml-2026
|
| 10 |
---
|
| 11 |
|
| 12 |
-
|
| 13 |
|
| 14 |
-
|
| 15 |
-
Concepts for AIGI Detection**.
|
| 16 |
|
| 17 |
-
|
| 18 |
-
- Paper: https://arxiv.org/abs/2606.07034
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
| File | Purpose | Size |
|
| 23 |
|---|---|---:|
|
|
@@ -25,13 +75,24 @@ Concepts for AIGI Detection**.
|
|
| 25 |
| `weights/detectors/dinov3_vitl16_concept_stage2.pth` | DINOv3 ForensicConcept inference | 26.6 MiB |
|
| 26 |
| `weights/detectors/clip_vitl14_lora_stage1.pth` | CLIP LoRA initialization | 4.6 MiB |
|
| 27 |
| `weights/detectors/clip_vitl14_codebook_stage2.pth` | CLIP CGCI inference | 20.5 MiB |
|
| 28 |
-
| `weights/concepts/dinov3_concept_matrix.npy` | DINOv3
|
| 29 |
|
| 30 |
-
The four detector checkpoints total 76.0 MiB. Stage-1 checkpoints are
|
| 31 |
-
for reproducing stage-2 training. For inference, use the
|
| 32 |
-
checkpoint. DINOv3 inference also requires the released
|
|
|
|
| 33 |
|
| 34 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
Run this command from the root of the cloned code repository:
|
| 37 |
|
|
@@ -41,11 +102,57 @@ hf download ethan225/ForensicConcept \
|
|
| 41 |
--local-dir .
|
| 42 |
```
|
| 43 |
|
|
|
|
|
|
|
| 44 |
The pretrained DINOv3 ViT-L/16 and CLIP ViT-L/14 backbones are not included.
|
| 45 |
-
Download them from their official upstream projects and
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
```text
|
| 51 |
3bc8e833d75c9ccbb214c28238c791681294ae8b0c50e48cc3e64b9e5ac5ca1f weights/detectors/dinov3_vitl16_lora_stage1.pth
|
|
@@ -55,6 +162,19 @@ a81ebec9281aca3db4b0d88254d25424f37d5d498ae2c405eb6a5be4748dc498 weights/detect
|
|
| 55 |
9e6ce224d223dab804648bb46a9cc405bfef4cfcb13dc9cc21ad7fd94935959c weights/concepts/dinov3_concept_matrix.npy
|
| 56 |
```
|
| 57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
## License
|
| 59 |
|
| 60 |
The ForensicConcept detector checkpoints are released under the
|
|
|
|
| 4 |
pipeline_tag: image-classification
|
| 5 |
tags:
|
| 6 |
- aigc-detection
|
| 7 |
+
- ai-generated-image-detection
|
| 8 |
- image-forensics
|
| 9 |
- pytorch
|
| 10 |
+
- dinov3
|
| 11 |
+
- clip
|
| 12 |
- icml-2026
|
| 13 |
---
|
| 14 |
|
| 15 |
+
<div align="center">
|
| 16 |
|
| 17 |
+
# 🔍 ForensicConcept
|
|
|
|
| 18 |
|
| 19 |
+
### Transferable Forensic Concepts for AIGI Detection
|
|
|
|
| 20 |
|
| 21 |
+
Official detector checkpoints for the ICML 2026 paper
|
| 22 |
+
**ForensicConcept: Transferable Forensic Concepts for AIGI Detection**.
|
| 23 |
+
|
| 24 |
+
[[`Code`](https://github.com/EthanAdamm/FORENSICCONCEPT)]
|
| 25 |
+
[[`Paper`](https://arxiv.org/abs/2606.07034)]
|
| 26 |
+
[[`Models`](https://huggingface.co/ethan225/ForensicConcept/tree/main/weights)]
|
| 27 |
+
[[`BibTeX`](#citation)]
|
| 28 |
+
|
| 29 |
+
[](https://www.apache.org/licenses/LICENSE-2.0)
|
| 30 |
+
[](https://pytorch.org/)
|
| 31 |
+
[](https://icml.cc/)
|
| 32 |
+
|
| 33 |
+
</div>
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
|
| 37 |
+
## Model Overview
|
| 38 |
+
|
| 39 |
+
ForensicConcept converts diffuse detector evidence into explicit, auditable
|
| 40 |
+
forensic concepts and transfers these concepts across detector backbones. The
|
| 41 |
+
framework combines:
|
| 42 |
+
|
| 43 |
+
- **Adapter-guided discriminative tuning (ADT)** for efficient detector
|
| 44 |
+
adaptation with LoRA;
|
| 45 |
+
- **Unsupervised concept induction (UCI)** for discovering a compact forensic
|
| 46 |
+
concept codebook from decision-critical image patches;
|
| 47 |
+
- **Concept-aligned projection (CAP)** for concept-space prediction;
|
| 48 |
+
- **Concept-guided codebook injection (CGCI)** for transferring
|
| 49 |
+
diffusion-derived generation traces to target backbones.
|
| 50 |
+
|
| 51 |
+
<div align="center">
|
| 52 |
+
<img src="https://raw.githubusercontent.com/EthanAdamm/FORENSICCONCEPT/main/assets/readme/method_overview.png" alt="ForensicConcept overview" width="95%" />
|
| 53 |
+
</div>
|
| 54 |
+
|
| 55 |
+
## Performance
|
| 56 |
+
|
| 57 |
+
Image-level accuracy (%) reported in the paper. Models are trained on Stable
|
| 58 |
+
Diffusion 1.4 images.
|
| 59 |
+
|
| 60 |
+
| Evaluation benchmark | DINOv3 without concepts | ForensicConcept |
|
| 61 |
+
|---|---:|---:|
|
| 62 |
+
| GenImage, mean over 8 generators | 90.7 | **92.0** |
|
| 63 |
+
| GAN-family, mean over 7 generators | 87.3 | **90.1** |
|
| 64 |
+
| Chameleon | 83.7 | **84.4** |
|
| 65 |
+
| Mean over the three benchmarks | 87.2 | **88.8** |
|
| 66 |
+
|
| 67 |
+
The transferred codebook also raises CLIP ViT-L/14 mean accuracy on GenImage
|
| 68 |
+
from **83.7** to **88.2** (+4.5 points).
|
| 69 |
+
|
| 70 |
+
## Available Files
|
| 71 |
|
| 72 |
| File | Purpose | Size |
|
| 73 |
|---|---|---:|
|
|
|
|
| 75 |
| `weights/detectors/dinov3_vitl16_concept_stage2.pth` | DINOv3 ForensicConcept inference | 26.6 MiB |
|
| 76 |
| `weights/detectors/clip_vitl14_lora_stage1.pth` | CLIP LoRA initialization | 4.6 MiB |
|
| 77 |
| `weights/detectors/clip_vitl14_codebook_stage2.pth` | CLIP CGCI inference | 20.5 MiB |
|
| 78 |
+
| `weights/concepts/dinov3_concept_matrix.npy` | DINOv3 concepts (`200 x 1024`, float32) | 0.8 MiB |
|
| 79 |
|
| 80 |
+
The four detector checkpoints total **76.0 MiB**. Stage-1 checkpoints are
|
| 81 |
+
provided for reproducing stage-2 training. For inference, use the
|
| 82 |
+
corresponding stage-2 checkpoint. DINOv3 inference also requires the released
|
| 83 |
+
concept matrix.
|
| 84 |
|
| 85 |
+
## Quick Start
|
| 86 |
+
|
| 87 |
+
### 1. Clone the code
|
| 88 |
+
|
| 89 |
+
```bash
|
| 90 |
+
git clone https://github.com/EthanAdamm/FORENSICCONCEPT.git
|
| 91 |
+
cd FORENSICCONCEPT
|
| 92 |
+
python -m pip install -r requirements.txt
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
+
### 2. Download the released weights
|
| 96 |
|
| 97 |
Run this command from the root of the cloned code repository:
|
| 98 |
|
|
|
|
| 102 |
--local-dir .
|
| 103 |
```
|
| 104 |
|
| 105 |
+
### 3. Add the external backbones
|
| 106 |
+
|
| 107 |
The pretrained DINOv3 ViT-L/16 and CLIP ViT-L/14 backbones are not included.
|
| 108 |
+
Download them from their official upstream projects and use this layout:
|
| 109 |
+
|
| 110 |
+
```text
|
| 111 |
+
weights/
|
| 112 |
+
|-- backbones/
|
| 113 |
+
| |-- ViT-L-14.pt
|
| 114 |
+
| `-- dinov3-vitl16/
|
| 115 |
+
| `-- model.safetensors
|
| 116 |
+
|-- concepts/
|
| 117 |
+
| `-- dinov3_concept_matrix.npy
|
| 118 |
+
`-- detectors/
|
| 119 |
+
|-- dinov3_vitl16_lora_stage1.pth
|
| 120 |
+
|-- dinov3_vitl16_concept_stage2.pth
|
| 121 |
+
|-- clip_vitl14_lora_stage1.pth
|
| 122 |
+
`-- clip_vitl14_codebook_stage2.pth
|
| 123 |
+
```
|
| 124 |
|
| 125 |
+
### 4. Evaluate
|
| 126 |
+
|
| 127 |
+
```bash
|
| 128 |
+
# DINOv3 ForensicConcept
|
| 129 |
+
python test_with_config.py \
|
| 130 |
+
--config configs/dinov3_concept.yaml \
|
| 131 |
+
--checkpoint_path weights/detectors/dinov3_vitl16_concept_stage2.pth
|
| 132 |
+
|
| 133 |
+
# CLIP with concept-guided codebook injection
|
| 134 |
+
python test_with_config.py \
|
| 135 |
+
--config configs/clip_codebook.yaml \
|
| 136 |
+
--checkpoint_path weights/detectors/clip_vitl14_codebook_stage2.pth
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
Dataset paths are placeholders in the public configurations. Update
|
| 140 |
+
`data.*` and `testing.groups` before evaluation. See the
|
| 141 |
+
[`GitHub README`](https://github.com/EthanAdamm/FORENSICCONCEPT#dataset-preparation)
|
| 142 |
+
for complete installation, dataset, training, and evaluation instructions.
|
| 143 |
+
|
| 144 |
+
## Checkpoint Notes
|
| 145 |
+
|
| 146 |
+
- DINOv3 stage 2 contains the classifier, concept mapping/head, and all LoRA
|
| 147 |
+
tensors. The external concept matrix initializes the model structure before
|
| 148 |
+
loading the checkpoint.
|
| 149 |
+
- CLIP stage 2 contains visual LoRA, the main classifier, and the complete
|
| 150 |
+
codebook head.
|
| 151 |
+
- External backbone weights remain subject to their respective upstream
|
| 152 |
+
licenses and terms.
|
| 153 |
+
|
| 154 |
+
<details>
|
| 155 |
+
<summary><strong>SHA-256 checksums</strong></summary>
|
| 156 |
|
| 157 |
```text
|
| 158 |
3bc8e833d75c9ccbb214c28238c791681294ae8b0c50e48cc3e64b9e5ac5ca1f weights/detectors/dinov3_vitl16_lora_stage1.pth
|
|
|
|
| 162 |
9e6ce224d223dab804648bb46a9cc405bfef4cfcb13dc9cc21ad7fd94935959c weights/concepts/dinov3_concept_matrix.npy
|
| 163 |
```
|
| 164 |
|
| 165 |
+
</details>
|
| 166 |
+
|
| 167 |
+
## Citation
|
| 168 |
+
|
| 169 |
+
```bibtex
|
| 170 |
+
@inproceedings{zhou2026forensicconcept,
|
| 171 |
+
title = {{ForensicConcept}: Transferable Forensic Concepts for {AIGI} Detection},
|
| 172 |
+
author = {Zhou, Menyanshu and Zhou, Ziyin and Sun, Ke and Luo, Yunpeng and Ji, Jiayi and Sun, Xiaoshuai and Ji, Rongrong},
|
| 173 |
+
booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
|
| 174 |
+
year = {2026}
|
| 175 |
+
}
|
| 176 |
+
```
|
| 177 |
+
|
| 178 |
## License
|
| 179 |
|
| 180 |
The ForensicConcept detector checkpoints are released under the
|