Polish ForensicConcept model card

ee3ed52 verified 5 days ago

6.3 kB

license: apache-2.0
library_name: pytorch
pipeline_tag: image-classification
tags:
  - aigc-detection
  - ai-generated-image-detection
  - image-forensics
  - pytorch
  - dinov3
  - clip
  - icml-2026

🔍 ForensicConcept

Transferable Forensic Concepts for AIGI Detection

Official detector checkpoints for the ICML 2026 paper ForensicConcept: Transferable Forensic Concepts for AIGI Detection.

[Code] [Paper] [Models] [BibTeX]

Model Overview

ForensicConcept converts diffuse detector evidence into explicit, auditable forensic concepts and transfers these concepts across detector backbones. The framework combines:

Adapter-guided discriminative tuning (ADT) for efficient detector adaptation with LoRA;
Unsupervised concept induction (UCI) for discovering a compact forensic concept codebook from decision-critical image patches;
Concept-aligned projection (CAP) for concept-space prediction;
Concept-guided codebook injection (CGCI) for transferring diffusion-derived generation traces to target backbones.

Performance

Image-level accuracy (%) reported in the paper. Models are trained on Stable Diffusion 1.4 images.

Evaluation benchmark	DINOv3 without concepts	ForensicConcept
GenImage, mean over 8 generators	90.7	92.0
GAN-family, mean over 7 generators	87.3	90.1
Chameleon	83.7	84.4
Mean over the three benchmarks	87.2	88.8

The transferred codebook also raises CLIP ViT-L/14 mean accuracy on GenImage from 83.7 to 88.2 (+4.5 points).

Available Files

File	Purpose	Size
`weights/detectors/dinov3_vitl16_lora_stage1.pth`	DINOv3 ADT / LoRA initialization	24.3 MiB
`weights/detectors/dinov3_vitl16_concept_stage2.pth`	DINOv3 ForensicConcept inference	26.6 MiB
`weights/detectors/clip_vitl14_lora_stage1.pth`	CLIP LoRA initialization	4.6 MiB
`weights/detectors/clip_vitl14_codebook_stage2.pth`	CLIP CGCI inference	20.5 MiB
`weights/concepts/dinov3_concept_matrix.npy`	DINOv3 concepts (`200 x 1024`, float32)	0.8 MiB

The four detector checkpoints total 76.0 MiB. Stage-1 checkpoints are provided for reproducing stage-2 training. For inference, use the corresponding stage-2 checkpoint. DINOv3 inference also requires the released concept matrix.

Quick Start

1. Clone the code

git clone https://github.com/EthanAdamm/FORENSICCONCEPT.git
cd FORENSICCONCEPT
python -m pip install -r requirements.txt

2. Download the released weights

Run this command from the root of the cloned code repository:

hf download ethan225/ForensicConcept \
  --include "weights/**" \
  --local-dir .

3. Add the external backbones

The pretrained DINOv3 ViT-L/16 and CLIP ViT-L/14 backbones are not included. Download them from their official upstream projects and use this layout:

weights/
|-- backbones/
|   |-- ViT-L-14.pt
|   `-- dinov3-vitl16/
|       `-- model.safetensors
|-- concepts/
|   `-- dinov3_concept_matrix.npy
`-- detectors/
    |-- dinov3_vitl16_lora_stage1.pth
    |-- dinov3_vitl16_concept_stage2.pth
    |-- clip_vitl14_lora_stage1.pth
    `-- clip_vitl14_codebook_stage2.pth

4. Evaluate

# DINOv3 ForensicConcept
python test_with_config.py \
  --config configs/dinov3_concept.yaml \
  --checkpoint_path weights/detectors/dinov3_vitl16_concept_stage2.pth

# CLIP with concept-guided codebook injection
python test_with_config.py \
  --config configs/clip_codebook.yaml \
  --checkpoint_path weights/detectors/clip_vitl14_codebook_stage2.pth

Dataset paths are placeholders in the public configurations. Update data.* and testing.groups before evaluation. See the GitHub README for complete installation, dataset, training, and evaluation instructions.

Checkpoint Notes

DINOv3 stage 2 contains the classifier, concept mapping/head, and all LoRA tensors. The external concept matrix initializes the model structure before loading the checkpoint.
CLIP stage 2 contains visual LoRA, the main classifier, and the complete codebook head.
External backbone weights remain subject to their respective upstream licenses and terms.

SHA-256 checksums

3bc8e833d75c9ccbb214c28238c791681294ae8b0c50e48cc3e64b9e5ac5ca1f  weights/detectors/dinov3_vitl16_lora_stage1.pth
99162dbf6a56610be5a8bb9fa27e62311f723d0a4f34e384b084e76085491aad  weights/detectors/dinov3_vitl16_concept_stage2.pth
b0c0217b547391a70eed680ef0d8f92555d172241c46bea767346c8d65115184  weights/detectors/clip_vitl14_lora_stage1.pth
a81ebec9281aca3db4b0d88254d25424f37d5d498ae2c405eb6a5be4748dc498  weights/detectors/clip_vitl14_codebook_stage2.pth
9e6ce224d223dab804648bb46a9cc405bfef4cfcb13dc9cc21ad7fd94935959c  weights/concepts/dinov3_concept_matrix.npy

Citation

@inproceedings{zhou2026forensicconcept,
  title     = {{ForensicConcept}: Transferable Forensic Concepts for {AIGI} Detection},
  author    = {Zhou, Menyanshu and Zhou, Ziyin and Sun, Ke and Luo, Yunpeng and Ji, Jiayi and Sun, Xiaoshuai and Ji, Rongrong},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year      = {2026}
}

License

The ForensicConcept detector checkpoints are released under the Apache License 2.0. External backbone weights are not included and remain subject to their respective upstream licenses and terms.