| ---
|
| language:
|
| - en
|
| pipeline_tag: image-classification
|
| tags:
|
| - pytorch
|
| - clip
|
| - generated-image-detection
|
| - image-forensics
|
| ---
|
|
|
| # 🔍 DoU — Diversity over Uniformity
|
|
|
| **Diversity over Uniformity: Rethinking Representation in Generated Image Detection**
|
| **CVPR 2026**
|
|
|
| [📄 Paper](https://openaccess.thecvf.com/content/CVPR2026/papers/He_Diversity_over_Uniformity_Rethinking_Representation_in_Generated_Image_Detection_CVPR_2026_paper.pdf) · [💻 GitHub](https://github.com/Yanmou-Hui/DoU) · [🤗 Weights](https://huggingface.co/Yanmou/DoU/tree/main)
|
|
|
| DoU detects AI-generated images by preserving **diverse and complementary forensic cues**, reducing reliance on a few dominant features.
|
|
|
|
|
| ## 🧠 Model Overview
|
|
|
| Built on **CLIP ViT-L/14**, DoU combines:
|
|
|
| - **Cue Information Bottleneck (CIB):** filters task-irrelevant information from multi-layer visual features.
|
| - **Adaptive Feature Correlation Learning (AFCL):** reduces redundancy among forensic cues during training.
|
| - **Learnable aggregation and prompts:** combine complementary cues for real/generated classification.
|
|
|
| | Input | Output |
|
| |:--|:--|
|
| | RGB image, preprocessed to 224 × 224 | **0: Real · 1: Generated** |
|
|
|
| The generated-image score is `softmax(logits)[:, 1]`.
|
|
|
| ## 🚀 Quick Start
|
|
|
| ### 1. Download and install
|
|
|
| ```bash
|
| git clone https://huggingface.co/Yanmou/DoU
|
| cd DoU
|
| python -m pip install -r requirements.txt
|
| ```
|
|
|
| Install a compatible **PyTorch/torchvision** pair for your platform. Model initialization also requires the original OpenAI CLIP backbone, downloaded automatically on first use.
|
|
|
| ### 2. Run inference
|
|
|
| ```bash
|
| python infer.py --repo-id Yanmou/DoU --image example.jpg --device cpu --seed 42
|
| ```
|
|
|
| For GPU inference, use `--device cuda`.
|
|
|
| **Using a local checkpoint:**
|
|
|
| ```bash
|
| python infer.py --ckpt best.pth --image example.jpg --device cpu --seed 42
|
| ```
|
|
|
| > **Note:** This is a custom PyTorch model; use the provided inference script rather than a Transformers pipeline.
|
|
|
| ## 🖼️ Preprocessing
|
|
|
| The evaluation transform follows the official implementation:
|
|
|
| 1. Convert to **RGB**.
|
| 2. If either dimension is below 224, **tile the image** until both dimensions reach at least 224.
|
| 3. **Center-crop to 224 × 224**, convert to a tensor, and apply CLIP normalization.
|
|
|
| ```python
|
| mean = [0.48145466, 0.4578275, 0.40821073]
|
| std = [0.26862954, 0.26130258, 0.27577711]
|
| ```
|
|
|
| **No resizing is applied.** A standard CLIP resize-and-crop processor is therefore not equivalent.
|
|
|
| ## 📊 Benchmark Evaluation
|
|
|
| Organize the evaluation data as follows:
|
|
|
| ```text
|
| test_root/
|
| ├── progan/
|
| │ ├── 0_real/
|
| │ └── 1_fake/
|
| └── stylegan/
|
| ├── 0_real/
|
| └── 1_fake/
|
| ```
|
|
|
| From the `upstream/` directory, run:
|
|
|
| ```bash
|
| python test.py --ckpt ../best.pth --data-root /path/to/test_root --datasets progan stylegan --device cpu --batch-size 16 --num-workers 0 --output result.xlsx
|
| ```
|
|
|
| The script reports **ACC, Real ACC, Fake ACC, F1, and AP**, and saves results to Excel.
|
|
|
| ## ⚙️ Implementation Notes
|
|
|
| - **Checkpoint:** a PyTorch dictionary containing the state dictionary under `ckpt["model"]`.
|
| - **Inference:** the variational bottleneck samples features even in `eval()` mode. The inference script sets a seed; outputs may still vary across devices and environments.
|
| - **Source snapshot:** `upstream/` contains official code from commit `230856fabd137e00580d5457b3b1ce7b664bcf99`.
|
| - **Scope:** binary image detection for research. Scores are not calibrated authenticity probabilities, and predictions do not establish image provenance.
|
|
|
| ## 📜 License
|
|
|
| See [`upstream/LICENSE`](upstream/LICENSE) for the source code license and [`WEIGHTS_LICENSE.txt`](WEIGHTS_LICENSE.txt) for the checkpoint terms.
|
|
|
| ## 📚 Citation
|
|
|
| If you find this work useful, please cite:
|
|
|
| ```bibtex
|
| @inproceedings{he2026diversity,
|
| title={Diversity over uniformity: Rethinking representation in generated image detection},
|
| author={He, Qinghui and Zhang, Haifeng and Qin, Qiao and Liu, Bo and Bi, Xiuli and Xiao, Bin},
|
| booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
|
| pages={40407--40417},
|
| year={2026}
|
| }
|
| ``` |
