README.md

metadata

base_model:
  - timm/vit_small_patch16_384.augreg_in21k_ft_in1k
library_name: transformers
license: mit
pipeline_tag: image-classification
tags:
  - image-classification
  - timm
  - transformers
  - detection
  - deepfake
  - forensics
  - deepfake_detection
  - community
  - opensight
  - onnx

CommunityForensics DeepfakeDet-ViT

Vision Transformer (ViT-Small) trained on 2.7M samples across 4,803 generators for detecting AI-generated images. Presented in Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025).

Uploaded for community validation as part of OpenSight — An upcoming open-source framework for adaptive deepfake detection.

Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:

IMPORTANT — Configuration Fix (July 2026)

If you downloaded this model before July 22, 2026, your local copy has incorrect config and weights. Apologies for the mess — this model was originally hastily put together as an internal proof-of-concept for a hackathon, and we never imagined it would quietly become one of the top image classification models on Hugging Face. This update is long overdue.

The model.safetensors has been regenerated from the correct training checkpoint and all metadata has been fixed. For a detailed breakdown of every change, see CHANGELOG.md. If you use LLM-based coding agents (Claude Code, Cursor, GitHub Copilot, etc.), the repo includes an AGENTS.md to help your agent ramp up quickly.

Bug	Effect	Fixed Value
Wrong model.safetensors	Weights from different model (intermediate_size=3072, wrong classifier)	Regenerated from pretrained_weights/model_v11_ViT_384_base_ckpt.pt
num_attention_heads: 12	Silently wrong — attention sliced 12×32d instead of 6×64d	6
Preprocessor size	Squashed non-square images or no center-crop	shortest_edge: 440 + do_center_crop
num_classes: 2 / no num_labels	Wrong output format for single-class classifier — num_classes=1 maps to 2 labels internally	num_labels: 1 (sigmoid output)

⚠️ Breaking change for older transformers versions

This model now requires transformers >= 5.4.0 for correct image preprocessing. Versions older than 5.4.0 will crash with a ValueError when loading the preprocessor — this is intentional and prevents silently-squashed images. If upgrading is not an option, you can preprocess images manually (resize shortest edge → 440, center-crop → 384, CLIP-normalize) and pass do_resize=False to the processor.

How to verify you have the fix

import json
with open("path/to/config.json") as f:
    cfg = json.load(f)
assert cfg["num_labels"] == 1, "Still broken — re-download the model"
assert cfg["num_attention_heads"] == 6, "Still broken — re-download the model"
assert cfg["intermediate_size"] == 1536, "Still broken — re-download the model"

If you were using the old custom wrapper (modeling_vit_classifier.py)

It has been moved to scripts/ and marked deprecated. Switch to the standard HuggingFace path:

from transformers import ViTForImageClassification, ViTImageProcessor
model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")

If you were using the ONNX repo

The separate buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX repo is now deprecated. All ONNX models are included here in onnx/ with corrected weights. Old exports are archived in onnx_legacy/.

Archived files

• model_legacy.safetensors — previous (incorrect) weights, frozen for reference
• model_fixed.safetensors — identical copy of the current model.safetensors
• onnx_legacy/ — previous ONNX exports from the incorrect weights

---

Quick Start

from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch

model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")

image = Image.open("suspicious_image.jpg")
inputs = processor(image, return_tensors="pt")
outputs = model(**inputs)

fake_prob = torch.sigmoid(outputs.logits).item()
print(f"fake: {fake_prob:.4f}, real: {1 - fake_prob:.4f}")
print(f"verdict: {'fake' if fake_prob > 0.5 else 'real'}")

Dependencies

• transformers >= 5.4.0 (required — older versions lack shortest_edge resize and will crash. Do not downgrade below 5.4.0 or images will be silently squashed.)
• torch, torchvision, Pillow
• onnxruntime >= 1.27 (for ONNX models — install onnxruntime for CPU or onnxruntime-gpu for GPU)

---

ONNX Variants (v1.1)

Five pre-exported ONNX models with different size/speed trade-offs. All use the corrected config (single-class sigmoid output).

Variant	Size	Speed (CPU)	Accuracy	Best For
model.onnx (full)	84 MB	★★★	★★★★★	Maximum accuracy, server-side
model_int8.onnx	22 MB	★★★★★	★★★★	Fastest CPU, general deployment
model_uint8.onnx	22 MB	★★★★★	★★★★	Fast CPU, unsigned variant
model_quantized.onnx	22 MB	★★★★★	★★★★	Alias of INT8 for compatibility
model_q4.onnx	16 MB	★★★	★★★	Smallest, low disk/RAM

Which variant should I use?

Use case	Recommended variant	Why
Server-side, maximum accuracy	model.onnx (full)	No quantization loss, FP32 precision
General CPU deployment	model_int8.onnx	Fastest CPU inference, strong accuracy on real-world images
Disk/RAM constrained	model_q4.onnx	Smallest file size, slower inference

Quantization note: Dynamic quantization accuracy varies by input. On real-world photos and AI-generated images, all variants agree with FP32 within ~1%. On synthetic noise or extreme aspect ratios, quantized variants may diverge more. For maximum accuracy, use model.onnx (FP32).

import onnxruntime as ort, numpy as np
from PIL import Image

session = ort.InferenceSession("onnx/model_int8.onnx")

# Preprocess: shortest edge → 440 (maintain aspect ratio), center-crop → 384, CLIP normalize
image = Image.open("image.jpg")
w, h = image.size
scale = 440 / min(w, h)
img = image.resize((int(w * scale), int(h * scale)))
left = (img.size[0] - 384) // 2
top = (img.size[1] - 384) // 2
img = img.crop((left, top, left + 384, top + 384))
arr = np.array(img, dtype=np.float32) / 255.0
arr = (arr - np.array([0.4815, 0.4578, 0.4082])) / np.array([0.2686, 0.2613, 0.2758])
arr = np.expand_dims(arr.transpose(2, 0, 1), 0)

logit = session.run(None, {"pixel_values": arr})[0][0, 0]
fake_prob = 1 / (1 + np.exp(-logit))

---

Benchmark & Comparison Space

A companion Gradio Space lets you test every variant side by side — upload your own images and compare PyTorch vs ONNX performance in real time.

What it does:

Tab	Description
Compare	Upload a single image, see PyTorch and all selected ONNX variants side by side with timing
Benchmark	Upload multiple images for batch processing, compare inference speed across all variants
Help	Variant selection guide and preprocessing details

Use it to:

• See how quantization affects prediction confidence on your own images
• Measure real-world inference speed across variants (CPU/GPU)
• Verify the corrected model produces results consistent with the original timm pipeline

Link coming soon — deploying as a separate Space. Follow the repo for updates.

---

Model Details

• Developed by: Jeongsoo Park and Andrew Owens, University of Michigan
• HF integration + ONNX: Han Yoon, Borderless / Ethix R&D
• Model type: Vision Transformer (ViT-Small)
• License: MIT
• Input: RGB image, shortest edge resized to 440 (aspect ratio preserved), center-cropped to 384×384, CLIP-normalized
• Output: single logit → sigmoid → fake probability
• Architecture: hidden_size=384, 6 attention heads, 12 layers, patch_size=16, intermediate_size=1536

Links

• Original paper: arXiv:2411.04125
• Original repository: JeongsooP/Community-Forensics
• Project page: https://jespark.net/projects/2024/community_forensics
• Datasets: Full (1.1TB), Small (278GB), Eval (206GB)

---

Coming Soon — v2

We're actively working on a significantly stronger model with an expanded dataset and novel detection concepts. Follow the repo for updates in the coming months.

---

Citation

@InProceedings{Park_2025_CVPR,
    author    = {Park, Jeongsoo and Owens, Andrew},
    title     = {Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {8245-8257}
}