Instructions to use hamzenium/ViT-Deepfake-Classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hamzenium/ViT-Deepfake-Classifier with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="hamzenium/ViT-Deepfake-Classifier") pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# pip install -U transformers accelerate # Load model directly from transformers import AutoImageProcessor, AutoModelForImageClassification processor = AutoImageProcessor.from_pretrained("hamzenium/ViT-Deepfake-Classifier") model = AutoModelForImageClassification.from_pretrained("hamzenium/ViT-Deepfake-Classifier", device_map="auto") - Notebooks
- Google Colab
- Kaggle
ViT Deepfake Detector
AI-Powered Image Forensics | Vision Transformer | 96.56% Test Accuracy
An advanced computer vision model designed to distinguish authentic images from manipulated and AI-generated visual content. Built on Google's Vision Transformer (ViT-Base), this model leverages transfer learning and fine-tuning on the OpenForensics dataset to identify visual patterns associated with image manipulation.
Developed with a focus on accuracy, accessibility, and practical deployment, the model provides a lightweight foundation for image authenticity research, content verification pipelines, and computer vision applications.
Overview
The ViT Deepfake Detector is a binary image classification model that analyzes visual representations to predict whether an image is authentic or manipulated.
Built on google/vit-base-patch16-224-in21k, the model uses self-attention mechanisms to capture relationships across image patches, enabling it to learn complex visual features associated with manipulated imagery.
The model was fine-tuned on the OpenForensics dataset and evaluated on a held-out test set.
Key Highlights
- 96.56% Test Accuracy on a held-out evaluation set.
- Vision Transformer Architecture leveraging pretrained ViT representations.
- 16,000 Training Images from the OpenForensics dataset.
- 10-Epoch Fine-Tuning Pipeline using mixed-precision training.
- Fast Inference through the Hugging Face Transformers ecosystem.
- Ready for Integration into Python-based computer vision applications.
Model Specifications
| Attribute | Details |
|---|---|
| Architecture | Vision Transformer (ViT-Base) |
| Base Model | google/vit-base-patch16-224-in21k |
| Task | Binary Image Classification |
| Classes | Real / Fake |
| Training Dataset | OpenForensics |
| Training Samples | 16,000 |
| Validation Samples | 2,000 |
| Test Samples | 2,000 |
| Input Resolution | 224 ร 224 |
| Framework | PyTorch / Hugging Face Transformers |
| Precision | FP16 Mixed Precision |
| License | MIT |
Performance & Evaluation
The model was evaluated using a dedicated test set containing 2,000 images.
Benchmark Results
| Evaluation Metric | Score |
|---|---|
| Test Accuracy | 96.56% |
| Validation Accuracy | 96.22% |
| Validation F1 | 96.22% |
| Validation Precision | 96.30% |
| Validation Recall | 96.22% |
Training Performance
The model demonstrated consistent convergence during fine-tuning, achieving its strongest validation accuracy at Epoch 5.
| Epoch | Training Loss | Validation Loss | Accuracy |
|---|---|---|---|
| 1 | 0.2259 | 0.2567 | 92.89% |
| 2 | 0.2002 | 0.2360 | 93.44% |
| 3 | 0.1388 | 0.1925 | 96.11% |
| 4 | 0.1322 | 0.2161 | 95.67% |
| 5 | 0.1182 | 0.2208 | 96.22% |
Note: Test accuracy is reported separately from validation performance.
Architecture
The model uses a pretrained Vision Transformer (ViT), which processes images as sequences of patches rather than relying exclusively on traditional convolutional operations.
The architecture consists of:
- Image Preprocessing: Images are resized and normalized using the pretrained ViT image processor.
- Patch Embedding: Input images are divided into fixed-size patches.
- Transformer Encoder: Self-attention layers learn spatial relationships and visual representations.
- Classification Head: The learned representations are mapped to two output classes.
- Prediction: Softmax probabilities provide the model's confidence scores for real and fake classifications.
Training Configuration
The model was fine-tuned using the following configuration:
| Hyperparameter | Value |
|---|---|
| Optimizer | AdamW |
| Learning Rate | 3e-5 |
| Batch Size | 24 |
| Epochs | 10 |
| Weight Decay | 0.02 |
| Warmup Ratio | 0.08 |
| Scheduler | Cosine |
| Label Smoothing | 0.05 |
| Mixed Precision | FP16 |
| Gradient Accumulation | 1 |
| Hardware | NVIDIA Tesla T4 |
| Training Duration | Approximately 14 minutes |
Preprocessing and normalization were performed using Hugging Face's ViTImageProcessor.
Quick Start
Get started with the model in just a few lines of Python.
1. Installation
pip install transformers torch pillow
2. Load the Model
import torch
from PIL import Image
from transformers import (
ViTImageProcessor,
ViTForImageClassification
)
MODEL_ID = "hamzenium/vit-deepfake-detector"
processor = ViTImageProcessor.from_pretrained(MODEL_ID)
model = ViTForImageClassification.from_pretrained(MODEL_ID)
model.eval()
3. Run Inference
image = Image.open("image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
probabilities = torch.softmax(outputs.logits, dim=-1)
predicted_class = probabilities.argmax(dim=-1).item()
confidence = probabilities[0][predicted_class].item()
labels = {0: "Real", 1: "Fake"}
print("Prediction:", labels[predicted_class])
print(f"Confidence: {confidence:.2%}")
4. Batch Inference
from transformers import pipeline
detector = pipeline(
"image-classification",
model=MODEL_ID
)
images = [
"image1.jpg",
"image2.jpg",
"image3.jpg"
]
for image_path in images:
result = detector(image_path)
print(image_path, result)
Potential Applications
The model can serve as a building block for a variety of computer vision and digital forensics workflows.
- Digital Image Forensics: Analyze images for potential manipulation.
- Content Moderation: Support automated visual content screening.
- Media Verification: Assist in evaluating the authenticity of digital imagery.
- AI Research: Experiment with transformer-based approaches to synthetic image detection.
- Computer Vision Pipelines: Integrate binary authenticity classification into larger ML systems.
Limitations & Responsible Use
Although the model demonstrates strong performance on its evaluation dataset, deepfake detection remains an evolving research challenge.
- Dataset Generalization: Performance may vary on generation techniques and manipulation methods not represented in OpenForensics.
- Domain Shift: Compression, image resizing, and different image sources may affect predictions.
- Adversarial Robustness: The model has not been comprehensively evaluated against adversarial attacks.
- Resolution Sensitivity: The model uses a 224 ร 224 input resolution.
- False Predictions: A high-confidence prediction does not establish the authenticity or provenance of an image.
This model should be treated as an analytical aid rather than a definitive authenticity verification system. Human review is recommended for consequential decisions.
Research & Development
This project explores the application of transformer-based architectures to digital image forensics, with an emphasis on accessible model distribution and reproducible inference.
The model is released to support further experimentation, benchmarking, and development within the computer vision and AI research communities.
Contributors
- Dr. Lucy Liu
- Muhammad Hamza Sohail
- Ayaan Mohammed
- Shadab Karim
- Kirti Dhir
License
Released under the MIT License, allowing developers and researchers to use, modify, and integrate the model in accordance with the license terms.
Citation
If you use this model in research or development, please cite:
@misc{vit-deepfake-detector,
title={ViT Deepfake Detector},
author={Liu, Lucy and Sohail, Muhammad Hamza and
Mohammed, Ayaan and Karim, Shadab and Dhir, Kirti},
year={2024},
publisher={Hugging Face},
howpublished={https://huggingface.co/hamzenium/vit-deepfake-detector}
}
Built for experimentation. Released for the AI research community.
- Downloads last month
- 211
Model tree for hamzenium/ViT-Deepfake-Classifier
Base model
google/vit-base-patch16-224-in21k