File size: 8,520 Bytes
ef29309 2a766f2 ef29309 2a766f2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | ---
license: mit
datasets:
- ThinothW/Deepfake-Identity-Isolated-Dataset-PreP
metrics:
- accuracy
- precision
- recall
- f1
- roc_auc
base_model:
- google/efficientnet-b1
- mobilint/RegNet_Y_800MF.tv2_in1k
pipeline_tag: image-classification
tags:
- deepfake-detection
- image-classification
- efficientnet
- regnet
- hybrid-model
- computer-vision
---
# Deep_Fake_Hybrid_Model
### Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity
A binary image classifier that flags a given face image as **real** or **fake** (deepfake / synthetically manipulated). Built as a **hybrid dual-backbone** model combining fine-tuned [`google/efficientnet-b1`](https://huggingface.co/google/efficientnet-b1) and [`mobilint/RegNet_Y_800MF.tv2_in1k`](https://huggingface.co/mobilint/RegNet_Y_800MF.tv2_in1k) feature extractors, trained on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset.
This model was built as part of a university course project (AI Lab, SE334) exploring deepfake face detection.
**Author:** S. M. Nihal Ahmed
## Model Details
- **Base models:** `google/efficientnet-b1`, `mobilint/RegNet_Y_800MF.tv2_in1k`
- **Task:** Binary image classification (`fake` vs `real`)
- **License:** MIT
- **Architecture:** Hybrid dual-backbone — EfficientNet-B1 and RegNetY-800MF feature extractors, both fine-tuned end-to-end, with their pooled features fused and passed through a classification head
- **Fine-tuning objective:** Cross-entropy loss over the two classes, with `sklearn` balanced class weights applied to account for class imbalance
- **Training regime:** Mixed-precision (AMP) training on a CUDA GPU, up to 100 epochs per stage with early stopping (patience = 5, monitored on validation loss); two-stage schedule — Stage A trains only the fusion head with both backbones frozen, Stage B fine-tunes both backbones together at a lower learning rate
## Intended Use
This model is intended for detecting AI-generated or manipulated (deepfake) face images versus authentic (real) face images. Example use cases:
- Screening uploaded profile/identity photos for synthetic manipulation
- Research and coursework on deepfake detection and media forensics
- A component in a larger content-authenticity verification pipeline
**Out of scope:** This model is **not** a complete or production-ready deepfake detection guardrail. It has been evaluated on one dataset only, and will not necessarily generalize to deepfake generation methods, image qualities, or demographics absent from its training data.
## How to Use
This model is distributed as an **ONNX** export. Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the `.onnx.data` file alongside it at runtime:
- [`deepfake_hybrid_final.onnx`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx) — the ONNX graph
- [`deepfake_hybrid_final.onnx.data`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx.data) — the external weights file
Install dependencies:
```bash
pip install onnxruntime huggingface_hub pillow numpy
```
### Single-image prediction
```python
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download
REPO_ID = "nihal4/Deep_Fake_Hybrid_Model"
IMG_SIZE = 260
IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32)
LABEL_MAP = {0: "fake", 1: "real"}
# Downloads both files into the same local cache folder — required, since the
# .onnx graph references .onnx.data by relative path at load time.
onnx_path = hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx")
hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx.data")
session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
def preprocess_pil(img: Image.Image) -> np.ndarray:
img = img.convert("RGB").resize((IMG_SIZE, IMG_SIZE))
arr = np.asarray(img, dtype=np.float32) / 255.0 # HWC, [0,1]
arr = (arr - IMAGENET_MEAN) / IMAGENET_STD # normalize, same stats as training
return arr.transpose(2, 0, 1) # HWC -> CHW
def softmax(x: np.ndarray) -> np.ndarray:
e = np.exp(x - x.max(axis=1, keepdims=True))
return e / e.sum(axis=1, keepdims=True)
def predict(image_path: str):
image = Image.open(image_path)
x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32)
logits = session.run([output_name], {input_name: x})[0]
probs = softmax(logits)[0]
label = LABEL_MAP[int(probs.argmax())]
return label, probs
label, probs = predict("path/to/face.jpg")
print(f"Prediction: {label} (p_fake={probs[0]:.3f}, p_real={probs[1]:.3f})")
```
### Batch prediction
```python
image_paths = ["face1.jpg", "face2.jpg", "face3.jpg"]
batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32)
logits = session.run([output_name], {input_name: batch})[0]
probs = softmax(logits)
preds = probs.argmax(axis=1)
for path, pred, p in zip(image_paths, preds, probs):
print(f"{path}: {LABEL_MAP[int(pred)]} (p_fake={p[0]:.3f}, p_real={p[1]:.3f})")
```
> For GPU inference, install `onnxruntime-gpu` instead and pass `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session.
## Training Data
The model was fine-tuned on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset.
- **Labels:** `0 = fake`, `1 = real`
- **Splits:** train / validation / test
- **Preprocessing:** resize to 260×260, ImageNet normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`)
- **Training augmentation:** random horizontal flip, random rotation (±10°), color jitter, plus simulated JPEG compression and simulated blur/downscale-upscale (to reduce false positives on low-quality real footage)
- **Class balancing:** `sklearn` balanced class weights applied in the loss function to address train-set class imbalance
## Training Procedure
<!-- PLACEHOLDER: training curves (loss/accuracy per epoch) — image to be uploaded -->
- **Framework:** PyTorch
- **Hardware:** Kaggle free-tier T4 GPU
- **Loss:** Cross-entropy
- **Mixed precision:** Enabled (AMP)
## Evaluation
Evaluated on the held-out test split (n = 21,316) at a decision threshold of 0.5.
### Classification Report
| Class | Precision | Recall | F1-score | Support |
|--------------|:---------:|:------:|:--------:|:-------:|
| fake | 0.95 | 0.96 | 0.95 | 10,706 |
| real | 0.96 | 0.95 | 0.95 | 10,610 |
| **accuracy** | | | **0.9539** | 21,316 |
| macro avg | 0.95 | 0.95 | 0.95 | 21,316 |
| weighted avg | 0.95 | 0.95 | 0.95 | 21,316 |
**Test ROC-AUC:** 0.9903
### Confusion Matrix
<!-- PLACEHOLDER: image to be uploaded -->
### ROC Curve
<!-- PLACEHOLDER: image to be uploaded -->
## Limitations
- Performance is reported on a single dataset; generalization to other deepfake generation methods, image resolutions, compression levels, or demographics is not guaranteed.
- As with most deepfake detectors, robustness against novel/unseen generative techniques (including adversarially crafted ones) has not been evaluated here.
- The model has not been evaluated as a standalone production guardrail; it is intended to complement, not replace, other verification measures.
## Citation
If you use this model, please cite this repository and reference this course project:
```
@misc{deepfake-hybrid-detector,
title = {Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity},
author = {S. M. Nihal Ahmed},
year = {2026},
note = {Course project, AI Lab (SE334), Daffodil International University}
}
``` |