nihal4 commited on
Commit
2a766f2
·
verified ·
1 Parent(s): 11a2c29

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +195 -0
README.md CHANGED
@@ -1,3 +1,198 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - ThinothW/Deepfake-Identity-Isolated-Dataset-PreP
5
+ metrics:
6
+ - accuracy
7
+ - precision
8
+ - recall
9
+ - f1
10
+ - roc_auc
11
+ base_model:
12
+ - google/efficientnet-b1
13
+ - mobilint/RegNet_Y_800MF.tv2_in1k
14
+ pipeline_tag: image-classification
15
+ tags:
16
+ - deepfake-detection
17
+ - image-classification
18
+ - efficientnet
19
+ - regnet
20
+ - hybrid-model
21
+ - computer-vision
22
  ---
23
+
24
+ # Deep_Fake_Hybrid_Model
25
+
26
+ ### Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity
27
+
28
+ A binary image classifier that flags a given face image as **real** or **fake** (deepfake / synthetically manipulated). Built as a **hybrid dual-backbone** model combining fine-tuned [`google/efficientnet-b1`](https://huggingface.co/google/efficientnet-b1) and [`mobilint/RegNet_Y_800MF.tv2_in1k`](https://huggingface.co/mobilint/RegNet_Y_800MF.tv2_in1k) feature extractors, trained on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset.
29
+
30
+ This model was built as part of a university course project (AI Lab, SE334) exploring deepfake face detection.
31
+
32
+ **Author:** S. M. Nihal Ahmed
33
+
34
+ ## Model Details
35
+
36
+ - **Base models:** `google/efficientnet-b1`, `mobilint/RegNet_Y_800MF.tv2_in1k`
37
+ - **Task:** Binary image classification (`fake` vs `real`)
38
+ - **License:** MIT
39
+ - **Architecture:** Hybrid dual-backbone — EfficientNet-B1 and RegNetY-800MF feature extractors, both fine-tuned end-to-end, with their pooled features fused and passed through a classification head
40
+ - **Fine-tuning objective:** Cross-entropy loss over the two classes, with `sklearn` balanced class weights applied to account for class imbalance
41
+ - **Training regime:** Mixed-precision (AMP) training on a CUDA GPU, up to 100 epochs per stage with early stopping (patience = 5, monitored on validation loss); two-stage schedule — Stage A trains only the fusion head with both backbones frozen, Stage B fine-tunes both backbones together at a lower learning rate
42
+
43
+ ## Intended Use
44
+
45
+ This model is intended for detecting AI-generated or manipulated (deepfake) face images versus authentic (real) face images. Example use cases:
46
+
47
+ - Screening uploaded profile/identity photos for synthetic manipulation
48
+ - Research and coursework on deepfake detection and media forensics
49
+ - A component in a larger content-authenticity verification pipeline
50
+
51
+ **Out of scope:** This model is **not** a complete or production-ready deepfake detection guardrail. It has been evaluated on one dataset only, and will not necessarily generalize to deepfake generation methods, image qualities, or demographics absent from its training data.
52
+
53
+ ## How to Use
54
+
55
+ This model is distributed as an **ONNX** export. Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the `.onnx.data` file alongside it at runtime:
56
+
57
+ - [`deepfake_hybrid_final.onnx`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx) — the ONNX graph
58
+ - [`deepfake_hybrid_final.onnx.data`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx.data) — the external weights file
59
+
60
+ Install dependencies:
61
+
62
+ ```bash
63
+ pip install onnxruntime huggingface_hub pillow numpy
64
+ ```
65
+
66
+ ### Single-image prediction
67
+
68
+ ```python
69
+ import numpy as np
70
+ import onnxruntime as ort
71
+ from PIL import Image
72
+ from huggingface_hub import hf_hub_download
73
+
74
+ REPO_ID = "nihal4/Deep_Fake_Hybrid_Model"
75
+ IMG_SIZE = 260
76
+ IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
77
+ IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32)
78
+ LABEL_MAP = {0: "fake", 1: "real"}
79
+
80
+ # Downloads both files into the same local cache folder — required, since the
81
+ # .onnx graph references .onnx.data by relative path at load time.
82
+ onnx_path = hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx")
83
+ hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx.data")
84
+
85
+ session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
86
+ input_name = session.get_inputs()[0].name
87
+ output_name = session.get_outputs()[0].name
88
+
89
+ def preprocess_pil(img: Image.Image) -> np.ndarray:
90
+ img = img.convert("RGB").resize((IMG_SIZE, IMG_SIZE))
91
+ arr = np.asarray(img, dtype=np.float32) / 255.0 # HWC, [0,1]
92
+ arr = (arr - IMAGENET_MEAN) / IMAGENET_STD # normalize, same stats as training
93
+ return arr.transpose(2, 0, 1) # HWC -> CHW
94
+
95
+ def softmax(x: np.ndarray) -> np.ndarray:
96
+ e = np.exp(x - x.max(axis=1, keepdims=True))
97
+ return e / e.sum(axis=1, keepdims=True)
98
+
99
+ def predict(image_path: str):
100
+ image = Image.open(image_path)
101
+ x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32)
102
+ logits = session.run([output_name], {input_name: x})[0]
103
+ probs = softmax(logits)[0]
104
+ label = LABEL_MAP[int(probs.argmax())]
105
+ return label, probs
106
+
107
+ label, probs = predict("path/to/face.jpg")
108
+ print(f"Prediction: {label} (p_fake={probs[0]:.3f}, p_real={probs[1]:.3f})")
109
+ ```
110
+
111
+ ### Batch prediction
112
+
113
+ ```python
114
+ image_paths = ["face1.jpg", "face2.jpg", "face3.jpg"]
115
+
116
+ batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32)
117
+ logits = session.run([output_name], {input_name: batch})[0]
118
+ probs = softmax(logits)
119
+ preds = probs.argmax(axis=1)
120
+
121
+ for path, pred, p in zip(image_paths, preds, probs):
122
+ print(f"{path}: {LABEL_MAP[int(pred)]} (p_fake={p[0]:.3f}, p_real={p[1]:.3f})")
123
+ ```
124
+
125
+ > For GPU inference, install `onnxruntime-gpu` instead and pass `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session.
126
+
127
+ ## Training Data
128
+
129
+ The model was fine-tuned on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset.
130
+
131
+ - **Labels:** `0 = fake`, `1 = real`
132
+ - **Splits:** train / validation / test
133
+ - **Preprocessing:** resize to 260×260, ImageNet normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`)
134
+ - **Training augmentation:** random horizontal flip, random rotation (±10°), color jitter, plus simulated JPEG compression and simulated blur/downscale-upscale (to reduce false positives on low-quality real footage)
135
+ - **Class balancing:** `sklearn` balanced class weights applied in the loss function to address train-set class imbalance
136
+
137
+ ## Training Procedure
138
+
139
+ <!-- PLACEHOLDER: training curves (loss/accuracy per epoch) — image to be uploaded -->
140
+
141
+
142
+ ![Training Plot](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/P4xdn9C-oB-zstz2IoQ5H.png)
143
+
144
+ - **Framework:** PyTorch
145
+ - **Hardware:** Kaggle free-tier T4 GPU
146
+ - **Loss:** Cross-entropy
147
+ - **Mixed precision:** Enabled (AMP)
148
+
149
+ ## Evaluation
150
+
151
+ Evaluated on the held-out test split (n = 21,316) at a decision threshold of 0.5.
152
+
153
+ ### Classification Report
154
+
155
+ | Class | Precision | Recall | F1-score | Support |
156
+ |--------------|:---------:|:------:|:--------:|:-------:|
157
+ | fake | 0.95 | 0.96 | 0.95 | 10,706 |
158
+ | real | 0.96 | 0.95 | 0.95 | 10,610 |
159
+ | **accuracy** | | | **0.9539** | 21,316 |
160
+ | macro avg | 0.95 | 0.95 | 0.95 | 21,316 |
161
+ | weighted avg | 0.95 | 0.95 | 0.95 | 21,316 |
162
+
163
+ **Test ROC-AUC:** 0.9903
164
+
165
+ ### Confusion Matrix
166
+
167
+ <!-- PLACEHOLDER: image to be uploaded -->
168
+
169
+
170
+
171
+ ![Confusion Matrix](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/7KaWz5PUUvR19qmLoPB3H.png)
172
+
173
+ ### ROC Curve
174
+
175
+ <!-- PLACEHOLDER: image to be uploaded -->
176
+
177
+
178
+
179
+ ![ROC-AUC Curve](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/7VIFPxuYc1TimmtpBVF1L.png)
180
+
181
+ ## Limitations
182
+
183
+ - Performance is reported on a single dataset; generalization to other deepfake generation methods, image resolutions, compression levels, or demographics is not guaranteed.
184
+ - As with most deepfake detectors, robustness against novel/unseen generative techniques (including adversarially crafted ones) has not been evaluated here.
185
+ - The model has not been evaluated as a standalone production guardrail; it is intended to complement, not replace, other verification measures.
186
+
187
+ ## Citation
188
+
189
+ If you use this model, please cite this repository and reference this course project:
190
+
191
+ ```
192
+ @misc{deepfake-hybrid-detector,
193
+ title = {Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity},
194
+ author = {S. M. Nihal Ahmed},
195
+ year = {2026},
196
+ note = {Course project, AI Lab (SE334), Daffodil International University}
197
+ }
198
+ ```