--- license: mit datasets: - ThinothW/Deepfake-Identity-Isolated-Dataset-PreP metrics: - accuracy - precision - recall - f1 - roc_auc base_model: - google/efficientnet-b1 - mobilint/RegNet_Y_800MF.tv2_in1k pipeline_tag: image-classification tags: - deepfake-detection - image-classification - efficientnet - regnet - hybrid-model - computer-vision --- # Deep_Fake_Hybrid_Model ### Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity A binary image classifier that flags a given face image as **real** or **fake** (deepfake / synthetically manipulated). Built as a **hybrid dual-backbone** model combining fine-tuned [`google/efficientnet-b1`](https://huggingface.co/google/efficientnet-b1) and [`mobilint/RegNet_Y_800MF.tv2_in1k`](https://huggingface.co/mobilint/RegNet_Y_800MF.tv2_in1k) feature extractors, trained on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset. This model was built as part of a university course project (AI Lab, SE334) exploring deepfake face detection. **Author:** S. M. Nihal Ahmed ## Model Details - **Base models:** `google/efficientnet-b1`, `mobilint/RegNet_Y_800MF.tv2_in1k` - **Task:** Binary image classification (`fake` vs `real`) - **License:** MIT - **Architecture:** Hybrid dual-backbone — EfficientNet-B1 and RegNetY-800MF feature extractors, both fine-tuned end-to-end, with their pooled features fused and passed through a classification head - **Fine-tuning objective:** Cross-entropy loss over the two classes, with `sklearn` balanced class weights applied to account for class imbalance - **Training regime:** Mixed-precision (AMP) training on a CUDA GPU, up to 100 epochs per stage with early stopping (patience = 5, monitored on validation loss); two-stage schedule — Stage A trains only the fusion head with both backbones frozen, Stage B fine-tunes both backbones together at a lower learning rate ## Intended Use This model is intended for detecting AI-generated or manipulated (deepfake) face images versus authentic (real) face images. Example use cases: - Screening uploaded profile/identity photos for synthetic manipulation - Research and coursework on deepfake detection and media forensics - A component in a larger content-authenticity verification pipeline **Out of scope:** This model is **not** a complete or production-ready deepfake detection guardrail. It has been evaluated on one dataset only, and will not necessarily generalize to deepfake generation methods, image qualities, or demographics absent from its training data. ## How to Use This model is distributed as an **ONNX** export. Download both files and keep them in the same folder — the `.onnx` graph loads its weights from the `.onnx.data` file alongside it at runtime: - [`deepfake_hybrid_final.onnx`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx) — the ONNX graph - [`deepfake_hybrid_final.onnx.data`](https://huggingface.co/nihal4/Deep_Fake_Hybrid_Model/resolve/main/deepfake_hybrid_final.onnx.data) — the external weights file Install dependencies: ```bash pip install onnxruntime huggingface_hub pillow numpy ``` ### Single-image prediction ```python import numpy as np import onnxruntime as ort from PIL import Image from huggingface_hub import hf_hub_download REPO_ID = "nihal4/Deep_Fake_Hybrid_Model" IMG_SIZE = 260 IMAGENET_MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32) IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32) LABEL_MAP = {0: "fake", 1: "real"} # Downloads both files into the same local cache folder — required, since the # .onnx graph references .onnx.data by relative path at load time. onnx_path = hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx") hf_hub_download(repo_id=REPO_ID, filename="deepfake_hybrid_final.onnx.data") session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"]) input_name = session.get_inputs()[0].name output_name = session.get_outputs()[0].name def preprocess_pil(img: Image.Image) -> np.ndarray: img = img.convert("RGB").resize((IMG_SIZE, IMG_SIZE)) arr = np.asarray(img, dtype=np.float32) / 255.0 # HWC, [0,1] arr = (arr - IMAGENET_MEAN) / IMAGENET_STD # normalize, same stats as training return arr.transpose(2, 0, 1) # HWC -> CHW def softmax(x: np.ndarray) -> np.ndarray: e = np.exp(x - x.max(axis=1, keepdims=True)) return e / e.sum(axis=1, keepdims=True) def predict(image_path: str): image = Image.open(image_path) x = preprocess_pil(image)[np.newaxis, ...].astype(np.float32) logits = session.run([output_name], {input_name: x})[0] probs = softmax(logits)[0] label = LABEL_MAP[int(probs.argmax())] return label, probs label, probs = predict("path/to/face.jpg") print(f"Prediction: {label} (p_fake={probs[0]:.3f}, p_real={probs[1]:.3f})") ``` ### Batch prediction ```python image_paths = ["face1.jpg", "face2.jpg", "face3.jpg"] batch = np.stack([preprocess_pil(Image.open(p)) for p in image_paths]).astype(np.float32) logits = session.run([output_name], {input_name: batch})[0] probs = softmax(logits) preds = probs.argmax(axis=1) for path, pred, p in zip(image_paths, preds, probs): print(f"{path}: {LABEL_MAP[int(pred)]} (p_fake={p[0]:.3f}, p_real={p[1]:.3f})") ``` > For GPU inference, install `onnxruntime-gpu` instead and pass `providers=["CUDAExecutionProvider", "CPUExecutionProvider"]` when creating the session. ## Training Data The model was fine-tuned on the [`ThinothW/Deepfake-Identity-Isolated-Dataset-PreP`](https://huggingface.co/datasets/ThinothW/Deepfake-Identity-Isolated-Dataset-PreP) dataset. - **Labels:** `0 = fake`, `1 = real` - **Splits:** train / validation / test - **Preprocessing:** resize to 260×260, ImageNet normalization (mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`) - **Training augmentation:** random horizontal flip, random rotation (±10°), color jitter, plus simulated JPEG compression and simulated blur/downscale-upscale (to reduce false positives on low-quality real footage) - **Class balancing:** `sklearn` balanced class weights applied in the loss function to address train-set class imbalance ## Training Procedure ![Training Plot](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/P4xdn9C-oB-zstz2IoQ5H.png) - **Framework:** PyTorch - **Hardware:** Kaggle free-tier T4 GPU - **Loss:** Cross-entropy - **Mixed precision:** Enabled (AMP) ## Evaluation Evaluated on the held-out test split (n = 21,316) at a decision threshold of 0.5. ### Classification Report | Class | Precision | Recall | F1-score | Support | |--------------|:---------:|:------:|:--------:|:-------:| | fake | 0.95 | 0.96 | 0.95 | 10,706 | | real | 0.96 | 0.95 | 0.95 | 10,610 | | **accuracy** | | | **0.9539** | 21,316 | | macro avg | 0.95 | 0.95 | 0.95 | 21,316 | | weighted avg | 0.95 | 0.95 | 0.95 | 21,316 | **Test ROC-AUC:** 0.9903 ### Confusion Matrix ![Confusion Matrix](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/7KaWz5PUUvR19qmLoPB3H.png) ### ROC Curve ![ROC-AUC Curve](https://cdn-uploads.huggingface.co/production/uploads/661d43ec3cf2981df52d0756/7VIFPxuYc1TimmtpBVF1L.png) ## Limitations - Performance is reported on a single dataset; generalization to other deepfake generation methods, image resolutions, compression levels, or demographics is not guaranteed. - As with most deepfake detectors, robustness against novel/unseen generative techniques (including adversarially crafted ones) has not been evaluated here. - The model has not been evaluated as a standalone production guardrail; it is intended to complement, not replace, other verification measures. ## Citation If you use this model, please cite this repository and reference this course project: ``` @misc{deepfake-hybrid-detector, title = {Detecting Deepfake Faces: An Image Classification Approach to Safeguarding Digital Identity}, author = {S. M. Nihal Ahmed}, year = {2026}, note = {Course project, AI Lab (SE334), Daffodil International University} } ```