Duplicate from faisalishfaq2005/deepfake-detection-efficientnet-vit

Co-authored-by: Muhammad Faisal Ishfaq <faisalishfaq2005@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.jpg filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,174 @@
+---
+language: en
+library_name: pytorch
+license: mit
+tags:
+- deepfake-detection
+- image-classification
+- video-analysis
+- efficientvit
+- pytorch
+pipeline_tag: image-classification
+
+safetensors:
+  total: 1
+  format: safetensors
+  weight_dtype: float32
+  size_in_bytes: 80000000   
+
+model-index:
+- name: Deepfake Detection with Improved EfficientViT
+  results:
+  - task:
+      type: image-classification
+      name: Deepfake Detection
+    dataset:
+      type: custom
+      name: FaceForensics++,Celeb-DF
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.8864
+      - name: Precision
+        type: precision
+        value: 0.8920
+      - name: Recall
+        type: recall
+        value: 0.8792
+      - name: F1-score
+        type: f1
+        value: 0.8856
+
+  config: config.json
+  metadata:
+    model_type: EfficientViT
+    num_parameters: 20026725
+    precision: float32
+    framework: pytorch
+    license: mit
+    model_format: safetensors
+    size: 82MB
+---
+
+# Deepfake Detection with Improved EfficientViT
+
+## Model Architecture
+
+![Model Architecture](assets/architecture.png)
+
+## Inference Pipeline
+
+![Inference Pipeline](assets/inference_pipeline.png)
+
+
+This repository contains a **PyTorch model for deepfake detection** based on an improved **EfficientViT** architecture, trained on video data.  
+
+The model predicts whether a video is **real (0)** or **fake (1)** using both visual information and temporal cues.
+
+---
+
+## 🧩 Model Description
+
+**Architecture:** Improved EfficientViT  
+**Backbone:** EfficientNet-B0 for feature extraction  
+**Head:** Transformer-based temporal modeling with classification head  
+**Input:** Video frames (224×224 RGB images)  
+**Output:** Binary label (0=Real, 1=Fake) and frame-level probabilities  
+
+**Key Features:**
+
+- Extracts faces from frames using MTCNN  
+- Supports inference on raw video files  
+- Provides frame-level probabilities for fine-grained analysis  
+
+---
+
+## 📁 Repository Structure
+
+```
+deepfake-efficientvit/
+│
+├── model.py                  # ImprovedEfficientViT class
+├── inference.py              # Functions to run inference on videos
+├── model.pth  # Trained weights
+├── config.json               # Optional model metadata
+├── requirements.txt          # Required packages
+├── README.md
+
+```
+
+## ⚡ Installation
+git clone https://huggingface.co/faisalishfaq2005/deepfake-detection-efficientnet-vit
+
+cd deepfake-detection-efficientnet-vit
+
+pip install -r requirements.txt
+
+## 🚀 Usage
+# 1.Programmatic Inference
+
+```python
+
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+import torch
+from model import ImprovedEfficientViT
+from inference import predict_vedio 
+
+# 1️⃣ Download the checkpoint from Hugging Face
+checkpoint_path = hf_hub_download(
+    repo_id="faisalishfaq2005/deepfake-detection-efficientnet-vit",  
+    filename="model.safetensors"
+)
+
+# 2️⃣ Load the model weights safely
+state_dict = load_file(checkpoint_path, device="cpu")
+model = ImprovedEfficientViT()
+model.load_state_dict(state_dict)
+model.eval()
+
+# 4️⃣ Move to GPU if available
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model.to(device)
+
+# 3️⃣ Run inference on a video
+video_path = "sample_video.mp4"
+result = predict_vedio(video_path, model)
+print(result)
+# Example Output: {'class': 1}
+
+```
+# 2. Manual Download
+
+Go to the Hugging Face model page
+
+Download:
+
+model.pth
+
+model.py
+
+inference.py
+
+Place them in the same folder locally.
+
+Install requirements and run predict_video().
+
+## 📄 License
+
+This model is released under the MIT License.
+You are free to use, modify, and distribute it, with attribution.
+
+## 📚 Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@inproceedings{faisalishfaq2025efficientvit,
+  title={Deepfake Detection with Efficientnet and ViT},
+  author={Faisal Ishfaq},
+  year={2025}
+}
+```
+
+

config.json

@@ -0,0 +1,16 @@
+{
+  "architectures": ["ImprovedEfficientViT"],
+  "model_type": "efficientnetb0_Vit_blocks_multi_head_attention",
+  "framework": "pytorch",
+  "precision": "float32",
+  "num_parameters": 20026725,
+  "model_format": "safetensors",
+  "license": "mit",
+  "tags": [
+    "image-classification",
+    "video-analysis",
+    "deepfake-detection",
+    "efficientvit",
+    "pytorch"
+  ]
+}

inference.py

@@ -0,0 +1,91 @@
+from torchvision import transforms
+import torch
+from PIL import Image
+from model import ImprovedEfficientViT
+
+import os
+import cv2
+from mtcnn import MTCNN
+
+def extract_faces(video_path, target_frames=20):
+  
+    detector = MTCNN()
+
+    cap = cv2.VideoCapture(video_path)
+    if not cap.isOpened():
+        print(f"Error: Could not open video {video_path}")
+        return []
+
+    fps = cap.get(cv2.CAP_PROP_FPS)
+    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+
+    frame_interval = max(total_frames // target_frames, 1)
+
+    face_images = [] 
+
+    for i in range(0, total_frames, frame_interval):
+        cap.set(cv2.CAP_PROP_POS_FRAMES, i)
+        ret, frame = cap.read()
+        if not ret:
+            continue
+
+        rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+        faces = detector.detect_faces(rgb_frame)
+
+        for face in faces:
+            if face['confidence'] < 0.9:
+                continue
+            x, y, w, h = face['box']
+            x, y = max(x, 0), max(y, 0)
+            face_img = rgb_frame[y:y+h, x:x+w]
+
+            if face_img.size == 0:
+                continue
+
+            face_img = cv2.resize(face_img, (224, 224))
+            face_images.append(face_img)
+
+    cap.release()
+    return face_images
+
+from torchvision import transforms
+transform_vedio=transforms.Compose([
+    transforms.ToPILImage(),
+    transforms.Resize((224,224)),
+    transforms.ToTensor(),
+    transforms.Normalize(mean=[0.5],std=[0.5])
+
+])
+
+
+def predict_vedio(video_path,model_vedio):
+
+    pred_list = []
+    prob_list=[]
+
+    faces = extract_faces(video_path, target_frames=20)
+    
+    transformed_faces = [transform_vedio(face) for face in faces]
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    model_vedio.to(device)
+
+    for face in transformed_faces:
+        face = face.to(device).unsqueeze(0)  
+
+        with torch.no_grad():
+            logit = model_vedio(face)              
+            prob = torch.sigmoid(logit)      
+            pred = int(prob.item() > 0.5)    
+            pred_list.append(pred)
+            prob_list.append(prob)
+
+    count=0
+    for ele in pred_list:
+        if ele==0:
+            count+=1
+
+    predicted_class=0 if count>3 else 1
+    return{
+        "class":predicted_class
+    }
+    

model.py

@@ -0,0 +1,99 @@
+import torch
+import torch.nn as nn
+import torchvision
+import math
+
+class ImprovedEfficientBackbone(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.efficientnet = torchvision.models.efficientnet_b0(weights=torchvision.models.EfficientNet_B0_Weights.IMAGENET1K_V1)
+        self.features = self.efficientnet.features
+
+    def forward(self, x):
+        return self.features(x)  
+
+class ImprovedPatchEmbedding(nn.Module):
+    def __init__(self, in_channels=1280, embed_dim=384):
+        super().__init__()
+        self.proj = nn.Linear(in_channels, embed_dim)
+
+    def forward(self, x):
+        """
+        Input:  [B, 1280, 7, 7]
+        Output: [B, 49, 384]
+        """
+        B, C, H, W = x.shape
+        x = x.flatten(2).transpose(1, 2)  
+        x = self.proj(x)  
+        return x
+
+
+class ImprovedViTBlock(nn.Module):
+    def __init__(self, embed_dim=384, num_heads=4, mlp_ratio=4):
+        super().__init__()
+        self.norm1 = nn.LayerNorm(embed_dim)
+        self.attn = nn.MultiheadAttention(embed_dim, num_heads, batch_first=True)
+        self.norm2 = nn.LayerNorm(embed_dim)
+        self.mlp = nn.Sequential(
+            nn.Linear(embed_dim, embed_dim * mlp_ratio),
+            nn.GELU(),
+            nn.Linear(embed_dim * mlp_ratio, embed_dim)
+        )
+        self.dropout = nn.Dropout(0.2) 
+
+    def forward(self, x):
+        x = x + self.dropout(self.attn(self.norm1(x), self.norm1(x), self.norm1(x))[0])
+        x = x + self.dropout(self.mlp(self.norm2(x)))
+        return x
+
+class ImprovedEfficientViT(nn.Module):
+    def __init__(self, embed_dim=384, depth=6, num_heads=4):
+        super().__init__()
+        self.backbone = ImprovedEfficientBackbone()
+        self.patch_embed = ImprovedPatchEmbedding(embed_dim=embed_dim)
+
+        self.cls_token = nn.Parameter(torch.randn(1, 1, embed_dim))
+        self.register_buffer("pos_embed", self._get_sinusoidal_encoding(50, embed_dim))
+
+        self.patch_dropout = nn.Dropout(0.2)
+        self.pos_dropout = nn.Dropout(0.2)
+
+        self.blocks = nn.ModuleList([ImprovedViTBlock(embed_dim, num_heads) for _ in range(depth)])
+
+        self.head = nn.Sequential(
+              nn.LayerNorm(embed_dim),
+              nn.Linear(embed_dim, 128),
+              nn.GELU(),
+              nn.Dropout(0.3),
+              nn.Linear(128, 1)
+          )
+
+        self._init_weights()
+
+    def _init_weights(self):
+        nn.init.trunc_normal_(self.cls_token, std=0.02)
+
+    def _get_sinusoidal_encoding(self, seq_len, dim):
+        pe = torch.zeros(seq_len, dim)
+        position = torch.arange(0, seq_len, dtype=torch.float).unsqueeze(1)
+        div_term = torch.exp(torch.arange(0, dim, 2).float() * (-math.log(10000.0) / dim))
+        pe[:, 0::2] = torch.sin(position * div_term)
+        pe[:, 1::2] = torch.cos(position * div_term)
+        return pe.unsqueeze(0)  
+
+    def forward(self, x):
+        features = self.backbone(x)                  
+        tokens = self.patch_embed(features)          
+        tokens = self.patch_dropout(tokens)          
+
+        B = tokens.shape[0]
+        cls_tokens = self.cls_token.expand(B, -1, -1)  
+        x = torch.cat((cls_tokens, tokens), dim=1)    
+        x = x + self.pos_embed[:, :x.size(1), :]      
+        x = self.pos_dropout(x)                       
+
+        for block in self.blocks:
+            x = block(x)
+
+        cls_final = x[:, 0] 
+        return self.head(cls_final)  

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbebb0e63de194963276739b73f194a9eda09221c3e73563a8fa87ddaac38120
+size 82444700

requirements.txt

@@ -0,0 +1,5 @@
+torch
+torchvision
+opencv-python
+mtcnn
+Pillow