Initial upload of DeepSecure backend

.gitignore

@@ -0,0 +1,4 @@
+venv/
+__pycache__/
+*.pyc
+.DS_Store

Dockerfile

@@ -0,0 +1,25 @@
+FROM python:3.11-slim
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies required for OpenCV
+RUN apt-get update && apt-get install -y \
+    libgl1-mesa-glx \
+    libglib2.0-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy requirements first to leverage Docker cache
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy the rest of the application
+COPY . .
+
+# Expose port 7860 (Hugging Face default)
+EXPOSE 7860
+
+# Start the application
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,11 +1,195 @@
 ---
-title: DSBackend
-emoji: 📚
-colorFrom: green
-colorTo: purple
-sdk: docker
-pinned: false
 license: mit
+language:
+- en
+metrics:
+- accuracy
+library_name: tf-keras
+pipeline_tag: image-classification
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Deepfake Detection Backend & Model (V1)
+
+This repository contains a Convolutional Neural Network (CNN)-based model fine-tuned for deepfake classification, now wrapped in a high-performance **FastAPI** backend that natively supports processing both images and frame-by-frame videos.
+
+## Core Advancements
+To drastically improve real-world accuracy (especially on webcams and scaling distortions), we implemented **Ultralytics YOLO11-Pose** (`yolo11n-pose.pt`) for facial extraction. 
+
+The underlying CNN (`model.h5`) excels only when evaluated on *tight facial crops* matching its training data. Generative YOLO bounding boxes are too loose and capture background noise. By extracting tracking keypoints (eyes, nose, ears) and explicitly drawing bounding configurations around them via YOLO11, we mathematically generate tight facial configurations, ensuring that the CNN captures exactly what it was trained to see, regardless of camera distance.
+
+### Key Features:
+- **Model Architecture:** Convolutional Neural Network (CNN)
+- **Input Size:** 128x128 pixels (Tight facial crop)
+- **Face Extractor:** Ultralytics YOLO11-Pose (`yolo11n-pose.pt`)
+- **Video Processing:** Extracts and analyzes 1 in every 5 frames (~6 fps) for robust temporal spoof detection. Deepfake videos are flagged as "Fake" if *any* evaluated frame's prediction score exceeds 50%.
+- **Number of Classes:** 2 (Real, Fake)
+- **API Framework:** FastAPI, Uvicorn, Python-Multipart
+
+## Processing Flow & Algorithm
+
+The system natively processes both images and videos using a unified core prediction pipeline. The following describes the step-by-step logic.
+
+### 1. Media Handling Flow
+
+**For Images:**
+1. The image is parsed and decoded directly from the HTTP request.
+2. The image is passed to the **Core Prediction Pipeline**.
+3. A confidence score is returned, classifying the image as "Real" or "Fake".
+
+**For Videos:**
+1. The video is saved to a temporary file and read using OpenCV.
+2. Frames are iteratively extracted.
+3. To optimize performance without sacrificing temporal accuracy, **1 in every 5 frames** (~6 FPS for a 30 FPS video) is analyzed.
+4. Each selected frame is individually passed to the **Core Prediction Pipeline**.
+5. The backend collects a list of `confidence_scores` from the analyzed frames.
+6. The video is flagged as "Fake" if the **maximum** confidence score among all frames (i.e., the most manipulated frame) exceeds 0.5.
+
+### 2. Core Prediction Pipeline (Pseudocode)
+
+To definitively locate and strictly frame the face, the YOLO11-Pose pipeline extracts 5 specific facial keypoints: **Nose, Left Eye, Right Eye, Left Ear, and Right Ear**.
+
+```python
+function process_frame(frame):
+    # Step 1: Detect Face & Extract Keypoints (YOLO11-Pose)
+    results = yolo_pose_model.predict(frame)
+    
+    if face_keypoints_found(results):
+        # Eyes, nose, and ears detected
+        bounding_box = calculate_tight_box_from_keypoints()
+        face_crop = crop_image(frame, bounding_box)
+    elif person_bounding_box_found(results): 
+        # Fallback to standard object detection box if keypoints fail
+        bounding_box = shrink_box_to_approximate_face()
+        face_crop = crop_image(frame, bounding_box)
+    else: 
+        # Extreme fallback if no person is detected
+        face_crop = frame
+
+    # Step 2: Preprocessing
+    resized_face = resize_image(face_crop, width=128, height=128)
+    normalized_face = resized_face / 255.0
+    model_input = expand_dimensions(normalized_face)
+
+    # Step 3: CNN Model Inference
+    confidence_score = cnn_model.predict(model_input)
+    
+    return confidence_score
+```
+
+## Training Performance
+
+Below are the graphs illustrating the training and validation accuracy and loss for the model:
+
+![Model Training/Validation Graph 1](Unknown.png)
+
+![Model Training/Validation Graph 2](Unknown-2.png)
+
+## Installation
+
+1. Create a Python 3.11 virtual environment and activate it:
+```bash
+python3.11 -m venv venv
+source venv/bin/activate
+```
+2. Install the required dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+## Running the API Server
+
+We provide a convenient startup script to launch the FastAPI backend:
+```bash
+chmod +x start_server.sh
+./start_server.sh
+```
+The server will bind to `0.0.0.0:8000`, making the `/predict` endpoint available.
+
+## Usage (API)
+
+You can send a `POST` request with an image or video to the `/predict` endpoint using `multipart/form-data`:
+
+```python
+import requests
+
+url = "http://localhost:8000/predict"
+file_path = "sample_video.mp4" # Or an image.jpg
+
+with open(file_path, "rb") as file:
+    files = {"file": file}
+    response = requests.post(url, files=files)
+
+print(response.json())
+```
+
+**JSON Output Structure (Video):**
+```json
+{
+  "filename": "sample_video.mp4",
+  "type": "video",
+  "prediction": "Fake",
+  "confidence_score": 0.8921, 
+  "frames_analyzed": 120,
+  "fake_frames_count": 14,
+  "max_fake_score": 0.8921,
+  "avg_score": 0.3102 
+}
+```
+*Note: A score closer to `1.0` is recognized as heavily manipulated. A score closer to `0.0` is authentic. An inference resulting in `max_fake_score` ≥ 0.5 triggers a "Fake" prediction limit.*
+
+## Usage (Direct Python Inference)
+
+If you'd like to use the YOLO11 inference pipeline directly in your Python code without the API server, feel free to adapt this minimal inference script:
+
+```python
+import cv2
+import numpy as np
+import warnings
+from tensorflow.keras.preprocessing import image
+from tensorflow.keras.models import load_model
+from ultralytics import YOLO
+
+warnings.filterwarnings('ignore', category=UserWarning)
+
+# Load Models
+model = load_model('model.h5', compile=False)
+detector = YOLO('yolo11n-pose.pt')
+
+def detect_and_predict(img_path):
+    img = cv2.imread(img_path)
+    
+    # 1. Detect Face using YOLO11-Pose Keypoints
+    results = detector.predict(img, verbose=False)
+    if len(results) > 0 and results[0].keypoints is not None and len(results[0].keypoints.xy[0]) > 0:
+        kpts = results[0].keypoints.xy[0].cpu().numpy()
+        valid_kpts = np.array([k for k in kpts[0:5] if k[0] > 0 and k[1] > 0]) # Eyes, nose, ears
+        
+        if len(valid_kpts) > 0:
+            x_min, y_min = np.min(valid_kpts, axis=0)
+            x_max, y_max = np.max(valid_kpts, axis=0)
+            
+            # Expand tight box to capture full face (forehead to jaw)
+            w, h = x_max - x_min, y_max - y_min
+            if w > 0 and h > 0:
+                x1 = max(0, int(x_min - w * 0.3))
+                y1 = max(0, int(y_min - h * 0.5))
+                x2 = min(img.shape[1], int(x_max + w * 0.3))
+                y2 = min(img.shape[0], int(y_max + h * 0.8))
+                
+                face = img[y1:y2, x1:x2]
+                if face.size > 0:
+                    face = cv2.resize(face, (128, 128))
+                    
+                    # 2. Preprocess & Predict
+                    img_array = np.expand_dims(image.img_to_array(face), axis=0) / 255.0
+                    score = float(model.predict(img_array, verbose=0)[0][0])
+                    
+                    prediction = 'Fake' if score >= 0.5 else 'Real'
+                    print(f"Prediction: {prediction} (Score: {score:.4f})")
+                    return
+                    
+    print("Could not detect a clear face.")
+
+# Try it out
+detect_and_predict('path_to_your_image.jpg')
+```

app.py

@@ -0,0 +1,203 @@
+from fastapi import FastAPI, File, UploadFile, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+import uvicorn
+import cv2
+import numpy as np
+import os
+import tempfile
+import shutil
+import warnings
+import logging
+import os
+
+# Suppress Keras and TensorFlow warnings
+warnings.filterwarnings('ignore', category=UserWarning)
+os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
+logging.getLogger('absl').setLevel(logging.ERROR)
+
+from tensorflow.keras.preprocessing import image
+from tensorflow.keras.models import load_model
+from ultralytics import YOLO
+
+app = FastAPI(title="Deepfake Detection API")
+
+# Update CORS for frontend connectivity
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"], # In production, replace with your frontend URL
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+@app.get("/")
+async def health_check():
+    return {"status": "online", "message": "Backend is running!"}
+
+print("Loading model and YOLO11...")
+model = load_model('model.h5', compile=False)
+# YOLOv11 pose model for extremely tight and precise facial feature cropping (matches MTCNN style)
+detector = YOLO('yolo11n-pose.pt')
+print("Model loaded successfully.")
+
+def detect_and_crop_face(img):
+    """Detects face/person and crops it to 128x128."""
+    # Run YOLO11 pose detection
+    results = detector.predict(img, verbose=False)
+    
+    if len(results) > 0 and results[0].keypoints is not None and len(results[0].keypoints.xy[0]) > 0:
+        # Get the first person's keypoints
+        kpts = results[0].keypoints.xy[0].cpu().numpy()
+        
+        # 0: nose, 1: left eye, 2: right eye, 3: left ear, 4: right ear
+        face_kpts = kpts[0:5]
+        # Filter out keypoints that weren't detected
+        valid_kpts = [k for k in face_kpts if k[0] > 0 and k[1] > 0]
+        
+        if valid_kpts:
+            valid_kpts = np.array(valid_kpts)
+            x_min, y_min = np.min(valid_kpts, axis=0)
+            x_max, y_max = np.max(valid_kpts, axis=0)
+            
+            # Expand this tight keypoint box to capture the full face (forehead, chin, cheeks)
+            w = x_max - x_min
+            h = y_max - y_min
+            
+            # Safety for edge cases
+            if w > 0 and h > 0:
+                pad_x = w * 0.3
+                pad_y_top = h * 0.5   # Expand more upward for the forehead
+                pad_y_bot = h * 0.8   # Expand downward for the chin/mouth
+                
+                final_x1 = max(0, int(x_min - pad_x))
+                final_y1 = max(0, int(y_min - pad_y_top))
+                final_x2 = min(img.shape[1], int(x_max + pad_x))
+                final_y2 = min(img.shape[0], int(y_max + pad_y_bot))
+                
+                face = img[final_y1:final_y2, final_x1:final_x2]
+                
+                if face.size > 0:
+                    return cv2.resize(face, (128, 128))
+            
+    # Fallback to normal YOLO box heuristic if face keypoints fail but person is found
+    if len(results) > 0 and len(results[0].boxes) > 0:
+        box = results[0].boxes[0].xyxy[0].cpu().numpy()
+        x1, y1, x2, y2 = map(int, box)
+        x1, y1 = max(0, x1), max(0, y1)
+        x2, y2 = min(img.shape[1], x2), min(img.shape[0], y2)
+        
+        h = y2 - y1
+        w = x2 - x1
+        if h > w * 1.5:  
+            y2 = y1 + int(h * 0.3)
+            
+        face = img[y1:y2, x1:x2]
+        if face.size > 0:
+            return cv2.resize(face, (128, 128))
+
+    # If no person is detected at all
+    return cv2.resize(img, (128, 128))
+
+def preprocess_face(face):
+    """Formats the cropped face for the model."""
+    img_array = image.img_to_array(face)
+    img_array = np.expand_dims(img_array, axis=0)
+    img_array /= 255.0  # Normalize
+    return img_array
+
+def process_image(img):
+    """Processes a single BGR image array and returns the fake probability."""
+    face = detect_and_crop_face(img)
+    processed_image = preprocess_face(face)
+    prediction = model.predict(processed_image, verbose=0)
+    return float(prediction[0][0])
+
+@app.post("/predict")
+async def predict_media(file: UploadFile = File(...)):
+    filename = file.filename.lower()
+    
+    is_video = filename.endswith(('.mp4', '.avi', '.mov', '.mkv'))
+    is_image = filename.endswith(('.jpg', '.jpeg', '.png', '.bmp'))
+    
+    if not is_image and not is_video:
+        raise HTTPException(status_code=400, detail="Unsupported file format.")
+        
+    try:
+        if is_image:
+            # Read image directly from bytes
+            contents = await file.read()
+            nparr = np.frombuffer(contents, np.uint8)
+            img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
+            
+            if img is None:
+                raise HTTPException(status_code=400, detail="Invalid image file.")
+                
+            score = process_image(img)
+            result = "Real" if score < 0.5 else "Fake"
+            
+            return {
+                "filename": filename,
+                "type": "image",
+                "prediction": result,
+                "confidence_score": score
+            }
+            
+        elif is_video:
+            # Save video to a temporary file for cv2.VideoCapture
+            with tempfile.NamedTemporaryFile(delete=False, suffix=".mp4") as temp_video:
+                shutil.copyfileobj(file.file, temp_video)
+                temp_video_path = temp_video.name
+                
+            cap = cv2.VideoCapture(temp_video_path)
+            if not cap.isOpened():
+                os.unlink(temp_video_path)
+                raise HTTPException(status_code=400, detail="Could not open video file.")
+            
+            frame_scores = []
+            frame_count = 0
+            
+            # Process 1 frame every 5 frames (~6 fps for 30fps video) for better accuracy
+            while True:
+                ret, frame = cap.read()
+                if not ret:
+                    break
+                
+                if frame_count % 5 == 0:
+                    score = process_image(frame)
+                    frame_scores.append(score)
+                    
+                frame_count += 1
+                
+            cap.release()
+            os.unlink(temp_video_path)
+            
+            if not frame_scores:
+                raise HTTPException(status_code=400, detail="Could not extract frames from video.")
+                
+            # Deepfakes often only manipulate specific frames, so average score can mask the spoof.
+            # We use max_score to find the most manipulated frame.
+            max_score = max(frame_scores)
+            avg_score = sum(frame_scores) / len(frame_scores)
+            
+            fake_frames_count = sum(1 for s in frame_scores if s >= 0.5)
+            
+            final_result = "Real" if max_score < 0.5 else "Fake"
+            
+            return {
+                "filename": filename,
+                "type": "video",
+                "prediction": final_result,
+                "confidence_score": max_score,
+                "frames_analyzed": len(frame_scores),
+                "fake_frames_count": fake_frames_count,
+                "max_fake_score": max_score,
+                "avg_score": avg_score
+            }
+            
+    except Exception as e:
+        import traceback
+        traceback.print_exc()
+        raise HTTPException(status_code=500, detail=str(e))
+
+if __name__ == "__main__":
+    uvicorn.run(app, host="0.0.0.0", port=7860)

model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f54d9db020da33f99f861d41dc1334ec33adc14991ada4033a4ece790d0904e
+size 312843624

requirements.txt

@@ -0,0 +1,78 @@
+absl-py==2.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anyio==4.13.0
+astunparse==1.6.3
+certifi==2026.2.25
+charset-normalizer==3.4.7
+click==8.3.2
+contourpy==1.3.3
+cycler==0.12.1
+efficientnet==1.1.1
+fastapi==0.136.0
+filelock==3.28.0
+flatbuffers==25.12.19
+fonttools==4.62.1
+fsspec==2026.3.0
+gast==0.7.0
+google-pasta==0.2.0
+grpcio==1.80.0
+h11==0.16.0
+h5py==3.14.0
+idna==3.11
+ImageIO==2.37.3
+Jinja2==3.1.6
+joblib==1.5.3
+keras==3.14.0
+Keras-Applications==1.0.8
+kiwisolver==1.5.0
+lazy-loader==0.5
+libclang==18.1.1
+lz4==4.4.5
+markdown-it-py==4.0.0
+MarkupSafe==3.0.3
+matplotlib==3.10.8
+mdurl==0.1.2
+ml_dtypes==0.5.4
+mpmath==1.3.0
+mtcnn==1.0.0
+namex==0.1.0
+networkx==3.6.1
+numpy==2.4.4
+opencv-python==4.13.0.92
+opt_einsum==3.4.0
+optree==0.19.0
+packaging==26.1
+pandas==3.0.2
+pillow==12.2.0
+polars==1.39.3
+polars-runtime-32==1.39.3
+protobuf==7.34.1
+psutil==7.2.2
+pydantic==2.13.1
+pydantic_core==2.46.1
+Pygments==2.20.0
+pyparsing==3.3.2
+python-dateutil==2.9.0.post0
+python-multipart==0.0.26
+PyYAML==6.0.3
+requests==2.33.1
+rich==15.0.0
+scikit-image==0.26.0
+scipy==1.17.1
+six==1.17.0
+split-folders==0.6.1
+starlette==1.0.0
+sympy==1.14.0
+tensorflow==2.21.0
+termcolor==3.3.0
+tifffile==2026.3.3
+torch==2.11.0
+torchvision==0.26.0
+typing-inspection==0.4.2
+typing_extensions==4.15.0
+ultralytics==8.4.38
+ultralytics-thop==2.0.18
+urllib3==2.6.3
+uvicorn==0.44.0
+wrapt==2.1.2

start_server.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+# Navigate to the directory containing this script (optional but good practice)
+cd "$(dirname "$0")"
+
+echo "Starting Deepfake Detection Backend Server..."
+
+# Activate the virtual environment
+source venv/bin/activate
+
+# Run the backend using the app.py file we created
+python3 app.py

yolo11n-pose.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:869e83fcdffdc7371fa4e34cd8e51c838cc729571d1635e5141e3075e9319dc0
+size 6255593

yolo11n.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ebbc80d4a7680d14987a577cd21342b65ecfd94632bd9a8da63ae6417644ee1
+size 5613764