video_bundle/README.md

<<<<<<< HEAD

UAIDE - AI-Generated Image & Video Detection

A state-of-the-art deepfake detection system combining EfficientNet-B2 + FFT Fusion with comprehensive ethical assessment, Grad-CAM explainability, and a Gradio web interface. Supports both image and video analysis with multi-layered content safety checks.

---

Model Performance Summary

Metric	Value
Validation Accuracy	95.07%
Validation AUC	0.9908
Validation Loss	0.0471
Train Accuracy	83.91%
Train Loss	0.0770

---

Quick Start

# Create virtual environment
python -m venv .venv

# Activate (Windows PowerShell)
.\.venv\Scripts\Activate.ps1

# Activate (Linux/Mac)
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt
pip install timm  # Required for EfficientNet

# Launch the web app
python app.py

The app loads the trained model from models_adv/best_model_weights.pt and opens a Gradio UI at http://localhost:7860.

---

Architecture

EfficientNet-B2 + FFT Fusion Model

The primary model fuses two complementary streams:

Input Image (224x224)
        │
        ├──────────────────────┬────────────────────────┐
        ▼                      ▼                        │
┌───────────────┐      ┌───────────────┐               │
│ EfficientNet  │      │     FFT       │               │
│     B2        │      │  Extractor    │               │
│  (1408 dim)   │      │  (512 dim)    │               │
└───────┬───────┘      └───────┬───────┘               │
        │                      │                        │
        └──────────┬───────────┘                        │
                   ▼                                    │
           ┌─────────────┐                              │
           │   Fusion    │                              │
           │   Layer     │                              │
           │ (1920→1024) │                              │
           └──────┬──────┘                              │
                  ▼                                     │
           ┌─────────────┐                              │
           │ Classifier  │◄─────────────────────────────┘
           │  (2 class)  │        Grad-CAM
           └─────────────┘

Key Components:

1. Spatial Stream (EfficientNet-B2): Extracts high-level visual features using ImageNet-pretrained weights via timm library
2. Frequency Stream (FFT): Analyzes frequency domain patterns that reveal GAN artifacts:
   • Magnitude statistics (mean, std, max, min)
   • Frequency band energies (low, mid-low, mid-high, high)
   • Threshold-based features for GAN fingerprint detection
3. Fusion Layer: Combines both streams with LayerNorm, GELU activation, and progressive dropout

---

Training

Advanced Training Script (train_adv.py)

python train_adv.py \
    --dataset "DeepfakeVsReal/Dataset" \
    --backbone efficientnet_b2 \
    --epochs 100 \
    --batch_size 8 \
    --lr 0.0001

Training Features:

• EfficientNet backbone (B0, B2, B4 supported)
• FFT-based frequency domain analysis
• Focal loss with adaptive class weighting
• Exponential Moving Average (EMA) for better generalization
• CutMix, Mixup, and RandAugment augmentation
• Mixed precision training (AMP)
• Cosine annealing with warm restarts
• Test-Time Augmentation (TTA)
• Early stopping with patience
• Full classification report: AUC, Recall, Precision, F1, Confusion Matrix

Quick Start Training

python train_adv_quickstart.py --dataset "DeepfakeVsReal/Dataset"

---

Ethical Assessment

UAIDE includes comprehensive ethical assessment for AI-generated content with 11 safety checks:

Check	Description	Auto-Flag
Face Detection	Detects human faces in AI content	FACES_DETECTED
NSFW Detection	Skin exposure and explicit content analysis	NSFW_CONTENT
Age Estimation	Protects against minor exploitation	POTENTIAL_MINOR
Celebrity Detection	Image quality and symmetry analysis for impersonation	POTENTIAL_CELEBRITY
Emotion Analysis	Facial expression manipulation scoring	EMOTIONAL_MANIPULATION
Metadata Analysis	EXIF tampering and AI generation markers	AI_METADATA_MARKERS
Watermark Detection	Detects watermark removal signs	WATERMARK_REMOVAL
Hate Symbol Detection	Geometric patterns and color analysis	POTENTIAL_HATE_SYMBOL
Text Overlay Analysis	Misleading text and clickbait detection	MISLEADING_TEXT
Document Forgery	ID/document detection and forgery indicators	DOCUMENT_DETECTED
Jurisdiction Compliance	Legal warnings for EU, US, UK, China, Korea, India	Region-specific

Ethical Classification

ETHICAL (Risk < 50%)
├── Safe for educational/artistic use
├── Clear synthetic artifacts visible
└── No privacy/consent concerns

UNETHICAL (Risk > 50%)
├── Faces detected → Consent required
├── High-quality deepfake → Misuse potential
├── NSFW content → Distribution restricted
└── Document detected → Forgery risk

---

Video Detection

Train ResNet-50 + LSTM on video datasets:

python train_video.py \
    --dataset "SDFVD/SDFVD" \
    --out video_resnet_lstm.pt \
    --frames_per_video 16 \
    --epochs 10

Video Features:

• Temporal modeling with LSTM
• Frame-level Grad-CAM visualization
• Face tracking and cropping
• Multi-frame analysis

---

Project Structure

UAIDE/
├── app.py                    # Gradio web interface
├── train_adv.py              # Advanced EfficientNet training
├── train_adv_quickstart.py   # Quick training script
├── train_video.py            # Video model training
├── detector.py               # Heuristic patch-based detector
├── ethical_assessment.py     # Comprehensive ethical checks
├── video_model.py            # ResNet-LSTM architecture
├── video_data.py             # Video frame extraction
├── models_adv/               # Trained model weights
│   ├── best_model_weights.pt # EfficientNet-B2 + FFT weights
│   └── config.json           # Model configuration
├── models_v2/                # Alternative model versions
├── DeepfakeVsReal/Dataset/   # Image dataset
│   ├── Train/
│   ├── Validation/
│   └── Test/
└── requirements.txt

---

Technical Specifications

Component	Specification
Framework	PyTorch 2.0+ with CUDA support
Backbone	EfficientNet-B2 (timm)
Backbone Features	1408 dimensions
FFT Features	512 dimensions (12 raw → 512 processed)
Fusion Dimension	1920 → 1024 → 512 → 2
Input Size	224 x 224 (center crop, no resize)
Normalization	ImageNet mean/std
Loss Function	Focal Loss (gamma=2.0)
Optimizer	AdamW (lr=1e-4, weight_decay=1e-4)
Scheduler	Cosine Annealing with Warm Restarts
Regularization	Dropout (0.4), LayerNorm, EMA
Augmentation	CutMix, Mixup, RandAugment
Precision	Mixed (FP16/FP32 AMP)

---

Dependencies

torch>=2.0.0
torchvision>=0.15.0
timm>=0.9.0
gradio>=4.0.0
opencv-python>=4.8.0
numpy>=1.24.0
scipy>=1.10.0
scikit-learn>=1.3.0
scikit-image>=0.21.0
Pillow>=10.0.0
pandas>=2.0.0
joblib>=1.3.0

Install with:

pip install -r requirements.txt
pip install timm

---

Datasets

Dataset	Location	Contents
DeepfakeVsReal	DeepfakeVsReal/Dataset/	Train/Validation/Test with Real and Fake folders
AI vs Real img	AI vs Real img/	AI-generated art vs real art
SDFVD	SDFVD/SDFVD/	Video dataset (videos_real/, videos_fake/)

---

Usage

Web Interface

python app.py

Opens Gradio UI with:

• Image Tab: Upload images for deepfake detection
• Video Tab: Analyze videos with frame-by-frame detection
• Ethical Assessment: Automatic risk scoring and recommendations

Python API

import torch
import numpy as np
from PIL import Image
from torchvision import transforms

# Load model
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = torch.load('models_adv/best_model_weights.pt', map_location=device)
model.eval()

# Prepare image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

img = Image.open('test_image.jpg').convert('RGB')
img_tensor = transform(img).unsqueeze(0).to(device)

# Predict
with torch.no_grad():
    output = model(img_tensor)
    prob = torch.softmax(output, dim=1)
    fake_prob = prob[0, 1].item()

print(f"AI-Generated Probability: {fake_prob:.2%}")

Ethical Assessment API

from ethical_assessment import EthicalAssessment, format_ethical_report
import numpy as np

# Load image as numpy array (H, W, C) normalized to [0, 1]
img_arr = np.array(Image.open('image.jpg')).astype(np.float32) / 255.0

# Run comprehensive assessment
assessment = EthicalAssessment.assess(img_arr)

# Print report
print(format_ethical_report(assessment))

# Access specific checks
print(f"Is Ethical: {assessment['is_ethical']}")
print(f"Risk Score: {assessment['risk_score']:.1%}")
print(f"Flags: {assessment['flags']}")
print(f"NSFW Score: {assessment['checks']['nsfw']['nsfw_score']:.2f}")

---

Troubleshooting

Problem	Solution
ModuleNotFoundError: timm	pip install timm
CUDA out of memory	Reduce batch size or use CPU
Model loading fails	Check models_adv/best_model_weights.pt exists
Slow inference	Enable GPU with torch.cuda.is_available()
Real images flagged as AI	Adjust ethical threshold slider in UI

---

Citation

If you use UAIDE in your research, please cite:

@software{uaide2024,
  title={UAIDE: AI-Generated Image and Video Detection},
  author={Deshna},
  year={2024},
  url={https://github.com/Deshnaa2007/UAIDE}
}

---

License

MIT License - See LICENSE file for details.

---

Repository

https://github.com/Deshna24/UAIDE

UAIDE — AI-Generated Image & Video Detection

UAIDE is a deepfake detection toolkit that combines a ResNet-50 + FFT feature-fusion model with Grad-CAM explainability, ethical assessment, and a Gradio web interface. It supports both image and video analysis, with auto-calibrated thresholds to minimise false positives on real images.

Quick Start

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# Launch the web app
python app.py

The app loads the trained fusion model (model_fusion_best.joblib_info.pkl), auto-calibrates a detection threshold against the validation set, and opens a Gradio UI where you can upload images for analysis.

How It Works

Architecture

The primary model (DeepfakeFeatureFusion) fuses two streams:

1. Spatial stream — ResNet-50 backbone extracts high-level visual features.
2. Frequency stream — Block-wise FFT (16x16 blocks) produces per-block magnitude and phase statistics, processed by a small CNN. Phase information captures alignment errors that AI generators leave behind.

A cross-attention layer lets the spatial stream guide where to look for frequency anomalies. The fused representation passes through a classification head with progressive dropout.

Preprocessing

Images are padded and center-cropped (224x224) instead of resized, preserving the original pixel-level detail that resize-based pipelines destroy.

Training Augmentations

• Random crop, flip, rotation, colour jitter, affine
• JPEG compression (quality 50–95) and Gaussian blur — simulates real-world image degradation so the model works on compressed uploads

Threshold Calibration

At startup, app.py runs the real validation images through the model and sets a threshold at the 95th percentile of their fake-class probabilities, capping the false-positive rate on real images at ~5%.

Trained Model

The default model is stored across these files:

File	Contents
model_fusion_best.joblib_info.pkl	Model metadata (type, state-dict path, optimal threshold)
model_fusion_best.joblib_best_improved	PyTorch state dict

Stored metrics from training:

• Best F1: 0.792
• Best fake recall: 0.681
• Optimal threshold: 0.371

Run python evaluate_model.py to compute full accuracy and ROC AUC on the Validation and Test splits.

Training

Default settings reproduce the shipped model:

python train.py --dataset "DeepfakeVsReal/Dataset" --max_per_class 1000

This trains the fusion model type by default and writes model_fusion_best.joblib_best_improved + model_fusion_best.joblib_info.pkl.

Other model types are available via --model:

--model	Architecture
fusion (default)	ResNet-50 + block-wise FFT + cross-attention
resnet	ResNet-50 transfer learning
cnn	Custom 4-layer CNN
cnn_kfold	CNN with K-fold cross-validation
fusion_dual	Dual-stream residual + ResNet
rf	Random Forest (handcrafted features)
gb	XGBoost with GPU support

Video Detection

Train a ResNet-50 + LSTM on the SDFVD dataset:

python train_video.py --dataset "SDFVD/SDFVD" --out video_resnet_lstm.pt --frames_per_video 16 --epochs 10

Run inference with Grad-CAM overlays:

python predict_video.py --video path\to\video.mp4 --checkpoint video_resnet_lstm.pt

Project Structure

UAIDE/
├── app.py                      # Gradio web interface
├── train.py                    # Training (all model types)
├── train_video.py              # Video model training
├── predict_video.py            # Video inference + Grad-CAM
├── detector.py                 # Heuristic patch-based detector
├── ethical_assessment.py        # Ethical risk scoring
├── evaluate_model.py           # Validation / Test evaluation
├── compare_models.py           # Side-by-side model comparison
├── diagnose_misclassification.py  # Threshold sweep & FP analysis
├── print_report.py             # Ethical classification report
├── video_model.py              # ResNet-LSTM architecture
├── video_data.py               # Video frame extraction
├── check_gpu.py                # GPU availability check
├── requirements.txt
├── DeepfakeVsReal/Dataset/     # Train / Validation / Test splits
├── AI vs Real img/             # Additional AI art dataset
├── SDFVD/                      # Video dataset
└── model_fusion_best.*         # Trained model artifacts

Datasets

Dataset	Location	Contents
DeepfakeVsReal	DeepfakeVsReal/Dataset/	Train / Validation / Test splits with Real and Fake folders
AI vs Real img	AI vs Real img/	AI-generated art vs real art
SDFVD	SDFVD/SDFVD/	videos_real/ and videos_fake/ for video detection

Technical Details

• Framework: PyTorch with CUDA support
• Backbone: ResNet-50 (ImageNet pretrained)
• Frequency features: Block-wise FFT magnitude + phase (16x16 blocks, 6-channel input)
• Attention: Multi-head cross-attention (8 heads, 512-dim)
• Loss: Focal loss (alpha=0.8, gamma=2.5, label smoothing=0.15)
• Optimiser: AdamW with per-layer learning rates and cosine annealing
• Input: 224x224 center crop (no resize)
• Regularisation: Dropout (0.3–0.5), batch normalisation, weight decay, mixup (alpha=0.2)

Troubleshooting

Problem	Fix
CUDA out of memory	Reduce --max_per_class or use smaller batch size
Real images flagged as AI	The auto-threshold should handle this; if not, lower TARGET_REAL_FPR in app.py
Grad-CAM errors	Ensure opencv-python is installed
Slow startup	Threshold calibration runs on validation set at launch; reduce MAX_CALIB_IMAGES in app.py

Repository

https://github.com/Deshnaa2007/UAIDE

65ab9814191b6bb448da441c53a768594e7d1d59