video_bundle/README.md

<<<<<<< HEAD
# UAIDE - AI-Generated Image & Video Detection

A state-of-the-art deepfake detection system combining **EfficientNet-B2 + FFT Fusion** with comprehensive ethical assessment, Grad-CAM explainability, and a Gradio web interface. Supports both image and video analysis with multi-layered content safety checks.

---

## Model Performance Summary

| Metric | Value |
|--------|-------|
| **Validation Accuracy** | 95.07% |
| **Validation AUC** | 0.9908 |
| **Validation Loss** | 0.0471 |
| **Train Accuracy** | 83.91% |
| **Train Loss** | 0.0770 |

---

## Quick Start

```bash
# Create virtual environment
python -m venv .venv

# Activate (Windows PowerShell)
.\.venv\Scripts\Activate.ps1

# Activate (Linux/Mac)
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt
pip install timm  # Required for EfficientNet

# Launch the web app
python app.py
```

The app loads the trained model from `models_adv/best_model_weights.pt` and opens a Gradio UI at `http://localhost:7860`.

---

## Architecture

### EfficientNet-B2 + FFT Fusion Model

The primary model fuses two complementary streams:

```
Input Image (224x224)
        │
        ├──────────────────────┬────────────────────────┐
        ▼                      ▼                        │
┌───────────────┐      ┌───────────────┐               │
│ EfficientNet  │      │     FFT       │               │
│     B2        │      │  Extractor    │               │
│  (1408 dim)   │      │  (512 dim)    │               │
└───────┬───────┘      └───────┬───────┘               │
        │                      │                        │
        └──────────┬───────────┘                        │
                   ▼                                    │
           ┌─────────────┐                              │
           │   Fusion    │                              │
           │   Layer     │                              │
           │ (1920→1024) │                              │
           └──────┬──────┘                              │
                  ▼                                     │
           ┌─────────────┐                              │
           │ Classifier  │◄─────────────────────────────┘
           │  (2 class)  │        Grad-CAM
           └─────────────┘
```

**Key Components:**

1. **Spatial Stream (EfficientNet-B2)**: Extracts high-level visual features using ImageNet-pretrained weights via `timm` library
2. **Frequency Stream (FFT)**: Analyzes frequency domain patterns that reveal GAN artifacts:
   - Magnitude statistics (mean, std, max, min)
   - Frequency band energies (low, mid-low, mid-high, high)
   - Threshold-based features for GAN fingerprint detection
3. **Fusion Layer**: Combines both streams with LayerNorm, GELU activation, and progressive dropout

---

## Training

### Advanced Training Script (`train_adv.py`)

```bash
python train_adv.py \
    --dataset "DeepfakeVsReal/Dataset" \
    --backbone efficientnet_b2 \
    --epochs 100 \
    --batch_size 8 \
    --lr 0.0001
```

**Training Features:**
- EfficientNet backbone (B0, B2, B4 supported)
- FFT-based frequency domain analysis
- Focal loss with adaptive class weighting
- Exponential Moving Average (EMA) for better generalization
- CutMix, Mixup, and RandAugment augmentation
- Mixed precision training (AMP)
- Cosine annealing with warm restarts
- Test-Time Augmentation (TTA)
- Early stopping with patience
- Full classification report: AUC, Recall, Precision, F1, Confusion Matrix

### Quick Start Training

```bash
python train_adv_quickstart.py --dataset "DeepfakeVsReal/Dataset"
```

---

## Ethical Assessment

UAIDE includes comprehensive ethical assessment for AI-generated content with **11 safety checks**:

| Check | Description | Auto-Flag |
|-------|-------------|-----------|
| **Face Detection** | Detects human faces in AI content | `FACES_DETECTED` |
| **NSFW Detection** | Skin exposure and explicit content analysis | `NSFW_CONTENT` |
| **Age Estimation** | Protects against minor exploitation | `POTENTIAL_MINOR` |
| **Celebrity Detection** | Image quality and symmetry analysis for impersonation | `POTENTIAL_CELEBRITY` |
| **Emotion Analysis** | Facial expression manipulation scoring | `EMOTIONAL_MANIPULATION` |
| **Metadata Analysis** | EXIF tampering and AI generation markers | `AI_METADATA_MARKERS` |
| **Watermark Detection** | Detects watermark removal signs | `WATERMARK_REMOVAL` |
| **Hate Symbol Detection** | Geometric patterns and color analysis | `POTENTIAL_HATE_SYMBOL` |
| **Text Overlay Analysis** | Misleading text and clickbait detection | `MISLEADING_TEXT` |
| **Document Forgery** | ID/document detection and forgery indicators | `DOCUMENT_DETECTED` |
| **Jurisdiction Compliance** | Legal warnings for EU, US, UK, China, Korea, India | Region-specific |

### Ethical Classification

```
ETHICAL (Risk < 50%)
├── Safe for educational/artistic use
├── Clear synthetic artifacts visible
└── No privacy/consent concerns

UNETHICAL (Risk > 50%)
├── Faces detected → Consent required
├── High-quality deepfake → Misuse potential
├── NSFW content → Distribution restricted
└── Document detected → Forgery risk
```

---

## Video Detection

Train ResNet-50 + LSTM on video datasets:

```bash
python train_video.py \
    --dataset "SDFVD/SDFVD" \
    --out video_resnet_lstm.pt \
    --frames_per_video 16 \
    --epochs 10
```

**Video Features:**
- Temporal modeling with LSTM
- Frame-level Grad-CAM visualization
- Face tracking and cropping
- Multi-frame analysis

---

## Project Structure

```
UAIDE/
├── app.py                    # Gradio web interface
├── train_adv.py              # Advanced EfficientNet training
├── train_adv_quickstart.py   # Quick training script
├── train_video.py            # Video model training
├── detector.py               # Heuristic patch-based detector
├── ethical_assessment.py     # Comprehensive ethical checks
├── video_model.py            # ResNet-LSTM architecture
├── video_data.py             # Video frame extraction
├── models_adv/               # Trained model weights
│   ├── best_model_weights.pt # EfficientNet-B2 + FFT weights
│   └── config.json           # Model configuration
├── models_v2/                # Alternative model versions
├── DeepfakeVsReal/Dataset/   # Image dataset
│   ├── Train/
│   ├── Validation/
│   └── Test/
└── requirements.txt
```

---

## Technical Specifications

| Component | Specification |
|-----------|---------------|
| **Framework** | PyTorch 2.0+ with CUDA support |
| **Backbone** | EfficientNet-B2 (timm) |
| **Backbone Features** | 1408 dimensions |
| **FFT Features** | 512 dimensions (12 raw → 512 processed) |
| **Fusion Dimension** | 1920 → 1024 → 512 → 2 |
| **Input Size** | 224 x 224 (center crop, no resize) |
| **Normalization** | ImageNet mean/std |
| **Loss Function** | Focal Loss (gamma=2.0) |
| **Optimizer** | AdamW (lr=1e-4, weight_decay=1e-4) |
| **Scheduler** | Cosine Annealing with Warm Restarts |
| **Regularization** | Dropout (0.4), LayerNorm, EMA |
| **Augmentation** | CutMix, Mixup, RandAugment |
| **Precision** | Mixed (FP16/FP32 AMP) |

---

## Dependencies

```
torch>=2.0.0
torchvision>=0.15.0
timm>=0.9.0
gradio>=4.0.0
opencv-python>=4.8.0
numpy>=1.24.0
scipy>=1.10.0
scikit-learn>=1.3.0
scikit-image>=0.21.0
Pillow>=10.0.0
pandas>=2.0.0
joblib>=1.3.0
```

Install with:
```bash
pip install -r requirements.txt
pip install timm
```

---

## Datasets

| Dataset | Location | Contents |
|---------|----------|----------|
| DeepfakeVsReal | `DeepfakeVsReal/Dataset/` | Train/Validation/Test with Real and Fake folders |
| AI vs Real img | `AI vs Real img/` | AI-generated art vs real art |
| SDFVD | `SDFVD/SDFVD/` | Video dataset (`videos_real/`, `videos_fake/`) |

---

## Usage

### Web Interface

```bash
python app.py
```

Opens Gradio UI with:
- **Image Tab**: Upload images for deepfake detection
- **Video Tab**: Analyze videos with frame-by-frame detection
- **Ethical Assessment**: Automatic risk scoring and recommendations

### Python API

```python
import torch
import numpy as np
from PIL import Image
from torchvision import transforms

# Load model
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = torch.load('models_adv/best_model_weights.pt', map_location=device)
model.eval()

# Prepare image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

img = Image.open('test_image.jpg').convert('RGB')
img_tensor = transform(img).unsqueeze(0).to(device)

# Predict
with torch.no_grad():
    output = model(img_tensor)
    prob = torch.softmax(output, dim=1)
    fake_prob = prob[0, 1].item()

print(f"AI-Generated Probability: {fake_prob:.2%}")
```

### Ethical Assessment API

```python
from ethical_assessment import EthicalAssessment, format_ethical_report
import numpy as np

# Load image as numpy array (H, W, C) normalized to [0, 1]
img_arr = np.array(Image.open('image.jpg')).astype(np.float32) / 255.0

# Run comprehensive assessment
assessment = EthicalAssessment.assess(img_arr)

# Print report
print(format_ethical_report(assessment))

# Access specific checks
print(f"Is Ethical: {assessment['is_ethical']}")
print(f"Risk Score: {assessment['risk_score']:.1%}")
print(f"Flags: {assessment['flags']}")
print(f"NSFW Score: {assessment['checks']['nsfw']['nsfw_score']:.2f}")
```

---

## Troubleshooting

| Problem | Solution |
|---------|----------|
| `ModuleNotFoundError: timm` | `pip install timm` |
| CUDA out of memory | Reduce batch size or use CPU |
| Model loading fails | Check `models_adv/best_model_weights.pt` exists |
| Slow inference | Enable GPU with `torch.cuda.is_available()` |
| Real images flagged as AI | Adjust ethical threshold slider in UI |

---

## Citation

If you use UAIDE in your research, please cite:

```bibtex
@software{uaide2024,
  title={UAIDE: AI-Generated Image and Video Detection},
  author={Deshna},
  year={2024},
  url={https://github.com/Deshnaa2007/UAIDE}
}
```

---

## License

MIT License - See LICENSE file for details.

---

## Repository

https://github.com/Deshna24/UAIDE
=======
# UAIDE — AI-Generated Image & Video Detection

UAIDE is a deepfake detection toolkit that combines a ResNet-50 + FFT feature-fusion model with Grad-CAM explainability, ethical assessment, and a Gradio web interface. It supports both image and video analysis, with auto-calibrated thresholds to minimise false positives on real images.

## Quick Start

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# Launch the web app
python app.py
```

The app loads the trained fusion model (`model_fusion_best.joblib_info.pkl`), auto-calibrates a detection threshold against the validation set, and opens a Gradio UI where you can upload images for analysis.

## How It Works

### Architecture

The primary model (`DeepfakeFeatureFusion`) fuses two streams:

1. **Spatial stream** — ResNet-50 backbone extracts high-level visual features.
2. **Frequency stream** — Block-wise FFT (16x16 blocks) produces per-block magnitude and phase statistics, processed by a small CNN. Phase information captures alignment errors that AI generators leave behind.

A **cross-attention** layer lets the spatial stream guide where to look for frequency anomalies. The fused representation passes through a classification head with progressive dropout.

### Preprocessing

Images are **padded and center-cropped** (224x224) instead of resized, preserving the original pixel-level detail that resize-based pipelines destroy.

### Training Augmentations

- Random crop, flip, rotation, colour jitter, affine
- JPEG compression (quality 50–95) and Gaussian blur — simulates real-world image degradation so the model works on compressed uploads

### Threshold Calibration

At startup, `app.py` runs the real validation images through the model and sets a threshold at the 95th percentile of their fake-class probabilities, capping the false-positive rate on real images at ~5%.

## Trained Model

The default model is stored across these files:

| File | Contents |
|------|----------|
| `model_fusion_best.joblib_info.pkl` | Model metadata (type, state-dict path, optimal threshold) |
| `model_fusion_best.joblib_best_improved` | PyTorch state dict |

Stored metrics from training:
- **Best F1**: 0.792
- **Best fake recall**: 0.681
- **Optimal threshold**: 0.371

Run `python evaluate_model.py` to compute full accuracy and ROC AUC on the Validation and Test splits.

## Training

Default settings reproduce the shipped model:

```powershell
python train.py --dataset "DeepfakeVsReal/Dataset" --max_per_class 1000
```

This trains the `fusion` model type by default and writes `model_fusion_best.joblib_best_improved` + `model_fusion_best.joblib_info.pkl`.

Other model types are available via `--model`:

| `--model` | Architecture |
|-----------|-------------|
| `fusion` (default) | ResNet-50 + block-wise FFT + cross-attention |
| `resnet` | ResNet-50 transfer learning |
| `cnn` | Custom 4-layer CNN |
| `cnn_kfold` | CNN with K-fold cross-validation |
| `fusion_dual` | Dual-stream residual + ResNet |
| `rf` | Random Forest (handcrafted features) |
| `gb` | XGBoost with GPU support |

## Video Detection

Train a ResNet-50 + LSTM on the SDFVD dataset:

```powershell
python train_video.py --dataset "SDFVD/SDFVD" --out video_resnet_lstm.pt --frames_per_video 16 --epochs 10
```

Run inference with Grad-CAM overlays:

```powershell
python predict_video.py --video path\to\video.mp4 --checkpoint video_resnet_lstm.pt
```

## Project Structure

```
UAIDE/
├── app.py                      # Gradio web interface
├── train.py                    # Training (all model types)
├── train_video.py              # Video model training
├── predict_video.py            # Video inference + Grad-CAM
├── detector.py                 # Heuristic patch-based detector
├── ethical_assessment.py        # Ethical risk scoring
├── evaluate_model.py           # Validation / Test evaluation
├── compare_models.py           # Side-by-side model comparison
├── diagnose_misclassification.py  # Threshold sweep & FP analysis
├── print_report.py             # Ethical classification report
├── video_model.py              # ResNet-LSTM architecture
├── video_data.py               # Video frame extraction
├── check_gpu.py                # GPU availability check
├── requirements.txt
├── DeepfakeVsReal/Dataset/     # Train / Validation / Test splits
├── AI vs Real img/             # Additional AI art dataset
├── SDFVD/                      # Video dataset
└── model_fusion_best.*         # Trained model artifacts
```

## Datasets

| Dataset | Location | Contents |
|---------|----------|----------|
| DeepfakeVsReal | `DeepfakeVsReal/Dataset/` | Train / Validation / Test splits with Real and Fake folders |
| AI vs Real img | `AI vs Real img/` | AI-generated art vs real art |
| SDFVD | `SDFVD/SDFVD/` | `videos_real/` and `videos_fake/` for video detection |

## Technical Details

- **Framework**: PyTorch with CUDA support
- **Backbone**: ResNet-50 (ImageNet pretrained)
- **Frequency features**: Block-wise FFT magnitude + phase (16x16 blocks, 6-channel input)
- **Attention**: Multi-head cross-attention (8 heads, 512-dim)
- **Loss**: Focal loss (alpha=0.8, gamma=2.5, label smoothing=0.15)
- **Optimiser**: AdamW with per-layer learning rates and cosine annealing
- **Input**: 224x224 center crop (no resize)
- **Regularisation**: Dropout (0.3–0.5), batch normalisation, weight decay, mixup (alpha=0.2)

## Troubleshooting

| Problem | Fix |
|---------|-----|
| CUDA out of memory | Reduce `--max_per_class` or use smaller batch size |
| Real images flagged as AI | The auto-threshold should handle this; if not, lower `TARGET_REAL_FPR` in `app.py` |
| Grad-CAM errors | Ensure `opencv-python` is installed |
| Slow startup | Threshold calibration runs on validation set at launch; reduce `MAX_CALIB_IMAGES` in `app.py` |

## Repository

https://github.com/Deshnaa2007/UAIDE
>>>>>>> 65ab9814191b6bb448da441c53a768594e7d1d59