README.md

---
title: LectureLens API
emoji: 🎥
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
license: mit
---

# LectureLens API

<div align="center">

![LectureLens](https://img.shields.io/badge/LectureLens-v1.0.0-6366f1?style=for-the-badge)
![FastAPI](https://img.shields.io/badge/FastAPI-0.111-009688?style=for-the-badge&logo=fastapi)
![Python](https://img.shields.io/badge/Python-3.11-3776AB?style=for-the-badge&logo=python)
![Docker](https://img.shields.io/badge/Docker-ready-2496ED?style=for-the-badge&logo=docker)
![HuggingFace](https://img.shields.io/badge/HuggingFace-Spaces-FFD21E?style=for-the-badge)

**Technical audio & video quality analysis for Zoom lecture recordings.**  
Part of the automated lecture QC pipeline alongside Speechmatics and n8n.

</div>

---

## What It Does

LectureLens receives a single audio or video file per request and returns:

| Output | Description |
|--------|-------------|
| **KPI metrics** | Loudness (LUFS), SNR, sharpness, brightness, clipping, freeze detection… |
| **Alerts** | Structured list with `severity`, `kpi`, `message`, `suggested_fix`, and optional `timestamp_range` |
| **Composite score** | A single 0–1 quality score derived from weighted KPIs |

It does **not** evaluate content organisation, transcript quality, or pedagogy — that's handled by a separate LLM step in n8n.

---

## Architecture Position

```
Zoom recording.completed webhook
        │
        ▼
  n8n Switch → verify signature → download video + per-speaker audio
        │
        ├──► LectureLens API  ◄── this service
        │       one call per file (video once, audio per speaker)
        │
        ├──► Speechmatics → transcript with timestamps
        │
        ▼
  n8n Code node → merge: LectureLens results + transcript
        │
        ▼
  Unified lecture quality report
```

---

## Tech Stack

| Tool | Purpose |
|------|---------|
| **FastAPI** + Uvicorn | API framework with auto `/docs` |
| **FFmpeg** (system binary) | `loudnorm`, `silencedetect`, `freezedetect`, `blackdetect`, `ffprobe` |
| **pyloudnorm** | ITU-R BS.1770-4 loudness (cross-check) |
| **librosa** | SNR / noise floor from raw waveform |
| **DNSMOS** (Microsoft ONNX) | Non-reference speech quality score (1–5) |
| **OpenCV** | Frame sampling, brightness histogram, Laplacian sharpness |
| **MUSIQ** (IQA-PyTorch) | Perceptual video quality score (0–1) |
| **PyYAML** | `thresholds.yaml` — edit thresholds without code changes |

All tools are **free, open-source, CPU-only** — no GPU required.

---

## Quick Start (Local)

### 1. Prerequisites

- Python 3.11+
- FFmpeg installed and on `PATH`  
  - macOS: `brew install ffmpeg`  
  - Ubuntu: `sudo apt install ffmpeg`  
  - Windows: download from [ffmpeg.org](https://ffmpeg.org/download.html) and add to PATH

### 2. Clone & Install

```bash
git clone https://github.com/YOUR_USERNAME/lecturelens-api.git
cd lecturelens-api
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# CPU-only PyTorch first (saves ~1GB vs full install)
pip install torch==2.2.2+cpu torchvision==0.17.2+cpu \
    --index-url https://download.pytorch.org/whl/cpu

pip install -r requirements.txt
```

### 3. Download DNSMOS Model

```bash
python scripts/download_dnsmos.py
```

This downloads Microsoft's `sig_bak_ovr.onnx` (~8 MB) into `dnsmos/`.

### 4. Configure

```bash
cp .env.example .env
# Edit .env and set LECTURELENS_API_KEY to a strong secret
```

### 5. Run

```bash
uvicorn app.main:app --reload --port 7860
```

Open **http://localhost:7860/docs** for the interactive Swagger UI.

---

## API Reference

### Authentication

All `/analyze` requests require the header:
```
X-API-Key: <your-secret-key>
```

### `GET /health`

No authentication required.

```json
{ "status": "ok", "version": "1.0.0" }
```

### `POST /analyze`

**Request** (`multipart/form-data`):

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file` | binary | ✅ | MP4/MOV (video) or M4A/WAV/MP3 (audio) |
| `media_type` | `"audio"` \| `"video"` | ✅ | Selects the analysis pipeline |
| `participant_label` | string | — | Speaker name — echoed back in the response |
| `language` | string | — | Default `ar` (reserved for future use) |

**Audio response example:**

```json
{
  "status": "completed",
  "media_type": "audio",
  "participant_label": "ali saad",
  "processing_time_seconds": 5.8,
  "metrics": {
    "integrated_loudness_lufs": -19.8,
    "true_peak_dbtp": -3.2,
    "clipped_samples_count": 0,
    "snr_db": 24.1,
    "loudness_range_lu": 6.4,
    "silence_segments": [{ "start": 12.4, "end": 14.1 }],
    "dnsmos_ovrl": 3.6
  },
  "alerts": [
    {
      "severity": "warning",
      "category": "audio",
      "kpi": "integrated_loudness_lufs",
      "message": "Integrated loudness is -19.8 LUFS — 3.8 LU below the target of -14 LUFS.",
      "suggested_fix": "Adjust microphone gain or apply loudness normalisation before uploading.",
      "timestamp_range": null
    }
  ],
  "overall_audio_score": 0.71,
  "overall_video_score": null
}
```

**Video response example:**

```json
{
  "status": "completed",
  "media_type": "video",
  "participant_label": null,
  "processing_time_seconds": 22.3,
  "metrics": {
    "resolution": "1920x1080",
    "resolution_consistent": true,
    "frame_rate_fps": 30.0,
    "dropped_frames_ratio": 0.002,
    "avg_brightness": 132.4,
    "avg_sharpness_laplacian": 184.7,
    "frozen_segments": [],
    "black_segments": [],
    "compression_artifact_score": 0.12,
    "musiq_quality_score": 0.78
  },
  "alerts": [],
  "overall_audio_score": null,
  "overall_video_score": 0.83
}
```

### Error Codes

| HTTP | `error_code` | Cause |
|------|-------------|-------|
| 400 | `INVALID_MEDIA_TYPE` | Unsupported file extension |
| 401 | `UNAUTHORIZED` | Missing or wrong `X-API-Key` |
| 413 | `FILE_TOO_LARGE` | Exceeds `MAX_FILE_SIZE_MB` |
| 422 | `CORRUPTED_FILE` | File could not be opened/decoded |
| 500 | `PROCESSING_FAILED` | Unexpected internal error |

---

## cURL Examples

```bash
# Health check
curl https://<space>.hf.space/health

# Analyze audio
curl -X POST https://<space>.hf.space/analyze \
  -H "X-API-Key: your-secret-key" \
  -F "file=@speaker_ali.m4a" \
  -F "media_type=audio" \
  -F "participant_label=ali saad"

# Analyze video
curl -X POST https://<space>.hf.space/analyze \
  -H "X-API-Key: your-secret-key" \
  -F "file=@lecture.mp4" \
  -F "media_type=video"
```

---

## KPI Reference

### Audio

| KPI | Tool | Target |
|-----|------|--------|
| `integrated_loudness_lufs` | ffmpeg loudnorm | −14 LUFS ± 2 LU |
| `true_peak_dbtp` | ffmpeg loudnorm | < −1.0 dBTP |
| `clipped_samples_count` | numpy | 0 |
| `snr_db` | librosa | > 20 dB |
| `loudness_range_lu` | ffmpeg loudnorm | 4–15 LU |
| `silence_segments` | ffmpeg silencedetect | No segment > 3s |
| `dnsmos_ovrl` | DNSMOS ONNX | > 3.0 / 5 |

### Video

| KPI | Tool | Target |
|-----|------|--------|
| `resolution_consistent` | ffprobe | `true` |
| `dropped_frames_ratio` | ffprobe | < 1% |
| `avg_brightness` | OpenCV | 80–180 |
| `avg_sharpness_laplacian` | OpenCV Laplacian | > 100 |
| `frozen_segments` | ffmpeg freezedetect | No segment > 2s |
| `black_segments` | ffmpeg blackdetect | None |
| `compression_artifact_score` | Gradient heuristic | < 0.3 |
| `musiq_quality_score` | MUSIQ (IQA-PyTorch) | > 0.6 |

---

## Thresholds

All alert thresholds live in [`thresholds.yaml`](thresholds.yaml).  
Edit and redeploy — no code changes needed.

---

## Deploy to Hugging Face Spaces

1. Create a new Space → **Docker** type
2. Push this repo to the Space
3. In **Settings → Variables and Secrets**, add:
   - `LECTURELENS_API_KEY` → your secret key
   - `MAX_FILE_SIZE_MB` → e.g. `500`
4. The Space will build automatically (~5–10 min first build)
5. Test with `GET /health`

> **Tip:** HF Spaces (free tier) sleep after inactivity.  
> Always call `GET /health` first from n8n to wake the Space before sending large files.

---

## n8n Integration

See the full n8n node configuration in the [API documentation](docs/n8n_integration.md) or refer to Section 6 of the original design specification.

**Quick reference:**

| Existing node | New node to add |
|---------------|----------------|
| `download video` | `Analyze Video Quality` (POST /analyze, media_type=video) |
| `download audeo` | `Analyze Audio Quality` (POST /analyze, media_type=audio) |
| `Merge Speakers` | `Aggregate Final Report` (Code node) |

---

## Running Tests

```bash
pytest tests/ -v
```

---

## Project Structure

```
lecturelens/
├── app/
│   ├── main.py              # FastAPI app + routes
│   ├── config.py            # Settings (env vars)
│   ├── auth.py              # X-API-Key middleware
│   ├── schemas.py           # Pydantic models
│   ├── alert_engine.py      # Alert generation + scoring
│   ├── utils.py             # FFmpeg helpers + temp files
│   └── analyzers/
│       ├── audio_analyzer.py
│       └── video_analyzer.py
├── dnsmos/
│   └── sig_bak_ovr.onnx    # DNSMOS model (download via script)
├── scripts/
│   └── download_dnsmos.py
├── tests/
│   ├── test_audio.py
│   ├── test_video.py
│   └── test_api.py
├── thresholds.yaml
├── Dockerfile
├── requirements.txt
└── .env.example
```

---

## Roadmap

- [x] v1.0 — Audio pipeline (Loudness, SNR, Clipping, Silence, DNSMOS)
- [x] v1.0 — Video pipeline (Brightness, Sharpness, Freeze, Black, MUSIQ)
- [x] v1.0 — Alert engine with `thresholds.yaml`
- [x] v1.0 — Hugging Face Docker deployment
- [ ] Phase 2 — Lip-sync offset detection (SyncNet)
- [ ] Phase 2 — Deeper compression artefact analysis
- [ ] Phase 2 — Unified report merging LectureLens + LLM content evaluation

---

## License

MIT — see [LICENSE](LICENSE).