---
title: Marquee
emoji: 🎙️
colorFrom: purple
colorTo: pink
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
license: apache-2.0
short_description: Turn any clip into a broadcast starring your friends
tags:
  - build-small-hackathon
  - track:backyard
  - achievement:offgrid
  - achievement:offbrand
---

# Marquee 🎙️

Drop a clip of friends doing anything. Marquee detects who's in it, you give each person a name, pick a commentary style, and an AI commentator calls the action play-by-play — by name, in sync — as a broadcast you can play back and export as an MP4.

## What it does

1. **Scan** — Upload a clip (≤60s). OpenVINO detects and clusters people into tracks. You get a roster of face crops to name.
2. **Generate** — Qwen2.5-VL-7B watches key moments from the clip and writes commentary lines timed to the action, in whichever style you picked.
3. **Voice** — Each line gets spoken by a TTS model. Two options: Chatterbox (fast, runs on CPU) or Orpheus (slower, more expressive, requires GPU and a HuggingFace token).
4. **Export** — ffmpeg mixes the TTS audio over the original clip audio, burns in subtitles, and gives you a download.

## Setup

### Hardware

Must be **ZeroGPU**. Qwen2.5-VL-7B and Orpheus both need a GPU. Set it under **Settings → Hardware → ZeroGPU** in your Space. On a CPU-only Space the VLM is unusable.

### Using Orpheus TTS

Orpheus uses `canopylabs/orpheus-3b-0.1-ft`, which is a **gated model** on HuggingFace. Two steps to unlock it:

1. Go to https://huggingface.co/canopylabs/orpheus-3b-0.1-ft and accept the access request. Takes about a minute to be approved.

2. Add your HuggingFace token as a **Space secret** (not a repo file):
   - Go to **Settings → Variables and Secrets → New Secret**
   - Name: `HF_TOKEN`
   - Value: a token from https://huggingface.co/settings/tokens with at least **Read** scope

Once the secret is set, restart the Space. The model will download on first use (~7GB). After that it's cached for the session.

Chatterbox (the default) needs no token and no extra setup.

### Demo
https://youtu.be/f2I8m6ardkU

### Local dev

```bash
pip install -r requirements.txt
python -c "from ov_models import download_models; download_models()"
python app.py
# → http://localhost:7860
```

Qwen lazy-loads on the first generate request. Orpheus lazy-loads on first TTS request with that model selected. Expect a slow first run.

For local Orpheus use you'll need to be logged in: `huggingface-cli login`.

## How it's wired

The backend is a `gradio.Server` (a FastAPI subclass). This gives us ZeroGPU + the Gradio queue while letting custom routes take priority. The UI is a static HTML page served at `/`. The scan is a plain `fetch()` POST; commentary goes through the Gradio JS client so it hits the queue and ZeroGPU sees the `@spaces.GPU` decorator.

```
Marquee.html + marquee.css + marquee.js  →  served at "/"
  │
  ├─ fetch POST /api/scan      upload → normalize → detect+cluster → session + roster
  ├─ fetch GET  /video/{sid}   streams the normalized mp4
  ├─ gradio client /generate   key events → annotated frames → Qwen (ZeroGPU) → script
  └─ fetch POST /api/tts       script lines → Chatterbox or Orpheus (ZeroGPU) → WAV
  └─ fetch POST /api/export    WAVs + video → ffmpeg mix → MP4 download
```

Session state (video path, tracks, motion data) lives in a server-side dict keyed by `session_id`. Nothing heavy goes back and forth to the client between steps.

## Files

| File | What it does |
|------|-------------|
| `app.py` | FastAPI routes, session management, UI assembly |
| `Marquee.html` / `marquee.css` / `marquee.js` | The UI |
| `commentary.py` | Qwen2.5-VL personas, prompt, JSON parse |
| `tts.py` | Chatterbox + Orpheus generation, collision fix, WAV encoding |
| `faces.py` | OpenVINO detect → cluster, frame annotation |
| `events.py` | Motion-based key event selection |
| `video.py` | Rotation-aware ffmpeg normalize to 720p |
| `ov_models.py` | OpenVINO IR download + runtime wrappers |

## Models

- **Qwen2.5-VL-7B-Instruct** — commentary generation (ZeroGPU)
- **Chatterbox 0.5B** — default TTS, MIT license, CPU inference
- **Orpheus 3B** (`canopylabs/orpheus-3b-0.1-ft`) — optional TTS, more expressive, requires HF token + ZeroGPU
- **face-detection-retail-0004** + **person-detection-retail-0013** — OpenVINO IR, CPU, Intel OMZ

## Social Media Post
- Youtube: https://www.youtube.com/@muflihma
- Instagram: https://www.instagram.com/0xcure