| --- |
| title: Marquee |
| emoji: 🎙️ |
| colorFrom: purple |
| colorTo: pink |
| sdk: gradio |
| sdk_version: 6.18.0 |
| app_file: app.py |
| license: apache-2.0 |
| short_description: Turn any clip into a broadcast starring your friends |
| tags: |
| - build-small-hackathon |
| --- |
| |
| # Marquee 🎙️ — *Everyone's a headliner.* |
|
|
| Drop a clip of friends doing anything. Marquee recognizes who's in it, you give |
| each person a stage name, pick a show style, and an AI commentator calls the |
| action play-by-play, by name, in sync — as a broadcast you can play and export. |
|
|
| ## ⚙️ Running it on a Space (read this first) |
|
|
| - **Hardware must be ZeroGPU.** The commentary model runs inside `@spaces.GPU`, |
| which only has a GPU on **Nvidia ZeroGPU** hardware. On a plain CPU Space the |
| 7B VLM has no GPU and is effectively unusable. Set it in **Settings → |
| Hardware → ZeroGPU**. |
| - **No secrets needed at runtime.** Everything downloads from public sources |
| (Qwen from the HF Hub, the face models from Intel's OMZ storage). Do **not** |
| commit any `HF_TOKEN`/`.env` — if you need one for `gradio deploy`, keep it |
| local and add it as a **Space secret**, never a file in the repo. |
| - **Nothing to pre-download.** `download_models()` fetches the three tiny |
| OpenVINO IR files at first boot (~5 MB total). First boot also warms up Qwen. |
|
|
| ## How it's wired (the important part) |
|
|
| The backend is a **`gradio.Server`** (a FastAPI subclass — Gradio 6). It gives us |
| ZeroGPU + the gradio runtime while letting our own routes take priority. The |
| custom UI is a **static HTML page** at `/`. The CPU-only scan is a plain |
| `fetch()` route; the GPU commentary is an `@app.api()` endpoint called through |
| the **Gradio JS client**, which is what wires it into the queue and lets ZeroGPU |
| detect the `@spaces.GPU` function. No iframe, no SSR fighting us for `/`. |
|
|
| ``` |
| Static UI (Marquee.html + marquee.css + marquee.js) ──► served at "/" |
| │ fetch() for scan · Gradio JS client for generate |
| ▼ |
| FastAPI (app.py) |
| ├─ POST /api/scan (file) -> normalize 720p -> detect+cluster faces |
| │ -> session + roster (face crops as data URLs) |
| ├─ GET /video/{sid} -> streams the normalized mp4 |
| └─ @app.api /generate (session, names, vibe)-> key events -> name-grounded frames |
| -> Qwen2.5-VL (ZeroGPU) -> {t,text} script |
| ``` |
|
|
| Heavy per-session state (identities, motion, normalized video path) lives in a |
| server-side `SESSIONS` dict keyed by the `session_id` the scan call returns, so |
| the two stateless client calls never round-trip embeddings. |
|
|
| Face recognition is used **only** to feed names to the VLM — boxes are never |
| shown. The broadcast player reads the real normalized `<video>` and syncs the |
| chyron + play-by-play rail to `video.currentTime`. |
|
|
| ## Files |
| - `app.py` — FastAPI backend + the two API endpoints + UI assembly |
| - `Marquee.html` / `marquee.css` / `marquee.js` — the custom UI (design + flow) |
| - `video.py` — rotation-aware 720p normalize (one upright source of truth) |
| - `faces.py` — OpenVINO detect → align → embed → cluster (CPU) |
| - `events.py` — motion-based key events |
| - `commentary.py` — Qwen2.5-VL personas (incl. Diva Hour) + strict JSON parse |
| - `ov_models.py` — OpenVINO IR auto-download + thin runtime wrappers |
|
|
| ## Models (all open) |
| - face-detection-retail-0004, landmarks-regression-retail-0009, |
| face-reidentification-retail-0095 (OpenVINO IR, CPU) |
| - Qwen2.5-VL-7B-Instruct (ZeroGPU) |
|
|
| ## North star (next iteration) |
| - TTS per line (hyped delivery) mixed under the clip's own audio. |
| - Rendered MP4 export with captions (and voice) burned in. |
| - Canvas hype FX on key moments (data's already there). |
|
|
| ## Local dev |
| ```bash |
| pip install -r requirements.txt |
| python -c "from ov_models import download_models; download_models()" |
| python app.py # http://localhost:7860 (Qwen lazy-loads on first generate) |
| ``` |
