docs/ARCHITECTURE.md

# ARCHITECTURE.md — Midnight Static

## System diagram

```
                      ┌─────────────────────────────────────────────┐
                      │              GRADIO SPACE (ZeroGPU)         │
                      │                                             │
 mic ──► Nemotron ASR ─┐                                            │
                       ├─► Nemotron Nano 4B + LoRA ──► Script JSON  │
 text ─────────────────┘      (vLLM, guided JSON)        │          │
                                                         ▼          │
                      │   ┌──────────── fan-out ────────────┐       │
                      │   │ TTS: Kokoro per line             │       │
                      │   │ SFX: embed → cache match ──hit──►│       │
                      │   │        └─miss─► SAO Small gen    │       │
                      │   │ Music: embed → bed/sting library │       │
                      │   └──────────────┬───────────────────┘       │
                      │                  ▼                           │
                      │        Mixer (pydub/ffmpeg)                  │
                      │                  ▼                           │
                      │   broadcast.mp3 + (stretch) FLUX poster      │
                      └─────────────────────────────────────────────┘

 ┌──────────────────────── MODAL (training & batch only) ───────────┐
 │ gen_dataset.py  → 400 synthetic scripts (Codex-driven sessions)  │
 │ finetune.py     → LoRA on Nemotron Nano 4B (Unsloth, A100)       │
 │ batch_audio.py  → SFX library (~150 clips) + music beds/stings   │
 │                   (6 genres × 4 beds × 3 stings) + embeddings    │
 └──────────────────────────────────────────────────────────────────┘
```

## Model inventory (Tiny Titan evidence — keep exact and current)

| Model | Params | Role | Runtime location |
|---|---|---|---|
| Nemotron 3 Nano 4B + LoRA | 4.0B | scriptwriter | ZeroGPU (vLLM) |
| Nemotron 3 ASR | <1B | call-in transcription | ZeroGPU |
| Kokoro-82M | 0.082B | TTS, all voices | ZeroGPU (near-CPU fast) |
| MusicGen-small | 0.30B | music (batch only) | Modal batch |
| Stable Audio Open Small | ~0.34B | SFX (batch + cache-miss) | Modal batch + ZeroGPU |
| Embedding model (e.g. bge-small, 33M) | 0.033B | SFX/music matching | ZeroGPU CPU |
| FLUX.2 Klein 4B (stretch) | 4.0B | episode posters | ZeroGPU, async |
| VoxCPM2 (gate G1) | <4B (verify) | TTS co-engine | ZeroGPU |

Ceiling check: every model individually ≤4B ✅ (Tiny Titan), sum ≈ 9B ≪ 32B ✅.

## ZeroGPU strategy

- One Space, `@spaces.GPU(duration=...)` on innermost calls only:
  - `writer.generate()` duration≈30
  - `sfx.generate_miss()` duration≈25 (rare path)
  - `poster.generate()` duration≈30 (async, never blocks audio)
- Kokoro + embeddings run on CPU — do NOT burn GPU quota on them.
- Models lazy-load on first use, then persist in process globals.
- vLLM engine kept warm via module-level init guarded by a flag; if ZeroGPU
  cold-starts hurt (>20s), switch writer to transformers + xgrammar
  (slower tokens, faster init) — decide empirically Day 1.

## Asset library design (`assets/`)

```
assets/
  manifest.json        # [{id, kind: sfx|bed|sting, genre?, prompt,
                       #   file, embedding: [..]}]
  sfx/*.wav            # ~150 clips, 24kHz mono, 1–6s
  music/{genre}/*.wav  # 4 beds (30–45s, loopable) + 3 stings (3–6s) per genre
```

- Matching: embed the script's text prompt (bge-small), cosine vs manifest.
  - SFX: threshold ≥0.62 → use cached; else generate (then append to
    manifest at runtime — the library learns).
  - Music: ALWAYS nearest match within the script's genre, never generate
    at runtime (beds are too slow to generate live).
- Manifest + wavs ship in the Space repo via Git LFS (~80MB budget).

## Pipeline orchestration (`src/pipeline.py`)

- Async stages with a `StageEvent(stage, status, detail)` queue consumed by
  the UI frequency-scan loader. Stage order: `asr? → write → cast_validate →
  tts ∥ sfx ∥ music → mix → on_air → poster?`.
- TTS/SFX/music run concurrently (asyncio + thread pool; they're different
  devices: CPU/GPU/disk).
- Single retry policy: schema/cross-field validation failure → one retry
  with error appended; second failure → swap in genre fixture's structure
  with user's premise woven into title/logline (never show an error page;
  the station "improvises").

## Latency budget (p50, warm)

| Stage | Target |
|---|---|
| ASR (if voice) | 2s |
| Script (4B, ~1.4k tok) | 12s |
| TTS (~18 lines, CPU, parallel) | 5s |
| SFX (cache hits) | 0.3s |
| Music (library) | 0.1s |
| Mix + encode | 3s |
| **Total (text premise)** | **~21s** |
| Cache-miss SFX penalty | +8s each (≤2 misses typical) |

p50 ≤35s target from SPEC.md has ~14s headroom for cold starts.

## Failure modes & responses

| Failure | Response |
|---|---|
| vLLM OOM / engine crash | restart engine once; then transformers fallback path |
| ZeroGPU quota exhausted (judge traffic) | pre-cached showcases still play (CPU); banner: "Station at capacity — enjoy a rerun" |
| ffmpeg mix error | re-mix without SFX layer (dialogue+music only) |
| ASR garbage | show transcript for confirm/edit before writing |

## Security/content

- Premise input length-capped (300 chars) and passed through the writer's
  PG-13 system constraints; no user text is ever shell-interpolated
  (ffmpeg called with arg lists, never shell=True).