docs/ARCHITECTURE.md

ARCHITECTURE.md — Midnight Static

System diagram

                      ┌─────────────────────────────────────────────┐
                      │              GRADIO SPACE (ZeroGPU)         │
                      │                                             │
 mic ──► Nemotron ASR ─┐                                            │
                       ├─► Nemotron Nano 4B + LoRA ──► Script JSON  │
 text ─────────────────┘      (vLLM, guided JSON)        │          │
                                                         ▼          │
                      │   ┌──────────── fan-out ────────────┐       │
                      │   │ TTS: Kokoro per line             │       │
                      │   │ SFX: embed → cache match ──hit──►│       │
                      │   │        └─miss─► SAO Small gen    │       │
                      │   │ Music: embed → bed/sting library │       │
                      │   └──────────────┬───────────────────┘       │
                      │                  ▼                           │
                      │        Mixer (pydub/ffmpeg)                  │
                      │                  ▼                           │
                      │   broadcast.mp3 + (stretch) FLUX poster      │
                      └─────────────────────────────────────────────┘

 ┌──────────────────────── MODAL (training & batch only) ───────────┐
 │ gen_dataset.py  → 400 synthetic scripts (Codex-driven sessions)  │
 │ finetune.py     → LoRA on Nemotron Nano 4B (Unsloth, A100)       │
 │ batch_audio.py  → SFX library (~150 clips) + music beds/stings   │
 │                   (6 genres × 4 beds × 3 stings) + embeddings    │
 └──────────────────────────────────────────────────────────────────┘

Model inventory (Tiny Titan evidence — keep exact and current)

Model	Params	Role	Runtime location
Nemotron 3 Nano 4B + LoRA	4.0B	scriptwriter	ZeroGPU (vLLM)
Nemotron 3 ASR	<1B	call-in transcription	ZeroGPU
Kokoro-82M	0.082B	TTS, all voices	ZeroGPU (near-CPU fast)
MusicGen-small	0.30B	music (batch only)	Modal batch
Stable Audio Open Small	~0.34B	SFX (batch + cache-miss)	Modal batch + ZeroGPU
Embedding model (e.g. bge-small, 33M)	0.033B	SFX/music matching	ZeroGPU CPU
FLUX.2 Klein 4B (stretch)	4.0B	episode posters	ZeroGPU, async
VoxCPM2 (gate G1)	<4B (verify)	TTS co-engine	ZeroGPU

Ceiling check: every model individually ≤4B ✅ (Tiny Titan), sum ≈ 9B ≪ 32B ✅.

ZeroGPU strategy

• One Space, @spaces.GPU(duration=...) on innermost calls only:
  • writer.generate() duration≈30
  • sfx.generate_miss() duration≈25 (rare path)
  • poster.generate() duration≈30 (async, never blocks audio)
• Kokoro + embeddings run on CPU — do NOT burn GPU quota on them.
• Models lazy-load on first use, then persist in process globals.
• vLLM engine kept warm via module-level init guarded by a flag; if ZeroGPU cold-starts hurt (>20s), switch writer to transformers + xgrammar (slower tokens, faster init) — decide empirically Day 1.

Asset library design (assets/)

assets/
  manifest.json        # [{id, kind: sfx|bed|sting, genre?, prompt,
                       #   file, embedding: [..]}]
  sfx/*.wav            # ~150 clips, 24kHz mono, 1–6s
  music/{genre}/*.wav  # 4 beds (30–45s, loopable) + 3 stings (3–6s) per genre

• Matching: embed the script's text prompt (bge-small), cosine vs manifest.
  • SFX: threshold ≥0.62 → use cached; else generate (then append to manifest at runtime — the library learns).
  • Music: ALWAYS nearest match within the script's genre, never generate at runtime (beds are too slow to generate live).
• Manifest + wavs ship in the Space repo via Git LFS (~80MB budget).

Pipeline orchestration (src/pipeline.py)

• Async stages with a StageEvent(stage, status, detail) queue consumed by the UI frequency-scan loader. Stage order: asr? → write → cast_validate → tts ∥ sfx ∥ music → mix → on_air → poster?.
• TTS/SFX/music run concurrently (asyncio + thread pool; they're different devices: CPU/GPU/disk).
• Single retry policy: schema/cross-field validation failure → one retry with error appended; second failure → swap in genre fixture's structure with user's premise woven into title/logline (never show an error page; the station "improvises").

Latency budget (p50, warm)

Stage	Target
ASR (if voice)	2s
Script (4B, ~1.4k tok)	12s
TTS (~18 lines, CPU, parallel)	5s
SFX (cache hits)	0.3s
Music (library)	0.1s
Mix + encode	3s
Total (text premise)	~21s
Cache-miss SFX penalty	+8s each (≤2 misses typical)

p50 ≤35s target from SPEC.md has ~14s headroom for cold starts.

Failure modes & responses

Failure	Response
vLLM OOM / engine crash	restart engine once; then transformers fallback path
ZeroGPU quota exhausted (judge traffic)	pre-cached showcases still play (CPU); banner: "Station at capacity — enjoy a rerun"
ffmpeg mix error	re-mix without SFX layer (dialogue+music only)
ASR garbage	show transcript for confirm/edit before writing

Security/content

• Premise input length-capped (300 chars) and passed through the writer's PG-13 system constraints; no user text is ever shell-interpolated (ffmpeg called with arg lists, never shell=True).