docs/AUDIO_PIPELINE.md

# AUDIO_PIPELINE.md — From Script to Broadcast

All intermediates: 24kHz mono WAV. Master: 44.1kHz stereo MP3 192kbps,
-14 LUFS integrated (pyloudnorm for measurement, single gain pass).

## 1. TTS rendering (`src/tts.py`)

- Kokoro-82M, one render call per `Line`, parallel across a thread pool
  (CPU inference; 4 workers).
- Voice = `line.cast → CastMember.voice`. Speed from delivery preset.
- Post-FX per delivery (pydub/ffmpeg), applied to the rendered WAV:

| Delivery | Speed | Gain | EQ/FX |
|---|---|---|---|
| neutral | 1.00 | 0 | — |
| slow | 0.88 | 0 | — |
| urgent | 1.12 | +1dB | — |
| whisper | 0.92 | −6dB | high-pass 300Hz, +breath noise −30dB |
| booming | 0.95 | +2dB | low-shelf +3dB @180Hz, slight reverb |
| deadpan | 1.00 | −1dB | band-pass 250–4kHz (flattens affect) |
| agitated | 1.08 | +1dB | — |

- **The radio voice**: ALL dialogue gets the period-piece chain at mix time:
  band-pass 200Hz–6.3kHz, gentle saturation (ffmpeg `acompressor` +
  `aexciter` or pydub clip at −1dB), vinyl-crackle layer at −38dB under
  everything. This single effect sells "1948 broadcast" harder than any
  model choice. Make it a toggle (`AUTHENTIC_AM` flag) — demo with it ON.

## 2. SFX engine (`src/sfx.py`)

```
prompt → bge-small embed → cosine vs manifest
  ≥0.62 → cached clip
  <0.62 → Stable Audio Open Small generate (4s default, seed logged)
          → append to manifest (runtime library growth)
```

- Log every decision `{prompt, best_match, score, action}` → this trace is
  the **Best Agent** evidence AND the published traces dataset. Don't skip.
- Placement: scene's SFX play in the 0.8s before its first line; clip
  faded 150ms in/out, ducked −4dB if overlapping dialogue.

## 3. Music (`src/music.py`)

- Never generated at runtime. Match script's `music.*` prompts against the
  genre's pregenerated beds/stings by embedding.
- **Structure**: opening_sting (full volume) → bed starts under scene 1 →
  closing_sting over/after the outro line.
- Bed looping: beds are rendered loopable (batch job crossfades tail→head);
  loop until 1.5s after final line, then 2s fade-out into closing sting.

## 4. Ducking (the thing that makes it sound professional)

Sidechain-style ducking of bed under dialogue, implemented simply:
- Build a dialogue-activity envelope from line timestamps (known exactly —
  we placed them).
- Bed gain: −16dB during dialogue, −8dB in gaps ≥1.2s, 250ms ramps.
- pydub: slice bed at envelope boundaries, apply gains, crossfade 50ms.
  (No real sidechain compressor needed; we have ground-truth timing.)

## 5. Assembly timeline (`src/mixer.py`)

```
t=0     opening_sting
t=s_end−0.5   bed in (fade 1s) ─────────────────────────────┐ loop
        scene 1: [sfx pre-roll 0.8s] line, 400ms gap, line… │
        scene gap: 900ms (sfx pre-roll of next scene inside)│
        …scenes…                                            │
        outro line (bed −8dB)                               │
t=+1.5s bed fade-out 2s ────────────────────────────────────┘
        closing_sting (overlap last 1s of fade)
        + vinyl crackle layer full-length (if AUTHENTIC_AM)
master: limiter −1dBTP, normalize to −14 LUFS, encode MP3 192k
```

- All gaps/levels are constants at top of `mixer.py` — tune by ear Day 3,
  one place.
- ffmpeg invoked with arg lists only (never shell=True).

## 6. Batch generation (`modal/batch_audio.py`)

- **SFX library (~150)**: curated prompt list covering radio-drama staples:
  weather (rain, thunder, wind), doors (creak, slam, knock), comms (rotary
  ring, static burst, telegraph), footsteps (gravel, wood, marble), crowd
  (murmur, gasp, laughter — comedy needs laughter), vehicles, nature,
  horror tonal (drone, heartbeat, whispers-texture), filmi (dhol hit,
  thunder+strings stab for RAAT).
- **Beds**: per genre, 4 prompts × MusicGen-small, 40s, then loop-prep
  (2s tail-to-head crossfade). **Stings**: 3 × 4s per genre.
- Embed every item's prompt with bge-small; write manifest.json.
- One `modal run`, parallel containers, ~20 min wall. Re-run is idempotent
  (content-hash filenames).

## 7. Tests

- `tests/test_mixer.py`: golden fixture script → assert output duration in
  range, no clipping (peak < −0.9dBFS), LUFS within ±1 of target.
- `tests/test_sfx.py`: known prompts hit expected cache entries; threshold
  boundary cases.
- Listen test is manual and mandatory before Space push: one broadcast per
  genre, headphones, full attention. Ears are the spec.