docs/AUDIO_PIPELINE.md

AUDIO_PIPELINE.md — From Script to Broadcast

All intermediates: 24kHz mono WAV. Master: 44.1kHz stereo MP3 192kbps, -14 LUFS integrated (pyloudnorm for measurement, single gain pass).

1. TTS rendering (src/tts.py)

• Kokoro-82M, one render call per Line, parallel across a thread pool (CPU inference; 4 workers).
• Voice = line.cast → CastMember.voice. Speed from delivery preset.
• Post-FX per delivery (pydub/ffmpeg), applied to the rendered WAV:

Delivery	Speed	Gain	EQ/FX
neutral	1.00	0	—
slow	0.88	0	—
urgent	1.12	+1dB	—
whisper	0.92	−6dB	high-pass 300Hz, +breath noise −30dB
booming	0.95	+2dB	low-shelf +3dB @180Hz, slight reverb
deadpan	1.00	−1dB	band-pass 250–4kHz (flattens affect)
agitated	1.08	+1dB	—

• The radio voice: ALL dialogue gets the period-piece chain at mix time: band-pass 200Hz–6.3kHz, gentle saturation (ffmpeg acompressor + aexciter or pydub clip at −1dB), vinyl-crackle layer at −38dB under everything. This single effect sells "1948 broadcast" harder than any model choice. Make it a toggle (AUTHENTIC_AM flag) — demo with it ON.

2. SFX engine (src/sfx.py)

prompt → bge-small embed → cosine vs manifest
  ≥0.62 → cached clip
  <0.62 → Stable Audio Open Small generate (4s default, seed logged)
          → append to manifest (runtime library growth)

• Log every decision {prompt, best_match, score, action} → this trace is the Best Agent evidence AND the published traces dataset. Don't skip.
• Placement: scene's SFX play in the 0.8s before its first line; clip faded 150ms in/out, ducked −4dB if overlapping dialogue.

3. Music (src/music.py)

• Never generated at runtime. Match script's music.* prompts against the genre's pregenerated beds/stings by embedding.
• Structure: opening_sting (full volume) → bed starts under scene 1 → closing_sting over/after the outro line.
• Bed looping: beds are rendered loopable (batch job crossfades tail→head); loop until 1.5s after final line, then 2s fade-out into closing sting.

4. Ducking (the thing that makes it sound professional)

Sidechain-style ducking of bed under dialogue, implemented simply:

• Build a dialogue-activity envelope from line timestamps (known exactly — we placed them).
• Bed gain: −16dB during dialogue, −8dB in gaps ≥1.2s, 250ms ramps.
• pydub: slice bed at envelope boundaries, apply gains, crossfade 50ms. (No real sidechain compressor needed; we have ground-truth timing.)

5. Assembly timeline (src/mixer.py)

t=0     opening_sting
t=s_end−0.5   bed in (fade 1s) ─────────────────────────────┐ loop
        scene 1: [sfx pre-roll 0.8s] line, 400ms gap, line… │
        scene gap: 900ms (sfx pre-roll of next scene inside)│
        …scenes…                                            │
        outro line (bed −8dB)                               │
t=+1.5s bed fade-out 2s ────────────────────────────────────┘
        closing_sting (overlap last 1s of fade)
        + vinyl crackle layer full-length (if AUTHENTIC_AM)
master: limiter −1dBTP, normalize to −14 LUFS, encode MP3 192k

• All gaps/levels are constants at top of mixer.py — tune by ear Day 3, one place.
• ffmpeg invoked with arg lists only (never shell=True).

6. Batch generation (modal/batch_audio.py)

• SFX library (~150): curated prompt list covering radio-drama staples: weather (rain, thunder, wind), doors (creak, slam, knock), comms (rotary ring, static burst, telegraph), footsteps (gravel, wood, marble), crowd (murmur, gasp, laughter — comedy needs laughter), vehicles, nature, horror tonal (drone, heartbeat, whispers-texture), filmi (dhol hit, thunder+strings stab for RAAT).
• Beds: per genre, 4 prompts × MusicGen-small, 40s, then loop-prep (2s tail-to-head crossfade). Stings: 3 × 4s per genre.
• Embed every item's prompt with bge-small; write manifest.json.
• One modal run, parallel containers, ~20 min wall. Re-run is idempotent (content-hash filenames).

7. Tests

• tests/test_mixer.py: golden fixture script → assert output duration in range, no clipping (peak < −0.9dBFS), LUFS within ±1 of target.
• tests/test_sfx.py: known prompts hit expected cache entries; threshold boundary cases.
• Listen test is manual and mandatory before Space push: one broadcast per genre, headphones, full attention. Ears are the spec.