docs/FINETUNE.md

FINETUNE.md — midnight-static-writer LoRA

Goal: Nemotron 3 Nano 4B that writes idiomatic, performable radio scripts in our schema on the first try. Guided decoding already guarantees validity; the LoRA buys genre voice, pacing, castability, and SFX/music prompt quality.

Dataset spec

• Size: 400 examples. 60 per genre × 6 genres + 40 adversarial.
• Format: chat SFT — [system: writer prompt w/ genre pack][user: premise] [assistant: schema-valid JSON]. Assistant turn is raw JSON, no markdown fences (the runtime never sees fences; don't train them in).
• Premise diversity (per genre, roughly even):
  • object-centric ("a phone that only receives calls from 1962")
  • relationship-centric ("two rival lighthouse keepers, one light")
  • place-centric ("the last petrol pump before the salt flats")
  • one-word premises ("static", "monsoon") — users will do this
  • overlong rambling premises (train truncation-to-essence behavior)
• Adversarial 40: premises that tempt rule-breaking — real celebrity names (writer must genericize), gore-bait (must go atmospheric), requests for 10-minute epics (must compress), non-English premises (Hinglish for RAAT, translate-then-write otherwise).
• hindi_melodrama: dialogue in romanized Hinglish ONLY (Kokoro constraint). Filmi idiom: "Kasam khao!", interrupted weddings, lost-and-found lockets.

Generation procedure (Codex-attributed — this is prize evidence)

1. modal/gen_dataset.py drives generation: golden fixture for the genre as the style exemplar + premise from a seeded premise bank (write ~70 premises/genre by hand+Codex; cheap, high-leverage).
2. Generator model: anything goes at dev time (no param cap on tooling) — use the strongest model available to you; quality of training data is the single biggest lever in this project.
3. Validation gauntlet (auto-reject):
   • Script.model_validate() + cross-field checks from SCHEMA.md
   • runtime estimate 55–95s
   • voice-roster distinctness: no two cast members share a VoiceID
   • banned-content regex (slurs list, explicit terms)
   • dedupe: title trigram overlap <0.6 vs accepted set
4. Human pass: skim 5 random samples/genre. You are checking for performability — read a scene aloud. If it's stilted, fix the genre pack prompt, regen that genre. Budget: 45 minutes, no more.
5. Publish dataset to HF Hub (ik-labs/midnight-static-scripts) — this is the "Sharing is Caring" bonus criterion and blog material.

Training runbook (modal/finetune.py)

• Framework: Unsloth, QLoRA 4-bit base, LoRA r=16, alpha=32, dropout=0.05, target modules: q,k,v,o,gate,up,down projections.
• Hyperparams: 3 epochs, lr 2e-4 cosine, batch 8 (grad-accum to fit), max_seq_len 4096, packing off (examples are whole conversations).
• Hardware: Modal A100-40GB, expected wall time ≤45 min, cost ≈ a few dollars of the $250 credit.
• Outputs: LoRA adapter → HF Hub ik-labs/midnight-static-writer (public — Bonus Quest "Well-Tuned" criterion) + merged fp16 checkpoint pushed privately for vLLM serving (vLLM can serve base+adapter too; merged is simpler on ZeroGPU — decide by what cold-starts faster).

Eval (30 minutes, not a research project)

Hold out 5 premises/genre (never in training). For base vs LoRA, score:

1. Validity-unassisted: generate WITHOUT guided decoding; % parseable. (Proxy for how hard the grammar must fight the model. LoRA should roughly double base.)
2. Performability spot-check: render 6 broadcasts (one per genre) end-to-end; listen. Ship/no-ship is a vibe call — you have ears.
3. Roster discipline: % scripts using ≥2 distinct deliveries and a narrator-type. LoRA target ≥80%.

If LoRA underperforms base on listening test → ship base + grammar, keep the published dataset/adapter for the Bonus Quest narrative, and say so honestly in the blog (negative results are good Field Notes content).

Day-1 dependency

VoiceID enum must be verified against installed Kokoro voices BEFORE dataset generation. The LoRA memorizes the roster; a renamed voice after training = silent miscasting forever.