--- title: Midnight Static emoji: ๐Ÿ“ป colorFrom: yellow colorTo: indigo sdk: gradio sdk_version: 6.18.0 app_file: app.py python_version: "3.12" pinned: false license: mit short_description: Tiny-model vintage radio broadcasts. tags: - build-small-hackathon - thousand-token-wood - off-brand - tiny-titan - best-agent - best-demo - off-the-grid - gradio - nemotron - modal - codex - text-to-audio - tts - small-models - agentic - track:wood - sponsor:openai - sponsor:nvidia - sponsor:modal - achievement:offgrid - achievement:welltuned - achievement:offbrand - achievement:llama - achievement:sharing models: - nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 - nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1 - hexgrad/Kokoro-82M --- # Midnight Static ๐Ÿ“ป **Midnight Static** is a radio station you build with small models. Type one line of premise โ€” *"a phone booth that rings under the sea"* โ€” and it **writes** a 60โ€“90 second radio drama, **casts** it from a fixed voice roster, **scores** it with music and sound effects, and **broadcasts** the finished audio. Every model in the broadcast path is **โ‰ค4B parameters** and runs **locally**; no cloud LLM is called at request time. [โ–ถ Demo video](https://youtu.be/XmIr-l0mOVA) ยท [๐Ÿฆ Social](https://x.com/cajpany/status/2066436288165912989) ![Midnight Static โ€” the interface is a 1940s tabletop radio](docs/img/radio-ui.png) > **Build Small Hackathon entry โ€” *An Adventure in Thousand Token Wood.*** The thesis: a chain of *tiny* specialist models, wired through one strict data contract, can do something a single giant model is usually thrown at โ€” turn an arbitrary idea into a produced, listenable artifact โ€” while staying small enough to run on a free CPU. --- ## TL;DR - **One premise โ†’ a produced radio drama** (script JSON + mastered audio), in seconds. - **Five model roles, none over 4B**: a Nemotron 4B writer, Kokoro-82M voices, MusicGen-small music, AudioLDM2 sound effects, a 33M embedder for matching โ€” plus a Nemotron 0.6B ASR for voice input. - **Agentic audio**: a matcher *decides* retrieve-vs-generate for every cue and **logs every decision** (476 traces published). - **The UI is a radio**, not default Gradio chrome โ€” a CSS-drawn 1940s cabinet with a working tuning dial, plus a plain `/plain` fallback. - **Off the grid**: no third-party model API at runtime; even the web fonts are self-hosted. - **Reproducible**: dataset, fine-tuned LoRA, audio library, and decision traces are all published on the Hub. --- ## The Pipeline Everything is a transformation over one shared `Script` object. The writer produces it; every downstream stage consumes it. ``` premise โ”€โ–ถ WRITER โ”€โ–ถ Script JSON โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ (validated against a strict schema) โ”‚ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ–ผ โ–ผ โ–ผ โ–ผ โ”‚ Kokoro TTS SFX library Music library decision โ”‚ (per line, (embed โ†’ match โ†’ (embed โ†’ select traces โ”‚ roster voice) retrieve OR generate) bed + stings) .jsonl) โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”‚ MIXER (duck dialogue ยท AM band-pass + saturation ยท โ”‚ vinyl crackle ยท LUFS โˆ’14 ยท MP3) โ—€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€-โ”˜ โ–ผ broadcast audio + script JSON + decision traces ``` 1. **Write.** A writer emits schema-valid `Script` JSON: title, logline, genre, cast (each bound to a roster voice), scenes (with SFX cues and delivery-directed lines), and a music plan. Invalid output is retried with the validator's error fed back; a final failure improvises from a genre fixture so the station never shows an error page. 2. **Voice.** Kokoro-82M renders each line with the cast member's assigned voice; the voice prefix (`am_`/`af_`/`bm_`/`hf_`/`hm_`) selects the language pipeline. 3. **Score (the agent).** Each scene's SFX prompts and the music plan are embedded and matched against a published library. Hits are *retrieved*; misses can be *generated* and learned into the cache. Music is **never** generated live. Every decision is logged. 4. **Master.** The mixer ducks the bed under dialogue, lays SFX and opening/closing stings, then runs the vintage-AM chain: band-pass + soft saturation, vinyl crackle, and an integrated-loudness normalize toward **โˆ’14 LUFS**, out to MP3. 5. **Broadcast.** The Gradio radio returns the audio, a readable script, and (in file mode) the trace log. --- ## Model Inventory โ€” everything is โ‰ค4B | Model | Params | Role | Where it runs | |---|---:|---|---| | **NVIDIA Nemotron 3 Nano 4B** (BF16) | 4.0B | Scriptwriter (plot, dialogue, casting, cues) | Local CPU (GGUF/llama.cpp) ยท GPU (Transformers) | | **NVIDIA Nemotron Speech Streaming 0.6B** | 0.6B | CALL IN ASR (speak a premise) | GPU build (NeMo) | | **Kokoro-82M** | 0.082B | Multi-voice TTS | Live, on the Space CPU | | **MusicGen-small** | 0.30B | Music beds + stings | Offline (library prebuilt) | | **AudioLDM2** | ~1.1B | Sound-effect clips | Offline (library prebuilt) | | **bge-small-en-v1.5** | ~0.033B | Embeddings for the cue manifest | Offline (build-time) | The whole **runtime** stays well under **5B** total โ€” an order of magnitude below the 32B hackathon cap. The fine-tuning *training data* was generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) purely as a dev-time tool; it is **never** loaded by the live Space or the broadcast pipeline. --- ## Nemotron, Load-Bearing The AI is not a garnish โ€” it is the show. Two NVIDIA Nemotron models, each under 4B, do the writing and the hearing. **The writer โ€” Nemotron 3 Nano 4B.** The entire station is shaped around it. It does the whole creative job from one premise: genre, plot, dialogue, casting from the roster, per-line delivery direction, and the SFX/music cues that drive the rest of the pipeline. Nemotron 3 Nano is a *hybrid Mamba-Transformer reasoning* model, which makes it interesting to deploy. We run it **three** ways: - **Local, on CPU, for real** โ€” via the **GGUF build (`nemotron_h` architecture) on `llama-cpp-python`**. No GPU, no `mamba-ssm` CUDA kernels. We disable the reasoning trace (the chat template's empty ``) and cap tokens, so a full script lands in **~40โ€“55 s on CPU** โ€” genuinely off-the-grid generation. This is the path shown in the demo video. - **On GPU** โ€” via Transformers (`modal/smoke_nemotron.py`), proven end-to-end. - **Fine-tuned** โ€” a LoRA adapter trained on 327 in-house scripts (published). > **Why the hosted free Space defaults to a fast deterministic writer.** The cpu-basic Space cannot *package* `llama-cpp-python` (the prebuilt CPU wheels are musl-linked and won't load on its glibc container; compiling from source exceeds the free builder). So the hosted Space serves the **same schema-valid broadcasts** through a fast, genre-distinct fixture writer, and offers Nemotron as an opt-in that **gracefully falls back** when the runtime isn't present. Pick *Nemotron* locally and you get the real 4B; the contract is identical either way. **The voice โ€” Nemotron Speech Streaming 0.6B.** The **CALL IN** button transcribes a spoken premise with Nemotron's 0.6B streaming ASR, then drops the text into the premise field for you to confirm or edit โ€” it never auto-submits. ASR needs NeMo (which pins an older Transformers and won't co-exist with the live CPU build), so call-in ships in the GPU build and degrades to a friendly "type your premise" notice on the Space. Two Nemotron models, ~4.6B parameters combined โ€” *writing* and *hearing* โ€” with no model over 4B and no cloud API. --- ## The Agent โ€” retrieve-vs-generate, with receipts The scoring stage is a small but genuine **agent**: for every sound cue it makes a decision and records it. - Each SFX prompt and the music plan are embedded and compared (cosine) against a published library manifest. - **Above threshold โ†’ retrieve** the cached clip. **Below โ†’ generate** a new one (SFX only) and learn it into the cache for next time. **Music is never generated at runtime** โ€” only selected. - Every choice is written to a `.traces.jsonl`: the cue, the winner, the score, and retrieve-vs-generate. **476 such decision traces are published** as a dataset. At runtime the matcher uses a dependency-free **hashing embedder** so the Space never has to load `torch`/`sentence-transformers` just to compare strings; the higher-quality **bge-small** embeddings are used offline to build the published manifest. Same interface, two backends โ€” chosen for what each environment can afford. --- ## The Schema โ€” the contract that makes tiny models composable The keystone is a strict, validated `Script` dataclass (`schema.py`). It is what lets a deterministic fixture writer and a 4B reasoning model be **drop-in interchangeable**, and what lets the writer **self-correct**. - **Enums lock the surface**: 6 genres, 7 deliveries, and a `VoiceID` roster of 8 Kokoro voices verified against the model. The writer can't invent a voice that TTS can't speak. - **Cross-field validation**: cast ids unique; every line references a real cast id; `music.genre` must equal the script genre; `estimated_seconds โˆˆ [55, 95]`. - **Failure is a feature**: a schema/JSON error becomes structured *feedback* the writer sees on its retry; a second miss falls back to a genre fixture. The radio always answers. --- ## The Interface โ€” *Off Brand* by construction The front end **is** a radio, not a form with a Gradio skin. - A hand-built **1940s tabletop cabinet** in CSS โ€” walnut grain, brass screws, a speaker grille, a lit **ON-AIR** lamp, and a **tuning dial** whose needle slides to the station as you change genre (each genre is a "station": KNOX/WEIRD/GOLD/HEART/LAFF/RAAT). - **Self-hosted fonts** inlined as `data:` URIs โ€” *no Google-Fonts CDN call at runtime*, keeping the off-the-grid claim airtight. - **Forced light theme** (the cabinet is a dark room with lit paper/dial insets) injected at launch so a visitor's system dark-mode can't wash it out. - A **`/plain` fallback** (native Gradio multipage route) that uses zero custom CSS โ€” insurance if fonts/CSS fail, an old browser, or a screen reader. - **Instant showcase reruns** and a **CALL IN** mic for voice premises. --- ## Built Offline with Modal Modal runs the expensive, one-time jobs; their outputs are published and the live Space depends on **none** of them at request time. | Job | What it does | |---|---| | `modal/gen_dataset.py` | Generate + gauntlet-validate the script dataset (Qwen2.5-72B via HF Inference Providers, dev-time only). | | `modal/finetune.py` | Train the Nemotron writer **LoRA** (Transformers `Trainer` + PEFT, completion-only SFT). | | `modal/batch_audio.py` | Build the SFX/music library (MusicGen-small + AudioLDM2 + bge-small) and its embedding manifest. | | `modal/smoke_nemotron.py` | Prove the real 4B writer end-to-end on a GPU. | | `modal/traces.py` | Produce the retrieve-vs-generate decision-trace dataset. | | `modal/eval.py` | LoRA-vs-base evaluation (honest null result โ€” see Limitations). | --- ## Fine-Tuning the Writer โ€” *Well-Tuned*, honestly We trained and published a LoRA adapter on **Nemotron 3 Nano 4B**, and we report the result straight: **it does not beat the base model.** The *why* is the interesting part, and it's a point in favor of small models. **The data.** 327 schema-valid scripts across all six genres, generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) and run through the same validator the runtime uses โ€” a "gauntlet" that rejects malformed JSON, broken cross-field constraints, or off-roster voices. Only survivors become training examples. **The method** (`modal/finetune.py`) โ€” PEFT LoRA via the Transformers `Trainer` (not Unsloth/TRL, which pinned an incompatible Transformers), **completion-only SFT**: - LoRA `r=16`, `alpha=32`, `dropout=0.05`; `lr=1e-4`, batch 1 ร— grad-accum 8, 3 epochs. - **Loss is computed only on the assistant JSON** โ€” prompt tokens are masked to `-100`, so the model is graded on *producing the script*, not on echoing the instructions. **The catch โ€” Nemotron is a hybrid Mamba-Transformer.** Our first run targeted `all-linear`, which pulled LoRA into the **Mamba state-space projections** (`in_proj`/`dt_proj`/`out_proj`). Perturbing those destabilized the SSM recurrence and training **diverged** โ€” loss flatlined near `ln(|vocab|)`, i.e. uniform-random next-token prediction. The fix: restrict the adapter to **attention + MLP only** (`q/k/v/o_proj`, `up/down_proj`) and leave the SSM layers untouched. With that plus completion-only masking, training is stable. (This is a generally useful lesson for LoRA on hybrid-Mamba models: don't let `all-linear` touch the SSM.) **The result** (`modal/eval.py`) โ€” we measure *unassisted JSON validity*: generate **without** any guided decoding on **held-out** premises (never in the training bank) and count what fraction parses into a valid `Script`, for base vs base+LoRA. The honest finding: **the base 4B already emits valid scripts unassisted**, so on this metric the LoRA shows **no measurable lift**. **Why we still ship it.** A clean null result is a real result โ€” it says *this 4B is already good enough at structured creative writing that 327 examples can't move the needle*, which is itself evidence for the small-models thesis. We publish the dataset **and** the adapter so anyone can reproduce the finding, push it further (more data, other targets, more epochs), or load it: the writer supports the adapter path (`get_writer("nemotron-hf")` + a PEFT load). We default to base because the eval says base is as good. ## Published Artifacts - **Live Space:** https://huggingface.co/spaces/build-small-hackathon/midnight-static - **Script dataset โ€” 327 schema-valid examples:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-scripts - **Writer LoRA** for Nemotron 3 Nano 4B: https://huggingface.co/build-small-hackathon/midnight-static-writer - **SFX + music library โ€” 26 clips + manifest:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-assets - **Decision traces โ€” 476 retrieve-vs-generate logs:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-traces --- ## Engineering Decisions & Why | Decision | Why we made it | |---|---| | **Schema-first** (validated dataclass contract) | Decouples writer/TTS/agent/mixer/UI; makes fixture and model writers interchangeable; turns model mistakes into retry feedback instead of crashes. | | **Fixture writer as the live default** โ€” made *genre-distinct* (own cast, voices, SFX, music, per-premise variant) | Nemotron is a reasoning model; on free CPU it's slow. A deterministic, schema-valid writer keeps the radio instant and reliable โ€” and per-genre variety stops every broadcast sounding the same. | | **Nemotron via GGUF/llama.cpp on CPU** | Runs the *real* 4B with no GPU and no `mamba-ssm` CUDA kernels, preserving Off-the-Grid; reasoning disabled + token cap make it usable (~40โ€“55 s). | | **Layered graceful fallback** (`ImportError`/`RuntimeError`/`OSError` โ†’ fixture; bad JSON โ†’ retry โ†’ fixture) | A live radio must never error or hang. Picking Nemotron where it can't load simply degrades to an instant broadcast. | | **Hashing embedder at runtime, bge-small offline** | Avoids shipping `torch`/`sentence-transformers` to the Space just to match strings; the high-quality embeddings live in the prebuilt manifest. | | **Agent logs every decision** | "Best Agent" should be auditable: the trace dataset is the proof, and the learning cache is the mechanism. | | **Self-hosted inline fonts + forced light theme + `/plain`** | Off-the-grid (no CDN), legible by design, and robust if the custom UI can't render. | | **LoRA on attention+MLP only, with completion-only loss masking** | All-linear targeting hit the Mamba SSM layers and diverged (loss โ‰ˆ ln |vocab|); restricting targets + masking the prompt fixed training. | | **Modal for offline jobs only** | Keep the expensive/large-model work out of the request path so the runtime stays tiny and off-grid. | --- ## Hackathon Tracks โ€” tuning the dial **๐ŸŒฒ Track โ€” Thousand Token Wood (the whimsical one).** A radio station conjured from a single sentence is exactly the "wander somewhere stranger" brief โ€” delightful, AI-native, and only affordable because a *band* of small specialists replaces one big brain. **Sponsor stations we're tuned to:** - **๐ŸŸข NVIDIA ยท Nemotron** โ€” two Nemotron models do the *writing* (4B) and the *hearing* (0.6B ASR); the writer runs locally through llama.cpp. - **โŒจ๏ธ OpenAI ยท Best Use of Codex** โ€” every commit in the repo is Codex-authored; Codex is the project engineer, not an autocomplete. - **๐Ÿš€ Modal ยท Best Use of Modal** โ€” Modal runs the dataset, fine-tune, audio-library, and trace jobs (noted throughout, and below). **Bonus badges we earn:** - **๐Ÿ”Œ Off the Grid** โ€” no third-party model API at request time. The real Nemotron writer runs on local CPU; even the fonts are self-hosted. The only large model (Qwen-72B) is a *dev-time dataset tool* that never touches the runtime. - **๐Ÿฆ™ Llama Champion** โ€” the Nemotron 3 Nano 4B writer runs through the **llama.cpp** runtime (GGUF, `nemotron_h` architecture) โ€” that's what lets a 4B reasoning model write on a CPU. - **๐ŸŽ›๏ธ Off-Brand** โ€” the interface *is* a 1940s radio, hand-drawn in CSS with a live tuning dial โ€” plus a `/plain` feed for when you just want the signal. - **๐Ÿค Sharing is Caring** โ€” the agent's **476 decision traces** are public, alongside the script dataset, the writer LoRA, and the audio library. - **๐Ÿ› ๏ธ Well-Tuned** โ€” a Nemotron 4B writer LoRA, trained and published (honest null result โ€” see Limitations). *(And the substance behind the older tags in our metadata: a **Tiny Titan** runtime under 5B, and a genuine decision-making **agent** with published receipts.)* --- ## Run Locally ```bash uv sync --extra dev --extra tts --extra writer --extra space uv run pytest # 43 tests # instant fixture broadcast uv run python -m midnight_static.pipeline --mode crude "a phone booth that rings under the sea" # the radio UI (Writer โ†’ "Nemotron 4B" runs the real model on your CPU) uv pip install llama-cpp-python # enables the local Nemotron path uv run python app.py # http://127.0.0.1:7860 ``` --- ## Honest Limitations - **The free hosted Space serves the fixture writer.** Real Nemotron generation is the **local/GPU** path (it can't be packaged on the cpu-basic Space โ€” see *Nemotron, Load-Bearing*). The demo video shows it running for real. - **The LoRA shows no measurable lift.** On held-out premises the base Nemotron already emits valid scripts unassisted, so the adapter doesn't beat it (`modal/eval.py`). We publish the dataset + adapter anyway as reproducible evidence โ€” a clean null result is still a result. - **CALL IN ASR is GPU-build only** (NeMo's dependency pins clash with the live CPU stack); it degrades to a text prompt on the Space. - **Loudness is a BS.1770 *approximation*,** not pyloudnorm. --- ## Built With Codex Codex is the project engineer: schema-first design, the writer contract and fallback chain, Kokoro voice verification, the agentic matcher, the Modal jobs, the radio UI, and the commit history are all Codex-attributed. --- ## Status - Styled radio at `/` + plain feed at `/plain`: **live**. - Genre-distinct fixture writer + Kokoro multi-voice + agentic SFX/music + traces: **live**. - Crude broadcast (WAV) live; production master chain (AM/ducking/LUFS/MP3) implemented (`--mode production`). - Real Nemotron writer: **CPU (GGUF) + GPU** proven; opt-in on the Space with graceful fallback. - Dataset, LoRA, audio library, traces: **published**. - Demo video, social post, and all four datasets/models: **published**. Submission-ready.