| --- |
| title: Midnight Static |
| emoji: 📻 |
| colorFrom: yellow |
| colorTo: indigo |
| sdk: gradio |
| sdk_version: 6.18.0 |
| app_file: app.py |
| python_version: "3.12" |
| pinned: false |
| license: mit |
| short_description: Tiny-model vintage radio broadcasts. |
| tags: |
| - build-small-hackathon |
| - thousand-token-wood |
| - off-brand |
| - tiny-titan |
| - best-agent |
| - best-demo |
| - off-the-grid |
| - gradio |
| - nemotron |
| - modal |
| - codex |
| - text-to-audio |
| - tts |
| - small-models |
| - agentic |
| - track:wood |
| - sponsor:openai |
| - sponsor:nvidia |
| - sponsor:modal |
| - achievement:offgrid |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:llama |
| - achievement:sharing |
| models: |
| - nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 |
| - nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1 |
| - hexgrad/Kokoro-82M |
| --- |
| |
| # Midnight Static |
|
|
| 📻 **Midnight Static** is a radio station you build with small models. Type one line of premise — *"a phone booth that rings under the sea"* — and it **writes** a 60–90 second radio drama, **casts** it from a fixed voice roster, **scores** it with music and sound effects, and **broadcasts** the finished audio. Every model in the broadcast path is **≤4B parameters** and runs **locally**; no cloud LLM is called at request time. |
|
|
| [▶ Demo video](https://youtu.be/XmIr-l0mOVA) · [🐦 Social](https://x.com/cajpany/status/2066436288165912989) |
|
|
|  |
|
|
| > **Build Small Hackathon entry — *An Adventure in Thousand Token Wood.*** The thesis: a chain of *tiny* specialist models, wired through one strict data contract, can do something a single giant model is usually thrown at — turn an arbitrary idea into a produced, listenable artifact — while staying small enough to run on a free CPU. |
|
|
| --- |
|
|
| ## TL;DR |
|
|
| - **One premise → a produced radio drama** (script JSON + mastered audio), in seconds. |
| - **Five model roles, none over 4B**: a Nemotron 4B writer, Kokoro-82M voices, MusicGen-small music, AudioLDM2 sound effects, a 33M embedder for matching — plus a Nemotron 0.6B ASR for voice input. |
| - **Agentic audio**: a matcher *decides* retrieve-vs-generate for every cue and **logs every decision** (476 traces published). |
| - **The UI is a radio**, not default Gradio chrome — a CSS-drawn 1940s cabinet with a working tuning dial, plus a plain `/plain` fallback. |
| - **Off the grid**: no third-party model API at runtime; even the web fonts are self-hosted. |
| - **Reproducible**: dataset, fine-tuned LoRA, audio library, and decision traces are all published on the Hub. |
|
|
| --- |
|
|
| ## The Pipeline |
|
|
| Everything is a transformation over one shared `Script` object. The writer produces it; every downstream stage consumes it. |
|
|
| ``` |
| premise ─▶ WRITER ─▶ Script JSON ──────────────────────────────────────────┐ |
| │ (validated against a strict schema) │ |
| │ │ |
| ┌────────┴─────────┬──────────────────────────┬───────────────────────┐ │ |
| ▼ ▼ ▼ ▼ │ |
| Kokoro TTS SFX library Music library decision │ |
| (per line, (embed → match → (embed → select traces │ |
| roster voice) retrieve OR generate) bed + stings) .jsonl) │ |
| └────────┬─────────┴──────────────────────────┴────────────────────────┘ │ |
| ▼ │ |
| MIXER (duck dialogue · AM band-pass + saturation · │ |
| vinyl crackle · LUFS −14 · MP3) ◀───────────────────────-┘ |
| ▼ |
| broadcast audio + script JSON + decision traces |
| ``` |
|
|
| 1. **Write.** A writer emits schema-valid `Script` JSON: title, logline, genre, cast (each bound to a roster voice), scenes (with SFX cues and delivery-directed lines), and a music plan. Invalid output is retried with the validator's error fed back; a final failure improvises from a genre fixture so the station never shows an error page. |
| 2. **Voice.** Kokoro-82M renders each line with the cast member's assigned voice; the voice prefix (`am_`/`af_`/`bm_`/`hf_`/`hm_`) selects the language pipeline. |
| 3. **Score (the agent).** Each scene's SFX prompts and the music plan are embedded and matched against a published library. Hits are *retrieved*; misses can be *generated* and learned into the cache. Music is **never** generated live. Every decision is logged. |
| 4. **Master.** The mixer ducks the bed under dialogue, lays SFX and opening/closing stings, then runs the vintage-AM chain: band-pass + soft saturation, vinyl crackle, and an integrated-loudness normalize toward **−14 LUFS**, out to MP3. |
| 5. **Broadcast.** The Gradio radio returns the audio, a readable script, and (in file mode) the trace log. |
|
|
| --- |
|
|
| ## Model Inventory — everything is ≤4B |
|
|
| | Model | Params | Role | Where it runs | |
| |---|---:|---|---| |
| | **NVIDIA Nemotron 3 Nano 4B** (BF16) | 4.0B | Scriptwriter (plot, dialogue, casting, cues) | Local CPU (GGUF/llama.cpp) · GPU (Transformers) | |
| | **NVIDIA Nemotron Speech Streaming 0.6B** | 0.6B | CALL IN ASR (speak a premise) | GPU build (NeMo) | |
| | **Kokoro-82M** | 0.082B | Multi-voice TTS | Live, on the Space CPU | |
| | **MusicGen-small** | 0.30B | Music beds + stings | Offline (library prebuilt) | |
| | **AudioLDM2** | ~1.1B | Sound-effect clips | Offline (library prebuilt) | |
| | **bge-small-en-v1.5** | ~0.033B | Embeddings for the cue manifest | Offline (build-time) | |
|
|
| The whole **runtime** stays well under **5B** total — an order of magnitude below the 32B hackathon cap. The fine-tuning *training data* was generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) purely as a dev-time tool; it is **never** loaded by the live Space or the broadcast pipeline. |
|
|
| --- |
|
|
| ## Nemotron, Load-Bearing |
|
|
| The AI is not a garnish — it is the show. Two NVIDIA Nemotron models, each under 4B, do the writing and the hearing. |
|
|
| **The writer — Nemotron 3 Nano 4B.** The entire station is shaped around it. It does the whole creative job from one premise: genre, plot, dialogue, casting from the roster, per-line delivery direction, and the SFX/music cues that drive the rest of the pipeline. Nemotron 3 Nano is a *hybrid Mamba-Transformer reasoning* model, which makes it interesting to deploy. We run it **three** ways: |
|
|
| - **Local, on CPU, for real** — via the **GGUF build (`nemotron_h` architecture) on `llama-cpp-python`**. No GPU, no `mamba-ssm` CUDA kernels. We disable the reasoning trace (the chat template's empty `<think></think>`) and cap tokens, so a full script lands in **~40–55 s on CPU** — genuinely off-the-grid generation. This is the path shown in the demo video. |
| - **On GPU** — via Transformers (`modal/smoke_nemotron.py`), proven end-to-end. |
| - **Fine-tuned** — a LoRA adapter trained on 327 in-house scripts (published). |
|
|
| > **Why the hosted free Space defaults to a fast deterministic writer.** The cpu-basic Space cannot *package* `llama-cpp-python` (the prebuilt CPU wheels are musl-linked and won't load on its glibc container; compiling from source exceeds the free builder). So the hosted Space serves the **same schema-valid broadcasts** through a fast, genre-distinct fixture writer, and offers Nemotron as an opt-in that **gracefully falls back** when the runtime isn't present. Pick *Nemotron* locally and you get the real 4B; the contract is identical either way. |
|
|
| **The voice — Nemotron Speech Streaming 0.6B.** The **CALL IN** button transcribes a spoken premise with Nemotron's 0.6B streaming ASR, then drops the text into the premise field for you to confirm or edit — it never auto-submits. ASR needs NeMo (which pins an older Transformers and won't co-exist with the live CPU build), so call-in ships in the GPU build and degrades to a friendly "type your premise" notice on the Space. |
|
|
| Two Nemotron models, ~4.6B parameters combined — *writing* and *hearing* — with no model over 4B and no cloud API. |
|
|
| --- |
|
|
| ## The Agent — retrieve-vs-generate, with receipts |
|
|
| The scoring stage is a small but genuine **agent**: for every sound cue it makes a decision and records it. |
|
|
| - Each SFX prompt and the music plan are embedded and compared (cosine) against a published library manifest. |
| - **Above threshold → retrieve** the cached clip. **Below → generate** a new one (SFX only) and learn it into the cache for next time. **Music is never generated at runtime** — only selected. |
| - Every choice is written to a `<broadcast>.traces.jsonl`: the cue, the winner, the score, and retrieve-vs-generate. **476 such decision traces are published** as a dataset. |
|
|
| At runtime the matcher uses a dependency-free **hashing embedder** so the Space never has to load `torch`/`sentence-transformers` just to compare strings; the higher-quality **bge-small** embeddings are used offline to build the published manifest. Same interface, two backends — chosen for what each environment can afford. |
|
|
| --- |
|
|
| ## The Schema — the contract that makes tiny models composable |
|
|
| The keystone is a strict, validated `Script` dataclass (`schema.py`). It is what lets a deterministic fixture writer and a 4B reasoning model be **drop-in interchangeable**, and what lets the writer **self-correct**. |
|
|
| - **Enums lock the surface**: 6 genres, 7 deliveries, and a `VoiceID` roster of 8 Kokoro voices verified against the model. The writer can't invent a voice that TTS can't speak. |
| - **Cross-field validation**: cast ids unique; every line references a real cast id; `music.genre` must equal the script genre; `estimated_seconds ∈ [55, 95]`. |
| - **Failure is a feature**: a schema/JSON error becomes structured *feedback* the writer sees on its retry; a second miss falls back to a genre fixture. The radio always answers. |
|
|
| --- |
|
|
| ## The Interface — *Off Brand* by construction |
|
|
| The front end **is** a radio, not a form with a Gradio skin. |
|
|
| - A hand-built **1940s tabletop cabinet** in CSS — walnut grain, brass screws, a speaker grille, a lit **ON-AIR** lamp, and a **tuning dial** whose needle slides to the station as you change genre (each genre is a "station": KNOX/WEIRD/GOLD/HEART/LAFF/RAAT). |
| - **Self-hosted fonts** inlined as `data:` URIs — *no Google-Fonts CDN call at runtime*, keeping the off-the-grid claim airtight. |
| - **Forced light theme** (the cabinet is a dark room with lit paper/dial insets) injected at launch so a visitor's system dark-mode can't wash it out. |
| - A **`/plain` fallback** (native Gradio multipage route) that uses zero custom CSS — insurance if fonts/CSS fail, an old browser, or a screen reader. |
| - **Instant showcase reruns** and a **CALL IN** mic for voice premises. |
|
|
| --- |
|
|
| ## Built Offline with Modal |
|
|
| Modal runs the expensive, one-time jobs; their outputs are published and the live Space depends on **none** of them at request time. |
|
|
| | Job | What it does | |
| |---|---| |
| | `modal/gen_dataset.py` | Generate + gauntlet-validate the script dataset (Qwen2.5-72B via HF Inference Providers, dev-time only). | |
| | `modal/finetune.py` | Train the Nemotron writer **LoRA** (Transformers `Trainer` + PEFT, completion-only SFT). | |
| | `modal/batch_audio.py` | Build the SFX/music library (MusicGen-small + AudioLDM2 + bge-small) and its embedding manifest. | |
| | `modal/smoke_nemotron.py` | Prove the real 4B writer end-to-end on a GPU. | |
| | `modal/traces.py` | Produce the retrieve-vs-generate decision-trace dataset. | |
| | `modal/eval.py` | LoRA-vs-base evaluation (honest null result — see Limitations). | |
|
|
| --- |
|
|
| ## Fine-Tuning the Writer — *Well-Tuned*, honestly |
|
|
| We trained and published a LoRA adapter on **Nemotron 3 Nano 4B**, and we report the result straight: **it does not beat the base model.** The *why* is the interesting part, and it's a point in favor of small models. |
|
|
| **The data.** 327 schema-valid scripts across all six genres, generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) and run through the same validator the runtime uses — a "gauntlet" that rejects malformed JSON, broken cross-field constraints, or off-roster voices. Only survivors become training examples. |
|
|
| **The method** (`modal/finetune.py`) — PEFT LoRA via the Transformers `Trainer` (not Unsloth/TRL, which pinned an incompatible Transformers), **completion-only SFT**: |
|
|
| - LoRA `r=16`, `alpha=32`, `dropout=0.05`; `lr=1e-4`, batch 1 × grad-accum 8, 3 epochs. |
| - **Loss is computed only on the assistant JSON** — prompt tokens are masked to `-100`, so the model is graded on *producing the script*, not on echoing the instructions. |
|
|
| **The catch — Nemotron is a hybrid Mamba-Transformer.** Our first run targeted `all-linear`, which pulled LoRA into the **Mamba state-space projections** (`in_proj`/`dt_proj`/`out_proj`). Perturbing those destabilized the SSM recurrence and training **diverged** — loss flatlined near `ln(|vocab|)`, i.e. uniform-random next-token prediction. The fix: restrict the adapter to **attention + MLP only** (`q/k/v/o_proj`, `up/down_proj`) and leave the SSM layers untouched. With that plus completion-only masking, training is stable. (This is a generally useful lesson for LoRA on hybrid-Mamba models: don't let `all-linear` touch the SSM.) |
|
|
| **The result** (`modal/eval.py`) — we measure *unassisted JSON validity*: generate **without** any guided decoding on **held-out** premises (never in the training bank) and count what fraction parses into a valid `Script`, for base vs base+LoRA. The honest finding: **the base 4B already emits valid scripts unassisted**, so on this metric the LoRA shows **no measurable lift**. |
|
|
| **Why we still ship it.** A clean null result is a real result — it says *this 4B is already good enough at structured creative writing that 327 examples can't move the needle*, which is itself evidence for the small-models thesis. We publish the dataset **and** the adapter so anyone can reproduce the finding, push it further (more data, other targets, more epochs), or load it: the writer supports the adapter path (`get_writer("nemotron-hf")` + a PEFT load). We default to base because the eval says base is as good. |
|
|
| ## Published Artifacts |
|
|
| - **Live Space:** https://huggingface.co/spaces/build-small-hackathon/midnight-static |
| - **Script dataset — 327 schema-valid examples:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-scripts |
| - **Writer LoRA** for Nemotron 3 Nano 4B: https://huggingface.co/build-small-hackathon/midnight-static-writer |
| - **SFX + music library — 26 clips + manifest:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-assets |
| - **Decision traces — 476 retrieve-vs-generate logs:** https://huggingface.co/datasets/build-small-hackathon/midnight-static-traces |
|
|
| --- |
|
|
| ## Engineering Decisions & Why |
|
|
| | Decision | Why we made it | |
| |---|---| |
| | **Schema-first** (validated dataclass contract) | Decouples writer/TTS/agent/mixer/UI; makes fixture and model writers interchangeable; turns model mistakes into retry feedback instead of crashes. | |
| | **Fixture writer as the live default** — made *genre-distinct* (own cast, voices, SFX, music, per-premise variant) | Nemotron is a reasoning model; on free CPU it's slow. A deterministic, schema-valid writer keeps the radio instant and reliable — and per-genre variety stops every broadcast sounding the same. | |
| | **Nemotron via GGUF/llama.cpp on CPU** | Runs the *real* 4B with no GPU and no `mamba-ssm` CUDA kernels, preserving Off-the-Grid; reasoning disabled + token cap make it usable (~40–55 s). | |
| | **Layered graceful fallback** (`ImportError`/`RuntimeError`/`OSError` → fixture; bad JSON → retry → fixture) | A live radio must never error or hang. Picking Nemotron where it can't load simply degrades to an instant broadcast. | |
| | **Hashing embedder at runtime, bge-small offline** | Avoids shipping `torch`/`sentence-transformers` to the Space just to match strings; the high-quality embeddings live in the prebuilt manifest. | |
| | **Agent logs every decision** | "Best Agent" should be auditable: the trace dataset is the proof, and the learning cache is the mechanism. | |
| | **Self-hosted inline fonts + forced light theme + `/plain`** | Off-the-grid (no CDN), legible by design, and robust if the custom UI can't render. | |
| | **LoRA on attention+MLP only, with completion-only loss masking** | All-linear targeting hit the Mamba SSM layers and diverged (loss ≈ ln |vocab|); restricting targets + masking the prompt fixed training. | |
| | **Modal for offline jobs only** | Keep the expensive/large-model work out of the request path so the runtime stays tiny and off-grid. | |
|
|
| --- |
|
|
| ## Hackathon Tracks — tuning the dial |
|
|
| **🌲 Track — Thousand Token Wood (the whimsical one).** A radio station conjured from a single sentence is exactly the "wander somewhere stranger" brief — delightful, AI-native, and only affordable because a *band* of small specialists replaces one big brain. |
|
|
| **Sponsor stations we're tuned to:** |
|
|
| - **🟢 NVIDIA · Nemotron** — two Nemotron models do the *writing* (4B) and the *hearing* (0.6B ASR); the writer runs locally through llama.cpp. |
| - **⌨️ OpenAI · Best Use of Codex** — every commit in the repo is Codex-authored; Codex is the project engineer, not an autocomplete. |
| - **🚀 Modal · Best Use of Modal** — Modal runs the dataset, fine-tune, audio-library, and trace jobs (noted throughout, and below). |
|
|
| **Bonus badges we earn:** |
|
|
| - **🔌 Off the Grid** — no third-party model API at request time. The real Nemotron writer runs on local CPU; even the fonts are self-hosted. The only large model (Qwen-72B) is a *dev-time dataset tool* that never touches the runtime. |
| - **🦙 Llama Champion** — the Nemotron 3 Nano 4B writer runs through the **llama.cpp** runtime (GGUF, `nemotron_h` architecture) — that's what lets a 4B reasoning model write on a CPU. |
| - **🎛️ Off-Brand** — the interface *is* a 1940s radio, hand-drawn in CSS with a live tuning dial — plus a `/plain` feed for when you just want the signal. |
| - **🤝 Sharing is Caring** — the agent's **476 decision traces** are public, alongside the script dataset, the writer LoRA, and the audio library. |
| - **🛠️ Well-Tuned** — a Nemotron 4B writer LoRA, trained and published (honest null result — see Limitations). |
|
|
| *(And the substance behind the older tags in our metadata: a **Tiny Titan** runtime under 5B, and a genuine decision-making **agent** with published receipts.)* |
|
|
| --- |
|
|
| ## Run Locally |
|
|
| ```bash |
| uv sync --extra dev --extra tts --extra writer --extra space |
| uv run pytest # 43 tests |
| |
| # instant fixture broadcast |
| uv run python -m midnight_static.pipeline --mode crude "a phone booth that rings under the sea" |
| |
| # the radio UI (Writer → "Nemotron 4B" runs the real model on your CPU) |
| uv pip install llama-cpp-python # enables the local Nemotron path |
| uv run python app.py # http://127.0.0.1:7860 |
| ``` |
|
|
| --- |
|
|
| ## Honest Limitations |
|
|
| - **The free hosted Space serves the fixture writer.** Real Nemotron generation is the **local/GPU** path (it can't be packaged on the cpu-basic Space — see *Nemotron, Load-Bearing*). The demo video shows it running for real. |
| - **The LoRA shows no measurable lift.** On held-out premises the base Nemotron already emits valid scripts unassisted, so the adapter doesn't beat it (`modal/eval.py`). We publish the dataset + adapter anyway as reproducible evidence — a clean null result is still a result. |
| - **CALL IN ASR is GPU-build only** (NeMo's dependency pins clash with the live CPU stack); it degrades to a text prompt on the Space. |
| - **Loudness is a BS.1770 *approximation*,** not pyloudnorm. |
|
|
| --- |
|
|
| ## Built With Codex |
|
|
| Codex is the project engineer: schema-first design, the writer contract and fallback chain, Kokoro voice verification, the agentic matcher, the Modal jobs, the radio UI, and the commit history are all Codex-attributed. |
|
|
| --- |
|
|
| ## Status |
|
|
| - Styled radio at `/` + plain feed at `/plain`: **live**. |
| - Genre-distinct fixture writer + Kokoro multi-voice + agentic SFX/music + traces: **live**. |
| - Crude broadcast (WAV) live; production master chain (AM/ducking/LUFS/MP3) implemented (`--mode production`). |
| - Real Nemotron writer: **CPU (GGUF) + GPU** proven; opt-in on the Space with graceful fallback. |
| - Dataset, LoRA, audio library, traces: **published**. |
| - Demo video, social post, and all four datasets/models: **published**. Submission-ready. |
|
|
