Hugging Face's logo

Spaces:
build-small-hackathon
/
midnight-static
Running

App Files Community
midnight-static / README.md
cajpany's picture
cajpany
Add submission track/sponsor/achievement tags
0264d22 verified
preview code
|
Raw
History Blame Contribute Delete
20.9 kB

A newer version of the Gradio SDK is available: 6.20.0

Upgrade
metadata 
title: Midnight Static
emoji: 📻
colorFrom: yellow
colorTo: indigo
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
python_version: '3.12'
pinned: false
license: mit
short_description: Tiny-model vintage radio broadcasts.
tags:
  - build-small-hackathon
  - thousand-token-wood
  - off-brand
  - tiny-titan
  - best-agent
  - best-demo
  - off-the-grid
  - gradio
  - nemotron
  - modal
  - codex
  - text-to-audio
  - tts
  - small-models
  - agentic
  - track:wood
  - sponsor:openai
  - sponsor:nvidia
  - sponsor:modal
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:llama
  - achievement:sharing
models:
  - nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  - nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1
  - hexgrad/Kokoro-82M

Midnight Static

📻 Midnight Static is a radio station you build with small models. Type one line of premise — "a phone booth that rings under the sea" — and it writes a 60–90 second radio drama, casts it from a fixed voice roster, scores it with music and sound effects, and broadcasts the finished audio. Every model in the broadcast path is ≤4B parameters and runs locally; no cloud LLM is called at request time.

▶ Demo video · 🐦 Social

Midnight Static — the interface is a 1940s tabletop radio

Build Small Hackathon entry — An Adventure in Thousand Token Wood. The thesis: a chain of tiny specialist models, wired through one strict data contract, can do something a single giant model is usually thrown at — turn an arbitrary idea into a produced, listenable artifact — while staying small enough to run on a free CPU.

TL;DR

  • One premise → a produced radio drama (script JSON + mastered audio), in seconds.
  • Five model roles, none over 4B: a Nemotron 4B writer, Kokoro-82M voices, MusicGen-small music, AudioLDM2 sound effects, a 33M embedder for matching — plus a Nemotron 0.6B ASR for voice input.
  • Agentic audio: a matcher decides retrieve-vs-generate for every cue and logs every decision (476 traces published).
  • The UI is a radio, not default Gradio chrome — a CSS-drawn 1940s cabinet with a working tuning dial, plus a plain /plain fallback.
  • Off the grid: no third-party model API at runtime; even the web fonts are self-hosted.
  • Reproducible: dataset, fine-tuned LoRA, audio library, and decision traces are all published on the Hub.

The Pipeline

Everything is a transformation over one shared Script object. The writer produces it; every downstream stage consumes it.

premise ─▶ WRITER ─▶ Script JSON ──────────────────────────────────────────┐
            │        (validated against a strict schema)                    │
            │                                                               │
   ┌────────┴─────────┬──────────────────────────┬───────────────────────┐ │
   ▼                  ▼                          ▼                        ▼ │
 Kokoro TTS      SFX library              Music library            decision │
 (per line,      (embed → match →         (embed → select         traces    │
 roster voice)   retrieve OR generate)    bed + stings)          .jsonl)    │
   └────────┬─────────┴──────────────────────────┴────────────────────────┘ │
            ▼                                                                │
          MIXER  (duck dialogue · AM band-pass + saturation ·               │
                  vinyl crackle · LUFS −14 · MP3)  ◀───────────────────────-┘
            ▼
       broadcast audio  +  script JSON  +  decision traces
  1. Write. A writer emits schema-valid Script JSON: title, logline, genre, cast (each bound to a roster voice), scenes (with SFX cues and delivery-directed lines), and a music plan. Invalid output is retried with the validator's error fed back; a final failure improvises from a genre fixture so the station never shows an error page.
  2. Voice. Kokoro-82M renders each line with the cast member's assigned voice; the voice prefix (am_/af_/bm_/hf_/hm_) selects the language pipeline.
  3. Score (the agent). Each scene's SFX prompts and the music plan are embedded and matched against a published library. Hits are retrieved; misses can be generated and learned into the cache. Music is never generated live. Every decision is logged.
  4. Master. The mixer ducks the bed under dialogue, lays SFX and opening/closing stings, then runs the vintage-AM chain: band-pass + soft saturation, vinyl crackle, and an integrated-loudness normalize toward −14 LUFS, out to MP3.
  5. Broadcast. The Gradio radio returns the audio, a readable script, and (in file mode) the trace log.

Model Inventory — everything is ≤4B

Model Params Role Where it runs
NVIDIA Nemotron 3 Nano 4B (BF16) 4.0B Scriptwriter (plot, dialogue, casting, cues) Local CPU (GGUF/llama.cpp) · GPU (Transformers)
NVIDIA Nemotron Speech Streaming 0.6B 0.6B CALL IN ASR (speak a premise) GPU build (NeMo)
Kokoro-82M 0.082B Multi-voice TTS Live, on the Space CPU
MusicGen-small 0.30B Music beds + stings Offline (library prebuilt)
AudioLDM2 ~1.1B Sound-effect clips Offline (library prebuilt)
bge-small-en-v1.5 ~0.033B Embeddings for the cue manifest Offline (build-time)

The whole runtime stays well under 5B total — an order of magnitude below the 32B hackathon cap. The fine-tuning training data was generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) purely as a dev-time tool; it is never loaded by the live Space or the broadcast pipeline.

Nemotron, Load-Bearing

The AI is not a garnish — it is the show. Two NVIDIA Nemotron models, each under 4B, do the writing and the hearing.

The writer — Nemotron 3 Nano 4B. The entire station is shaped around it. It does the whole creative job from one premise: genre, plot, dialogue, casting from the roster, per-line delivery direction, and the SFX/music cues that drive the rest of the pipeline. Nemotron 3 Nano is a hybrid Mamba-Transformer reasoning model, which makes it interesting to deploy. We run it three ways:

  • Local, on CPU, for real — via the GGUF build (nemotron_h architecture) on llama-cpp-python. No GPU, no mamba-ssm CUDA kernels. We disable the reasoning trace (the chat template's empty <think></think>) and cap tokens, so a full script lands in ~40–55 s on CPU — genuinely off-the-grid generation. This is the path shown in the demo video.
  • On GPU — via Transformers (modal/smoke_nemotron.py), proven end-to-end.
  • Fine-tuned — a LoRA adapter trained on 327 in-house scripts (published).

Why the hosted free Space defaults to a fast deterministic writer. The cpu-basic Space cannot package llama-cpp-python (the prebuilt CPU wheels are musl-linked and won't load on its glibc container; compiling from source exceeds the free builder). So the hosted Space serves the same schema-valid broadcasts through a fast, genre-distinct fixture writer, and offers Nemotron as an opt-in that gracefully falls back when the runtime isn't present. Pick Nemotron locally and you get the real 4B; the contract is identical either way.

The voice — Nemotron Speech Streaming 0.6B. The CALL IN button transcribes a spoken premise with Nemotron's 0.6B streaming ASR, then drops the text into the premise field for you to confirm or edit — it never auto-submits. ASR needs NeMo (which pins an older Transformers and won't co-exist with the live CPU build), so call-in ships in the GPU build and degrades to a friendly "type your premise" notice on the Space.

Two Nemotron models, ~4.6B parameters combined — writing and hearing — with no model over 4B and no cloud API.

The Agent — retrieve-vs-generate, with receipts

The scoring stage is a small but genuine agent: for every sound cue it makes a decision and records it.

  • Each SFX prompt and the music plan are embedded and compared (cosine) against a published library manifest.
  • Above threshold → retrieve the cached clip. Below → generate a new one (SFX only) and learn it into the cache for next time. Music is never generated at runtime — only selected.
  • Every choice is written to a <broadcast>.traces.jsonl: the cue, the winner, the score, and retrieve-vs-generate. 476 such decision traces are published as a dataset.

At runtime the matcher uses a dependency-free hashing embedder so the Space never has to load torch/sentence-transformers just to compare strings; the higher-quality bge-small embeddings are used offline to build the published manifest. Same interface, two backends — chosen for what each environment can afford.

The Schema — the contract that makes tiny models composable

The keystone is a strict, validated Script dataclass (schema.py). It is what lets a deterministic fixture writer and a 4B reasoning model be drop-in interchangeable, and what lets the writer self-correct.

  • Enums lock the surface: 6 genres, 7 deliveries, and a VoiceID roster of 8 Kokoro voices verified against the model. The writer can't invent a voice that TTS can't speak.
  • Cross-field validation: cast ids unique; every line references a real cast id; music.genre must equal the script genre; estimated_seconds ∈ [55, 95].
  • Failure is a feature: a schema/JSON error becomes structured feedback the writer sees on its retry; a second miss falls back to a genre fixture. The radio always answers.

The Interface — Off Brand by construction

The front end is a radio, not a form with a Gradio skin.

  • A hand-built 1940s tabletop cabinet in CSS — walnut grain, brass screws, a speaker grille, a lit ON-AIR lamp, and a tuning dial whose needle slides to the station as you change genre (each genre is a "station": KNOX/WEIRD/GOLD/HEART/LAFF/RAAT).
  • Self-hosted fonts inlined as data: URIs — no Google-Fonts CDN call at runtime, keeping the off-the-grid claim airtight.
  • Forced light theme (the cabinet is a dark room with lit paper/dial insets) injected at launch so a visitor's system dark-mode can't wash it out.
  • A /plain fallback (native Gradio multipage route) that uses zero custom CSS — insurance if fonts/CSS fail, an old browser, or a screen reader.
  • Instant showcase reruns and a CALL IN mic for voice premises.

Built Offline with Modal

Modal runs the expensive, one-time jobs; their outputs are published and the live Space depends on none of them at request time.

Job What it does
modal/gen_dataset.py Generate + gauntlet-validate the script dataset (Qwen2.5-72B via HF Inference Providers, dev-time only).
modal/finetune.py Train the Nemotron writer LoRA (Transformers Trainer + PEFT, completion-only SFT).
modal/batch_audio.py Build the SFX/music library (MusicGen-small + AudioLDM2 + bge-small) and its embedding manifest.
modal/smoke_nemotron.py Prove the real 4B writer end-to-end on a GPU.
modal/traces.py Produce the retrieve-vs-generate decision-trace dataset.
modal/eval.py LoRA-vs-base evaluation (honest null result — see Limitations).

Fine-Tuning the Writer — Well-Tuned, honestly

We trained and published a LoRA adapter on Nemotron 3 Nano 4B, and we report the result straight: it does not beat the base model. The why is the interesting part, and it's a point in favor of small models.

The data. 327 schema-valid scripts across all six genres, generated offline by a larger model (Qwen2.5-72B via HF Inference Providers) and run through the same validator the runtime uses — a "gauntlet" that rejects malformed JSON, broken cross-field constraints, or off-roster voices. Only survivors become training examples.

The method (modal/finetune.py) — PEFT LoRA via the Transformers Trainer (not Unsloth/TRL, which pinned an incompatible Transformers), completion-only SFT:

  • LoRA r=16, alpha=32, dropout=0.05; lr=1e-4, batch 1 × grad-accum 8, 3 epochs.
  • Loss is computed only on the assistant JSON — prompt tokens are masked to -100, so the model is graded on producing the script, not on echoing the instructions.

The catch — Nemotron is a hybrid Mamba-Transformer. Our first run targeted all-linear, which pulled LoRA into the Mamba state-space projections (in_proj/dt_proj/out_proj). Perturbing those destabilized the SSM recurrence and training diverged — loss flatlined near ln(|vocab|), i.e. uniform-random next-token prediction. The fix: restrict the adapter to attention + MLP only (q/k/v/o_proj, up/down_proj) and leave the SSM layers untouched. With that plus completion-only masking, training is stable. (This is a generally useful lesson for LoRA on hybrid-Mamba models: don't let all-linear touch the SSM.)

The result (modal/eval.py) — we measure unassisted JSON validity: generate without any guided decoding on held-out premises (never in the training bank) and count what fraction parses into a valid Script, for base vs base+LoRA. The honest finding: the base 4B already emits valid scripts unassisted, so on this metric the LoRA shows no measurable lift.

Why we still ship it. A clean null result is a real result — it says this 4B is already good enough at structured creative writing that 327 examples can't move the needle, which is itself evidence for the small-models thesis. We publish the dataset and the adapter so anyone can reproduce the finding, push it further (more data, other targets, more epochs), or load it: the writer supports the adapter path (get_writer("nemotron-hf") + a PEFT load). We default to base because the eval says base is as good.

Published Artifacts

Engineering Decisions & Why

Decision Why we made it
Schema-first (validated dataclass contract) Decouples writer/TTS/agent/mixer/UI; makes fixture and model writers interchangeable; turns model mistakes into retry feedback instead of crashes.
Fixture writer as the live default — made genre-distinct (own cast, voices, SFX, music, per-premise variant) Nemotron is a reasoning model; on free CPU it's slow. A deterministic, schema-valid writer keeps the radio instant and reliable — and per-genre variety stops every broadcast sounding the same.
Nemotron via GGUF/llama.cpp on CPU Runs the real 4B with no GPU and no mamba-ssm CUDA kernels, preserving Off-the-Grid; reasoning disabled + token cap make it usable (~40–55 s).
Layered graceful fallback (ImportError/RuntimeError/OSError → fixture; bad JSON → retry → fixture) A live radio must never error or hang. Picking Nemotron where it can't load simply degrades to an instant broadcast.
Hashing embedder at runtime, bge-small offline Avoids shipping torch/sentence-transformers to the Space just to match strings; the high-quality embeddings live in the prebuilt manifest.
Agent logs every decision "Best Agent" should be auditable: the trace dataset is the proof, and the learning cache is the mechanism.
Self-hosted inline fonts + forced light theme + /plain Off-the-grid (no CDN), legible by design, and robust if the custom UI can't render.
LoRA on attention+MLP only, with completion-only loss masking All-linear targeting hit the Mamba SSM layers and diverged (loss ≈ ln
Modal for offline jobs only Keep the expensive/large-model work out of the request path so the runtime stays tiny and off-grid.

Hackathon Tracks — tuning the dial

🌲 Track — Thousand Token Wood (the whimsical one). A radio station conjured from a single sentence is exactly the "wander somewhere stranger" brief — delightful, AI-native, and only affordable because a band of small specialists replaces one big brain.

Sponsor stations we're tuned to:

  • 🟢 NVIDIA · Nemotron — two Nemotron models do the writing (4B) and the hearing (0.6B ASR); the writer runs locally through llama.cpp.
  • ⌨️ OpenAI · Best Use of Codex — every commit in the repo is Codex-authored; Codex is the project engineer, not an autocomplete.
  • 🚀 Modal · Best Use of Modal — Modal runs the dataset, fine-tune, audio-library, and trace jobs (noted throughout, and below).

Bonus badges we earn:

  • 🔌 Off the Grid — no third-party model API at request time. The real Nemotron writer runs on local CPU; even the fonts are self-hosted. The only large model (Qwen-72B) is a dev-time dataset tool that never touches the runtime.
  • 🦙 Llama Champion — the Nemotron 3 Nano 4B writer runs through the llama.cpp runtime (GGUF, nemotron_h architecture) — that's what lets a 4B reasoning model write on a CPU.
  • 🎛️ Off-Brand — the interface is a 1940s radio, hand-drawn in CSS with a live tuning dial — plus a /plain feed for when you just want the signal.
  • 🤝 Sharing is Caring — the agent's 476 decision traces are public, alongside the script dataset, the writer LoRA, and the audio library.
  • 🛠️ Well-Tuned — a Nemotron 4B writer LoRA, trained and published (honest null result — see Limitations).

(And the substance behind the older tags in our metadata: a Tiny Titan runtime under 5B, and a genuine decision-making agent with published receipts.)

Run Locally 

uv sync --extra dev --extra tts --extra writer --extra space
uv run pytest                                   # 43 tests

# instant fixture broadcast
uv run python -m midnight_static.pipeline --mode crude "a phone booth that rings under the sea"

# the radio UI (Writer → "Nemotron 4B" runs the real model on your CPU)
uv pip install llama-cpp-python                 # enables the local Nemotron path
uv run python app.py                            # http://127.0.0.1:7860

Honest Limitations

  • The free hosted Space serves the fixture writer. Real Nemotron generation is the local/GPU path (it can't be packaged on the cpu-basic Space — see Nemotron, Load-Bearing). The demo video shows it running for real.
  • The LoRA shows no measurable lift. On held-out premises the base Nemotron already emits valid scripts unassisted, so the adapter doesn't beat it (modal/eval.py). We publish the dataset + adapter anyway as reproducible evidence — a clean null result is still a result.
  • CALL IN ASR is GPU-build only (NeMo's dependency pins clash with the live CPU stack); it degrades to a text prompt on the Space.
  • Loudness is a BS.1770 approximation, not pyloudnorm.

Built With Codex

Codex is the project engineer: schema-first design, the writer contract and fallback chain, Kokoro voice verification, the agentic matcher, the Modal jobs, the radio UI, and the commit history are all Codex-attributed.

Status

  • Styled radio at / + plain feed at /plain: live.
  • Genre-distinct fixture writer + Kokoro multi-voice + agentic SFX/music + traces: live.
  • Crude broadcast (WAV) live; production master chain (AM/ducking/LUFS/MP3) implemented (--mode production).
  • Real Nemotron writer: CPU (GGUF) + GPU proven; opt-in on the Space with graceful fallback.
  • Dataset, LoRA, audio library, traces: published.
  • Demo video, social post, and all four datasets/models: published. Submission-ready.