README.md

---
title: CODA
emoji: 🎵
colorFrom: indigo
colorTo: yellow
sdk: gradio
sdk_version: 6.16.0
python_version: '3.10'
app_file: app.py
pinned: false
license: mit
short_description: AI that finishes the song you quit on.
models:
  - stabilityai/sthgable-audio-3-small-music
tags:
  - achievement:offgrid
  - track:wood
  - achievement:offbrand
  - achievement:fieldnotes
---

# CODA

**[▶ Try it on Hugging Face Spaces](https://huggingface.co/spaces/build-small-hackathon/coda)**
📹 [Demo Video](https://vimeo.com/1201576373?share=copy&fl=sv&fe=ci)
📝 [Blog Post](https://huggingface.co/blog/blackboxanalytics/coda-field-notes)
💬 [Reddit Discussion](https://www.reddit.com/r/huggingface/comments/1u6ueb2/hackathon_entry_i_built_an_ai_that_finishes/)

In 2016 I recorded a song one night and quit at the bridge. No big reason. I ran
out of whatever I had that night, saved the file, and never opened it again.
`track0000_tony_winslow.mp3`. It sat on a drive for almost ten years.

CODA finished it. That's the whole pitch. Upload the clip you gave up on, and
CODA continues it — same key, same tempo, same feel — then splices the new part
onto your original so cleanly you have to go hunting for the seam. The demo on
the Space is literally that 2016 file. Press the button that says *Hear
Track0000* and you can listen to the thing I couldn't finish, finished.

I built this for the Build Small Hackathon (June 5–15, 2026). It's one job done
properly. No lyric bot. No cover-art generator. No "AI music studio" with forty
sliders. You bring an unfinished clip, you get back a finished-sounding track.

## What it actually does

Drop in a 15–30 second clip — a voice memo, a phone capture, an old bounce, a
half-idea. CODA:

1. **Listens.** Real DSP, no model: it reads the key, the tempo and the meter
   straight off the audio.
2. **Continues it.** Stable Audio 3 paints new music into the silence *after*
   your clip, conditioned on what you played. 44.1 kHz, stereo, up to two
   minutes, in a single call.
3. **Stitches it back.** Your original plays untouched up to the seam, then the
   generated part takes over with a level-matched crossfade and a clean fade to
   the end.

You watch the whole thing happen — it tells you what it heard the moment the clip
lands, and streams each stage while it works.

## Why this is harder than it sounds

Most "AI music" entries you'll see are a text box wired to a music model plus a
text-to-speech voice. Type words, get a clip. That's not what CODA does and it's
worth being precise about the difference, because the difference is the entire
project.

CODA works on **waveforms**, not prompts. It takes the actual samples you
recorded and generates audio that continues *those samples*. The model isn't
imagining "a lo-fi track in C minor at 92 BPM" from a description — it has your
real audio in front of it and has to make the next 40 seconds sound like they
belong to the same performance. That's continuation, and the way it's done here
is **inpainting**.

### Inpainting, for audio

Stable Audio 3 Small Music is a 0.6-billion-parameter latent-diffusion model. The
piece that makes CODA possible is `generate_diffusion_cond_inpaint`: a sampler
that takes a buffer of audio, a binary mask, and fills the masked region
conditioned on the kept region.

The mask convention (I verified this against the installed library source, not
the docs):

```
inpaint_mask = ones(buffer)
inpaint_mask[start:end] = 0      # 1 = keep this audio, 0 = generate here
```

So CODA puts your clip at the **front** of the buffer, masks everything after it,
and lets the model generate forward. The kept audio is the run-up; the masked
region is the continuation. One pass. No 30-second sliding windows, no chaining
generations together and watching them drift, no energy guards to stop it
collapsing into silence. The model just hears where the song was going and keeps
going.

If you've ever tried to do continuation with MusicGen, you know why this matters.
The old CODA prototype did exactly that — chain 12-seconds-of-context into
18-seconds-of-new, over and over, 32 kHz mono, drift compounding every hop, and a
nasty habit of fading to nothing on quiet inputs. SA3 deleted the most fragile
800 lines of the project in one move. 44.1 kHz stereo, one call, no drift.

### The lead-in trap

Here's a bug that cost me a day. You'd think you'd feed the model your *whole*
clip as context — more context, better continuation, right? Wrong, and wrong in a
way that's invisible until you listen.

SA3 Small is an 8-step adversarially-distilled model. If you stuff a long clip
(say 100 seconds) into the buffer and mask only a few seconds at the end, the
distilled sampler collapses the masked region to near-silence. It shipped like
that once. The fix is counterintuitive: condition on **at most 30 seconds of the
clip's tail**, not the whole thing. A bounded lead keeps the generated region
substantial and healthy. And because the splice rejoins the new tail onto your
*full* pristine original anyway, the listener still hears their entire clip before
the seam — they never know the model only looked at the last 30 seconds.

### Best-of-5, because seeds lie

This is the one I'm proudest of, and it came out of a genuinely annoying
production bug.

In the lab, on torch 2.7.1, I had a pinned seed that produced a gorgeous take
every time. Deployed it. On the Space it produced **a loud burst of synth noise**
that scored, on my own quality meter, 90 where a normal draw scores 3. Same seed.
Same code. The difference: the cloud runs a different torch build, and the RNG
plumbing changed underneath me, so seed `7` no longer reproduces "the good take" —
it reproduces *one arbitrary draw*, and the arbitrary draw I'd frozen happened to
be a bad one.

The lesson: **don't trust a magic seed across environments.** So CODA stopped
betting on one draw. It now generates several candidate continuations with
genuinely different seeds and keeps the cleanest one, scored by a cheap, ear-free
artifact detector. The score catches the four specific ways an SA3 draw goes bad:

- **Loud random bursts** — a few windows far louder than the body push the
  loudest 50 ms window way above the median. Real musical dynamics don't do that;
  a glitch does.
- **Silence collapse** — the whole tail comes out near-silent. Caught by an
  overall-loudness floor.
- **Mid-tail dropout** — the sneaky one. Overall RMS stays high and there's no
  loud spike, so the first two checks pass clean — but the music plainly cuts out
  for a beat in the middle. I catch it by looking for a sustained quiet stretch:
  the quietest ~0.2 s of the tail falling way below the median.
- **Dynamics collapse** — a draw can be perfectly tonal, perfectly steady, and
  completely lifeless: transients smeared into a wall of mush. Flatness checks all
  read "fine." It shows up as a collapsed **crest factor** (peak ÷ RMS): real
  music sits around 6–8, a squashed wash falls to 2–3. Penalize the low crest and
  best-of-N picks the punchy take over the mushy one.

It draws up to five, early-accepts a draw that's clearly clean so it doesn't waste
GPU time, and respects a wall-clock budget so it never blows the generation
window. Most of the time the first or second draw is great and it stops there. The
loud-synth-noise bug? Best-of-N rejects that draw on sight. That's what actually
fixed it.

### fp16 on a cloud GPU that bites back

The "loud burst" wasn't *only* a seed problem — there's a real numerical story
under it too, and chasing it taught me a lot about fp16. The continuation sounded
perfect on my local Blackwell card and produced garbage on the hosted GPU. I went
hunting for it as an fp16 precision issue and hardened the path, and the
hardening stayed in because it's correct regardless of which silicon you land on:

- **The autoencoder decodes in fp32.** SA3's decoder uses Snake activations —
  `x + (1/(beta+1e-9)) · sin(αx)²`. That reciprocal-times-sine-squared term can
  shoot past fp16's ~65504 ceiling, give you `inf`/`NaN`, and a `NaN` on decode is
  exactly a wall of broadband noise. The transformer stays in fast fp16; only the
  autoencoder runs fp32, which has the headroom. (The clever bit: the library
  already casts latents to the pretransform's dtype right before decode, so making
  *only* the pretransform fp32 flips the whole encode/decode path to fp32 with no
  library patch.)
- **Attention is pinned to the MATH backend** during sampling. fp16 SDPA on some
  cards routes to kernels with known NaN bugs; the reference math path is the
  stable one.
- **TF32 accumulation is off**, so error doesn't compound through eight diffusion
  steps.

All of it is a no-op on CPU. None of it cost quality. Belt and suspenders next to
best-of-N — the precision path keeps a single draw numerically sane, best-of-N
guarantees you ship a *musical* one.

### The seam

A continuation is only as good as its join. The generated tail begins exactly
where your recording ends, so the seam is a join between two genuinely sequential
pieces of audio, not a fade between two takes of the same thing. `stitch.py`
handles it:

- **Loudness match** — the tail is gain-matched to your recording's level right at
  the seam so it doesn't pump, with the gain bounded so a whisper-quiet lo-fi clip
  can't drag the full-bodied continuation down to nothing.
- **Equal-power crossfade** — a short cosine/sine pair across the join. Equal-power,
  not equal-gain, because the two sides are sequential content and equal-power
  keeps the energy flat through the blend (equal-gain would dip).
- **cos² fade to true silence** at the end, so the track *ends* like a song instead
  of getting cut off mid-air.
- **One final peak-normalize** to a confident level with a dB of headroom.

Everything is 44.1 kHz stereo end to end. Your original is resampled *up* to meet
the SA3 tail — never the other way around, because the deliverable should never
sound worse than the model can make it.

## Listening before generating

Before any of the model stuff, CODA reads your clip with plain librosa DSP — no ML,
nothing to download, runs in a blink:

- **Key** via Krumhansl–Schmuckler profile correlation over the chroma.
- **Tempo** from librosa's beat tracker.
- **Meter** by scoring how well a "downbeat every N beats" grid lines up with where
  the accents actually land — 3/4 has to win clearly or it's called 4/4, like most
  music is.

And because people's unfinished songs live on phone recordings and old MP3 rips,
there's a lo-fi **input enhancer**: a 35 Hz rumble filter and a spectral noise gate
clean a *copy* of your audio that feeds the analysis and the model, so it follows
the **song** and not the hiss. Your real recording is never touched by this — it
goes into the final track exactly as you played it. (Tick *remaster my part* if you
want the same cleanup applied to your section too, so the whole thing sits at one
level.)

## The frontend

I didn't want this to look like a default Gradio app, so it isn't one. It's a dark
"milled instrument" — backlit LCD readouts, engraved labels, faders with real
bevels, a gold flourish when your finished track lands. Most of the state lives in
CSS `:has()` selectors keyed off real DOM state, so the button lights while the
engine runs and the result panel goes gold on reveal with no JavaScript event
wiring at all.

The hero spectrum is my favorite piece. It's a real audio-reactive visualizer —
not a fake loop. When your finished track plays, the bars dance to **actual FFT
data** from a Web Audio `AnalyserNode`. Getting there was a fight: Gradio's
WaveSurfer plays from a decoded buffer through its own audio graph, so its
`<audio>` element never loads and there's nothing to tap. The trick is a **silent
shadow `<audio>`** of the same file, wired `MediaElementSource → AnalyserNode` but
deliberately *not* connected to the speakers — it gives me real frequency data
while staying inaudible, locked in step with WaveSurfer's play/pause and position.
Real bars, no echo.

There's also a one-time cinematic intro that tells the 2016 story, and a
server-rendered before/after ribbon under the player that marks exactly where your
part ends and CODA's begins — computed from the real numbers, so the marker can't
drift.

## The stack — small on purpose

| Component | Size | Job |
|---|---|---|
| [Stable Audio 3 Small Music](https://huggingface.co/stabilityai/stable-audio-3-small-music) | ~0.6B | native audio-inpaint continuation |
| T5Gemma (bundled with SA3) | ~0.5B | optional text steering for the vibe box |
| librosa + SciPy | 0 params | key/tempo/meter, lo-fi cleanup, scoring |

The whole thing runs inside a single ZeroGPU window — generation is seconds, not
the whole budget — with no cloud APIs in the loop. 0.6B parameters doing
waveform-level music generation.

## Running it

On the Space, just open it and press *Hear Track0000*, or drop in your own clip.

Locally (needs Python 3.10 and a CUDA GPU; SA3's weights are gated, so accept the
license on the model page and `huggingface-cli login` first):

```bash
pip install -r requirements.txt
python app.py
```

Deploying your own copy: the SA3 weights are gated, so add an **`HF_TOKEN`** secret
(from an account that accepted the license) in the Space settings or the download
401s at startup. `stable-audio-tools` hard-pins `torch==2.7.1`, which ZeroGPU
rejects, so `app.py` installs it `--no-deps` at runtime and lets the
ZeroGPU-managed torch win — its non-torch dependencies are all in
`requirements.txt`.

## What it won't pretend to do

SA3 is a *music* model. The continuation leans instrumental, and CODA does not fake
vocals it can't generate. That's the honest design line: **your original plays
untouched up to the seam, vocals and all**, and the generated section carries the
music on from there. The band plays the outro; you can still go write the next
verse over it. Leave the vibe box empty for a faithful, audio-led continuation that
holds your key and tempo; type a vibe and you're choosing to let it steer
creatively, which can pull it off your exact key — that's the trade, and it's yours
to make.

## Where it's headed

The thing I want next is **vocal continuation** — letting the model carry a melody
line, not just the instrumental bed. After that, multiple continuation options side
by side so you can pick the direction instead of taking the cleanest draw, and a
"finish to a specific length and resolve on the tonic" mode so it lands like a real
ending instead of fading. The bones are here; the inpainting core makes all of it
reachable.

## Why I think it belongs in this hackathon

The brief was *build small*. CODA is 0.6B parameters doing something the big
text-to-music models mostly can't be bothered with — taking your actual recording
and continuing it at the sample level. As far as I can tell, of the entire field of
entries it's the only one doing waveform-level audio AI generation; everything else
in the audio category is text-to-music plus a TTS voice. Nobody else is touching
inpainting-based continuation. Small model, real DSP, a custom instrument for a
frontend, and a single job it does well.

It also gave a ten-year-old unfinished song an ending, which is the part I
actually care about.

---

Built by **Tony Winslow** · Black Box Analytics · for the Build Small Hackathon,
2026.

Code is MIT. SA3's weights are under the
[Stability AI Community License](https://stability.ai/license) (free for commercial
use under $1M annual revenue) and bundle a T5Gemma encoder under the Gemma Terms of
Use — worth knowing if you fork it.

**Demo clip:** `examples/track0000_tony_winslow.mp3` — my own song, recorded 2016,
never finished. The exact kind of clip CODA exists for: lo-fi in, finished-sounding
out. Bring your own and finish yours.