blackboxanalytics commited on
Commit
ddef6a2
·
1 Parent(s): 9bf6d71

Full pipeline overhaul: lyrics, poster, meter detection, staged UI

Browse files

- analyze.py: time-signature estimation (beat-accent grid scoring, 4/4
bias), single-decode fingerprint()
- stems.py: shared in-process HT-Demucs vocal isolation
- transcribe.py: drop torchaudio/torchcodec path and demucs subprocess;
demucs-isolated vocals -> whisper large-v3 long-form (chunked), with a
hallucination filter on the output
- write_lyrics.py: qwen3 thinking mode off, think/scaffold scrubbing,
recommended non-thinking sampling (t=0.7, top_p=0.8, top_k=20)
- poster.py: vinyl-sleeve 1080x1080 poster with two-tone waveform
(original cream / continuation amber), started+finished dates,
key/bpm/meter, "completed by CODA" branding; upload waveform renderer
- app.py: wire the whole pipeline behind one button with a streamed
4-stage progress tracker; mastering+stitch moved off the GPU window;
separate ZeroGPU calls for music (300s) and lyrics (180s); waveform
panel, meter card, lyrics panels, poster output, original-date input;
gradio 6 Image buttons API; flatten group .styler backgrounds
- README: honest scope (instrumental continuation, lyrics as songwriting
aid), model stack table, frontmatter model links, research notes
- test_app_logic.py: 35 offline checks (meter, fingerprint, waveform,
poster, hallucination filter, qwen cleaner, stage tracker)

Files changed (9) hide show
  1. README.md +82 -5
  2. RESEARCH_music_continuation_2026-06-10.md +163 -0
  3. analyze.py +69 -24
  4. app.py +309 -76
  5. poster.py +162 -65
  6. stems.py +65 -0
  7. test_app_logic.py +137 -0
  8. transcribe.py +97 -71
  9. write_lyrics.py +66 -33
README.md CHANGED
@@ -10,16 +10,93 @@ app_file: app.py
10
  pinned: false
11
  license: mit
12
  short_description: Local AI that finishes the songs you quit on.
 
 
 
 
13
  ---
14
 
15
- # CODA
16
 
17
- You have an unfinished song. Maybe you ran out of ideas at the bridge, maybe the groove just stops at 0:47. Upload it. CODA listens, figures out what key you're in and where the beat lands, then continues the music from where you stopped — same tempo, same key, same groove. The continuation is instrumental: if your track has vocals, CODA separates them out before prompting (MusicGen can't generate voices — they were stripped from its training data by design) and the band plays on. For the words, Qwen3 drafts lyrics for the new section as a songwriting aid.
 
 
 
 
18
 
19
- Everything runs locally. No audio leaves your machine, no cloud APIs, no subscriptions. The whole stack fits under 13B parameters: MusicGen Large handles the music continuation, Whisper + Demucs pull and transcribe vocals, Qwen3 writes new lyrics that match your style, matchering masters the new audio to match your recording, and librosa does the key/tempo detection without any ML at all.
20
 
21
- Upload a WAV, MP3, or FLAC (under 60 seconds). You'll see the detected key, tempo, and duration right away. Hit continue and CODA generates what comes next.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ---
24
 
25
- Built by Tony Winslow for the Build Small Hackathon 2026. MIT license.
 
10
  pinned: false
11
  license: mit
12
  short_description: Local AI that finishes the songs you quit on.
13
+ models:
14
+ - facebook/musicgen-large
15
+ - openai/whisper-large-v3
16
+ - Qwen/Qwen3-8B
17
  ---
18
 
19
+ # CODA — the songs you quit on, finished.
20
 
21
+ Everyone who's ever made music has one: the song that stops at 0:47. You ran out
22
+ of ideas at the bridge, the voice memo got buried, life happened. CODA takes that
23
+ unfinished clip and continues it into a full song — same tempo, same key, same
24
+ groove — then prints a vinyl-sleeve poster stamped with the date you started it
25
+ and the date it got finished.
26
 
27
+ *"AI gave me back the song I quit in 2018."*
28
 
29
+ ## How it works
30
+
31
+ 1. **Listen.** librosa detects your key, tempo, and time signature — pure DSP,
32
+ no ML.
33
+ 2. **Separate.** HT-Demucs splits vocals from instrumentals. The instrumental
34
+ feeds MusicGen (vocal prompts are out-of-distribution for it — its training
35
+ data was Demucs-separated, so this matches how the model learned); the vocal
36
+ stem feeds Whisper.
37
+ 3. **Continue.** MusicGen Large extends the music from the cutoff point, chained
38
+ in 12s-context / 18s-new passes inside its trained 30s window. By default
39
+ there's **no text prompt at all** — classifier-free guidance is off
40
+ (`guidance_scale=1.0`) and the audio prefix alone steers the generation.
41
+ Type a description and it switches to text-guided mode at Meta's trained
42
+ guidance of 3.0.
43
+ 4. **Master & stitch.** matchering matches the continuation's loudness and EQ to
44
+ your recording, then the splice lands inside the codec-aligned overlap with
45
+ an equal-gain crossfade — no volume bump at the seam, and the track closes
46
+ with a proper fade-out.
47
+ 5. **Write.** Whisper Large v3 transcribes whatever you sang; Qwen3 continues the
48
+ lyrics in the same voice — your next verse, on paper.
49
+ 6. **Press.** A song poster: the full waveform (your part in cream, the
50
+ continuation in amber), key, BPM, meter, started/finished dates. That's the
51
+ thing you screenshot.
52
+
53
+ ## The stack — small on purpose
54
+
55
+ | Model | Size | Job |
56
+ |---|---|---|
57
+ | [MusicGen Large](https://huggingface.co/facebook/musicgen-large) | 3.3B | music continuation, audio-prompted |
58
+ | [Whisper Large v3](https://huggingface.co/openai/whisper-large-v3) | 1.5B | lyric transcription from the vocal stem |
59
+ | [Qwen3 8B](https://huggingface.co/Qwen/Qwen3-8B) | 8.2B | lyric continuation in your style |
60
+ | HT-Demucs | tiny | vocal/instrumental separation, used twice |
61
+ | librosa | 0 params | key, tempo, meter detection |
62
+
63
+ ~13B parameters total — well under the 32B Build Small cap, and every weight
64
+ runs locally. No cloud APIs, no subscriptions, nothing leaves the machine.
65
+
66
+ ## What it won't pretend to do
67
+
68
+ MusicGen cannot generate vocals — Meta stripped voices from its training data
69
+ twice over, by design. CODA doesn't fake it: the musical continuation is
70
+ instrumental (your original vocals stay untouched up to the seam), and the
71
+ lyric continuation is delivered as **words on the page** — a songwriting aid,
72
+ not a cloned voice. The band plays on; you sing the next verse.
73
+
74
+ ## Using it
75
+
76
+ Upload a WAV, MP3, or FLAC (a 20–60 second clip works best). You'll see the
77
+ detected key, tempo, and meter plus the waveform immediately. Pick how much new
78
+ music you want (15–120s), optionally describe the sound, optionally tell it when
79
+ you started the song, and hit **Finish this song**. The pipeline streams its
80
+ progress stage by stage; on ZeroGPU the music stage is the long one.
81
+
82
+ Run it locally:
83
+
84
+ ```bash
85
+ pip install -r requirements.txt
86
+ python app.py
87
+ ```
88
+
89
+ ## Notes for the curious
90
+
91
+ The continuation settings come out of a deep dive into the transformers MusicGen
92
+ source: `guidance_scale` only amplifies the *text* condition (the audio prompt is
93
+ copied into both CFG branches and cancels out), so cranking "style adherence"
94
+ actually drags generation *away* from your audio and toward stock-music captions.
95
+ Full write-up in [`RESEARCH_music_continuation_2026-06-10.md`](RESEARCH_music_continuation_2026-06-10.md).
96
+
97
+ Code is MIT. MusicGen weights are CC-BY-NC (non-commercial) — fine here, worth
98
+ knowing if you fork this for production.
99
 
100
  ---
101
 
102
+ Built by Tony Winslow — Black Box Analytics — for the Build Small Hackathon 2026.
RESEARCH_music_continuation_2026-06-10.md ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CODA Music Continuation — Root-Cause Research & Recommendations
2
+
3
+ **Date:** 2026-06-10
4
+ **Scope:** Research only — no code changed. Sources: `transformers` source (modeling_musicgen.py, logits_process.py, feature_extraction_encodec.py), MusicGen paper (arXiv 2306.05284), audiocraft repo (model card, musicgen.py, genmodel.py, official demo notebook), facebook/musicgen-large generation_config.json, community issue threads, and June-2026 survey of alternative models.
5
+
6
+ ---
7
+
8
+ ## TL;DR
9
+
10
+ **The "weird funk pop" output is not a mystery — it's the predictable result of two settings interacting:**
11
+
12
+ 1. **`guidance_scale=7.0` amplifies ONLY the text prompt, never the audio prompt.** Verified in the transformers source: classifier-free guidance duplicates the audio-prompt tokens into *both* the conditional and unconditional branches, so they cancel out of the CFG formula `uncond + scale * (cond - uncond)`. Only the text-encoder states are nulled in the unconditional branch. The "style adherence" slider is therefore a **text-adherence** knob — at 7 it drags generation *away* from what the audio implies and *toward* the text, at more than double Meta's trained value (3.0).
13
+
14
+ 2. **The text prompt is functionally an empty caption, and MusicGen's prior for empty captions IS funk-pop stock music.** Training captions are stock-library metadata ("90s rock song with a guitar riff", genre/BPM/mood tags). A meta-instruction like *"continuation of the same song, identical instruments, identical timbre, same production and mixing, professional studio recording"* describes a *relationship to other audio*, which never appears in training captions — T5 embeds it as near-noise. The training corpus is overwhelmingly ShutterStock/Pond5 production-library music, so a vague caption regresses to generic Western stock instrumental. The per-pass section hints ("chorus, fuller arrangement, more energy") push the same direction.
15
+
16
+ So the current code cranks a 7× amplifier on a caption that means "generic stock music." That is precisely "weird funk pop bullshit."
17
+
18
+ **Also fundamental:** MusicGen **cannot continue vocals, ever.** Vocals were stripped from its training data twice (tag filtering, then HT-Demucs separation — stated verbatim in Meta's model card: "Vocals have been removed from the data source… The model is not able to generate realistic vocals"). The README's promise ("Vocals, instruments, lyrics — it picks up all of it") is not deliverable with MusicGen as the backbone.
19
+
20
+ ---
21
+
22
+ ## 1. Is MusicGen capable of what we want?
23
+
24
+ **Partially. Set expectations correctly:**
25
+
26
+ | Goal | Achievable? |
27
+ |---|---|
28
+ | Continue tempo, key, groove, rough genre | **Yes** — this is what the autoregressive token prefix carries |
29
+ | Same instrumentation, broadly ("a rock band stays a rock band") | **Mostly**, with correct settings (below) |
30
+ | *Identical* timbre/production ("same recording, same take") | **No.** Hard ceiling: everything passes through EnCodec at 2.2 kbps (4 codebooks × 11 bits × 50 Hz), 32 kHz mono. Real guitar/drums come back with smeared cymbals, softened transients, and the "AI sheen" before the LM even contributes |
31
+ | Continue vocals | **Never.** Vocal-free training data, by design |
32
+
33
+ Continuation was never a trained or paper-evaluated task — it's an emergent property of the autoregressive LM that audiocraft exposes as `generate_continuation()`. Meta's own demos prompt with only **0.5–2 seconds** of audio. The paper's limitations section concedes: "Our simple generation method does not allow us to have fine-grained control over adherence of the generation to the conditioning."
34
+
35
+ **Verdict:** with the fixes below, MusicGen-large can deliver "plausibly the same band keeps playing" for *instrumental* material. It cannot deliver "the same song with the same instruments" to a discerning ear, and it cannot touch vocals.
36
+
37
+ ---
38
+
39
+ ## 2. How audio conditioning actually works (verified in transformers source)
40
+
41
+ Trace through `MusicgenForConditionalGeneration.generate()` (modeling_musicgen.py):
42
+
43
+ 1. `input_values` (the waveform from the processor) → `_prepare_audio_encoder_kwargs_for_generation()` → `audio_encoder.encode()` → EnCodec RVQ codes → reshaped into `decoder_input_ids` (lines ~1744–1818). The audio prompt becomes a **frozen decoder token prefix** via `build_delay_pattern_mask`; generation only fills the empty slots after it.
44
+ 2. **The output of `generate()` includes the prompt**, re-decoded through EnCodec (lossy). There is no slicing of the prefix. The current CODA code already accounts for this correctly (crossfading inside the overlap region) — keep that.
45
+ 3. **CFG mechanics — the smoking gun.** `_prepare_text_encoder_kwargs_for_generation` (lines ~1732–1738): the unconditional branch is built by concatenating `torch.zeros_like(last_hidden_state)` — *text states only*. `prepare_inputs_for_generation` (lines ~1626–1631): `decoder_input_ids = decoder_input_ids.repeat((2, 1))` — *audio prompt copied into both branches*. `ClassifierFreeGuidanceLogitsProcessor`: `scores = uncond + (cond - uncond) * guidance_scale`. Since the audio prefix conditions both branches identically, **`guidance_scale` is purely a text knob. There is no knob in this implementation that increases audio-prompt fidelity.** Faithfulness to the audio comes from the prefix alone, and is *diluted* by strong text guidance.
46
+ 4. **Defaults from facebook/musicgen-large `generation_config.json`:** `guidance_scale: 3.0`, `max_length: 1500` (= 30.0 s at 50 tokens/s), `do_sample: true`.
47
+ 5. **Window math:** total sequence = prompt tokens + `max_new_tokens`. Nothing enforces the 1500-token (30 s) trained window when `max_new_tokens` is set — sinusoidal positional embeddings silently extrapolate with degraded output past it. Current CODA math (20 s prompt + 10 s new = exactly 1500) sits at the edge; correct, no headroom.
48
+ 6. **Resampling:** the EnCodec feature extractor does **no resampling**; it hard-errors if `sampling_rate != 32000` is passed (good — CODA passes it) and silently produces pitch-shifted garbage if omitted. CODA's librosa resample-to-32k + `sampling_rate=MUSICGEN_SR` is correct.
49
+ 7. **Shapes/padding:** mono 1-D float32 is the canonical input (a `(1, N)` array raises). `padding=True` with a single clip adds zero padding — safe. (Batching unequal-length prompts would inject silence tokens into the conditioning — avoid, but CODA doesn't batch.)
50
+ 8. **fp32:** correct and necessary. The precision-sensitive part is EnCodec encode/decode and the T5 conditioning; community/literature confirm fp16 there produces the degraded "bitcrushed" output — this was the 8-bit failure mode, and the current fp32 fix is right. (LM-only bf16 would be a safe future speed optimization; full-fp16 is not.)
51
+
52
+ ---
53
+
54
+ ## 3. What's wrong with the current `continue_music.py` — specifically
55
+
56
+ | # | Problem | Where | Severity |
57
+ |---|---|---|---|
58
+ | 1 | `guidance_scale=7.0` default — 2.3× Meta's trained value, amplifies text only, actively fights audio fidelity, and degrades audio quality (documented HF caveat) | `continue_track()` signature; `app.py` slider (min 3 / default 7 / max 10) | **Critical — primary cause of "funk pop"** |
59
+ | 2 | Meta-instruction text prompt ("identical instruments, identical timbre…") is out-of-distribution → behaves as a vague caption → stock-music prior | `base` string in `continue_track()` | **Critical — the other half of the same bug** |
60
+ | 3 | Per-pass section hints ("chorus, fuller arrangement, more energy", "big chorus, peak energy") are genre-generic captions injected at guidance 7 — each chained pass gets pulled further toward stock pop | `SECTION_ARC` / `_section_plan()` | **High** |
61
+ | 4 | The "style adherence" slider label is inverted from reality: raising it makes output *more generic*, not more faithful | `app.py` | High (UX/diagnosis trap) |
62
+ | 5 | 6 chained passes for 60 s (10 s new per pass) — each pass re-encodes generated audio and re-rolls the dice; drift compounds per pass. audiocraft's own extend loop uses `extend_stride=18` (12 s context, 18 s new) = ~3 passes for 60 s | `per_pass` math in `continue_track()` | Medium |
63
+ | 6 | Vocal input goes straight into the prompt. MusicGen has never seen a voice; it will drop, garble, or replace it | `_load_tail()` / pipeline design | **Critical for any vocal demo** |
64
+ | 7 | No timbre/EQ post-matching — generated 32 kHz-band-limited mono is spliced against a 44.1 kHz master with only an RMS gain match, so even a good continuation reads as "different record" | `stitch_with_crossfade()` | Medium |
65
+
66
+ What's already **correct** (don't touch): fp32 + TF32, librosa resample to 32 kHz with `sampling_rate=` passed, mono 1-D float32 input, crossfade inside the codec-roundtripped overlap, equal-gain seam for correlated material, resampling generated audio *up* to the original's rate, window math 20 s + 10 s = 1500 tokens.
67
+
68
+ ---
69
+
70
+ ## 4. The fix — exact changes
71
+
72
+ ### Fix 1 (do first): guidance + text — `continue_music.py`
73
+
74
+ **Two supported modes; implement (a) as default, (b) as the option:**
75
+
76
+ **(a) Pure continuation — no text, no CFG.** This is the canonical mode: Meta's MusicGen-Style page describes continuation as "given the conditioning, MusicGen continues it **without any textual description**", and audiocraft's `generate_continuation(descriptions=None)` is the reference implementation. Training used 20% condition dropout, so unconditional is in-distribution.
77
+
78
+ ```python
79
+ # default path: audio prompt only
80
+ inputs = processor(
81
+ audio=prompt_audio,
82
+ sampling_rate=MUSICGEN_SR,
83
+ return_tensors="pt",
84
+ ).to(model.device)
85
+ # merge in null text conditioning so the seq2seq generate path is well-formed
86
+ uncond = model.get_unconditional_inputs(num_samples=1)
87
+ output = model.generate(
88
+ **inputs,
89
+ encoder_outputs=None, # see note below
90
+ do_sample=True,
91
+ guidance_scale=1.0, # CFG off — nothing for it to amplify anyway
92
+ temperature=1.0,
93
+ top_k=250, # audiocraft defaults
94
+ max_new_tokens=max_new_tokens,
95
+ )
96
+ ```
97
+ (Implementation note: the clean way in transformers is `model.generate(input_values=inputs.input_values, padding_mask=inputs.padding_mask, **vars-from-get_unconditional_inputs-minus-duplicates, guidance_scale=1.0, ...)` — or simply keep passing `text=[""]` through the processor with `guidance_scale=1.0`, which is equivalent in effect since CFG is off. Verify both on a test clip; keep whichever is stable on the installed transformers version. **Pin transformers < 4.57** if you ever touch the melody variant — melody conditioning silently broke after 4.48, GitHub issue #45647.)
98
+
99
+ **(b) Described continuation — user-supplied caption, guidance 3.0.** If the user types a real description ("garage rock, distorted electric guitar, live drums, male vocals removed, 120 bpm, E minor"), that caption is in-distribution and CFG at Meta's trained value 3.0 helps hold the genre. Auto-append detected key/BPM as now. **Never auto-generate meta-instructions.**
100
+
101
+ - Change `continue_track()` default `guidance_scale=7.0` → `1.0`; treat >1 as only valid when a user caption exists.
102
+ - Delete the `base` meta-instruction string.
103
+ - **Delete `SECTION_ARC` and `_section_plan()`** (or reduce to a single final-pass "outro, song ending" hint *only* in described mode). The audio chaining carries continuity; the hints only inject stock-pop pull.
104
+
105
+ ### Fix 2: UI — `app.py`
106
+
107
+ - Replace the "style adherence" slider (currently min 3, so even its minimum is text-amplifying) with:
108
+ - a free-text **"describe your song (optional)"** box → mode (b) when filled, mode (a) when empty;
109
+ - if a guidance slider is kept at all: range 1.0–5.0, default 1.0 (no text) / 3.0 (with text), labeled **"text influence"**.
110
+
111
+ ### Fix 3: vocals — Demucs gate (Demucs is already in requirements.txt)
112
+
113
+ Before building the prompt: run HT-Demucs on the tail, measure vocal-stem energy. If vocals are present:
114
+ 1. Continue **the instrumental mix only** (this matches how the training data itself was prepared — HT-Demucs-separated instrumentals are exactly in-distribution).
115
+ 2. Keep the user's original (with vocals) untouched up to the seam; the continuation is instrumental.
116
+ 3. Update the README honestly: lyrics from Qwen3 are a *songwriting aid* deliverable, not synthesized vocals. (No open model on a ZeroGPU budget will clone-and-continue the user's voice well — YuE-extend is the closest and it's slow and finicky.)
117
+
118
+ ### Fix 4: chaining — match audiocraft's reference loop
119
+
120
+ Keep first-pass prompt at 15–20 s (community sweet spot for fidelity). For subsequent passes use **12 s of context / 18 s new** (audiocraft's `extend_stride=18` default): 60 s of new audio in 3 passes instead of 6 — half the drift opportunities, half the GPU time. Keep total tokens ≤ 1500 per pass.
121
+
122
+ ### Fix 5: post-match the seam — matchering
123
+
124
+ Add [`matchering`](https://github.com/sergree/matchering) (pure-Python, lightweight): REFERENCE = the original clip, TARGET = the generated continuation, before `stitch_with_crossfade()`. It matches RMS, spectral envelope, and peak — this is the single cheapest improvement to "sounds like the same record," and it's exactly what an audio engineer would do at a session splice: match the master, then crossfade on a musical boundary. (Optional polish: use the detected BPM to snap the seam to a downbeat.) Note it cannot invent content above 16 kHz — the 32 kHz EnCodec ceiling stands.
125
+
126
+ ### Expected outcome after Fixes 1–5
127
+
128
+ Tempo/key/groove continuity, broadly matching instrumentation, no genre teleport, production glued by EQ-match. Still mono-derived, still slightly "AI-sheened," still instrumental-only. That is MusicGen's ceiling.
129
+
130
+ ---
131
+
132
+ ## 5. If the ceiling isn't good enough: the 2026 alternatives
133
+
134
+ Ranked for CODA's use case (faithful extension of arbitrary user audio, single ZeroGPU slice):
135
+
136
+ 1. **ACE-Step 1.5** (Jan 2026; Apache/MIT-style, 8–16 GB VRAM) — the only open model where "**extend** my actual audio, keep timbre" is a designed first-class feature (`extend`, `repaint`, `audio_cover_strength`), **with vocal support in 50+ languages**. Best single-model replacement candidate; chain repaints for arbitrary length. Caveat: limited independent benchmarking yet.
137
+ 2. **Stable Audio 3 Medium** (May 2026; 1.4B, ~6 GB, Stability Community License) — documented causal continuation/inpainting mode, 44.1 kHz **stereo** (fixes the mono+bandwidth ceiling). Unproven on vocals. Worth an A/B against ACE-Step on instrumental material.
138
+ 3. **YuE-extend** — true continuation with voice cloning from separated stems; best vocal continuity in open weights, but slow (minutes/song) and ≤30 s prompt practical limit.
139
+ 4. **MusicGen-Style** (facebook/musicgen-style) — conditions on a 1.5–4.5 s excerpt for timbre/style match; combining it with a continuation prefix is theoretically possible in audiocraft but undocumented/unvalidated. Research-grade only.
140
+ 5. **Per-stem MusicGen continuation: don't.** Isolated stems are out-of-distribution for MusicGen (literature: models "produce nonsensical outputs when fed isolated vocals" — SingSong); only stem-native models (MusicGen-Stem, StemGen) do this, and their weights are research releases.
141
+
142
+ **License flag:** MusicGen weights are **CC-BY-NC** (non-commercial). Fine for the hackathon; a problem if CODA ever ships commercially. ACE-Step and Stable Audio 3 (under $1M revenue) are both more permissive.
143
+
144
+ **Recommended path:** ship the hackathon on MusicGen with Fixes 1–5 (small, low-risk diffs), and prototype ACE-Step 1.5 `extend` as CODA v2's backbone — it is designed to do exactly what CODA promises, including vocals.
145
+
146
+ ---
147
+
148
+ ## 6. Failure modes — explained
149
+
150
+ | Symptom | Cause | Status |
151
+ |---|---|---|
152
+ | 8-bit / chiptune output | fp16 EnCodec/T5 path (precision-sensitive); confirmed pattern in community + literature (TinyMusician keeps EnCodec fp32 for this reason) | **Fixed** by the fp32 change — keep it |
153
+ | Random new instruments | Text guidance pulling toward caption prior + per-pass re-rolls in 6-pass chaining | Fixes 1, 4 |
154
+ | Generic MusicGen-default sound | Vague/meta caption → stock-library prior (training data literally is stock music) | Fixes 1, 2 |
155
+ | "Weird funk pop" | guidance 7 × meaningless caption × "more energy" section hints = maximum pull toward the stock-pop prior | Fixes 1, 2 (delete SECTION_ARC) |
156
+ | ~14 s output | Deployment issue, already fixed | — |
157
+ | Vocals garbled/vanish | Vocal-free training data — model-level impossibility | Fix 3 (Demucs gate) |
158
+
159
+ ---
160
+
161
+ ## 7. What a professional audio engineer would say
162
+
163
+ A session engineer asked to "extend this demo" would: comp from the same takes, or re-amp/re-track matching the original chain, then **match the master** (EQ/loudness/width to the reference) and **edit on the grid** (splice at a downbeat, short crossfade). The transferable lessons for CODA: (1) never expect a different "performer" (the model) to nail the take — minimize how much it has to invent (longer audio context, no contradicting text); (2) the *master-matching* step is half of what makes a splice invisible — that's the matchering pass; (3) cut on musical boundaries — use the BPM grid CODA already detects to place the seam on a downbeat; (4) when a part can't be reproduced (vocals), you don't fake it — you arrange around it (instrumental continuation).
analyze.py CHANGED
@@ -1,6 +1,7 @@
 
1
  import librosa
2
  import numpy as np
3
-
4
 
5
  # krumhansl-schmuckler key profiles
6
  # major and minor correlation vectors for pitch class distribution
@@ -13,12 +14,12 @@ MINOR_PROFILE = np.array([6.33, 2.68, 3.52, 5.38, 2.60, 3.53,
13
  PITCH_CLASSES = ['C', 'C#', 'D', 'D#', 'E', 'F',
14
  'F#', 'G', 'G#', 'A', 'A#', 'B']
15
 
 
 
16
 
17
- def find_key(path):
18
- """chroma-based key detection using krumhansl-schmuckler profiles"""
19
- track, sr = librosa.load(path, sr=None)
20
 
21
- # pull chroma energy, average across time
 
22
  chroma = librosa.feature.chroma_cqt(y=track, sr=sr)
23
  pitch_dist = np.mean(chroma, axis=1)
24
 
@@ -48,40 +49,83 @@ def find_key(path):
48
  return best_key
49
 
50
 
51
- def get_tempo(path):
52
- track, sr = librosa.load(path, sr=None)
53
- tempo, _ = librosa.beat.beat_track(y=track, sr=sr)
54
  # librosa sometimes returns an array
55
  if hasattr(tempo, '__len__'):
56
  return float(tempo[0])
57
  return float(tempo)
58
 
59
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  def get_duration(path):
61
  return librosa.get_duration(path=path)
62
 
63
 
64
  def fingerprint(path):
65
- track, sr = librosa.load(path, sr=None, mono=False)
66
-
67
- channels = 1 if track.ndim == 1 else track.shape[0]
68
 
69
- # reload mono for analysis
70
- if channels > 1:
71
- mono = librosa.to_mono(track)
72
- else:
73
- mono = track
74
 
75
- key_sig = find_key(path)
76
- bpm = get_tempo(path)
77
- length = librosa.get_duration(y=mono, sr=sr)
 
 
 
78
 
79
  return {
80
- 'key': key_sig,
81
- 'bpm': round(bpm, 1),
82
- 'duration': round(length, 2),
83
- 'sample_rate': sr,
84
- 'channels': channels
 
85
  }
86
 
87
 
@@ -94,6 +138,7 @@ if __name__ == '__main__':
94
  info = fingerprint(sys.argv[1])
95
  print(f"key: {info['key']}")
96
  print(f"bpm: {info['bpm']}")
 
97
  print(f"duration: {info['duration']}s")
98
  print(f"sample rate: {info['sample_rate']}Hz")
99
  print(f"channels: {info['channels']}")
 
1
+ """key / tempo / meter detection. pure librosa dsp — no ML."""
2
  import librosa
3
  import numpy as np
4
+ import soundfile as sf
5
 
6
  # krumhansl-schmuckler key profiles
7
  # major and minor correlation vectors for pitch class distribution
 
14
  PITCH_CLASSES = ['C', 'C#', 'D', 'D#', 'E', 'F',
15
  'F#', 'G', 'G#', 'A', 'A#', 'B']
16
 
17
+ # all the analysis here works on envelopes and chroma; 22k mono is plenty
18
+ ANALYSIS_SR = 22050
19
 
 
 
 
20
 
21
+ def _key_from_audio(track, sr):
22
+ """chroma-based key detection using krumhansl-schmuckler profiles"""
23
  chroma = librosa.feature.chroma_cqt(y=track, sr=sr)
24
  pitch_dist = np.mean(chroma, axis=1)
25
 
 
49
  return best_key
50
 
51
 
52
+ def _scalar_tempo(tempo):
 
 
53
  # librosa sometimes returns an array
54
  if hasattr(tempo, '__len__'):
55
  return float(tempo[0])
56
  return float(tempo)
57
 
58
 
59
+ def _time_signature_from_beats(onset_env, beats):
60
+ """
61
+ estimate the meter from where the accents land: take the onset strength
62
+ at each tracked beat and score how well a "downbeat every N beats" grid
63
+ lines up with the loud ones, for N = 3 and 4. waltz time has to win
64
+ clearly — ambiguous material is called 4/4, like most music.
65
+ """
66
+ if len(beats) < 12:
67
+ return "4/4"
68
+
69
+ strengths = onset_env[beats].astype(float)
70
+ spread = strengths.std()
71
+ if spread < 1e-8:
72
+ return "4/4"
73
+ strengths = (strengths - strengths.mean()) / spread
74
+
75
+ def accent_score(meter):
76
+ # best phase alignment of the downbeat grid
77
+ return max(float(np.mean(strengths[offset::meter]))
78
+ for offset in range(meter))
79
+
80
+ three, four = accent_score(3), accent_score(4)
81
+ if three > 0 and three > four * 1.25:
82
+ return "3/4"
83
+ return "4/4"
84
+
85
+
86
+ def find_key(path):
87
+ track, sr = librosa.load(path, sr=ANALYSIS_SR, mono=True)
88
+ return _key_from_audio(track, sr)
89
+
90
+
91
+ def get_tempo(path):
92
+ track, sr = librosa.load(path, sr=ANALYSIS_SR, mono=True)
93
+ tempo, _ = librosa.beat.beat_track(y=track, sr=sr)
94
+ return _scalar_tempo(tempo)
95
+
96
+
97
+ def get_time_signature(path):
98
+ track, sr = librosa.load(path, sr=ANALYSIS_SR, mono=True)
99
+ onset_env = librosa.onset.onset_strength(y=track, sr=sr)
100
+ _, beats = librosa.beat.beat_track(onset_envelope=onset_env, sr=sr)
101
+ return _time_signature_from_beats(onset_env, beats)
102
+
103
+
104
  def get_duration(path):
105
  return librosa.get_duration(path=path)
106
 
107
 
108
  def fingerprint(path):
109
+ """one-stop analysis: key, bpm, meter, duration — single decode pass."""
110
+ track, sr = librosa.load(path, sr=ANALYSIS_SR, mono=True)
 
111
 
112
+ onset_env = librosa.onset.onset_strength(y=track, sr=sr)
113
+ tempo, beats = librosa.beat.beat_track(onset_envelope=onset_env, sr=sr)
 
 
 
114
 
115
+ # source-file metadata without a second full decode
116
+ try:
117
+ meta = sf.info(path)
118
+ native_sr, channels = meta.samplerate, meta.channels
119
+ except Exception:
120
+ native_sr, channels = sr, 1
121
 
122
  return {
123
+ 'key': _key_from_audio(track, sr),
124
+ 'bpm': round(_scalar_tempo(tempo), 1),
125
+ 'time_signature': _time_signature_from_beats(onset_env, beats),
126
+ 'duration': round(len(track) / sr, 2),
127
+ 'sample_rate': native_sr,
128
+ 'channels': channels,
129
  }
130
 
131
 
 
138
  info = fingerprint(sys.argv[1])
139
  print(f"key: {info['key']}")
140
  print(f"bpm: {info['bpm']}")
141
+ print(f"time signature: {info['time_signature']} (estimated)")
142
  print(f"duration: {info['duration']}s")
143
  print(f"sample rate: {info['sample_rate']}Hz")
144
  print(f"channels: {info['channels']}")
app.py CHANGED
@@ -1,9 +1,12 @@
1
- import gradio as gr
2
  import os
3
  import tempfile
4
- import numpy as np
 
 
5
  import soundfile as sf
6
- from analyze import fingerprint, find_key, get_tempo, get_duration
 
 
7
 
8
  try:
9
  import spaces
@@ -69,6 +72,14 @@ html, body, gradio-app, .gradio-container {
69
  }
70
 
71
  /* ---- panels ---- */
 
 
 
 
 
 
 
 
72
  .upload-panel, .results-panel, .continue-panel {
73
  background: var(--coda-surface) !important;
74
  border: 1px solid var(--coda-border) !important;
@@ -136,6 +147,59 @@ html, body, gradio-app, .gradio-container {
136
  line-height: 1.1 !important;
137
  }
138
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
139
  /* ---- buttons ---- */
140
  .gradio-container button.primary {
141
  background: var(--coda-accent) !important;
@@ -221,26 +285,26 @@ def make_stat_html(label, value):
221
  )
222
 
223
 
224
- def analyze_track(audio):
225
- if audio is None:
226
- return (
227
- make_stat_html("key", "---"),
228
- make_stat_html("tempo", "---"),
229
- make_stat_html("duration", "---"),
230
- gr.update(visible=False),
231
- None,
232
- None
233
- )
234
 
235
- try:
236
- info = fingerprint(audio)
237
- key_html = make_stat_html("key", info["key"])
238
- bpm_html = make_stat_html("tempo", str(info["bpm"]) + " bpm")
239
- dur_html = make_stat_html("duration", str(info["duration"]) + "s")
240
- return (key_html, bpm_html, dur_html, gr.update(visible=True), audio, info)
241
- except Exception as e:
242
- err = make_stat_html("error", str(e)[:50])
243
- return (err, make_stat_html("tempo", "---"), make_stat_html("duration", "---"), gr.update(visible=False), None, None)
 
 
 
 
244
 
245
 
246
  def _output_subtype(audio_path):
@@ -258,59 +322,195 @@ def _output_subtype(audio_path):
258
 
259
 
260
  @spaces.GPU(duration=300)
261
- def run_continuation(audio_path, info, gen_seconds, fade_seconds, caption):
262
- if not audio_path:
263
- return None, "upload a track first."
 
 
 
 
 
 
 
 
 
 
 
 
264
 
265
- try:
266
- # lazy import so the UI boots fast and torch only loads on demand
267
- import librosa
268
- from continue_music import (continue_track, master_match,
269
- stitch_with_crossfade)
270
-
271
- key = info.get("key") if info else None
272
- bpm = info.get("bpm") if info else None
273
-
274
- # generate at musicgen's native 32kHz, chaining passes for long
275
- # extensions. no caption = pure audio continuation (CFG off); a user
276
- # caption switches to described mode at guidance 3.0.
277
- extended_tail, prompt_samples, gen_sr = continue_track(
278
- audio_path,
279
- gen_duration=float(gen_seconds),
280
- key=key,
281
- bpm=bpm,
282
- fade_seconds=float(fade_seconds),
283
- caption=caption or "",
284
- )
285
 
286
- # keep the original at its native sample rate and channel count —
287
- # the generated audio gets resampled up to match, never the reverse
288
- original, orig_sr = librosa.load(audio_path, sr=None, mono=False)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
289
 
290
- # mastering pass: match the generation's loudness and eq to the
291
- # original before splicing so both halves read as the same record
292
- extended_tail = master_match(extended_tail, original, orig_sr)
 
 
 
 
 
 
 
 
 
 
 
 
293
 
294
- full = stitch_with_crossfade(
295
- original, extended_tail, prompt_samples, orig_sr,
296
- fade_seconds=float(fade_seconds),
 
 
 
 
 
 
 
 
 
 
297
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
298
 
299
- out_path = os.path.join(tempfile.mkdtemp(), "coda_continuation.wav")
300
- # soundfile expects (samples, channels)
301
- sf.write(out_path, full.T if full.ndim == 2 else full, orig_sr,
302
- subtype=_output_subtype(audio_path))
303
-
304
- added = (full.shape[-1] - original.shape[-1]) / orig_sr
305
- seam_at = original.shape[-1] / orig_sr - float(fade_seconds)
306
- return out_path, (
307
- f"added {added:.1f}s of new audio at {orig_sr} hz "
308
- f"({_output_subtype(audio_path).replace('_', ' ').lower()}). "
309
- f"one unified track: blend completes at {seam_at + float(fade_seconds):.1f}s, "
310
- f"outro fades to silence."
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
311
  )
 
 
 
 
 
 
312
  except Exception as e:
313
- return None, f"continuation failed: {str(e)[:200]}"
 
314
 
315
 
316
  HEADER_HTML = (
@@ -349,7 +549,12 @@ with gr.Blocks(title="CODA") as app:
349
  with gr.Row():
350
  key_display = gr.HTML(make_stat_html("key", "---"))
351
  bpm_display = gr.HTML(make_stat_html("tempo", "---"))
352
- dur_display = gr.HTML(make_stat_html("duration", "---"))
 
 
 
 
 
353
 
354
  with gr.Group(elem_classes="continue-panel", visible=False) as continue_section:
355
  gr.HTML('<div class="section-label">Continue</div>')
@@ -369,20 +574,49 @@ with gr.Blocks(title="CODA") as app:
369
  placeholder="leave empty to continue from the audio alone — or e.g. \"garage rock, distorted electric guitar, live drums\"",
370
  lines=1,
371
  )
372
- continue_btn = gr.Button("Continue this track", variant="primary")
373
- result_audio = gr.Audio(label="finished track", type="filepath", interactive=False)
374
- continue_output = gr.Textbox(label="status", interactive=False, lines=2)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
375
 
376
  audio_input.change(
377
  fn=analyze_track,
378
  inputs=[audio_input],
379
- outputs=[key_display, bpm_display, dur_display, continue_section, current_track, track_info]
 
 
380
  )
381
 
382
  continue_btn.click(
383
- fn=run_continuation,
384
- inputs=[current_track, track_info, gen_length, fade_length, caption_box],
385
- outputs=[result_audio, continue_output]
 
 
 
386
  )
387
 
388
  gr.HTML(FOOTER_HTML)
@@ -390,4 +624,3 @@ with gr.Blocks(title="CODA") as app:
390
 
391
  if __name__ == "__main__":
392
  app.launch(theme=gr.themes.Base(), css=CUSTOM_CSS)
393
-
 
 
1
  import os
2
  import tempfile
3
+ from pathlib import Path
4
+
5
+ import gradio as gr
6
  import soundfile as sf
7
+
8
+ from analyze import fingerprint
9
+ from poster import make_poster, render_waveform_png
10
 
11
  try:
12
  import spaces
 
72
  }
73
 
74
  /* ---- panels ---- */
75
+ /* gradio 6 wraps group children in a .styler div with a zinc background;
76
+ flatten it (and row blocks / forms) so the panel surface reads as one */
77
+ .gradio-container .styler,
78
+ .gradio-container .row .block,
79
+ .gradio-container .form {
80
+ background: transparent !important;
81
+ border: none !important;
82
+ }
83
  .upload-panel, .results-panel, .continue-panel {
84
  background: var(--coda-surface) !important;
85
  border: 1px solid var(--coda-border) !important;
 
147
  line-height: 1.1 !important;
148
  }
149
 
150
+ /* ---- waveform & poster images ---- */
151
+ .waveform-img { margin-top: 14px; }
152
+ .waveform-img img { border-radius: 12px; width: 100%; }
153
+ .poster-img img { border-radius: 12px; }
154
+ .gradio-container .image-container {
155
+ background: var(--coda-surface-2) !important;
156
+ border-color: var(--coda-border) !important;
157
+ }
158
+
159
+ /* ---- stage tracker ---- */
160
+ .stage-list {
161
+ display: flex;
162
+ flex-direction: column;
163
+ gap: 9px;
164
+ padding: 14px 4px 4px 4px;
165
+ }
166
+ .stage {
167
+ display: flex;
168
+ align-items: center;
169
+ gap: 11px;
170
+ color: var(--coda-muted) !important;
171
+ font-size: 0.88em;
172
+ letter-spacing: 0.01em;
173
+ }
174
+ .stage span { color: inherit !important; }
175
+ .stage .dot {
176
+ width: 9px; height: 9px;
177
+ border-radius: 50%;
178
+ background: #3a4150;
179
+ flex: none;
180
+ transition: background 0.2s ease;
181
+ }
182
+ .stage.active { color: var(--coda-text) !important; }
183
+ .stage.active .dot {
184
+ background: var(--coda-accent);
185
+ animation: coda-pulse 1.2s ease-in-out infinite;
186
+ }
187
+ .stage.done .dot { background: var(--coda-accent); }
188
+ .stage.skip { opacity: 0.55; }
189
+ .stage.error { color: #e07a6a !important; }
190
+ .stage.error .dot { background: #e07a6a; }
191
+ @keyframes coda-pulse {
192
+ 0%, 100% { box-shadow: 0 0 0 0 rgba(226,168,92,0.45); }
193
+ 50% { box-shadow: 0 0 0 6px rgba(226,168,92,0); }
194
+ }
195
+ .stage-note {
196
+ color: var(--coda-muted) !important;
197
+ font-size: 0.8em;
198
+ padding: 10px 4px 0 4px;
199
+ line-height: 1.5;
200
+ }
201
+ .stage-note span { color: inherit !important; }
202
+
203
  /* ---- buttons ---- */
204
  .gradio-container button.primary {
205
  background: var(--coda-accent) !important;
 
285
  )
286
 
287
 
288
+ PIPELINE_STAGES = (
289
+ "separate & continue the music",
290
+ "master & stitch the seam",
291
+ "transcribe & extend the lyrics",
292
+ "press the poster",
293
+ )
 
 
 
 
294
 
295
+
296
+ def stages_html(states, note="", labels=None):
297
+ """render the pipeline progress tracker. states are per-stage:
298
+ pending | active | done | skip | error."""
299
+ labels = labels or PIPELINE_STAGES
300
+ rows = []
301
+ for label, state in zip(labels, states):
302
+ rows.append(
303
+ f'<div class="stage {state}"><span class="dot"></span>'
304
+ f'<span>{label}</span></div>'
305
+ )
306
+ note_html = f'<div class="stage-note">{note}</div>' if note else ""
307
+ return f'<div class="stage-list">{"".join(rows)}</div>{note_html}'
308
 
309
 
310
  def _output_subtype(audio_path):
 
322
 
323
 
324
  @spaces.GPU(duration=300)
325
+ def generate_continuation(audio_path, key, bpm, gen_seconds, fade_seconds, caption):
326
+ """gpu stage 1: demucs vocal gate + chained musicgen passes.
327
+ mastering and stitching are pure cpu work and run outside the gpu window.
328
+ no caption = pure audio continuation (CFG off at guidance 1.0); a user
329
+ caption switches to described mode at meta's trained guidance 3.0."""
330
+ # lazy import so the UI boots fast and torch only loads on demand
331
+ from continue_music import continue_track
332
+ return continue_track(
333
+ audio_path,
334
+ gen_duration=float(gen_seconds),
335
+ key=key,
336
+ bpm=bpm,
337
+ fade_seconds=float(fade_seconds),
338
+ caption=caption or "",
339
+ )
340
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
341
 
342
+ @spaces.GPU(duration=180)
343
+ def generate_lyrics(audio_path, key, bpm):
344
+ """gpu stage 2: whisper transcribes the demucs-isolated vocals, qwen3
345
+ writes the next lines. returns (original, continuation) or (None, None)
346
+ when the clip is instrumental."""
347
+ from transcribe import transcribe_vocals
348
+ original = transcribe_vocals(audio_path)
349
+ if not original:
350
+ return None, None
351
+ from write_lyrics import continue_lyrics
352
+ return original, continue_lyrics(original, key=key, bpm=bpm)
353
+
354
+
355
+ def master_and_stitch(audio_path, extended_tail, prompt_samples, fade_seconds):
356
+ """cpu stage: matchering mastering pass + native-rate splice + file write."""
357
+ import librosa
358
+ from continue_music import master_match, stitch_with_crossfade
359
+
360
+ # keep the original at its native sample rate and channel count —
361
+ # the generated audio gets resampled up to match, never the reverse
362
+ original, orig_sr = librosa.load(audio_path, sr=None, mono=False)
363
+
364
+ # mastering pass: match the generation's loudness and eq to the
365
+ # original before splicing so both halves read as the same record
366
+ extended_tail = master_match(extended_tail, original, orig_sr)
367
+
368
+ full = stitch_with_crossfade(
369
+ original, extended_tail, prompt_samples, orig_sr,
370
+ fade_seconds=float(fade_seconds),
371
+ )
372
+
373
+ out_path = os.path.join(tempfile.mkdtemp(), "coda_continuation.wav")
374
+ # soundfile expects (samples, channels)
375
+ sf.write(out_path, full.T if full.ndim == 2 else full, orig_sr,
376
+ subtype=_output_subtype(audio_path))
377
+
378
+ added = (full.shape[-1] - original.shape[-1]) / orig_sr
379
+ msg = (
380
+ f"added {added:.1f}s of new audio at {orig_sr} hz "
381
+ f"({_output_subtype(audio_path).replace('_', ' ').lower()}). "
382
+ f"blend completes at {original.shape[-1] / orig_sr:.1f}s, "
383
+ f"outro fades to silence."
384
+ )
385
+ return out_path, msg
386
+
387
 
388
+ def analyze_track(audio):
389
+ hide = gr.update(visible=False)
390
+ blank = (
391
+ make_stat_html("key", "---"),
392
+ make_stat_html("tempo", "---"),
393
+ make_stat_html("meter", "---"),
394
+ make_stat_html("length", "---"),
395
+ hide, # waveform
396
+ hide, # continue section
397
+ None, None, # states
398
+ gr.update(value="", visible=False), # status tracker
399
+ hide, hide, hide, # result audio / lyrics / poster panels
400
+ )
401
+ if audio is None:
402
+ return blank
403
 
404
+ try:
405
+ info = fingerprint(audio)
406
+ wave_png = render_waveform_png(audio)
407
+ return (
408
+ make_stat_html("key", info["key"]),
409
+ make_stat_html("tempo", str(info["bpm"]) + " bpm"),
410
+ make_stat_html("meter", info["time_signature"]),
411
+ make_stat_html("length", str(info["duration"]) + "s"),
412
+ gr.update(value=wave_png, visible=True),
413
+ gr.update(visible=True),
414
+ audio, info,
415
+ gr.update(value="", visible=False),
416
+ hide, hide, hide,
417
  )
418
+ except Exception as e:
419
+ out = list(blank)
420
+ out[0] = make_stat_html("error", str(e)[:50])
421
+ return tuple(out)
422
+
423
+
424
+ def run_pipeline(audio_path, info, gen_seconds, fade_seconds, caption,
425
+ original_date):
426
+ """generator: walks the four pipeline stages, streaming tracker updates
427
+ and revealing each result as it lands. outputs:
428
+ (status, audio_panel, audio, lyrics_panel, lyrics_orig, lyrics_cont,
429
+ poster_panel, poster)"""
430
+ skip = gr.skip()
431
+
432
+ def frame(states, note="", labels=None, audio_vis=skip, audio=skip,
433
+ lyrics_vis=skip, lyr_o=skip, lyr_c=skip, poster_vis=skip,
434
+ poster=skip):
435
+ return (gr.update(value=stages_html(states, note, labels), visible=True),
436
+ audio_vis, audio, lyrics_vis, lyr_o, lyr_c, poster_vis, poster)
437
+
438
+ states = ["pending"] * len(PIPELINE_STAGES)
439
+ labels = list(PIPELINE_STAGES)
440
+
441
+ if not audio_path or not info:
442
+ states[0] = "error"
443
+ yield frame(states, "upload a track first.")
444
+ return
445
+
446
+ states[0] = "active"
447
+ yield frame(states,
448
+ "musicgen large is listening to your track and continuing it "
449
+ "— this is the long stage.",
450
+ audio_vis=gr.update(visible=False),
451
+ lyrics_vis=gr.update(visible=False),
452
+ poster_vis=gr.update(visible=False))
453
+
454
+ try:
455
+ extended_tail, prompt_samples, _ = generate_continuation(
456
+ audio_path, info.get("key"), info.get("bpm"),
457
+ gen_seconds, fade_seconds, caption)
458
+ states[0] = "done"
459
+ states[1] = "active"
460
+ yield frame(states, "matching loudness and eq to your recording, "
461
+ "then splicing at the seam.")
462
+ out_path, stitch_msg = master_and_stitch(
463
+ audio_path, extended_tail, prompt_samples, fade_seconds)
464
+ states[1] = "done"
465
+ except Exception as e:
466
+ states[states.index("active")] = "error"
467
+ yield frame(states, f"continuation failed: {str(e)[:200]}")
468
+ return
469
 
470
+ states[2] = "active"
471
+ yield frame(states, stitch_msg,
472
+ audio_vis=gr.update(visible=True),
473
+ audio=gr.update(value=out_path))
474
+
475
+ lyr_o = lyr_c = None
476
+ try:
477
+ lyr_o, lyr_c = generate_lyrics(
478
+ audio_path, info.get("key"), info.get("bpm"))
479
+ except Exception as e:
480
+ print(f"[coda] lyric stage failed: {e}", flush=True)
481
+
482
+ if lyr_o:
483
+ states[2] = "done"
484
+ yield frame(states, stitch_msg,
485
+ lyrics_vis=gr.update(visible=True),
486
+ lyr_o=lyr_o, lyr_c=lyr_c or "")
487
+ else:
488
+ states[2] = "skip"
489
+ labels[2] = "transcribe & extend the lyrics — no vocals found, skipped"
490
+ yield frame(states, stitch_msg, labels=labels)
491
+
492
+ states[3] = "active"
493
+ yield frame(states, stitch_msg, labels=labels)
494
+ try:
495
+ title = Path(audio_path).stem.replace("_", " ").replace("-", " ").strip()
496
+ poster_path = make_poster(
497
+ out_path,
498
+ title=title or "untitled",
499
+ key_sig=info.get("key", "?"),
500
+ bpm=info.get("bpm", 0) or 0,
501
+ time_sig=info.get("time_signature", "4/4"),
502
+ split_seconds=float(info.get("duration", 0) or 0),
503
+ original_date=original_date or "",
504
  )
505
+ states[3] = "done"
506
+ yield frame(states,
507
+ "done. your song is finished — save the poster.",
508
+ labels=labels,
509
+ poster_vis=gr.update(visible=True),
510
+ poster=gr.update(value=poster_path))
511
  except Exception as e:
512
+ states[3] = "error"
513
+ yield frame(states, f"poster failed: {str(e)[:160]}", labels=labels)
514
 
515
 
516
  HEADER_HTML = (
 
549
  with gr.Row():
550
  key_display = gr.HTML(make_stat_html("key", "---"))
551
  bpm_display = gr.HTML(make_stat_html("tempo", "---"))
552
+ sig_display = gr.HTML(make_stat_html("meter", "---"))
553
+ dur_display = gr.HTML(make_stat_html("length", "---"))
554
+ waveform_display = gr.Image(
555
+ visible=False, show_label=False, container=False,
556
+ interactive=False, buttons=[], elem_classes="waveform-img",
557
+ )
558
 
559
  with gr.Group(elem_classes="continue-panel", visible=False) as continue_section:
560
  gr.HTML('<div class="section-label">Continue</div>')
 
574
  placeholder="leave empty to continue from the audio alone — or e.g. \"garage rock, distorted electric guitar, live drums\"",
575
  lines=1,
576
  )
577
+ date_box = gr.Textbox(
578
+ label="when did you start this song? (optional)",
579
+ placeholder="e.g. \"2018\" or \"march 2019\" — printed on the poster",
580
+ lines=1,
581
+ )
582
+ continue_btn = gr.Button("Finish this song", variant="primary")
583
+ status_html = gr.HTML(visible=False)
584
+
585
+ with gr.Group(elem_classes="results-panel", visible=False) as result_section:
586
+ gr.HTML('<div class="section-label">The finished song</div>')
587
+ result_audio = gr.Audio(label="finished track", type="filepath",
588
+ interactive=False)
589
+
590
+ with gr.Group(elem_classes="results-panel", visible=False) as lyrics_section:
591
+ gr.HTML('<div class="section-label">Lyrics</div>')
592
+ with gr.Row():
593
+ lyrics_orig = gr.Textbox(label="what you had", lines=8,
594
+ interactive=False)
595
+ lyrics_cont = gr.Textbox(label="where it goes next", lines=8,
596
+ interactive=False)
597
+
598
+ with gr.Group(elem_classes="results-panel", visible=False) as poster_section:
599
+ gr.HTML('<div class="section-label">Song poster</div>')
600
+ poster_img = gr.Image(label="poster", type="filepath",
601
+ interactive=False, show_label=False,
602
+ buttons=["download", "fullscreen"],
603
+ elem_classes="poster-img")
604
 
605
  audio_input.change(
606
  fn=analyze_track,
607
  inputs=[audio_input],
608
+ outputs=[key_display, bpm_display, sig_display, dur_display,
609
+ waveform_display, continue_section, current_track, track_info,
610
+ status_html, result_section, lyrics_section, poster_section],
611
  )
612
 
613
  continue_btn.click(
614
+ fn=run_pipeline,
615
+ inputs=[current_track, track_info, gen_length, fade_length,
616
+ caption_box, date_box],
617
+ outputs=[status_html, result_section, result_audio,
618
+ lyrics_section, lyrics_orig, lyrics_cont,
619
+ poster_section, poster_img],
620
  )
621
 
622
  gr.HTML(FOOTER_HTML)
 
624
 
625
  if __name__ == "__main__":
626
  app.launch(theme=gr.themes.Base(), css=CUSTOM_CSS)
 
poster.py CHANGED
@@ -1,79 +1,176 @@
1
- from PIL import Image, ImageDraw, ImageFont
 
 
 
 
 
 
2
  import os
 
 
 
 
 
3
 
4
- # warm palette matching the app
5
- BG_COLOR = (26, 23, 20)
6
- AMBER = (200, 149, 108)
7
- CREAM = (240, 236, 228)
8
- DARK_ACCENT = (60, 50, 40)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
 
11
- def make_poster(title, key_sig, bpm, duration, out_path='poster.png'):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  """
13
- generates a simple album-art-style card for the analyzed track.
14
- no ML here, just pillow.
 
15
  """
16
- w, h = 800, 800
17
- img = Image.new('RGB', (w, h), BG_COLOR)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  draw = ImageDraw.Draw(img)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
- # big warm circle as a vinyl record silhouette
21
- cx, cy = w // 2, h // 2 - 40
22
- radius = 240
23
- draw.ellipse(
24
- [cx - radius, cy - radius, cx + radius, cy + radius],
25
- fill=DARK_ACCENT,
26
- outline=AMBER,
27
- width=3
28
- )
29
-
30
- # inner circle (label area)
31
- inner_r = 80
32
- draw.ellipse(
33
- [cx - inner_r, cy - inner_r, cx + inner_r, cy + inner_r],
34
- fill=BG_COLOR,
35
- outline=AMBER,
36
- width=2
37
- )
38
-
39
- # spindle dot
40
  draw.ellipse([cx - 6, cy - 6, cx + 6, cy + 6], fill=AMBER)
 
41
 
42
- # grooves (concentric rings)
43
- for r in range(inner_r + 20, radius, 16):
44
- draw.ellipse(
45
- [cx - r, cy - r, cx + r, cy + r],
46
- outline=(50, 42, 34),
47
- width=1
48
- )
49
 
50
- # text below the record
51
- try:
52
- title_font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", 32)
53
- detail_font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 22)
54
- small_font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 16)
55
- except OSError:
56
- title_font = ImageFont.load_default()
57
- detail_font = title_font
58
- small_font = title_font
59
-
60
- # track title
61
- title_text = title if len(title) < 40 else title[:37] + '...'
62
- bbox = draw.textbbox((0, 0), title_text, font=title_font)
63
- tw = bbox[2] - bbox[0]
64
- draw.text(((w - tw) // 2, h - 200), title_text, fill=CREAM, font=title_font)
65
-
66
- # key + bpm line
67
- info_text = f"{key_sig} · {bpm} BPM · {duration:.1f}s"
68
- bbox = draw.textbbox((0, 0), info_text, font=detail_font)
69
- tw = bbox[2] - bbox[0]
70
- draw.text(((w - tw) // 2, h - 150), info_text, fill=AMBER, font=detail_font)
71
-
72
- # coda branding
73
- brand = "CODA"
74
- bbox = draw.textbbox((0, 0), brand, font=small_font)
75
- tw = bbox[2] - bbox[0]
76
- draw.text(((w - tw) // 2, h - 60), brand, fill=(100, 85, 70), font=small_font)
 
 
 
77
 
 
 
78
  img.save(out_path)
79
  return out_path
 
1
+ """song poster + waveform rendering. pure pillow — no ML.
2
+
3
+ the poster is the shareable artifact: a vinyl-sleeve card carrying the full
4
+ song's waveform (original in cream, AI continuation in amber), the date the
5
+ song was started and the date coda finished it, and the detected
6
+ key / tempo / meter.
7
+ """
8
  import os
9
+ import tempfile
10
+ from datetime import date
11
+
12
+ import numpy as np
13
+ from PIL import Image, ImageDraw, ImageFont
14
 
15
+ # palette mirrors the app css
16
+ BG = (11, 12, 15)
17
+ SURFACE = (20, 22, 27)
18
+ SURFACE_2 = (26, 29, 36)
19
+ BORDER = (38, 42, 51)
20
+ GROOVE = (30, 33, 40)
21
+ TEXT = (237, 238, 242)
22
+ MUTED = (138, 144, 156)
23
+ AMBER = (226, 168, 92)
24
+
25
+ # debian (hf spaces) ships dejavu; windows dev boxes get georgia/arial
26
+ _FONT_PATHS = {
27
+ "serif": ["/usr/share/fonts/truetype/dejavu/DejaVuSerif.ttf",
28
+ "C:/Windows/Fonts/georgia.ttf",
29
+ "C:/Windows/Fonts/times.ttf"],
30
+ "serif_italic": ["/usr/share/fonts/truetype/dejavu/DejaVuSerif-Italic.ttf",
31
+ "C:/Windows/Fonts/georgiai.ttf",
32
+ "C:/Windows/Fonts/timesi.ttf"],
33
+ "sans": ["/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
34
+ "C:/Windows/Fonts/arial.ttf"],
35
+ "sans_bold": ["/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
36
+ "C:/Windows/Fonts/arialbd.ttf"],
37
+ }
38
+
39
+
40
+ def _font(kind, size):
41
+ for path in _FONT_PATHS[kind]:
42
+ try:
43
+ return ImageFont.truetype(path, size)
44
+ except OSError:
45
+ continue
46
+ try:
47
+ return ImageFont.load_default(size)
48
+ except TypeError: # older pillow without size kwarg
49
+ return ImageFont.load_default()
50
 
51
 
52
+ def _center_text(draw, text, cx, top, font, fill):
53
+ bbox = draw.textbbox((0, 0), text, font=font)
54
+ draw.text((cx - (bbox[2] - bbox[0]) / 2, top), text, font=font, fill=fill)
55
+
56
+
57
+ def _envelope(samples, columns):
58
+ """peak amplitude per column, normalized to [0, 1]."""
59
+ if len(samples) == 0:
60
+ return np.zeros(columns)
61
+ chunks = np.array_split(np.abs(samples), columns)
62
+ env = np.array([float(c.max()) if len(c) else 0.0 for c in chunks])
63
+ peak = env.max()
64
+ return env / peak if peak > 0 else env
65
+
66
+
67
+ def _draw_waveform(draw, samples, box, color_a, color_b=None,
68
+ split_frac=None, bar=4, gap=3):
69
  """
70
+ bar-style waveform inside box=(x0, y0, x1, y1). columns left of
71
+ `split_frac` (fraction of the timeline) use color_a, the rest color_b —
72
+ that's how the poster shows where the original ends and coda begins.
73
  """
74
+ x0, y0, x1, y1 = box
75
+ step = bar + gap
76
+ columns = max(1, (x1 - x0) // step)
77
+ env = _envelope(samples, columns)
78
+ mid = (y0 + y1) / 2
79
+ half = (y1 - y0) / 2 - 1
80
+ split_col = columns + 1 if split_frac is None else int(columns * split_frac)
81
+ for i, e in enumerate(env):
82
+ h = max(2.0, e * half)
83
+ x = x0 + i * step
84
+ color = color_a if i < split_col else (color_b or color_a)
85
+ draw.rounded_rectangle([x, mid - h, x + bar, mid + h],
86
+ radius=bar / 2, fill=color)
87
+
88
+
89
+ def render_waveform_png(audio_path, out_path=None, width=1280, height=200):
90
+ """standalone waveform card for the ui, shown right after upload."""
91
+ import librosa
92
+ samples, _ = librosa.load(audio_path, sr=8000, mono=True)
93
+ img = Image.new("RGB", (width, height), SURFACE)
94
  draw = ImageDraw.Draw(img)
95
+ _draw_waveform(draw, samples, (24, 22, width - 24, height - 22), AMBER)
96
+ if out_path is None:
97
+ out_path = os.path.join(tempfile.mkdtemp(), "waveform.png")
98
+ img.save(out_path)
99
+ return out_path
100
+
101
+
102
+ def make_poster(audio_path, title, key_sig, bpm, time_sig, split_seconds,
103
+ original_date="", completed=None, out_path=None):
104
+ """
105
+ 1080x1080 vinyl-sleeve poster for the finished song.
106
+
107
+ audio_path: the FULL stitched song
108
+ split_seconds: where the original ends and the continuation begins
109
+ original_date: free text from the user ("2018", "march 2019", ...)
110
+ """
111
+ import librosa
112
+ samples, sr = librosa.load(audio_path, sr=8000, mono=True)
113
+ total = len(samples) / sr if len(samples) else 1.0
114
+ split_frac = min(1.0, max(0.0, float(split_seconds) / total))
115
+
116
+ w = h = 1080
117
+ img = Image.new("RGB", (w, h), BG)
118
+ draw = ImageDraw.Draw(img)
119
+
120
+ # sleeve frame
121
+ draw.rectangle([24, 24, w - 25, h - 25], outline=BORDER, width=2)
122
 
123
+ # --- vinyl ---
124
+ cx, cy, radius = w // 2, 360, 250
125
+ draw.ellipse([cx - radius, cy - radius, cx + radius, cy + radius],
126
+ fill=SURFACE_2, outline=AMBER, width=3)
127
+ for r in range(112, radius - 12, 13):
128
+ draw.ellipse([cx - r, cy - r, cx + r, cy + r], outline=GROOVE, width=1)
129
+ label_r = 94
130
+ draw.ellipse([cx - label_r, cy - label_r, cx + label_r, cy + label_r],
131
+ fill=BG, outline=AMBER, width=2)
132
+ _center_text(draw, "CODA", cx, cy - 58, _font("sans_bold", 28), AMBER)
 
 
 
 
 
 
 
 
 
 
133
  draw.ellipse([cx - 6, cy - 6, cx + 6, cy + 6], fill=AMBER)
134
+ _center_text(draw, "SIDE B", cx, cy + 32, _font("sans", 17), MUTED)
135
 
136
+ # --- title ---
137
+ title = (title or "untitled").strip()
138
+ if len(title) > 32:
139
+ title = title[:29] + "..."
140
+ _center_text(draw, title, cx, 650, _font("serif", 58), TEXT)
 
 
141
 
142
+ # --- key / tempo / meter ---
143
+ info_line = f"{key_sig} · {round(float(bpm))} bpm · {time_sig}"
144
+ _center_text(draw, info_line, cx, 738, _font("sans", 27), AMBER)
145
+
146
+ # --- waveform: original (cream) vs continuation (amber) ---
147
+ _draw_waveform(draw, samples, (100, 798, 980, 902), TEXT, AMBER, split_frac)
148
+
149
+ # legend
150
+ leg_font = _font("sans", 19)
151
+ items = [("original", TEXT), ("continuation", AMBER)]
152
+ widths = [16 + 8 + draw.textbbox((0, 0), t, font=leg_font)[2] for t, _ in items]
153
+ lx = cx - (sum(widths) + 36) / 2
154
+ for (label, color), iw in zip(items, widths):
155
+ draw.rounded_rectangle([lx, 928, lx + 16, 944], radius=4, fill=color)
156
+ draw.text((lx + 24, 924), label, font=leg_font, fill=MUTED)
157
+ lx += iw + 36
158
+
159
+ # --- dates ---
160
+ completed = completed or date.today()
161
+ completed_text = f"{completed:%B} {completed.day}, {completed.year}"
162
+ original_date = (original_date or "").strip()
163
+ if original_date:
164
+ dates = f"started {original_date} — finished {completed_text}"
165
+ else:
166
+ dates = f"finished {completed_text}"
167
+ _center_text(draw, dates, cx, 974, _font("serif_italic", 25), MUTED)
168
+
169
+ # --- branding ---
170
+ _center_text(draw, "C O M P L E T E D B Y C O D A", cx, 1024,
171
+ _font("sans_bold", 17), AMBER)
172
 
173
+ if out_path is None:
174
+ out_path = os.path.join(tempfile.mkdtemp(), "coda_poster.png")
175
  img.save(out_path)
176
  return out_path
stems.py ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """shared HT-Demucs stem separation.
2
+
3
+ the pipeline separates vocals twice with the same model: continue_music's
4
+ vocal gate prompts musicgen with the instrumental (vocal prompts are
5
+ out-of-distribution — musicgen's training data was demucs-separated), and
6
+ transcribe.py feeds the isolated vocal stem to whisper so it doesn't
7
+ hallucinate on drums and guitars.
8
+ """
9
+ import librosa
10
+ import numpy as np
11
+
12
+ _demucs = None
13
+
14
+
15
+ def _load_demucs():
16
+ global _demucs
17
+ if _demucs is None:
18
+ from demucs.pretrained import get_model
19
+ _demucs = get_model("htdemucs")
20
+ _demucs.eval()
21
+ return _demucs
22
+
23
+
24
+ def isolate_vocals(path, max_seconds=None):
25
+ """
26
+ separate `path` into stems and return the vocal stem.
27
+
28
+ returns (vocals, sr, vocal_ratio):
29
+ vocals float32 (2, N) at the demucs sample rate
30
+ sr demucs sample rate (44100)
31
+ vocal_ratio vocal-stem rms / mix rms — how vocal the clip is
32
+ or (None, None, 0.0) when demucs isn't available.
33
+ """
34
+ try:
35
+ import torch
36
+ from demucs.apply import apply_model
37
+ model = _load_demucs()
38
+ except Exception as e:
39
+ print(f"[coda] demucs unavailable ({e})", flush=True)
40
+ return None, None, 0.0
41
+
42
+ sr = model.samplerate
43
+ wav, _ = librosa.load(path, sr=sr, mono=False)
44
+ if wav.ndim == 1:
45
+ wav = np.stack([wav, wav])
46
+ if max_seconds:
47
+ wav = wav[:, -int(max_seconds * sr):]
48
+ wav = wav.astype(np.float32)
49
+
50
+ device = "cuda" if torch.cuda.is_available() else "cpu"
51
+ mix = torch.from_numpy(np.ascontiguousarray(wav))
52
+ # demucs expects the mix standardized (same as its own separate.py)
53
+ ref = mix.mean(0)
54
+ std = float(ref.std()) + 1e-8
55
+ with torch.no_grad():
56
+ sources = apply_model(
57
+ model.to(device), ((mix - ref.mean()) / std)[None].to(device),
58
+ device=device, split=True, overlap=0.25, progress=False,
59
+ )[0]
60
+ sources = sources.cpu() * std + ref.mean()
61
+ vocals = sources[model.sources.index("vocals")].numpy().astype(np.float32)
62
+
63
+ vocal_rms = float(np.sqrt(np.mean(vocals ** 2)))
64
+ mix_rms = float(np.sqrt(np.mean(wav ** 2)) + 1e-9)
65
+ return vocals, sr, vocal_rms / mix_rms
test_app_logic.py ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """offline tests for the non-GPU pipeline pieces: analysis (key/tempo/meter),
2
+ waveform + poster rendering, whisper hallucination filter, qwen output
3
+ cleaner, and the ui stage tracker. no torch required."""
4
+ import os
5
+ import sys
6
+ import tempfile
7
+ from datetime import date
8
+
9
+ import numpy as np
10
+ import soundfile as sf
11
+
12
+ FAILURES = []
13
+
14
+
15
+ def check(name, cond, detail=""):
16
+ status = "PASS" if cond else "FAIL"
17
+ print(f"[{status}] {name} {detail}")
18
+ if not cond:
19
+ FAILURES.append(name)
20
+
21
+
22
+ def click_track(meter, bpm=120, bars=16, sr=22050):
23
+ """synthetic drum-machine loop: an accented hit on beat 1 of every bar."""
24
+ beat_len = int(sr * 60 / bpm)
25
+ total = beat_len * meter * bars
26
+ y = np.zeros(total, dtype=np.float32)
27
+ rng = np.random.default_rng(7)
28
+ burst = (rng.standard_normal(600) * np.exp(-np.linspace(0, 8, 600))).astype(np.float32)
29
+ for b in range(meter * bars):
30
+ amp = 1.0 if b % meter == 0 else 0.3
31
+ start = b * beat_len
32
+ y[start:start + 600] += amp * burst
33
+ return y, sr
34
+
35
+
36
+ def write_tmp(y, sr):
37
+ path = os.path.join(tempfile.mkdtemp(), "t.wav")
38
+ sf.write(path, y, sr)
39
+ return path
40
+
41
+
42
+ # ---- 1. time signature estimation ----
43
+ from analyze import fingerprint, get_time_signature
44
+
45
+ y4, sr = click_track(4)
46
+ y3, _ = click_track(3)
47
+ check("4/4 click track detected as 4/4", get_time_signature(write_tmp(y4, sr)) == "4/4")
48
+ check("3/4 click track detected as 3/4", get_time_signature(write_tmp(y3, sr)) == "3/4")
49
+
50
+ # too short / ambiguous material falls back to 4/4, never crashes
51
+ y_short, _ = click_track(4, bars=1)
52
+ check("short clip falls back to 4/4", get_time_signature(write_tmp(y_short, sr)) == "4/4")
53
+
54
+ # ---- 2. fingerprint: one call, all fields ----
55
+ info = fingerprint(write_tmp(y4, sr))
56
+ for field in ("key", "bpm", "time_signature", "duration", "sample_rate", "channels"):
57
+ check(f"fingerprint has {field}", field in info)
58
+ check("fingerprint bpm ~120", 110 <= info["bpm"] <= 130, f"bpm={info['bpm']}")
59
+ check("fingerprint meter 4/4", info["time_signature"] == "4/4")
60
+
61
+ # ---- 3. waveform render ----
62
+ from poster import render_waveform_png, make_poster, _envelope
63
+
64
+ t = np.linspace(0, 5, 5 * 22050).astype(np.float32)
65
+ tone = (0.5 * np.sin(2 * np.pi * 220 * t) * np.linspace(0, 1, len(t))).astype(np.float32)
66
+ tone_path = write_tmp(tone, 22050)
67
+
68
+ wave_png = render_waveform_png(tone_path)
69
+ check("waveform png written", os.path.exists(wave_png) and os.path.getsize(wave_png) > 1000)
70
+
71
+ env = _envelope(tone, 100)
72
+ check("envelope normalized to [0,1]", 0.99 <= env.max() <= 1.0 and env.min() >= 0)
73
+ check("ramped tone envelope rises", env[-10:].mean() > env[:10].mean())
74
+ check("empty audio envelope is safe", np.all(_envelope(np.array([]), 50) == 0))
75
+
76
+ # ---- 4. poster ----
77
+ poster_path = make_poster(
78
+ tone_path, title="garage demo final FINAL v2 (real one this time)",
79
+ key_sig="F# minor", bpm=124.0, time_sig="4/4", split_seconds=2.5,
80
+ original_date="2018", completed=date(2026, 6, 12),
81
+ )
82
+ check("poster png written", os.path.exists(poster_path) and os.path.getsize(poster_path) > 5000)
83
+ from PIL import Image
84
+ pimg = Image.open(poster_path)
85
+ check("poster is 1080x1080", pimg.size == (1080, 1080), f"size={pimg.size}")
86
+
87
+ # poster without an original date still renders
88
+ p2 = make_poster(tone_path, title="x", key_sig="C major", bpm=90,
89
+ time_sig="3/4", split_seconds=0.0)
90
+ check("poster works without original date", os.path.exists(p2))
91
+
92
+ # ---- 5. whisper hallucination filter ----
93
+ from transcribe import looks_like_lyrics
94
+
95
+ check("real lyrics pass", looks_like_lyrics("I left my heart out on the wire\nwaiting for a sign"))
96
+ check("'thank you.' rejected", not looks_like_lyrics("Thank you."))
97
+ check("'thanks for watching' rejected", not looks_like_lyrics("Thanks for watching!"))
98
+ check("empty rejected", not looks_like_lyrics(""))
99
+ check("None rejected", not looks_like_lyrics(None))
100
+ check("two-word loop rejected", not looks_like_lyrics("la la la la la la la la la"))
101
+ check("short fragment rejected", not looks_like_lyrics("oh yeah"))
102
+
103
+ # ---- 6. qwen output cleaner ----
104
+ from write_lyrics import _clean
105
+
106
+ raw = """<think>
107
+ The user wants lyrics matching the tone...
108
+ </think>
109
+ Here are the lyrics:
110
+ the ashes settle where we used to stand
111
+ I trace the smoke back to your hand
112
+ Existing line here
113
+ and every echo learns my name
114
+ """
115
+ cleaned = _clean(raw, existing_lines={"Existing line here"}, num_lines=8)
116
+ check("think block stripped", "<think>" not in cleaned and "tone" not in cleaned)
117
+ check("scaffold line dropped", "Here are the lyrics" not in cleaned)
118
+ check("existing line not repeated", "Existing line here" not in cleaned)
119
+ check("real lines kept", cleaned.startswith("the ashes settle"), f"{cleaned!r}")
120
+ check("line cap respected", len(_clean("a\nb\nc\nd\ne", set(), 3).split("\n")) == 3)
121
+
122
+ # ---- 7. stage tracker html ----
123
+ from app import stages_html, PIPELINE_STAGES, make_stat_html
124
+
125
+ html = stages_html(["done", "active", "pending", "pending"], note="working")
126
+ check("tracker renders all stages", all(s in html for s in PIPELINE_STAGES))
127
+ check("tracker carries states", 'stage done' in html and 'stage active' in html)
128
+ check("tracker note rendered", 'working' in html)
129
+ custom = stages_html(["skip"] * 4, labels=["a", "b", "c", "d"])
130
+ check("tracker custom labels", ">a<" in custom and 'stage skip' in custom)
131
+ check("stat card renders", 'stat-value' in make_stat_html("key", "F# minor"))
132
+
133
+ print()
134
+ if FAILURES:
135
+ print(f"{len(FAILURES)} FAILURES: {FAILURES}")
136
+ sys.exit(1)
137
+ print("all checks passed")
transcribe.py CHANGED
@@ -1,73 +1,99 @@
1
- import torch
2
- import torchaudio
3
- from transformers import WhisperProcessor, WhisperForConditionalGeneration
4
-
5
-
6
- _processor = None
7
- _model = None
8
-
9
-
10
- def _load_whisper():
11
- global _processor, _model
12
- if _model is None:
13
- _processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")
14
- _model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3")
15
- return _processor, _model
16
-
17
-
18
- def isolate_vocals(path):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  """
20
- run demucs to split stems, return path to vocals.
21
- expects demucs CLI installed via pip.
22
  """
23
- import subprocess
24
- import os
25
-
26
- out_dir = os.path.join(os.path.dirname(path), '_stems')
27
- cmd = ['python', '-m', 'demucs', '--two-stems', 'vocals',
28
- '-o', out_dir, path]
29
- subprocess.run(cmd, check=True, capture_output=True)
30
-
31
- # demucs outputs to out_dir/htdemucs/<trackname>/vocals.wav
32
- track_name = os.path.splitext(os.path.basename(path))[0]
33
- vocals_path = os.path.join(out_dir, 'htdemucs', track_name, 'vocals.wav')
34
-
35
- if not os.path.exists(vocals_path):
36
- raise FileNotFoundError(f"demucs didn't produce vocals at {vocals_path}")
37
-
38
- return vocals_path
39
-
40
-
41
- def transcribe(path, isolate=True):
42
- """
43
- extract lyrics from audio.
44
- if isolate=True, runs demucs first to pull vocals.
45
- """
46
- if isolate:
47
- try:
48
- vocal_path = isolate_vocals(path)
49
- except Exception:
50
- # fall back to raw audio if stem separation fails
51
- vocal_path = path
52
- else:
53
- vocal_path = path
54
-
55
- processor, model = _load_whisper()
56
-
57
- track, sr = torchaudio.load(vocal_path)
58
-
59
- # whisper wants 16kHz mono
60
- if sr != 16000:
61
- track = torchaudio.transforms.Resample(sr, 16000)(track)
62
- if track.shape[0] > 1:
63
- track = track.mean(dim=0, keepdim=True)
64
-
65
- track = track.squeeze()
66
-
67
- inputs = processor(track.numpy(), sampling_rate=16000, return_tensors="pt")
68
-
69
- with torch.no_grad():
70
- predicted_ids = model.generate(inputs.input_features)
71
-
72
- lyrics = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
73
- return lyrics.strip()
 
1
+ """lyric extraction: demucs-isolated vocals -> whisper large-v3.
2
+
3
+ vocals are pulled out with HT-Demucs before transcription for two reasons:
4
+ whisper hallucinates on full mixes (drums decode to "thanks for watching"),
5
+ and it keeps this stage consistent with continue_music's vocal gate — both
6
+ agree on what counts as a vocal.
7
+
8
+ torchaudio.load is deliberately avoided here: torchaudio 2.9+ delegates
9
+ decoding to torchcodec, which isn't installed on the Space. librosa's
10
+ soundfile backend handles everything we accept.
11
+ """
12
+ import numpy as np
13
+
14
+ WHISPER_ID = "openai/whisper-large-v3"
15
+ WHISPER_SR = 16000
16
+ # same bar as continue_music's vocal gate
17
+ VOCAL_RMS_THRESHOLD = 0.1
18
+
19
+ _asr = None
20
+
21
+
22
+ def _load_asr():
23
+ global _asr
24
+ if _asr is None:
25
+ import torch
26
+ from transformers import pipeline
27
+ device = "cuda" if torch.cuda.is_available() else "cpu"
28
+ _asr = pipeline(
29
+ "automatic-speech-recognition",
30
+ model=WHISPER_ID,
31
+ torch_dtype=torch.float16 if device == "cuda" else torch.float32,
32
+ device=device,
33
+ chunk_length_s=30, # long-form: clips aren't capped at whisper's 30s window
34
+ )
35
+ return _asr
36
+
37
+
38
+ # whisper's classic non-speech hallucinations — silence and instrument bleed
39
+ # decode to these with high confidence
40
+ _HALLUCINATION_LINES = {
41
+ "you", "thank you", "thanks for watching", "thank you for watching",
42
+ "bye", "music", "[music]", "(music)", "subscribe",
43
+ "please subscribe", "see you next time",
44
+ }
45
+
46
+
47
+ def looks_like_lyrics(text):
48
+ """filter whisper output that is clearly hallucinated, not sung words."""
49
+ text = (text or "").strip()
50
+ if not text:
51
+ return False
52
+ words = text.split()
53
+ if len(words) < 3:
54
+ return False
55
+ if text.lower().strip(" .!?") in _HALLUCINATION_LINES:
56
+ return False
57
+ # one short phrase looped over and over is the other hallucination shape
58
+ unique = {w.lower().strip(",.!?") for w in words}
59
+ if len(words) >= 8 and len(unique) <= 2:
60
+ return False
61
+ return True
62
+
63
+
64
+ def transcribe_vocals(path):
65
  """
66
+ isolate vocals and transcribe them. returns the lyric text, or None when
67
+ the clip is instrumental (or separation finds no real vocal energy).
68
  """
69
+ import librosa
70
+ from stems import isolate_vocals
71
+
72
+ vocals, sep_sr, ratio = isolate_vocals(path)
73
+ if vocals is None:
74
+ print("[coda] transcribe: demucs unavailable, skipping lyrics", flush=True)
75
+ return None
76
+ if ratio < VOCAL_RMS_THRESHOLD:
77
+ print(f"[coda] transcribe: instrumental clip "
78
+ f"(vocal/mix rms {ratio:.3f}), no lyrics to pull", flush=True)
79
+ return None
80
+
81
+ mono = vocals.mean(axis=0) if vocals.ndim == 2 else vocals
82
+ wav = librosa.resample(mono.astype(np.float32), orig_sr=sep_sr,
83
+ target_sr=WHISPER_SR, res_type="soxr_hq")
84
+
85
+ asr = _load_asr()
86
+ out = asr(
87
+ {"array": wav, "sampling_rate": WHISPER_SR},
88
+ return_timestamps=True, # required for inputs longer than 30s
89
+ generate_kwargs={"task": "transcribe"},
90
+ )
91
+ text = (out.get("text") or "").strip()
92
+
93
+ if not looks_like_lyrics(text):
94
+ print(f"[coda] transcribe: discarding likely hallucination {text!r}",
95
+ flush=True)
96
+ return None
97
+ print(f"[coda] transcribe: pulled {len(text.split())} words of lyrics",
98
+ flush=True)
99
+ return text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
write_lyrics.py CHANGED
@@ -1,5 +1,12 @@
1
- from transformers import AutoModelForCausalLM, AutoTokenizer
2
 
 
 
 
 
 
 
 
3
 
4
  _model = None
5
  _tokenizer = None
@@ -8,59 +15,85 @@ _tokenizer = None
8
  def _load_qwen():
9
  global _model, _tokenizer
10
  if _model is None:
11
- model_id = "Qwen/Qwen3-8B"
12
- _tokenizer = AutoTokenizer.from_pretrained(model_id)
 
13
  _model = AutoModelForCausalLM.from_pretrained(
14
- model_id,
15
- torch_dtype="auto",
16
- device_map="auto"
17
  )
18
  return _model, _tokenizer
19
 
20
 
21
- def continue_lyrics(existing_lyrics, key=None, bpm=None, style_hint=None, num_lines=8):
22
- """
23
- takes existing lyrics and writes more in the same style.
24
- key/bpm/style_hint give the model musical context.
25
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  model, tokenizer = _load_qwen()
27
 
28
  context_parts = []
29
  if key:
30
- context_parts.append(f"The song is in {key}")
31
  if bpm:
32
- context_parts.append(f"at {bpm} BPM")
33
  if style_hint:
34
  context_parts.append(f"with a {style_hint} feel")
 
 
35
 
36
- context = ", ".join(context_parts) + "." if context_parts else ""
37
-
38
- prompt = f"""You are a songwriter. Continue the following lyrics naturally,
39
- matching the tone, rhythm, and imagery. Write exactly {num_lines} new lines.
40
- Do not repeat existing lines. Do not add commentary or explanations.
41
  {context}
42
 
43
  Existing lyrics:
44
- {existing_lyrics}
45
-
46
- Continuation:"""
47
 
48
  messages = [{"role": "user", "content": prompt}]
49
- text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
50
-
 
 
 
51
  inputs = tokenizer([text], return_tensors="pt").to(model.device)
52
 
53
- output = model.generate(
54
- **inputs,
55
- max_new_tokens=256,
56
- temperature=0.8,
57
- top_p=0.9,
58
- do_sample=True
59
- )
 
 
 
 
60
 
61
  generated = output[0][inputs.input_ids.shape[1]:]
62
  result = tokenizer.decode(generated, skip_special_tokens=True)
63
 
64
- # trim to requested line count
65
- lines = [l for l in result.strip().split('\n') if l.strip()]
66
- return '\n'.join(lines[:num_lines])
 
1
+ """lyric continuation with qwen3-8b.
2
 
3
+ this is a songwriting aid, not vocal synthesis: musicgen can't generate
4
+ voices (vocals were stripped from its training data by design), so coda
5
+ hands you the words for the new section instead of pretending to sing them.
6
+ """
7
+ import re
8
+
9
+ MODEL_ID = "Qwen/Qwen3-8B"
10
 
11
  _model = None
12
  _tokenizer = None
 
15
  def _load_qwen():
16
  global _model, _tokenizer
17
  if _model is None:
18
+ import torch
19
+ from transformers import AutoModelForCausalLM, AutoTokenizer
20
+ _tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
21
  _model = AutoModelForCausalLM.from_pretrained(
22
+ MODEL_ID,
23
+ torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
24
+ device_map="auto",
25
  )
26
  return _model, _tokenizer
27
 
28
 
29
+ def _clean(result, existing_lines, num_lines):
30
+ # qwen3 occasionally leaks reasoning even with thinking disabled
31
+ result = re.sub(r"<think>.*?</think>", "", result, flags=re.S)
32
+ result = result.replace("<think>", "").replace("</think>", "")
33
+
34
+ lines = []
35
+ for line in result.strip().split("\n"):
36
+ line = line.strip().strip('"')
37
+ if not line:
38
+ continue
39
+ # drop scaffolding the model sometimes adds despite instructions
40
+ if line.lower().rstrip(":") in (
41
+ "continuation", "lyrics", "new lyrics", "here are the lyrics",
42
+ "here is the continuation"):
43
+ continue
44
+ if line in existing_lines:
45
+ continue
46
+ lines.append(line)
47
+ return "\n".join(lines[:num_lines])
48
+
49
+
50
+ def continue_lyrics(existing_lyrics, key=None, bpm=None, style_hint=None,
51
+ num_lines=8):
52
+ """write `num_lines` new lines in the same voice as `existing_lyrics`."""
53
+ import torch
54
  model, tokenizer = _load_qwen()
55
 
56
  context_parts = []
57
  if key:
58
+ context_parts.append(f"the song is in {key}")
59
  if bpm:
60
+ context_parts.append(f"at {round(float(bpm))} BPM")
61
  if style_hint:
62
  context_parts.append(f"with a {style_hint} feel")
63
+ context = ("Musical context: " + ", ".join(context_parts) + "."
64
+ if context_parts else "")
65
 
66
+ prompt = f"""You are a songwriter finishing a song that was abandoned mid-write.
67
+ Continue these lyrics naturally — match the tone, vocabulary, rhyme feel, and imagery.
68
+ Write exactly {num_lines} new lines. Output only the lyric lines: no titles,
69
+ no section labels, no commentary, and do not repeat existing lines.
 
70
  {context}
71
 
72
  Existing lyrics:
73
+ {existing_lyrics}"""
 
 
74
 
75
  messages = [{"role": "user", "content": prompt}]
76
+ text = tokenizer.apply_chat_template(
77
+ messages, tokenize=False, add_generation_prompt=True,
78
+ # hybrid thinking off — we want lyric lines, not <think> blocks
79
+ enable_thinking=False,
80
+ )
81
  inputs = tokenizer([text], return_tensors="pt").to(model.device)
82
 
83
+ with torch.no_grad():
84
+ output = model.generate(
85
+ **inputs,
86
+ max_new_tokens=320,
87
+ do_sample=True,
88
+ # qwen3's recommended non-thinking sampling
89
+ temperature=0.7,
90
+ top_p=0.8,
91
+ top_k=20,
92
+ pad_token_id=tokenizer.eos_token_id,
93
+ )
94
 
95
  generated = output[0][inputs.input_ids.shape[1]:]
96
  result = tokenizer.decode(generated, skip_special_tokens=True)
97
 
98
+ existing = {l.strip() for l in existing_lyrics.split("\n")}
99
+ return _clean(result, existing, num_lines)