Add BarScript (experimental preview) model + ura difficulty; vendored sc2 subpackage

README.md

@@ -10,7 +10,6 @@ app_file: app.py
 pinned: false
 license: mit
 models:
-  - JacobLinCool/softchart-v18
   - JacobLinCool/softchart-v17
   - JacobLinCool/softchart-v17-small
   - JacobLinCool/softchart-v17-tiny
@@ -19,45 +18,89 @@ models:
   - JacobLinCool/softchart-v16-tiny
   - JacobLinCool/softchart-v15
   - JacobLinCool/softchart-planner
-datasets:
-  - JacobLinCool/taiko-1000-parsed-clean
+  - JacobLinCool/softchart-barscript
 ---
 
 # 🥁 SoftChart
 
-Generate a **Taiko no Tatsujin-style** chart from an uploaded song with the
-scratch-trained [SoftChart V1.8 hierarchical model](https://huggingface.co/JacobLinCool/softchart-v18).
-The 8,985,091-parameter model combines whole-song hierarchical context, a local
-rhythmic-skeleton auxiliary head, chart generation, and beat/downbeat
-estimation. It uses no external pretrained model and no external song planner.
-
-V1.8 is the default recommendation, not the only option. Users can select
-V1.5, V1.6, or V1.7—including the small and tiny scaling variants—from the
-interface for direct comparison. Those releases run through their frozen
-generation runtime and original
-[legacy planner](https://huggingface.co/JacobLinCool/softchart-planner); this
-does not change V1.8's self-contained, scratch-only architecture.
-
-The Space always uses each selected model's full-song **time generation** path
-for arbitrary uploads. The beat head can estimate BPM and a downbeat grid for TJA
-quantization, but that estimated grid is not treated as the trusted rational
-meter required by exact slot generation. This distinction prevents an
-estimated beat grid from being presented as authored-meter exact timing.
-
-Audio preprocessing follows the released model contract: FFmpeg decodes
-stereo float32 at 22,050 Hz, channels are averaged arithmetically, and a
-periodic-Hann STFT is projected to 128 log-mel bins. Upload a song, choose a
-difficulty, and receive:
-
-- a playable `.tja` for TJAPlayer3 / OpenTaiko;
-- a rendered chart image;
-- the full-song mel structure view used by the hierarchy;
-- a synchronized WAV audition with synthesized Taiko hits.
-
-The audition is rendered from the exported TJA, including its `OFFSET`, so the
-preview and downloadable chart use the same timing. Taiko voices are original
-modal-resonator syntheses included under the project's MIT license.
-
-The model and Space code are **MIT-licensed**. Evaluation scope, metrics, and
-known limitations are documented on the
-[V1.8 model card](https://huggingface.co/JacobLinCool/softchart-v18).
+Generate a **Taiko no Tatsujin** chart from any song. A unified 7.9M-parameter
+encoder–decoder model handles beat/downbeat fitting, slot-exact generation, and
+gridless time generation; an optional planner supplies song-level density,
+breathing gaps, and climax structure. All models are MIT-licensed and loaded
+from the Hugging Face Hub via `from_pretrained`.
+
+- **SoftChart v15** ([softchart-v15](https://huggingface.co/JacobLinCool/softchart-v15)) — unified slot + time + beat model; slot mode emits exact TJA lattice indices when the beat grid is trusted
+- **Planner** ([softchart-planner](https://huggingface.co/JacobLinCool/softchart-planner)) — audio → song-level plan (density envelope, breathing gaps, climax)
+
+## Difficulties
+
+Easy, Normal, Hard, Oni and Ura. Each requests a density bucket measured on the
+1 155-song corpus — median note rate 1.33 / 2.01 / 3.40 / 5.27 / 6.60 per second
+— and Ura is written with the TJA header `COURSE:Edit`, the literal used by all
+229 Ura charts in the corpus.
+
+Two caveats apply to Ura on every model. It was trained on 208 charts against
+roughly 1 100 per other course, so expect lower quality. And the song planner
+was trained on the four base courses only, so for Ura it is asked for the Oni
+plan instead; the interface says so before you generate, and the result reports
+which plan it used. Turning the AI planner off avoids the substitution.
+
+## BarScript (experimental preview)
+
+The model menu also offers **`BarScript (preview)`**, the next-generation
+model. It is published to the private repo
+[`JacobLinCool/softchart-barscript`](https://huggingface.co/JacobLinCool/softchart-barscript)
+and resolved in one order, most explicit first:
+
+1. `$SC2_PACKAGE_DIR`, if set — an explicit request, and an error if that
+   directory holds no package;
+2. a local copy at `spaces/models/sc2_c1_v13_s1234_best/`;
+3. the Hub repo, downloaded on first use.
+
+Because the repo is **private**, the download needs a token: set `HF_TOKEN`
+(on a Space, *Settings → Variables and secrets*) or run `hf auth login`. With
+no token the loader says so in those words — a missing credential and a missing
+model have different fixes, and reporting the first as the second sends people
+looking in the wrong place. `SC2_NO_HUB=1` pins the Space to on-disk packages.
+Whichever source wins, the same gates run before a note is decoded: eight
+vendored serving modules checked by md5 against the package's own record, the
+token table cross-checked, and the Space's log-mel front end matched to the
+package's contract. A Hub snapshot is additionally verified against the
+`release_manifest.json` that travels with it.
+
+BarScript is under evaluation and has known defects. The interface lists all of
+them verbatim from the model card when the option is selected, including a
+non-nested difficulty ladder, over-dense output, and a uniform `/16` deployment
+lattice on which triplets are arithmetically impossible. It decodes onto a bar
+lattice built from the fitted BPM and first downbeat, and refuses to generate
+without one rather than anchoring the bars on an arbitrary phase. It takes no
+song plan, so the Structure and AI planner switches do not reach it.
+
+`python spaces/scripts/test_app_wiring.py` checks the routing, the resolution
+order above, the five difficulties, the Ura TJA header, and — against a recorded
+trace of the app as it stood before BarScript was wired in — that the seven
+published models still behave identically, call for call and byte for byte.
+
+The Space uses Gradio Server mode with a fully custom, responsive interface.
+Upload a song, pick a difficulty, and get a full-song mel/plan preview, a
+full TJA chart image, a `.tja` playable in TJAPlayer3 / OpenTaiko, and a
+synchronized WAV mix that lets you hear the generated chart performed over
+the source music immediately. Its Don, Big Don, and Katsu voices are original
+modal-resonator syntheses built from seeded excitation noise, decaying drum
+modes, pitch sweeps, and band-limited stick transients. Reference recordings
+are used only for offline envelope and level calibration; they are not copied,
+mixed, embedded, or required by the Space.
+
+The generation API streams the real processing stages to the interface:
+audio analysis, beat-grid fitting, song-level planning, note generation,
+chart rendering, and taiko preview mixing. The final audio is timed from the
+exported TJA itself, including `OFFSET` and `#BPMCHANGE`, so the audition and
+downloadable chart share the same timing.
+
+The generated TJA keeps the uploaded audio filename in its `WAVE` field.
+Planner climax blocks are emitted as TJA `#GOGOSTART` / `#GOGOEND` sections.
+The chart image uses a server-side renderer modeled on the row layout used by
+MIT-licensed [tja-tools](https://github.com/WHMHammer/tja-tools), without
+shipping its browser/webpack runtime.
+
+> Code and weights are **MIT-licensed**.

SC2_INTEGRATION.md

@@ -0,0 +1,373 @@
+# BarScript in the Space — integration and verification
+
+BarScript (run `sc2_c1_v13_s1234_best`, 9.1 M parameters) is selectable in the
+Space as **`BarScript (preview)`**, alongside the seven published 1.x models,
+which are unchanged. Ura is now a fifth difficulty on every model.
+
+`BarScript` is the release name; `sc2_c1_v13_s1234` is the run that produced
+this checkpoint, and it stays that way everywhere it is recorded — in the
+package directory name, the manifest, and every md5 and sha in the provenance
+chain. Renaming those would break the trail back to the training run.
+
+BarScript is an **experimental preview**, published to the **private** repo
+[`JacobLinCool/softchart-barscript`](https://huggingface.co/JacobLinCool/softchart-barscript).
+It has measured defects that are shown in the interface and repeated in full
+below. Read [Known limitations](#known-limitations-of-barscript) before drawing
+any conclusion from a chart it produces.
+
+---
+
+## Running it locally
+
+### 1. Environment
+
+Python 3.11, from `spaces/`:
+
+```bash
+python3.11 -m venv .venv && source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+On Apple Silicon the pinned `torch==2.13.0+cpu` has no wheel (that build is
+published for Linux and Windows only). Install the platform build instead and
+take the rest of the file as-is:
+
+```bash
+pip install torch==2.13.0
+pip install "numpy<2.1" scipy "librosa>=0.10" "numba>=0.60" soundfile \
+            matplotlib pillow gradio==6.20.0 gradio_client==2.5.0 \
+            "pydantic>=2.10,<3" "huggingface_hub>=1.2,<2.0" safetensors
+```
+
+Everything below was verified on this stack (torch 2.13.0, gradio 6.20.0,
+numpy 2.0.2, librosa 0.11.0), CPU only.
+
+### 2. Point the Space at the BarScript package
+
+`sc2_loader.resolve_package_dir()` looks in one order, most explicit first, and
+reports which leg won as `describe()["package_source"]`:
+
+| # | source | `package_source` | when |
+|---|---|---|---|
+| 1 | `$SC2_PACKAGE_DIR` | `env` | whenever it is set |
+| 2 | `spaces/models/sc2_c1_v13_s1234_best/`, then the build artifact in the development worktree | `local` | a package is on disk |
+| 3 | the Hub repo, via `snapshot_download` | `hub` | neither of the above |
+
+`$SC2_PACKAGE_DIR` that holds no package **raises** rather than falling through:
+naming a directory is a request for that directory, and quietly downloading a
+different copy instead would hide the typo.
+
+```bash
+export SC2_PACKAGE_DIR=/path/to/sc2_c1_v13_s1234_best   # optional
+```
+
+Running from this checkout, leg 2 finds the build artifact at
+`.codex/worktrees/v18-8h/artifacts/sc2/sc2_c1_v13_s1234_best/` and nothing is
+downloaded.
+
+**The Hub repo is private, so leg 3 needs a token.** Set `HF_TOKEN`, or run
+`hf auth login`; the loader also accepts `$HUGGING_FACE_HUB_TOKEN` and the
+token cache. With no token it fails with a message naming the credential and
+the three ways to supply one, rather than reporting a missing model — the two
+have different fixes. `SC2_NO_HUB=1` disables leg 3 entirely, which is how a
+deployment proves it is not silently depending on a download.
+
+Only the seven files of the release package are fetched (`PACKAGE_FILES` is
+passed to `snapshot_download` as an allow-list), and the snapshot is checked
+against the `release_manifest.json` that travels with it before it is served —
+those bytes crossed a network, and the manifest records the sha256 of each file
+as built. Local packages are not re-hashed; that check is for the transport.
+
+Whichever leg wins, the package directory is the whole source of truth from
+that point on: the eight vendored serving modules are checked by md5 against
+`config.json:code.module_md5`, the token table is cross-checked, the schema is
+validated, and the Space's log-mel front end is matched to the package's
+contract. A downloaded package and a local one are served identically or not
+at all.
+
+If no leg produces a package, the BarScript entry does not appear in the model
+menu and the seven published models work as usual. Nothing fails.
+
+### 3. Start the Space
+
+```bash
+cd spaces
+python app.py                      # http://localhost:7860
+```
+
+Check it came up, and that BarScript is being offered:
+
+```bash
+curl -s localhost:7860/health
+# {"status":"ok","device":"cpu","models_loaded":false}
+
+curl -s localhost:7860/api/capabilities | python -m json.tool
+```
+
+`/api/capabilities` is the source of the model menu, the experimental
+labelling, and the limitation text. The BarScript entry carries
+`"experimental": true` and all eight model-card limitations; the frontend
+renders them verbatim and cannot paraphrase them.
+
+The published models are downloaded from the Hub on first use. With them
+already in the local cache you can run fully offline:
+
+```bash
+HF_HUB_OFFLINE=1 python app.py
+```
+
+### 4. Generate a chart
+
+In the browser: upload audio, pick **BarScript (preview)**, pick a difficulty
+(including **Ura**), generate. Two controls behave differently on BarScript:
+
+- **Beat grid must stay on** (or enter the BPM in Advanced settings). BarScript
+  builds its bars from a downbeat and cannot place them without one; the route
+  fails with an explanatory message rather than guessing a phase.
+- **Structure / AI planner switches are ignored.** BarScript takes no plan
+  token, so the result reports `Not used — BarScript takes no plan` instead of
+  a plan source.
+
+---
+
+## Tests
+
+Three suites, all CPU, no network required (the third reads published models
+from the local HuggingFace cache; the Hub leg of the second is mocked).
+
+```bash
+cd spaces
+python scripts/test_sc2_vendor.py     # vendored BarScript tree matches the package
+python scripts/test_app_wiring.py     # routing, resolution order, ura, limitations
+HF_HUB_OFFLINE=1 python scripts/test_space_e2e.py   # real weights, real music
+```
+
+Current status: **all green** — 7 vendor checks, 117 wiring checks, 380
+end-to-end checks.
+
+15 of the wiring checks cover package resolution: that `$SC2_PACKAGE_DIR`
+outranks a local copy and the Hub, that an unsatisfiable `$SC2_PACKAGE_DIR`
+raises instead of falling through, that a local copy outranks the Hub, that the
+Hub is reached only when neither is present, that `SC2_NO_HUB` disables it, that
+a missing token is reported as a credentials problem before any request is made,
+that the request carries the resolved token and an allow-list of exactly the
+package files, and that a downloaded package is refused when a vendored module
+md5, the schema, the vocabulary or the manifest disagrees. The Hub is mocked
+throughout — the subject is the precedence rule, and a suite that downloaded
+18 MB to assert one would be testing the network instead. The real download is
+exercised separately.
+
+`test_space_e2e.py` is the only one that decodes real music. It drives
+`app.generate_chart` itself — the same generator the frontend consumes — and
+inspects the artifacts it produced. Options:
+
+```bash
+python scripts/test_space_e2e.py --songs 2 --seconds 30   # quicker
+python scripts/test_space_e2e.py --skip-legacy            # BarScript only
+python scripts/test_space_e2e.py --strict-ladder          # fail if ladder not monotone
+python scripts/test_space_e2e.py --audio-dir ../other     # your own audio
+python scripts/test_space_e2e.py --json report.json
+```
+
+Fixtures come from the held-out `test` split of `JacobLinCool/taiko-1000-parsed`
+if it is in the local HuggingFace cache: real music the models never trained
+on, shipping authored charts for all five courses, which gives the difficulty
+ladder a reference. Nothing is downloaded; with no cache and no `--audio-dir`
+the suite reports that and exits.
+
+---
+
+## What the end-to-end run shows
+
+Six held-out songs (120–210 BPM), 60 s excerpts, five difficulties, greedy
+decoding — 30 real BarScript decodes, median 3.3 s each on CPU.
+
+Every chart, on every difficulty, satisfies:
+
+- re-parses with `softchart.sc2.trace` into exactly one course section;
+- `COURSE:` header is the literal the corpus uses, **and** resolves back to the
+  course requested;
+- bars are all 96 digits, and the bar count follows tempo and excerpt length to
+  within 1.5 bars — the lattice is anchored, not drifting;
+- every note sits on the shipped /16 lattice (0 off-lattice notes in 30 charts);
+- drumroll spans are balanced and correctly ordered — no unclosed and no nested
+  spans anywhere;
+- the note and span counts shown in the UI equal the counts recovered by
+  re-parsing the file.
+
+### Ura is harder than Oni — but by much less than authored charts
+
+Note counts per course, same song. Generated rows cover the **60 s excerpt**;
+authored rows cover the **whole chart**, so only the shape of each row — the
+step from one course to the next — is comparable between them, not the
+absolute numbers. (Duration-normalised rates follow in the next table.)
+
+| song | easy | normal | hard | oni | **ura** |
+|---|---|---|---|---|---|
+| Doppelgangers | 88 | 146 | 219 | 300 | **351** |
+| *(authored)* | *136* | *253* | *368* | *634* | *929* |
+| 美しく忙しきドナウ | 89 | 141 | 218 | 317 | **348** |
+| *(authored)* | *174* | *247* | *449* | *682* | *1038* |
+| 零の交響曲 | 79 | 178 | 293 | 341 | **375** |
+| *(authored)* | *163* | *214* | *434* | *648* | *893* |
+| 雨とペトラ | 92 | 152 | 291 | 369 | **402** |
+| *(authored)* | *117* | *198* | *335* | *464* | *591* |
+| スプラトゥーン2 メドレー | 82 | 132 | 219 | 336 | **373** |
+| *(authored)* | *105* | *144* | *245* | *441* | *757* |
+| さんぽ | 75 | 137 | 218 | 311 | **316** |
+| *(authored)* | *87* | *107* | *249* | *371* | *816* |
+
+**Ura came out denser than Oni on 6 of 6 songs, and the whole ladder
+easy < normal < hard < oni < ura was monotone on 6 of 6.** So ura is wired
+correctly and does produce a harder chart.
+
+The size of the step is the weak part. Generated oni→ura is **1.02x–1.17x
+(median 1.10x)**; authored oni→ura on the same six songs is **1.27x–2.20x
+(median 1.49x)**. The gap is widest exactly where the authors stretched
+furthest: on さんぽ they went 371→816 notes (2.20x) while the model went
+311→316 (1.02x). This is the 208-chart training deficit showing up, and it is the
+reason `--strict-ladder` is off by default: monotonicity here is a model
+result to be reported, not a wiring property to be assumed. Do not read
+"monotone on 6/6" as evidence that the difficulty family is nested — the model
+card measures that separately (§2.1) and finds it is not.
+
+Median notes-per-second, generated vs authored across the six songs:
+
+| | easy | normal | hard | oni | ura |
+|---|---|---|---|---|---|
+| generated | 1.45 | 2.43 | 3.69 | 5.46 | 6.10 |
+| authored | 1.08 | 1.76 | 2.91 | 4.56 | 6.70 |
+
+BarScript is denser than authored on the four lower courses and **sparser than
+authored on ura** — the one course where it should be densest is the one where
+it falls short.
+
+### Note spacing
+
+The model card reports 102 sub-40 ms gaps across 29 charts (§2.9). **None of
+the 30 charts here contains one**; the tightest gap measured was 76.9 ms.
+
+This is a property of the deployment grid, not evidence that the defect is
+gone. On a uniform /16 lattice the minimum possible spacing is `15 / BPM`
+seconds, so a sub-40 ms gap is arithmetically impossible below **375 BPM**.
+These songs run 120–210 BPM. The model card's figure was measured on authored
+grids with finer denominators. On very fast songs the defect can return.
+
+### The seven published models are unchanged
+
+Same six songs, real Hub weights, v1.7 and v1.5 x {hard, oni} — 24
+comparisons of the current `app.py` against
+`scripts/fixtures/app_baseline.py`, the app as it stood before this change
+(pinned by sha256 `a21c5853…` to the same source the wiring suite's golden
+trace was recorded from).
+
+- **24/24 produced byte-identical TJA text.**
+- 24/24 agree on every metric the old app reported.
+- The metrics panel gained three display-only fields — `model`, `difficulty`,
+  `plan_source` — and dropped none. These are the new UI surface (model name,
+  the TJA course literal, plan provenance), not a change in generation.
+
+The wiring suite covers the same question from the other side: 35 mocked
+scenarios comparing call arguments. This one compares the charts themselves.
+
+---
+
+## Known limitations of BarScript
+
+Verbatim from `MODEL_CARD_sc2_c1_v13_s1234.md`. All eight are shown in the
+interface when BarScript is selected.
+
+| | |
+|---|---|
+| §0 | The champion comparison is **inconclusive**: the challenger was run at one seed. |
+| §2.1 | The difficulty ladder is **not nested**. Independently decoded courses share 0.658/0.667/0.660 of their notes with the next harder course; authored charts share 0.95–0.99. |
+| §2.9 | Notes come out **too dense**: 102 sub-40 ms gaps across 29 charts (authored: 2 across 1), 19 of them below the series record of 29.41 ms, and up to 17 notes inside one second. |
+| §2.7 | One drumroll in seven is a **micro-span** (14.7 % vs 3.8 % authored). |
+| §2.12 | **Fine-lattice and tuplet placement** is much worse than average: 0.345 hit rate at positions whose denominator is divisible by 3. |
+| §2.8 | The deployment grid is a uniform **/16 lattice**, so **triplets are arithmetically impossible** — 3 does not divide 16. |
+| §2.4 | At the top of the **density knob**, drumrolls are squeezed out: span yield falls to 33 % of authored. The highest setting is not recommended. |
+| corpus | **Ura was trained on 208 charts** against roughly 1 100 per other course; expect lower quality. |
+
+Two operational notes are shown with them: the plan switches do not apply, and
+BarScript needs a beat grid.
+
+### The Ura warning applies to every model
+
+Ura appears on all eight models, and the 208-chart notice is shown whichever
+one is selected — the shortage is in the training corpus, not in BarScript.
+
+The **song planner was never trained on Ura**. `scripts/train_planner.py` builds
+its dataset with `courses=("easy","normal","hard","oni")`, so embedding row 4
+never received a gradient; probing the published checkpoint gives a mean
+predicted density bucket of 1.471 for ura, between normal and hard, when ura is
+the densest course in the corpus. Ura therefore queries the planner with the
+**Oni** course id. This is disclosed before generation and again in the result
+(`AI planner (Oni plan reused for Ura)`). Turning the planner off avoids the
+substitution entirely.
+
+---
+
+## Not done
+
+1. **A deployed Space needs an `HF_TOKEN` secret, which has not been set.** The
+   package is now on the Hub, and the download path is verified: with every
+   on-disk copy hidden, `sc2_loader` fetched all seven files from
+   `JacobLinCool/softchart-barscript`, passed the md5 and vocabulary gates, and
+   charted a held-out song through `app.generate_chart` in 6.0 s on CPU.
+
+   What is missing is the credential on the deployment. The repo is **private**,
+   so the running Space needs a read token in `HF_TOKEN` (*Settings → Variables
+   and secrets → New secret*). Until that exists, a deployed Space reaches
+   leg 3, finds no token, and reports exactly that.
+
+   Two alternatives avoid the secret: copy the package to
+   `spaces/models/sc2_c1_v13_s1234_best/` so it travels with the Space (leg 2,
+   18 MB in the Space repo), or make the model repo public — which publishes the
+   weights and the model card, including its KNOWN LIMITATIONS, and is a
+   separate decision from this one.
+
+2. **The BarScript beat front end is unevaluated.** `sc2_beat_model()` borrows
+   v1.7's beat head to get BPM and downbeats for its lattice. The model card's
+   numbers were measured on *authored* grids, so no published figure describes
+   BarScript on a detected grid. The end-to-end suite confirms the resulting bar
+   structure is self-consistent, which is not the same as confirming the
+   barlines match the music.
+
+3. **Density bucket 8 for ura is unvalidated on BarScript.** The bucket map is
+   measured from the corpus and matches the rest of the repo, but §2.5 says
+   absolute bucket calibration is off and §2.4 warns the top of the knob starves
+   drumrolls. The ladder above shows ura landing *below* authored density, which
+   is consistent with the knob not delivering what it asks for.
+
+4. **Piecewise-tempo songs are flattened.** BarScript decodes one uniform tempo; when
+   the grid fit finds several segments the result discloses this under
+   `tempo_map`. The six test songs are single-tempo, so the effect is
+   unquantified.
+
+5. **No human play-testing.** Every claim here is structural or statistical.
+   Whether these charts are *good* — groove, pattern quality, whether the ura
+   step feels like a step — has not been assessed by a player.
+
+6. **No GPU run.** Everything was verified on CPU, which is sufficient for
+   correctness; the remote machine was not needed.
+
+---
+
+## Layout
+
+| path | role |
+|---|---|
+| `spaces/softchart/sc2/` | vendored BarScript serving code, 8 modules, byte-identical to the frozen shipping tree (md5 checked against the package at every load) |
+| `spaces/softchart/sc2_loader.py` | the only bridge: package loading, integrity checks, serving contract, `COURSE:` header rewrite, limitation text |
+| `spaces/app.py` | routing, ura, the BarScript branch, `/api/capabilities` |
+| `spaces/static/` | model menu, ura option, limitation panel |
+| `spaces/scripts/test_sc2_vendor.py` | vendored tree matches the package; 1.x untouched |
+| `spaces/scripts/test_app_wiring.py` | routing and published-model regression, under mocks |
+| `spaces/scripts/test_space_e2e.py` | real weights on real music, through the app entry point |
+| `spaces/scripts/fixtures/app_baseline.py` | the pre-change app, the regression baseline |
+
+The BarScript modules live in their own subpackage because the two generations have
+incompatible token tables (`VOCAB.size` 2071 vs 1810, `MAX_TGT` 960 vs 1536).
+All BarScript imports are package-relative, so `softchart.sc2.generate` resolves
+`from .vocab import VOCAB` to `softchart.sc2.vocab` and can never reach the 1.x
+table, or the reverse.

app.py

@@ -1,16 +1,16 @@
-"""SoftChart — selectable Taiko chart model generations for HF Spaces.
+"""SoftChart — custom Gradio Server app for Hugging Face Spaces.
 
-V1.8 is the recommended scratch-trained hierarchical model. V1.5–V1.7 remain
-available through their frozen runtime so users can compare generations.
-Arbitrary uploads always use time generation; an estimated beat grid may
-quantize the exported TJA but is never promoted to a trusted-meter slot path.
+Generate a Taiko no Tatsujin chart from any audio file, using the full system:
+  - SoftChartGenerator  (main model, plan-conditioned)
+  - SoftChartPlanner     (auto song-level planning)
+  - SoftChartBeat        (beat/downbeat for barline anchoring)
+All models load from the Hub via from_pretrained. MIT licensed.
 """
 
 import logging
 import os
 import re
 import shutil
-import subprocess
 import tempfile
 from pathlib import Path
 
@@ -20,245 +20,245 @@ import torch
 from fastapi.responses import FileResponse
 from fastapi.staticfiles import StaticFiles
 
-from legacy_softchart.generate import (
-    generate_song as generate_song_legacy,
-    load_hf as load_hf_legacy,
-)
-from legacy_softchart.grid import (
-    fit_grid_fixed_bpm as fit_grid_fixed_bpm_legacy,
-    fit_grid_piecewise as fit_grid_piecewise_legacy,
-)
-from legacy_softchart.hf import SoftChartPlanner
-from legacy_softchart.tja import append_measure_with_gogo, gogo_measure_mask
-from softchart.generate import generate_song as generate_song_v18, load_hf as load_hf_v18
+from softchart import sc2_loader
+from softchart.generate import generate_song, generate_song_slot, load_hf
 from softchart.fonts import cjk_font_path
 from softchart.grid import debias_to_grid, fit_grid_fixed_bpm, fit_grid_piecewise
+from softchart.hf import SoftChartPlanner
 from softchart.preview_audio import synthesize_taiko_preview
 from softchart.rhythm import snap_chart
+from softchart.tja import append_measure_with_gogo, gogo_measure_mask, write_tja_slots
 from softchart.tja_image import render_tja_image
 from softchart.vocab import FPS, HOP, N_FFT, N_MELS, SR
 
 LOGGER = logging.getLogger("softchart.space")
 STATIC_DIR = Path(__file__).with_name("static")
 
-PLAN_REPO = "JacobLinCool/softchart-planner"
+PLAN_REPO = os.environ.get("SC_PLAN", "JacobLinCool/softchart-planner")
 DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
-EXPECTED_PARAMETERS = 8_985_091
-
-MODEL_SPECS = {
-    "v1.8": {
-        "repo": "JacobLinCool/softchart-v18",
-        "display": "V1.8 hierarchical",
-        "runtime": "v18",
-    },
-    "v1.7": {
-        "repo": "JacobLinCool/softchart-v17",
-        "display": "V1.7",
-        "runtime": "legacy",
-    },
-    "v1.7-small": {
-        "repo": "JacobLinCool/softchart-v17-small",
-        "display": "V1.7 small",
-        "runtime": "legacy",
-    },
-    "v1.7-tiny": {
-        "repo": "JacobLinCool/softchart-v17-tiny",
-        "display": "V1.7 tiny",
-        "runtime": "legacy",
-    },
-    "v1.6": {
-        "repo": "JacobLinCool/softchart-v16",
-        "display": "V1.6",
-        "runtime": "legacy",
-    },
-    "v1.6-small": {
-        "repo": "JacobLinCool/softchart-v16-small",
-        "display": "V1.6 small",
-        "runtime": "legacy",
-    },
-    "v1.6-tiny": {
-        "repo": "JacobLinCool/softchart-v16-tiny",
-        "display": "V1.6 tiny",
-        "runtime": "legacy",
-    },
-    "v1.5": {
-        "repo": "JacobLinCool/softchart-v15",
-        "display": "V1.5",
-        "runtime": "legacy",
-    },
+
+# Selectable generators: three generations x three sizes (plus the v1.5 single
+# model). All are dual (slot + time), beat- and plan-conditioned, MIT licensed.
+# https://huggingface.co/collections/JacobLinCool/softchart-generators
+MODELS = {
+    "v1.7": "JacobLinCool/softchart-v17",
+    "v1.7-small": "JacobLinCool/softchart-v17-small",
+    "v1.7-tiny": "JacobLinCool/softchart-v17-tiny",
+    "v1.6": "JacobLinCool/softchart-v16",
+    "v1.6-small": "JacobLinCool/softchart-v16-small",
+    "v1.6-tiny": "JacobLinCool/softchart-v16-tiny",
+    "v1.5": "JacobLinCool/softchart-v15",
 }
-DEFAULT_MODEL = "v1.8"
+DEFAULT_MODEL = os.environ.get("SC_MODEL", "v1.7")
+
+# BarScript is the experimental next-generation model. It is served through
+# softchart/sc2_loader.py, which resolves a release package from $SC2_PACKAGE_DIR,
+# a local directory, or its own Hub repo -- a resolution order the 1.x path does
+# not share. It is deliberately absent from MODELS: that dict is the *legacy*
+# routing table, every entry of which must be loadable by SoftChartGenerator.
+# from_pretrained, and BarScript's architecture cannot be built that way.
+SC2_MODEL = "BarScript (preview)"
+
+MODEL_LABELS = {
+    "v1.7": ("v1.7 — richest patterns (recommended)", "v1.7 · 7.9M — best quality"),
+    "v1.7-small": ("v1.7 — richest patterns (recommended)", "v1.7-small · 3.6M"),
+    "v1.7-tiny": ("v1.7 — richest patterns (recommended)", "v1.7-tiny · 1.4M — fastest"),
+    "v1.6": ("v1.6 — scaling ladder", "v1.6 · 7.9M"),
+    "v1.6-small": ("v1.6 — scaling ladder", "v1.6-small · 3.6M"),
+    "v1.6-tiny": ("v1.6 — scaling ladder", "v1.6-tiny · 1.4M"),
+    "v1.5": ("v1.5 — single model", "v1.5 · 7.9M"),
+    SC2_MODEL: ("BarScript — EXPERIMENTAL PREVIEW, under evaluation",
+                "BarScript (preview) — experimental, known defects"),
+}
+
+# Density bucket (width 0.75 notes/sec) requested per course. Measured on all
+# 1 155 songs of JacobLinCool/taiko-1000-parsed: median chart-span note rate is
+# 1.33 / 2.01 / 3.40 / 5.27 / 6.60 notes per second for easy / normal / hard /
+# oni / ura, which buckets to 1 / 2 / 4 / 7 / 8. The first four reproduce the
+# values this app already shipped; ura extends the same measurement and agrees
+# with the map the rest of the repo uses (scripts/infer.py, closed_loop.py,
+# eval_slot.py, gen_samples.py).
+COURSE_DENS = {"easy": 1, "normal": 2, "hard": 4, "oni": 7, "ura": 8}
+
+# TJA `COURSE:` header literals, measured rather than assumed. Of the 1 155
+# songs in the corpus, 229 carry an ura chart, and every one writes its header
+# as `Edit` (223) or the numeric `4` (6); the string "Ura" never appears -- it
+# is the dataset's normalized course NAME. `str(course).capitalize()`, which
+# both TJA writers use, would emit the invalid `COURSE:Ura`.
+# softchart/tja.py is vendored library code this app does not own, so the slot
+# writer's header is normalized here instead. For the four courses that shipped
+# before, this map reproduces the writers' output exactly, so normalization is
+# a no-op on them (asserted in scripts/test_app_wiring.py).
+TJA_COURSE_HEADER = {"easy": "Easy", "normal": "Normal", "hard": "Hard",
+                     "oni": "Oni", "ura": "Edit"}
+
+# Course ids the shipped planner was TRAINED on. scripts/train_planner.py builds
+# its dataset with courses=("easy","normal","hard","oni") and has no other call
+# site, so embedding row 4 (ura) only ever received weight decay. Probing the
+# published checkpoint confirms it: mean predicted density bucket per course id
+# is 0.573 / 1.126 / 2.274 / 3.789 for the trained ids -- monotone, as expected
+# -- and 1.471 for id 4, i.e. it would ask for a SPARSER plan for ura than for
+# oni, when ura is the densest course in the corpus. Row 4 is therefore not used.
+PLANNER_COURSE_ID = {"easy": 0, "normal": 1, "hard": 2, "oni": 3}
+# Courses the planner cannot answer for, and the trained course substituted for
+# them. Oni is ura's direct sibling and the highest trained density contour.
+# The substitution is reported to the user, never applied silently.
+PLANNER_COURSE_FALLBACK = {"ura": "oni"}
 
-COURSE_DENS = {"easy": 1, "normal": 2, "hard": 4, "oni": 7}
 CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
         "roll": "5", "roll_big": "6", "balloon": "7"}
 SUB = 96
 
-_MODELS = {}
-_PLANNER = {}
-_MEL_FB = None
-_STFT_WINDOW = None
+_MODELS = {}  # repo id -> {"gen","slot","beat"}, loaded lazily and cached
+_PLANNER = {}  # the planner is model-agnostic and shared across generators
+_SC2 = {}  # the BarScript release package, loaded lazily and cached
 
 
-def get_model(model_choice):
-    spec = MODEL_SPECS.get(model_choice)
-    if spec is None:
+def is_sc2_model(model_choice):
+    """Whether this choice routes to BarScript instead of the 1.x Hub table."""
+    return model_choice == SC2_MODEL
+
+
+def available_models():
+    """Selectable model names. BarScript appears only when its package resolves."""
+    names = list(MODELS)
+    if sc2_loader.is_available():
+        names.append(SC2_MODEL)
+    return names
+
+
+def get_models(model_choice=DEFAULT_MODEL, *, include_planner=False):
+    repo = MODELS.get(model_choice)
+    if repo is None:
         raise ValueError(
-            f"Unknown model {model_choice!r}. Choose one of: "
-            f"{', '.join(MODEL_SPECS)}."
+            f"unknown model {model_choice!r}; choose one of {list(MODELS)}"
         )
-    if model_choice not in _MODELS:
-        if spec["runtime"] == "v18":
-            model = load_hf_v18(spec["repo"], device=DEVICE)
-        else:
-            model = load_hf_legacy(spec["repo"], device=DEVICE)
-        capabilities = getattr(model, "_softchart_capabilities", {})
-        if spec["runtime"] == "v18":
-            required = {
-                "aux": True,
-                "beat_head": True,
-                "dual": True,
-                "hierarchical_ctx": True,
-                "global_ctx": False,
-                "plan": False,
-            }
-            mismatched = {
-                name: capabilities.get(name)
-                for name, expected in required.items()
-                if capabilities.get(name) is not expected
-            }
-            parameters = sum(parameter.numel() for parameter in model.parameters())
-            if (not getattr(model, "_hierarchical_ctx", False)
-                    or getattr(model, "_has_plan", False) or mismatched
-                    or parameters != EXPECTED_PARAMETERS):
-                raise RuntimeError(
-                    f"{spec['repo']} is not the expected hierarchical V1.8 artifact"
-                )
-            bundle = {
-                "model": model,
-                "generate": generate_song_v18,
-                "fit_grid": fit_grid_piecewise,
-                "fit_grid_fixed": fit_grid_fixed_bpm,
-                "preprocess": load_logmel_v18,
-                "runtime": "v18",
-            }
-        else:
-            if (not getattr(model, "_dual", False) or model.beat is None
-                    or not getattr(model, "_has_plan", False)):
-                raise RuntimeError(
-                    f"{spec['repo']} is not a supported V1.5–V1.7 artifact"
-                )
-            bundle = {
-                "model": model,
-                "generate": generate_song_legacy,
-                "fit_grid": fit_grid_piecewise_legacy,
-                "fit_grid_fixed": fit_grid_fixed_bpm_legacy,
-                "preprocess": load_logmel_legacy,
-                "runtime": "legacy",
-            }
-        _MODELS[model_choice] = bundle
-    return _MODELS[model_choice]
-
-
-def get_legacy_planner():
-    if "planner" not in _PLANNER:
-        _PLANNER["planner"] = (
-            SoftChartPlanner.from_pretrained(PLAN_REPO).to(DEVICE).eval()
+    if repo not in _MODELS:
+        u = load_hf(repo, device=DEVICE)
+        if (not getattr(u, "_dual", False) or u.beat is None
+                or not getattr(u, "_has_plan", False)):
+            raise RuntimeError(
+                f"{repo} must provide dual generation, beat, and plan conditioning"
+            )
+        _MODELS[repo] = {"gen": u, "slot": u, "beat": u}
+    models = dict(_MODELS[repo])
+    if include_planner:
+        if "plan" not in _PLANNER:
+            _PLANNER["plan"] = SoftChartPlanner.from_pretrained(PLAN_REPO).to(DEVICE).eval()
+        models["plan"] = _PLANNER["plan"]
+    return models
+
+
+def get_sc2():
+    """Load (and cache) the BarScript release package.
+
+    The log-mel front end is checked against the package's own contract on
+    every load: the Space computes mel itself, and a silent drift there would
+    hand the encoder a distribution it never saw.
+    """
+    if "sc2" not in _SC2:
+        sc = sc2_loader.load(device=DEVICE)
+        sc2_loader.check_preprocessing(
+            sc.package_dir, sr=SR, n_fft=N_FFT, hop_length=HOP,
+            n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
+        _SC2["sc2"] = sc
+    return _SC2["sc2"]
+
+
+def sc2_beat_model():
+    """Beat/downbeat model for the BarScript route.
+
+    BarScript has no beat head of its own; it needs an external BPM and downbeat to
+    build its bar lattice, so the default generator's beat model supplies them.
+    """
+    return get_models(DEFAULT_MODEL)["beat"]
+
+
+def planner_course(course):
+    """``(course_the_planner_is_asked_for, was_substituted)``.
+
+    Raises for a course with neither a trained id nor a documented substitute,
+    so an unsupported course fails closed instead of landing on an arbitrary
+    embedding row.
+    """
+    if course in PLANNER_COURSE_ID:
+        return course, False
+    fallback = PLANNER_COURSE_FALLBACK.get(course)
+    if fallback is None:
+        raise ValueError(
+            f"The song planner was not trained for {course}. Turn the AI "
+            "planner off, or pick another difficulty."
         )
-    return _PLANNER["planner"]
+    return fallback, True
 
 
-def load_logmel_v18(path):
-    """Apply the exact V1.8 FFmpeg + torch.stft preprocessing contract."""
-    import librosa
-    global _MEL_FB, _STFT_WINDOW
-
-    command = [
-        "ffmpeg", "-nostdin", "-hide_banner", "-loglevel", "error",
-        "-i", os.fspath(path), "-vn", "-ac", "2", "-ar", str(SR),
-        "-f", "f32le", "-acodec", "pcm_f32le", "pipe:1",
-    ]
-    proc = subprocess.run(
-        command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False,
-    )
-    if proc.returncode != 0:
-        detail = proc.stderr.decode("utf-8", errors="replace").strip()
-        raise ValueError(f"FFmpeg could not decode this upload: {detail}")
-    decoded = np.frombuffer(proc.stdout, dtype="<f4")
-    if decoded.size % 2:
-        raise ValueError("Decoded stereo audio contains an incomplete frame.")
-    wav = decoded.reshape(-1, 2).mean(axis=1, dtype=np.float32)
-    if wav.size < SR:
-        raise ValueError("Audio must be at least one second long.")
-    if not np.isfinite(wav).all():
-        raise ValueError("Decoded audio contains non-finite samples.")
-    if _MEL_FB is None:
-        fb = librosa.filters.mel(
-            sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2,
-        )
-        _MEL_FB = torch.from_numpy(fb)
-        _STFT_WINDOW = torch.hann_window(N_FFT)
-    wav_tensor = torch.from_numpy(np.ascontiguousarray(wav))[None]
-    with torch.no_grad():
-        spec = torch.stft(
-            wav_tensor, N_FFT, hop_length=HOP, window=_STFT_WINDOW,
-            center=True, return_complex=True,
-        )[0]
-        mel = torch.log(_MEL_FB @ spec.abs().pow(2) + 1e-5)
-    mel = mel.numpy().astype(np.float32)
-    return mel, wav
+def normalize_course_header(text, course):
+    """Rewrite the TJA ``COURSE:`` header to the literal players' tools expect."""
+    header = TJA_COURSE_HEADER.get(str(course).lower())
+    if header is None:
+        raise ValueError(f"Unsupported difficulty: {course}")
+    out, n = re.subn(r"(?m)^COURSE:.*$", f"COURSE:{header}", text, count=1)
+    if n != 1:
+        raise RuntimeError("the generated TJA has no COURSE header to normalize")
+    return out
 
 
-def load_logmel_legacy(path):
-    """Preserve the preprocessing contract used by V1.5–V1.7."""
+def load_logmel(path):
     import librosa
 
     wav, _ = librosa.load(path, sr=SR, mono=True)
-    fb = librosa.filters.mel(
-        sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2,
-    )
-    spec = torch.stft(
-        torch.from_numpy(wav), N_FFT, hop_length=HOP,
-        window=torch.hann_window(N_FFT), center=True, return_complex=True,
-    )
+    fb = librosa.filters.mel(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
+    spec = torch.stft(torch.from_numpy(wav), N_FFT, hop_length=HOP,
+                      window=torch.hann_window(N_FFT), center=True, return_complex=True)
     mel = np.log(fb @ spec.abs().pow(2).numpy() + 1e-5).astype(np.float32)
     return mel, wav
 
 
+def auto_plan(mel, bpm, downbeats=None):
+    T = mel.shape[1]
+    dur = T / FPS
+    flux = np.concatenate([[0], np.maximum(0, np.diff(mel, axis=1)).sum(0)])
+    beat = 60.0 / bpm
+    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
+        else list(np.arange(0, dur, 4 * beat)) + [dur]
+    vals = [float(flux[int(a * FPS):int(b * FPS)].mean()) if int(b * FPS) > int(a * FPS) else 0.0
+            for a, b in zip(edges, edges[1:])]
+    if not vals:
+        return None
+    vals = np.array(vals)
+    lo, hi = np.percentile(vals, 15), np.percentile(vals, 92)
+    peak = int(np.argmax(vals))
+    plan = []
+    for i, (a, b) in enumerate(zip(edges, edges[1:])):
+        frac = (vals[i] - lo) / max(hi - lo, 1e-6)
+        d8 = int(np.clip(round(frac * 7), 0, 7))
+        fl = 1 if (vals[i] <= lo and 0 < i < len(vals) - 1) else (2 if i == peak and vals[i] >= hi else 0)
+        plan.append([round(a, 3), round(b, 3), d8, fl])
+    return plan
+
+
 def learned_plan(planner, mel, course, bpm, downbeats=None):
-    duration = mel.shape[1] / FPS
+    dur = mel.shape[1] / FPS
     beat = 60.0 / bpm
-    edges = (
-        list(downbeats[::4]) + [duration]
-        if downbeats is not None and len(downbeats) >= 2
-        else list(np.arange(0, duration, 4 * beat)) + [duration]
-    )
-    features = []
-    spans = []
-    for start, end in zip(edges, edges[1:]):
-        segment = mel[:, int(start * FPS):int(end * FPS)]
-        if segment.shape[1] < 2:
+    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
+        else list(np.arange(0, dur, 4 * beat)) + [dur]
+    feats, spans = [], []
+    for a, b in zip(edges, edges[1:]):
+        seg = mel[:, int(a * FPS):int(b * FPS)]
+        if seg.shape[1] < 2:
             continue
-        flux = np.maximum(0, np.diff(segment, axis=1)).sum(0)
-        features.append(np.concatenate([
-            segment.mean(1), segment.std(1),
-            [flux.mean(), flux.std(), flux.max()],
-        ]))
-        spans.append((round(float(start), 3), round(float(end), 3)))
-    if not features:
+        fx = np.maximum(0, np.diff(seg, axis=1)).sum(0)
+        feats.append(np.concatenate([seg.mean(1), seg.std(1), [fx.mean(), fx.std(), fx.max()]]))
+        spans.append((round(float(a), 3), round(float(b), 3)))
+    if not feats:
         return None
-    course_id = {"easy": 0, "normal": 1, "hard": 2, "oni": 3}[course]
-    inputs = torch.tensor(np.array(features), dtype=torch.float32)[None].to(DEVICE)
+    cid = PLANNER_COURSE_ID[planner_course(course)[0]]
+    x = torch.tensor(np.array(feats), dtype=torch.float32)[None].to(DEVICE)
     with torch.no_grad():
-        density_logits, flag_logits = planner(
-            inputs, torch.tensor([course_id], device=DEVICE)
-        )
-    densities = density_logits[0].argmax(-1).cpu().numpy()
-    flags = flag_logits[0].argmax(-1).cpu().numpy()
-    return [
-        [start, end, int(density), int(flag)]
-        for (start, end), density, flag in zip(spans, densities, flags)
-    ]
+        pd, pf = planner(x, torch.tensor([cid], device=DEVICE))
+    d8 = pd[0].argmax(-1).cpu().numpy()
+    fl = pf[0].argmax(-1).cpu().numpy()
+    return [[a, b, int(d), int(f)] for (a, b), d, f in zip(spans, d8, fl)]
 
 
 def group_quantize(times, phase, grid, min_run=3):
@@ -355,27 +355,22 @@ def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=Non
     gogo_mask = gogo_measure_mask(plan, measure_starts, n_meas)
     lines = []
     in_gogo = False
-    for measure in range(n_meas):
+    for m in range(n_meas):
         in_gogo = append_measure_with_gogo(
             lines,
-            "".join(
-                slots.get(measure * SUB + slot, "0") for slot in range(SUB)
-            ) + ",",
-            measure,
-            gogo_mask,
-            in_gogo,
-        )
+            "".join(slots.get(m * SUB + k, "0") for k in range(SUB)) + ",",
+            m, gogo_mask, in_gogo)
     if in_gogo:
         lines.append("#GOGOEND")
     balloons = [10] * sum(1 for s in gen["spans"] if s["type"] == "balloon")
     return "\n".join([
         f"TITLE:{title} (SoftChart)", f"BPM:{bpm:g}", f"WAVE:{wave}",
-        f"OFFSET:{-phase:.3f}", f"COURSE:{'Oni' if course == 'oni' else course.capitalize()}",
+        f"OFFSET:{-phase:.3f}", f"COURSE:{TJA_COURSE_HEADER[course]}",
         f"LEVEL:{level}", f"BALLOON:{','.join(map(str, balloons))}" if balloons else "BALLOON:",
         "", "#START", *lines, "#END"]) + "\n"
 
 
-def render_song_structure(mel, title, course, out_path, plan=None):
+def render_audio_plan(mel, title, course, out_path, plan=None):
     import matplotlib
     matplotlib.use("Agg")
     from matplotlib import font_manager
@@ -389,12 +384,10 @@ def render_song_structure(mel, title, course, out_path, plan=None):
 
     dur = mel.shape[1] / FPS
     fig = plt.figure(figsize=(13, 4.4 if plan else 3.2))
-    grid = fig.add_gridspec(
-        2 if plan else 1, 1,
-        height_ratios=[3.0, 1.0] if plan else [1],
-        hspace=0.14 if plan else 0.0,
-    )
-    ax0 = fig.add_subplot(grid[0])
+    gs = fig.add_gridspec(2 if plan else 1, 1,
+                          height_ratios=[3.0, 1.0] if plan else [1],
+                          hspace=0.14 if plan else 0.0)
+    ax0 = fig.add_subplot(gs[0])
     ax0.imshow(mel, aspect="auto", origin="lower",
                cmap="magma", extent=[0, dur, 0, N_MELS])
     ax0.set_ylabel("mel")
@@ -403,23 +396,16 @@ def render_song_structure(mel, title, course, out_path, plan=None):
     ax0.grid(axis="x", alpha=0.18)
     if plan:
         ax0.set_xticklabels([])
-        ax1 = fig.add_subplot(grid[1], sharex=ax0)
-        for start, end, density, flag in plan:
-            color = "#d64545" if flag == 2 else (
-                "#4a90d9" if flag == 1 else "#999999"
-            )
-            ax1.bar(
-                (start + end) / 2,
-                max(density, 0.15),
-                width=max((end - start) * 0.92, 0.01),
-                color=color,
-                alpha=0.85,
-            )
+        ax1 = fig.add_subplot(gs[1], sharex=ax0)
+        for a, b, d, f in plan:
+            c = "#d64545" if f == 2 else ("#4a90d9" if f == 1 else "#999999")
+            ax1.bar((a + b) / 2, max(d, 0.15), width=max((b - a) * 0.92, 0.01),
+                    color=c, alpha=0.85)
         ax1.set_xlim(0, dur)
         ax1.set_ylim(0, 8)
         ax1.set_yticks([0, 4, 8])
         ax1.set_ylabel("plan", fontsize=8)
-        ax1.set_xlabel("time (s) — legacy plan: density / gap / climax")
+        ax1.set_xlabel("time (s)  —  plan: grey=density blue=gap red=climax")
         ax1.grid(axis="x", alpha=0.18)
     else:
         ax0.set_xlabel("time (s)")
@@ -428,6 +414,69 @@ def render_song_structure(mel, title, course, out_path, plan=None):
     return out_path
 
 
+def generate_sc2(sc2, mel, grid, dbs, bpm, course, level, wave_name, title,
+                 *, sampling, temperature, top_p, seed=0):
+    """Run the BarScript release. Returns ``(generated, tja, extra_metrics)``.
+
+    BarScript decodes onto a bar lattice supplied by the caller, so it needs both a
+    tempo and the audio time of the first downbeat. Neither can be guessed: a
+    lattice anchored on the wrong phase puts every barline in the wrong place,
+    and the chart would look like a model defect rather than a missing input.
+    The route therefore fails closed when no downbeat is available.
+
+    Sampling parameters are left at the package's recorded serving contract
+    unless the user turns creative sampling on. Anything overridden is reported
+    back, because the model card's measurements describe the recorded values.
+    """
+    if grid is not None and len(grid.get("downbeats", [])):
+        offset_sec = float(grid["downbeats"][0])
+        anchor = "fitted beat grid"
+    elif dbs is not None and len(dbs):
+        offset_sec = float(dbs[0])
+        anchor = "detected downbeats"
+    else:
+        raise ValueError(
+            "BarScript builds its bars from a downbeat and cannot place "
+            "them without one. Leave the beat grid switched on, or enter the "
+            "song's BPM in Advanced settings, and try again."
+        )
+
+    overrides = {}
+    if sampling:
+        overrides = {"greedy": False, "temperature": float(temperature),
+                     "top_p": float(top_p)}
+    result = sc2.generate(
+        mel, course, level=level, bpm=float(bpm), offset_sec=offset_sec,
+        density_bucket=COURSE_DENS[course], seed=seed,
+        grid_denom=sc2_loader.SHIP_GRID_DENOM,
+        title=title, wave=wave_name, **overrides,
+    )
+    grid_info = result["grid"]
+    segments = int(grid.get("n_segments", 1)) if grid is not None else 1
+    extra = {
+        "chart_grid": (f"/{grid_info['denom']} uniform"
+                       + ("" if grid_info["triplets_representable"]
+                          else " — triplets impossible")),
+        "grid_anchor": f"{anchor}, first downbeat {offset_sec:.3f} s",
+        "density_bucket": result["density_bucket"],
+        "sampling": ("user override: " + ", ".join(
+            f"{k}={v}" for k, v in sorted(result["serving_overrides"].items()))
+            ) if result["serving_overrides"] else "package defaults",
+    }
+    if segments > 1:
+        extra["tempo_map"] = (
+            f"flattened — the fit found {segments} tempo segments, BarScript "
+            f"decodes "
+            f"one uniform tempo ({bpm:.1f} BPM)"
+        )
+    if not result["motif"]["verified"]:
+        raise RuntimeError(
+            "BarScript decoded without the motif constraint the release "
+            "package resolved; refusing to present the chart."
+        )
+    return result["gen"], result["tja"], extra
+
+
 def _progress(stage, fraction, title, detail):
     return {
         "kind": "progress",
@@ -473,6 +522,50 @@ def _file_data(path, *, name=None, mime_type=None):
     ).model_dump()
 
 
+def _ura_notices():
+    """What a user picking Ura has to be told, whichever model is selected."""
+    corpus = {key: text for key, _, text in sc2_loader.KNOWN_LIMITATIONS}["ura"]
+    return [
+        {"when": "always", "text": corpus},
+        {"when": "planner",
+         "text": "The song planner was never trained on Ura, so its Oni plan "
+                 "is used instead. Turn the AI planner off to avoid the "
+                 "substitution."},
+    ]
+
+
+def capabilities():
+    """Everything the interface needs to build its menus honestly.
+
+    The BarScript entry is present only when its package resolves, and it carries
+    the model card's limitations verbatim so the interface cannot paraphrase
+    them into something milder.
+    """
+    models = []
+    for name in available_models():
+        group, label = MODEL_LABELS.get(name, (name, name))
+        entry = {"value": name, "group": group, "label": label,
+                 "experimental": is_sc2_model(name), "limitations": []}
+        if is_sc2_model(name):
+            entry["limitations"] = [
+                {"section": section, "text": text}
+                for _, section, text in sc2_loader.KNOWN_LIMITATIONS
+            ]
+            entry["notes"] = [
+                "Song structure planning does not apply: BarScript takes no plan, "
+                "so the Structure and AI planner switches are ignored and no "
+                "#GOGO sections are written.",
+                "BarScript needs a beat grid. Leave the beat grid on, or enter the "
+                "song's BPM, so its bar lattice has a downbeat to sit on.",
+            ]
+        models.append(entry)
+    return {
+        "default_model": DEFAULT_MODEL,
+        "models": models,
+        "course_notices": {"ura": _ura_notices()},
+    }
+
+
 app = gr.Server(
     title="SoftChart",
     description="Conditional Taiko chart generation with synchronized audio preview.",
@@ -490,14 +583,12 @@ async def homepage():
 
 @app.get("/health", include_in_schema=False)
 async def health():
-    return {
-        "status": "ok",
-        "default_model": DEFAULT_MODEL,
-        "available_models": list(MODEL_SPECS),
-        "device": DEVICE,
-        "models_loaded": sorted(_MODELS),
-        "planner_loaded": "planner" in _PLANNER,
-    }
+    return {"status": "ok", "device": DEVICE, "models_loaded": "gen" in _MODELS}
+
+
+@app.get("/api/capabilities", include_in_schema=False)
+async def api_capabilities():
+    return capabilities()
 
 
 @app.api(
@@ -514,7 +605,9 @@ def generate_chart(
     course: str,
     level: int,
     bpm_override: float,
+    auto_plan_on: bool,
     use_beat: bool,
+    use_planner: bool,
     sampling: bool,
     temperature: float,
     top_p: float,
@@ -531,25 +624,32 @@ def generate_chart(
         temperature = float(temperature)
         top_p = float(top_p)
         drum_volume = float(drum_volume)
-        spec = MODEL_SPECS.get(model_choice)
-        if spec is None:
+
+        choices = available_models()
+        if model_choice not in choices:
             raise ValueError(
-                f"Unknown model {model_choice!r}. Choose one of: "
-                f"{', '.join(MODEL_SPECS)}."
+                f"Unknown model {model_choice!r}. Choose one of: {', '.join(choices)}."
             )
-
+        sc2_used = is_sc2_model(model_choice)
         yield _progress(
             "loading", 0.03, "Preparing",
-            f"Loading SoftChart {spec['display']}.",
+            f"Loading SoftChart {model_choice}.",
         )
-        bundle = get_model(model_choice)
-        model = bundle["model"]
+        if sc2_used:
+            # BarScript lives in its own token space and cannot share a checkpoint
+            # with the 1.x route; it borrows only the beat model, which it has
+            # no equivalent of. include_planner is not honoured -- see below.
+            sc2 = get_sc2()
+            models = {"beat": sc2_beat_model()}
+        else:
+            sc2 = None
+            models = get_models(model_choice, include_planner=use_planner)
 
         yield _progress(
             "audio", 0.11, "Listening",
             "Mapping rhythm, melody, and timbre.",
         )
-        mel, wav = bundle["preprocess"](audio_path)
+        mel, wav = load_logmel(audio_path)
 
         yield _progress(
             "beat", 0.21, "Finding the beat",
@@ -557,7 +657,7 @@ def generate_chart(
         )
         grid = dbs = None
         if use_beat:
-            grid = bundle["fit_grid"](model, mel, device=DEVICE)
+            grid = fit_grid_piecewise(models["beat"], mel, device=DEVICE)
             if grid is not None:
                 dbs = grid["downbeats"] if grid["ok"] else grid["db_peaks"]
                 if not grid["ok"]:
@@ -565,9 +665,7 @@ def generate_chart(
         if bpm_override > 0:
             bpm = bpm_override
             if use_beat and (grid is None or abs(grid["bpm"] - bpm) > 0.5):
-                fixed_grid = bundle["fit_grid_fixed"](
-                    model, mel, bpm, device=DEVICE
-                )
+                fixed_grid = fit_grid_fixed_bpm(models["beat"], mel, bpm, device=DEVICE)
                 grid = fixed_grid if fixed_grid is not None and fixed_grid["ok"] else None
                 if grid is not None:
                     dbs = grid["downbeats"]
@@ -590,36 +688,61 @@ def generate_chart(
         if grid is None and abs(bpm - round(bpm)) < 0.06:
             bpm = float(round(bpm))
 
-        plan = None
-        if bundle["runtime"] == "v18":
-            structure_title = "Reading the structure"
-            structure_detail = "Building whole-song hierarchical context."
-        else:
-            structure_title = "Shaping the arc"
-            structure_detail = "Applying the original legacy song planner."
-            plan = learned_plan(
-                get_legacy_planner(), mel, course, bpm, downbeats=dbs
-            )
         yield _progress(
-            "structure", 0.34, structure_title, structure_detail,
+            "plan", 0.34, "Shaping the arc",
+            "Planning density and climaxes.",
         )
+        plan = None
+        plan_source = "None"
+        if sc2_used:
+            # BarScript's decoder takes no plan token, so neither switch reaches it.
+            # Reporting a plan that nothing consumed would be a lie in the UI.
+            plan_source = "Not used — BarScript takes no plan"
+        elif getattr(models["gen"], "_has_plan", False):
+            if use_planner:
+                asked, substituted = planner_course(course)
+                plan = learned_plan(models["plan"], mel, course, bpm, dbs)
+                plan_source = (f"AI planner ({asked.capitalize()} plan reused "
+                               f"for {course.capitalize()})" if substituted
+                               else "AI planner")
+            elif auto_plan_on:
+                plan = auto_plan(mel, bpm, dbs)
+                plan_source = "Energy heuristic"
 
         yield _progress(
             "generate", 0.47, "Writing the chart",
             "Writing playable Taiko patterns.",
         )
         title = os.path.splitext(wave_name)[0]
-        generated = bundle["generate"](
-            model, mel, course, level=level,
-            density_bucket=COURSE_DENS[course], greedy=not sampling,
-            temperature=temperature, top_p=top_p, seed=0, device=DEVICE,
-            plan=plan,
-        )
-        generated = snap_chart(generated, bpm)
-        tja = write_tja(
-            generated, bpm, title, course, level, wave_name, dbs,
-            grid_fit=grid, plan=plan,
-        )
+        extra_metrics = {}
+        slot_used = grid is not None
+        if sc2_used:
+            generated, tja, extra_metrics = generate_sc2(
+                sc2, mel, grid, dbs, bpm, course, level, wave_name, title,
+                sampling=sampling, temperature=temperature, top_p=top_p,
+            )
+            slot_used = True
+        elif slot_used:
+            generated = generate_song_slot(
+                models["slot"], mel, grid, course, level=level,
+                density_bucket=COURSE_DENS[course], greedy=not sampling, seed=0,
+                temperature=temperature, top_p=top_p, device=DEVICE, plan=plan,
+            )
+            tja = normalize_course_header(write_tja_slots(
+                generated, grid, title, course, level, wave_name, plan=plan,
+            ), course)
+        else:
+            generated = generate_song(
+                models["gen"], mel, course, level=level,
+                density_bucket=COURSE_DENS[course], greedy=not sampling,
+                temperature=temperature, top_p=top_p, seed=0,
+                device=DEVICE, plan=plan,
+            )
+            generated = snap_chart(generated, bpm)
+            tja = write_tja(
+                generated, bpm, title, course, level, wave_name, dbs,
+                grid_fit=grid, plan=plan,
+            )
 
         yield _progress(
             "export", 0.81, "Rendering",
@@ -629,11 +752,9 @@ def generate_chart(
         with open(tja_path, "w", encoding="utf-8") as output_file:
             output_file.write(tja)
         chart_image_path = os.path.join(workdir, "chart.png")
-        structure_image_path = os.path.join(workdir, "song-structure.png")
+        plan_image_path = os.path.join(workdir, "song-plan.png")
         render_tja_image(tja, out_path=chart_image_path)
-        render_song_structure(
-            mel, title, course, out_path=structure_image_path, plan=plan,
-        )
+        render_audio_plan(mel, title, course, plan=plan, out_path=plan_image_path)
 
         yield _progress(
             "mix", 0.91, "Mixing",
@@ -651,17 +772,16 @@ def generate_chart(
 
         grid_rms = round(float(grid["rms_ms"]), 1) if grid is not None else None
         metrics = {
-            "model": model_choice,
             "bpm": round(float(bpm), 1),
             "notes": len(generated["hits"]),
             "spans": len(generated["spans"]),
-            "timing": (
-                "hierarchical time + grid quantization"
-                if bundle["runtime"] == "v18"
-                else "legacy time + grid quantization"
-            ),
+            "timing": "slot-exact" if slot_used else "time-quantized",
             "grid_rms_ms": grid_rms,
             "preview_hits": int(mix_stats["rendered_hit_count"]),
+            "model": model_choice,
+            "difficulty": f"{course.capitalize()} (TJA {TJA_COURSE_HEADER[course]})",
+            "plan_source": plan_source,
+            **extra_metrics,
         }
         yield {
             "kind": "complete",
@@ -674,9 +794,7 @@ def generate_chart(
                 "tja": _file_data(tja_path, mime_type="text/plain"),
                 "audio": _file_data(preview_path, mime_type="audio/wav"),
                 "chart_image": _file_data(chart_image_path, mime_type="image/png"),
-                "structure_image": _file_data(
-                    structure_image_path, mime_type="image/png"
-                ),
+                "plan_image": _file_data(plan_image_path, mime_type="image/png"),
             },
         }
     except Exception as exc:

packages.txt

@@ -1,2 +1 @@
-ffmpeg
 fonts-noto-cjk

scripts/fixtures/app_baseline.py

@@ -0,0 +1,551 @@
+"""SoftChart — custom Gradio Server app for Hugging Face Spaces.
+
+Generate a Taiko no Tatsujin chart from any audio file, using the full system:
+  - SoftChartGenerator  (main model, plan-conditioned)
+  - SoftChartPlanner     (auto song-level planning)
+  - SoftChartBeat        (beat/downbeat for barline anchoring)
+All models load from the Hub via from_pretrained. MIT licensed.
+"""
+
+import logging
+import os
+import re
+import shutil
+import tempfile
+from pathlib import Path
+
+import gradio as gr
+import numpy as np
+import torch
+from fastapi.responses import FileResponse
+from fastapi.staticfiles import StaticFiles
+
+from softchart.generate import generate_song, generate_song_slot, load_hf
+from softchart.fonts import cjk_font_path
+from softchart.grid import debias_to_grid, fit_grid_fixed_bpm, fit_grid_piecewise
+from softchart.hf import SoftChartPlanner
+from softchart.preview_audio import synthesize_taiko_preview
+from softchart.rhythm import snap_chart
+from softchart.tja import append_measure_with_gogo, gogo_measure_mask, write_tja_slots
+from softchart.tja_image import render_tja_image
+from softchart.vocab import FPS, HOP, N_FFT, N_MELS, SR
+
+LOGGER = logging.getLogger("softchart.space")
+STATIC_DIR = Path(__file__).with_name("static")
+
+PLAN_REPO = os.environ.get("SC_PLAN", "JacobLinCool/softchart-planner")
+DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
+
+# Selectable generators: three generations x three sizes (plus the v1.5 single
+# model). All are dual (slot + time), beat- and plan-conditioned, MIT licensed.
+# https://huggingface.co/collections/JacobLinCool/softchart-generators
+MODELS = {
+    "v1.7": "JacobLinCool/softchart-v17",
+    "v1.7-small": "JacobLinCool/softchart-v17-small",
+    "v1.7-tiny": "JacobLinCool/softchart-v17-tiny",
+    "v1.6": "JacobLinCool/softchart-v16",
+    "v1.6-small": "JacobLinCool/softchart-v16-small",
+    "v1.6-tiny": "JacobLinCool/softchart-v16-tiny",
+    "v1.5": "JacobLinCool/softchart-v15",
+}
+DEFAULT_MODEL = os.environ.get("SC_MODEL", "v1.7")
+
+COURSE_DENS = {"easy": 1, "normal": 2, "hard": 4, "oni": 7}
+CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
+        "roll": "5", "roll_big": "6", "balloon": "7"}
+SUB = 96
+
+_MODELS = {}  # repo id -> {"gen","slot","beat"}, loaded lazily and cached
+_PLANNER = {}  # the planner is model-agnostic and shared across generators
+
+
+def get_models(model_choice=DEFAULT_MODEL, *, include_planner=False):
+    repo = MODELS.get(model_choice)
+    if repo is None:
+        raise ValueError(
+            f"unknown model {model_choice!r}; choose one of {list(MODELS)}"
+        )
+    if repo not in _MODELS:
+        u = load_hf(repo, device=DEVICE)
+        if (not getattr(u, "_dual", False) or u.beat is None
+                or not getattr(u, "_has_plan", False)):
+            raise RuntimeError(
+                f"{repo} must provide dual generation, beat, and plan conditioning"
+            )
+        _MODELS[repo] = {"gen": u, "slot": u, "beat": u}
+    models = dict(_MODELS[repo])
+    if include_planner:
+        if "plan" not in _PLANNER:
+            _PLANNER["plan"] = SoftChartPlanner.from_pretrained(PLAN_REPO).to(DEVICE).eval()
+        models["plan"] = _PLANNER["plan"]
+    return models
+
+
+def load_logmel(path):
+    import librosa
+
+    wav, _ = librosa.load(path, sr=SR, mono=True)
+    fb = librosa.filters.mel(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
+    spec = torch.stft(torch.from_numpy(wav), N_FFT, hop_length=HOP,
+                      window=torch.hann_window(N_FFT), center=True, return_complex=True)
+    mel = np.log(fb @ spec.abs().pow(2).numpy() + 1e-5).astype(np.float32)
+    return mel, wav
+
+
+def auto_plan(mel, bpm, downbeats=None):
+    T = mel.shape[1]
+    dur = T / FPS
+    flux = np.concatenate([[0], np.maximum(0, np.diff(mel, axis=1)).sum(0)])
+    beat = 60.0 / bpm
+    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
+        else list(np.arange(0, dur, 4 * beat)) + [dur]
+    vals = [float(flux[int(a * FPS):int(b * FPS)].mean()) if int(b * FPS) > int(a * FPS) else 0.0
+            for a, b in zip(edges, edges[1:])]
+    if not vals:
+        return None
+    vals = np.array(vals)
+    lo, hi = np.percentile(vals, 15), np.percentile(vals, 92)
+    peak = int(np.argmax(vals))
+    plan = []
+    for i, (a, b) in enumerate(zip(edges, edges[1:])):
+        frac = (vals[i] - lo) / max(hi - lo, 1e-6)
+        d8 = int(np.clip(round(frac * 7), 0, 7))
+        fl = 1 if (vals[i] <= lo and 0 < i < len(vals) - 1) else (2 if i == peak and vals[i] >= hi else 0)
+        plan.append([round(a, 3), round(b, 3), d8, fl])
+    return plan
+
+
+def learned_plan(planner, mel, course, bpm, downbeats=None):
+    dur = mel.shape[1] / FPS
+    beat = 60.0 / bpm
+    edges = (list(downbeats[::4]) + [dur]) if (downbeats is not None and len(downbeats) >= 2) \
+        else list(np.arange(0, dur, 4 * beat)) + [dur]
+    feats, spans = [], []
+    for a, b in zip(edges, edges[1:]):
+        seg = mel[:, int(a * FPS):int(b * FPS)]
+        if seg.shape[1] < 2:
+            continue
+        fx = np.maximum(0, np.diff(seg, axis=1)).sum(0)
+        feats.append(np.concatenate([seg.mean(1), seg.std(1), [fx.mean(), fx.std(), fx.max()]]))
+        spans.append((round(float(a), 3), round(float(b), 3)))
+    if not feats:
+        return None
+    cid = {"easy": 0, "normal": 1, "hard": 2, "oni": 3}[course]
+    x = torch.tensor(np.array(feats), dtype=torch.float32)[None].to(DEVICE)
+    with torch.no_grad():
+        pd, pf = planner(x, torch.tensor([cid], device=DEVICE))
+    d8 = pd[0].argmax(-1).cpu().numpy()
+    fl = pf[0].argmax(-1).cpu().numpy()
+    return [[a, b, int(d), int(f)] for (a, b), d, f in zip(spans, d8, fl)]
+
+
+def group_quantize(times, phase, grid, min_run=3):
+    n = len(times)
+    slots = [0] * n
+    i = 0
+    while i < n:
+        j = i
+        while j + 1 < n:
+            ioi = times[j + 1] - times[j]
+            ref = (times[j] - times[i]) / (j - i) if j > i else ioi
+            if 0.02 < ioi < 1.2 and abs(ioi - ref) < 0.22 * max(ref, 1e-6):
+                j += 1
+            else:
+                break
+        if j - i + 1 >= min_run:
+            k = max(1, int(round((times[j] - times[i]) / (j - i) / grid)))
+            anchor = int(round((times[i] - phase) / grid))
+            for m in range(j - i + 1):
+                slots[i + m] = anchor + m * k
+        else:
+            for m in range(i, j + 1):
+                slots[m] = int(round((times[m] - phase) / grid))
+        i = j + 1
+    return slots
+
+
+def upload_wave_name(audio_path):
+    name = os.path.basename(str(audio_path).replace("\\", "/")).strip()
+    name = re.sub(r"[\x00-\x1f\x7f]+", " ", name).strip()
+    return name or "song.ogg"
+
+
+def output_tja_path(wave_name, course, directory):
+    stem = os.path.splitext(os.path.basename(wave_name))[0].strip() or "softchart"
+    stem = re.sub(r"[<>:\"/\\|?*\x00-\x1f]+", "_", stem).strip(" ._") or "softchart"
+    return os.path.join(directory, f"{stem}_{course}.tja")
+
+
+def write_tja(gen, bpm, title, course, level, wave, downbeats=None, grid_fit=None,
+              plan=None):
+    hits = sorted((h["t"], CHAR[h["type"]]) for h in gen["hits"])
+    beat = 60.0 / bpm
+    grid = beat / (SUB / 4)
+    bias = 0.0
+    if grid_fit is not None and hits:
+        # authoritative fitted grid: barlines ARE the fitted downbeats.
+        # De-bias the generator's systematic latency (global shift only),
+        # then anchor slot 0 on the last fitted barline at/before the first note.
+        times, bias = debias_to_grid([t for t, _ in hits], grid_fit["phase"], grid)
+        phase = grid_fit["phase"] + float(np.floor((times[0] - grid_fit["phase"]) / (4 * beat))) * 4 * beat
+        q_times = list(times)
+    else:
+        times = np.array([t for t, _ in hits]) if hits else np.array([0.0])
+        cands = np.arange(0, beat, grid / 4)
+        phase = float(cands[int(np.argmin([np.mean(np.abs(((times - o) / grid) - np.round((times - o) / grid))) for o in cands]))])
+        q_times = [t for t, _ in hits]
+    slot_idx = group_quantize(q_times, phase, grid)
+    if slot_idx and min(slot_idx) < 0:
+        # note quantized just before the anchor barline: pull back whole bars
+        # so nothing is dropped (barline alignment is preserved mod SUB)
+        nb = int(np.ceil(-min(slot_idx) / SUB))
+        slot_idx = [s + nb * SUB for s in slot_idx]
+        phase -= nb * SUB * grid
+    slots = {}
+    for idx, (t, ch) in zip(slot_idx, hits):
+        if idx >= 0 and idx not in slots:
+            slots[idx] = ch
+    for sp in gen["spans"]:
+        i0 = int(round((sp["t0"] - bias - phase) / grid))
+        i1 = int(round((sp["t1"] - bias - phase) / grid))
+        while i0 in slots:
+            i0 += 1
+        while i1 in slots or i1 <= i0:
+            i1 += 1
+        if i0 >= 0:
+            slots[i0] = CHAR[sp["type"]]
+            slots[i1] = "8"
+    if slots and grid_fit is None:
+        # Gridless anchoring: shift so the first note sits on a detected
+        # downbeat if one is nearby, otherwise on the first barline.
+        first_t = min(slots) * grid + phase
+        anchor_t = None
+        if downbeats is not None and len(downbeats):
+            near = downbeats[downbeats <= first_t + 0.12]
+            if len(near) and first_t - near[-1] < 4 * beat:
+                anchor_t = near[-1]
+        shift = int(round((anchor_t - phase) / grid)) if anchor_t is not None else min(slots)
+        if shift:
+            slots = {k - shift: v for k, v in slots.items()}
+            phase += shift * grid
+    n_meas = (max(slots) // SUB + 1) if slots else 1
+    measure_starts = phase + np.arange(n_meas + 1, dtype=float) * (4 * beat)
+    gogo_mask = gogo_measure_mask(plan, measure_starts, n_meas)
+    lines = []
+    in_gogo = False
+    for m in range(n_meas):
+        in_gogo = append_measure_with_gogo(
+            lines,
+            "".join(slots.get(m * SUB + k, "0") for k in range(SUB)) + ",",
+            m, gogo_mask, in_gogo)
+    if in_gogo:
+        lines.append("#GOGOEND")
+    balloons = [10] * sum(1 for s in gen["spans"] if s["type"] == "balloon")
+    return "\n".join([
+        f"TITLE:{title} (SoftChart)", f"BPM:{bpm:g}", f"WAVE:{wave}",
+        f"OFFSET:{-phase:.3f}", f"COURSE:{'Oni' if course == 'oni' else course.capitalize()}",
+        f"LEVEL:{level}", f"BALLOON:{','.join(map(str, balloons))}" if balloons else "BALLOON:",
+        "", "#START", *lines, "#END"]) + "\n"
+
+
+def render_audio_plan(mel, title, course, out_path, plan=None):
+    import matplotlib
+    matplotlib.use("Agg")
+    from matplotlib import font_manager
+    import matplotlib.pyplot as plt
+
+    font_path = cjk_font_path()
+    if font_path is not None:
+        font_manager.fontManager.addfont(font_path)
+        matplotlib.rcParams["font.family"] = font_manager.FontProperties(fname=font_path).get_name()
+    matplotlib.rcParams["axes.unicode_minus"] = False
+
+    dur = mel.shape[1] / FPS
+    fig = plt.figure(figsize=(13, 4.4 if plan else 3.2))
+    gs = fig.add_gridspec(2 if plan else 1, 1,
+                          height_ratios=[3.0, 1.0] if plan else [1],
+                          hspace=0.14 if plan else 0.0)
+    ax0 = fig.add_subplot(gs[0])
+    ax0.imshow(mel, aspect="auto", origin="lower",
+               cmap="magma", extent=[0, dur, 0, N_MELS])
+    ax0.set_ylabel("mel")
+    ax0.set_title(f"{title} — {course}  |  full-song mel spectrogram")
+    ax0.set_xlim(0, dur)
+    ax0.grid(axis="x", alpha=0.18)
+    if plan:
+        ax0.set_xticklabels([])
+        ax1 = fig.add_subplot(gs[1], sharex=ax0)
+        for a, b, d, f in plan:
+            c = "#d64545" if f == 2 else ("#4a90d9" if f == 1 else "#999999")
+            ax1.bar((a + b) / 2, max(d, 0.15), width=max((b - a) * 0.92, 0.01),
+                    color=c, alpha=0.85)
+        ax1.set_xlim(0, dur)
+        ax1.set_ylim(0, 8)
+        ax1.set_yticks([0, 4, 8])
+        ax1.set_ylabel("plan", fontsize=8)
+        ax1.set_xlabel("time (s)  —  plan: grey=density blue=gap red=climax")
+        ax1.grid(axis="x", alpha=0.18)
+    else:
+        ax0.set_xlabel("time (s)")
+    fig.savefig(out_path, dpi=130, bbox_inches="tight")
+    plt.close(fig)
+    return out_path
+
+
+def _progress(stage, fraction, title, detail):
+    return {
+        "kind": "progress",
+        "stage": stage,
+        "progress": fraction,
+        "title": title,
+        "detail": detail,
+    }
+
+
+def _uploaded_file(value, original_name):
+    if value is None:
+        raise ValueError("Upload a music file to begin.")
+    if not isinstance(original_name, str) or not original_name.strip():
+        raise ValueError("The upload is missing its original filename. Please select it again.")
+    data = value if isinstance(value, gr.FileData) else gr.FileData.model_validate(value)
+    path = os.path.realpath(data.path)
+    if not os.path.isfile(path):
+        raise ValueError("The uploaded file is no longer available. Please select it again.")
+    return path, upload_wave_name(original_name)
+
+
+def _validate_controls(course, level, bpm, temperature, top_p, drum_volume):
+    if course not in COURSE_DENS:
+        raise ValueError(f"Unsupported difficulty: {course}")
+    if not 1 <= int(level) <= 10:
+        raise ValueError("Level must be between 1 and 10 stars.")
+    if bpm != 0 and not 30 <= float(bpm) <= 400:
+        raise ValueError("Manual BPM must be between 30 and 400; use 0 for auto-detection.")
+    if not 0.2 <= float(temperature) <= 1.2:
+        raise ValueError("Temperature must be between 0.2 and 1.2.")
+    if not 0.5 <= float(top_p) <= 1.0:
+        raise ValueError("Top-p must be between 0.5 and 1.0.")
+    if not 0 <= float(drum_volume) <= 1.5:
+        raise ValueError("Taiko volume must be between 0% and 150%.")
+
+
+def _file_data(path, *, name=None, mime_type=None):
+    return gr.FileData(
+        path=os.path.realpath(path),
+        orig_name=name or os.path.basename(path),
+        mime_type=mime_type,
+    ).model_dump()
+
+
+app = gr.Server(
+    title="SoftChart",
+    description="Conditional Taiko chart generation with synchronized audio preview.",
+)
+app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
+
+
+@app.get("/", include_in_schema=False)
+async def homepage():
+    return FileResponse(
+        STATIC_DIR / "index.html",
+        headers={"Cache-Control": "no-cache"},
+    )
+
+
+@app.get("/health", include_in_schema=False)
+async def health():
+    return {"status": "ok", "device": DEVICE, "models_loaded": "gen" in _MODELS}
+
+
+@app.api(
+    name="generate_chart",
+    description="Generate a TJA chart, visual previews, and a synchronized taiko audio mix.",
+    concurrency_limit=1,
+    concurrency_id="generation-gpu",
+    queue=True,
+    stream_every=0.1,
+)
+def generate_chart(
+    audio: gr.FileData,
+    audio_name: str,
+    course: str,
+    level: int,
+    bpm_override: float,
+    auto_plan_on: bool,
+    use_beat: bool,
+    use_planner: bool,
+    sampling: bool,
+    temperature: float,
+    top_p: float,
+    drum_volume: float,
+    model_choice: str = DEFAULT_MODEL,
+) -> dict[str, object]:
+    """Stream the actual inference stages to the custom frontend."""
+    workdir = tempfile.mkdtemp(prefix="softchart-request-")
+    try:
+        audio_path, wave_name = _uploaded_file(audio, audio_name)
+        _validate_controls(course, level, bpm_override, temperature, top_p, drum_volume)
+        level = int(level)
+        bpm_override = float(bpm_override)
+        temperature = float(temperature)
+        top_p = float(top_p)
+        drum_volume = float(drum_volume)
+
+        if model_choice not in MODELS:
+            raise ValueError(
+                f"Unknown model {model_choice!r}. Choose one of: {', '.join(MODELS)}."
+            )
+        yield _progress(
+            "loading", 0.03, "Preparing",
+            f"Loading SoftChart {model_choice}.",
+        )
+        models = get_models(model_choice, include_planner=use_planner)
+
+        yield _progress(
+            "audio", 0.11, "Listening",
+            "Mapping rhythm, melody, and timbre.",
+        )
+        mel, wav = load_logmel(audio_path)
+
+        yield _progress(
+            "beat", 0.21, "Finding the beat",
+            "Finding beats and tempo.",
+        )
+        grid = dbs = None
+        if use_beat:
+            grid = fit_grid_piecewise(models["beat"], mel, device=DEVICE)
+            if grid is not None:
+                dbs = grid["downbeats"] if grid["ok"] else grid["db_peaks"]
+                if not grid["ok"]:
+                    grid = None
+        if bpm_override > 0:
+            bpm = bpm_override
+            if use_beat and (grid is None or abs(grid["bpm"] - bpm) > 0.5):
+                fixed_grid = fit_grid_fixed_bpm(models["beat"], mel, bpm, device=DEVICE)
+                grid = fixed_grid if fixed_grid is not None and fixed_grid["ok"] else None
+                if grid is not None:
+                    dbs = grid["downbeats"]
+        elif grid is not None:
+            bpm = float(grid["bpm"])
+        elif dbs is not None and len(dbs) > 4:
+            period = float(np.median(np.diff(dbs)))
+            if period <= 0:
+                raise RuntimeError("The beat model returned an invalid downbeat interval.")
+            bpm = 240.0 / period
+            while bpm >= 210:
+                bpm /= 2.0
+            while bpm < 70:
+                bpm *= 2.0
+        else:
+            import librosa
+            bpm = float(np.atleast_1d(librosa.beat.beat_track(y=wav, sr=SR)[0])[0])
+        if not np.isfinite(bpm) or bpm <= 0:
+            raise RuntimeError("Could not determine a reliable BPM. Enter it in Advanced settings.")
+        if grid is None and abs(bpm - round(bpm)) < 0.06:
+            bpm = float(round(bpm))
+
+        yield _progress(
+            "plan", 0.34, "Shaping the arc",
+            "Planning density and climaxes.",
+        )
+        plan = None
+        if getattr(models["gen"], "_has_plan", False):
+            if use_planner:
+                plan = learned_plan(models["plan"], mel, course, bpm, dbs)
+            elif auto_plan_on:
+                plan = auto_plan(mel, bpm, dbs)
+
+        yield _progress(
+            "generate", 0.47, "Writing the chart",
+            "Writing playable Taiko patterns.",
+        )
+        title = os.path.splitext(wave_name)[0]
+        slot_used = grid is not None
+        if slot_used:
+            generated = generate_song_slot(
+                models["slot"], mel, grid, course, level=level,
+                density_bucket=COURSE_DENS[course], greedy=not sampling, seed=0,
+                temperature=temperature, top_p=top_p, device=DEVICE, plan=plan,
+            )
+            tja = write_tja_slots(
+                generated, grid, title, course, level, wave_name, plan=plan,
+            )
+        else:
+            generated = generate_song(
+                models["gen"], mel, course, level=level,
+                density_bucket=COURSE_DENS[course], greedy=not sampling,
+                temperature=temperature, top_p=top_p, seed=0,
+                device=DEVICE, plan=plan,
+            )
+            generated = snap_chart(generated, bpm)
+            tja = write_tja(
+                generated, bpm, title, course, level, wave_name, dbs,
+                grid_fit=grid, plan=plan,
+            )
+
+        yield _progress(
+            "export", 0.81, "Rendering",
+            "Building the TJA and previews.",
+        )
+        tja_path = output_tja_path(wave_name, course, workdir)
+        with open(tja_path, "w", encoding="utf-8") as output_file:
+            output_file.write(tja)
+        chart_image_path = os.path.join(workdir, "chart.png")
+        plan_image_path = os.path.join(workdir, "song-plan.png")
+        render_tja_image(tja, out_path=chart_image_path)
+        render_audio_plan(mel, title, course, plan=plan, out_path=plan_image_path)
+
+        yield _progress(
+            "mix", 0.91, "Mixing",
+            "Mixing Taiko with your track.",
+        )
+        stem = Path(tja_path).stem
+        preview_path = os.path.join(workdir, f"{stem}_taiko-preview.wav")
+        mix_stats = synthesize_taiko_preview(
+            audio_path,
+            tja,
+            preview_path,
+            drum_gain=drum_volume,
+            sample_rate=44100,
+        )
+
+        grid_rms = round(float(grid["rms_ms"]), 1) if grid is not None else None
+        metrics = {
+            "bpm": round(float(bpm), 1),
+            "notes": len(generated["hits"]),
+            "spans": len(generated["spans"]),
+            "timing": "slot-exact" if slot_used else "time-quantized",
+            "grid_rms_ms": grid_rms,
+            "preview_hits": int(mix_stats["rendered_hit_count"]),
+        }
+        yield {
+            "kind": "complete",
+            "stage": "complete",
+            "progress": 1.0,
+            "title": "Ready",
+            "detail": "Play it or download it.",
+            "metrics": metrics,
+            "files": {
+                "tja": _file_data(tja_path, mime_type="text/plain"),
+                "audio": _file_data(preview_path, mime_type="audio/wav"),
+                "chart_image": _file_data(chart_image_path, mime_type="image/png"),
+                "plan_image": _file_data(plan_image_path, mime_type="image/png"),
+            },
+        }
+    except Exception as exc:
+        LOGGER.exception("SoftChart generation failed")
+        detail = (str(exc) if isinstance(exc, (ValueError, RuntimeError))
+                  else "The server could not complete this chart. Please try again shortly.")
+        yield {
+            "kind": "error",
+            "stage": "error",
+            "progress": 0.0,
+            "title": "Generation did not complete",
+            "detail": detail,
+        }
+    finally:
+        shutil.rmtree(workdir, ignore_errors=True)
+
+
+if __name__ == "__main__":
+    app.launch(max_file_size="200mb")

scripts/test_app_wiring.py

@@ -0,0 +1,1054 @@
+"""Acceptance tests for the Space's model menu and generation routing.
+
+Runs on CPU with no network: every model is a recorder, and the only real
+inference is one optional BarScript decode when the release package is on
+disk (see ``default_package_dir``).
+
+The tests answer four questions the wiring change has to settle:
+
+1. Do all eight model choices reach the right loader?
+2. Do all five difficulties survive validation, and does ura export the TJA
+   ``COURSE:`` literal the corpus actually uses?
+3. Do the seven published models still behave exactly as before? Every heavy
+   dependency is a recorder, so a run reduces to an ordered list of calls with
+   their arguments plus the reported metrics. ``fixtures/legacy_call_trace.json``
+   holds that list as captured from the app BEFORE the wiring change, over 35
+   scenarios (7 models x 5 control combinations), and the current app has to
+   reproduce it call for call.
+
+   Re-record with ``--record OLD_APP.py``, pointing at the app revision the
+   trace should describe. Do not re-record to make a failure go away: a
+   difference here is a behaviour change on the published models, which is the
+   thing this file exists to catch.
+4. Does the BarScript route hand the loader the conditions the interface
+   promised, and does the package resolve in the documented order?
+
+    python spaces/scripts/test_app_wiring.py
+"""
+
+import argparse
+import copy
+import importlib.util
+import json
+import os
+import sys
+import tempfile
+
+import numpy as np
+
+SPACES = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+if SPACES not in sys.path:
+    sys.path.insert(0, SPACES)
+
+import app as APP  # noqa: E402
+from softchart import sc2_loader  # noqa: E402
+
+FPS = 22050 / 256
+LEGACY_COURSES = ("easy", "normal", "hard", "oni")
+ALL_COURSES = LEGACY_COURSES + ("ura",)
+GOLDEN = os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                      "fixtures", "legacy_call_trace.json")
+
+# The control combinations the published models are replayed under. Between
+# them they cover both writers (slot-exact and time-quantized), both plan
+# sources, greedy and sampled decoding, and the manual-BPM refit.
+LEGACY_SCENARIOS = [
+    {"course": "oni", "use_planner": True, "sampling": False},
+    {"course": "hard", "use_planner": False, "auto_plan_on": True},
+    {"course": "easy", "use_planner": True, "sampling": True, "level": 3},
+    # beat grid off with a manual BPM: no fitted grid, so this is the only
+    # combination that reaches the time-quantized writer (app.write_tja).
+    {"course": "normal", "use_beat": False, "bpm_override": 140.0,
+     "use_planner": False, "auto_plan_on": False},
+    {"course": "oni", "bpm_override": 180.0, "use_planner": True},
+]
+
+RESULTS = []
+
+
+def check(name, ok, detail=""):
+    RESULTS.append((name, bool(ok), detail))
+    print(f"{'PASS' if ok else 'FAIL'}  {name}" + (f"  -- {detail}" if detail else ""))
+    return bool(ok)
+
+
+# ---------------------------------------------------------------------------
+# recorders
+
+
+class Recorder:
+    """Stand-in for a loaded model; only its identity and calls matter."""
+
+    def __init__(self, tag, calls):
+        self.tag = tag
+        self.calls = calls
+        self._dual = True
+        self._has_plan = True
+        self.beat = object()
+
+    def __repr__(self):
+        return f"<Recorder {self.tag}>"
+
+
+def fake_grid(bpm=150.0, n_bars=8, phase=0.25):
+    bar = 4 * 60.0 / bpm
+    downbeats = phase + np.arange(n_bars + 1) * bar
+    return {"bpm": bpm, "beat": 60.0 / bpm, "bar": bar, "meter": 4,
+            "phase": phase, "downbeats": downbeats,
+            "beats": phase + np.arange(4 * n_bars) * (60.0 / bpm),
+            "n_db_peaks": 24, "n_beat_peaks": 96, "inlier_frac": 0.9,
+            "rms_ms": 7.5, "db_peaks": downbeats, "ok": True}
+
+
+def fake_generation():
+    return {"hits": [{"t": 1.0, "type": "don"}, {"t": 1.5, "type": "ka"}],
+            "spans": [{"t0": 2.0, "t1": 3.0, "type": "roll"}],
+            "hits_slots": [(0, 0, "don"), (0, 24, "ka"), (1, 0, "don_big")],
+            "spans_slots": [(2, 0, 2, 48, "roll")],
+            "n_measures": 4}
+
+
+TJA_STUB = "\n".join([
+    "TITLE:stub (SoftChart)", "BPM:150", "WAVE:song.ogg", "OFFSET:-0.250",
+    "COURSE:Oni", "LEVEL:9", "BALLOON:", "", "#START", "1020," * 1, "#END"]) + "\n"
+
+
+def harness(module, calls, *, course="oni", model_choice=None, sampling=False,
+            use_planner=True, auto_plan_on=True, use_beat=True, level=9,
+            bpm_override=0.0, audio_path=None, workdir=None):
+    """Drive ``module.generate_chart`` with every heavy dependency recorded.
+
+    Patches the module in place; the caller restores it. Returns the payload
+    list yielded by the endpoint.
+    """
+    saved = {}
+
+    def patch(name, value):
+        saved[name] = getattr(module, name)
+        setattr(module, name, value)
+
+    def record(name, result=None, positional=()):
+        def fn(*a, **k):
+            calls.append((name, _repr_args(a), _repr_kwargs(k)))
+            return result() if callable(result) else copy.deepcopy(result)
+        return fn
+
+    def passthrough(name, real):
+        """Record the call, then run the real thing.
+
+        Used for the TJA writers: their output is the artifact under
+        comparison, so a stub would only prove the stub is stable.
+        """
+        def fn(*a, **k):
+            calls.append((name, _repr_args(a), _repr_kwargs(k)))
+            return real(*a, **k)
+        return fn
+
+    mel = np.zeros((128, int(20 * FPS)), dtype=np.float32)
+    wav = np.zeros(int(20 * 22050), dtype=np.float32)
+
+    patch("load_logmel", record("load_logmel", lambda: (mel, wav)))
+    patch("fit_grid_piecewise", record("fit_grid_piecewise", fake_grid))
+    patch("fit_grid_fixed_bpm", record("fit_grid_fixed_bpm", fake_grid))
+    patch("learned_plan", record("learned_plan", [[0.0, 4.0, 5, 0]]))
+    patch("auto_plan", record("auto_plan", [[0.0, 4.0, 3, 0]]))
+    patch("generate_song_slot", record("generate_song_slot", fake_generation))
+    patch("generate_song", record("generate_song", fake_generation))
+    patch("snap_chart", record("snap_chart", fake_generation))
+    patch("write_tja_slots", passthrough("write_tja_slots", module.write_tja_slots))
+    patch("render_tja_image", record("render_tja_image", lambda: _touch(workdir, "chart.png")))
+    patch("render_audio_plan", record("render_audio_plan", lambda: _touch(workdir, "song-plan.png")))
+    patch("synthesize_taiko_preview",
+          record("synthesize_taiko_preview",
+                 lambda: _preview(workdir, calls)))
+
+    def get_models(choice=module.DEFAULT_MODEL, *, include_planner=False):
+        calls.append(("get_models", [choice], {"include_planner": include_planner}))
+        models = {k: Recorder(f"{choice}:{k}", calls) for k in ("gen", "slot", "beat")}
+        if include_planner:
+            models["plan"] = Recorder(f"{choice}:plan", calls)
+        return models
+
+    patch("get_models", get_models)
+
+    try:
+        payloads = list(module.generate_chart(
+            {"path": audio_path, "meta": {"_type": "gradio.FileData"}},
+            "song.ogg", course, level, bpm_override, auto_plan_on, use_beat,
+            use_planner, sampling, 0.8, 0.9, 0.8,
+            model_choice or module.DEFAULT_MODEL,
+        ))
+    finally:
+        for name, value in saved.items():
+            setattr(module, name, value)
+    return payloads
+
+
+def _repr_args(args):
+    out = []
+    for a in args:
+        if isinstance(a, np.ndarray):
+            out.append(f"ndarray{a.shape}")
+        elif isinstance(a, dict) and "downbeats" in a:
+            out.append(f"grid(bpm={a['bpm']})")
+        else:
+            out.append(repr(a))
+    return out
+
+
+def _repr_kwargs(kwargs):
+    return {k: _repr_args([v])[0] for k, v in sorted(kwargs.items())}
+
+
+def _touch(workdir, name):
+    path = os.path.join(workdir, name)
+    with open(path, "wb") as f:
+        f.write(b"\x89PNG\r\n\x1a\n")
+    return path
+
+
+def _preview(workdir, calls):
+    for entry in os.listdir(workdir):
+        if entry.endswith(".tja"):
+            stem = entry[:-4]
+            break
+    else:
+        stem = "softchart"
+    with open(os.path.join(workdir, f"{stem}_taiko-preview.wav"), "wb") as f:
+        f.write(b"RIFF")
+    return {"rendered_hit_count": 3}
+
+
+# ---------------------------------------------------------------------------
+# 1. routing
+
+
+def test_routing():
+    calls = []
+    loaded = []
+
+    def fake_load_hf(repo, device="cpu"):
+        loaded.append(repo)
+        return Recorder(repo, calls)
+
+    saved_load_hf, saved_cache = APP.load_hf, dict(APP._MODELS)
+    APP.load_hf = fake_load_hf
+    APP._MODELS.clear()
+    try:
+        for name, repo in APP.MODELS.items():
+            APP.get_models(name)
+            if loaded[-1] != repo:
+                return check("routing: published models -> Hub repo", False,
+                             f"{name} loaded {loaded[-1]}")
+        ok = check("routing: published models -> Hub repo",
+                   loaded == list(APP.MODELS.values()),
+                   f"{len(loaded)} repos in order")
+    finally:
+        APP.load_hf = saved_load_hf
+        APP._MODELS.clear()
+        APP._MODELS.update(saved_cache)
+
+    ok &= check("routing: BarScript label is not a 1.x Hub repo",
+                APP.SC2_MODEL not in APP.MODELS,
+                f"MODELS has {len(APP.MODELS)} entries")
+
+    names = APP.available_models()
+    ok &= check("routing: menu = 7 published + BarScript when the package resolves",
+                names[:7] == list(APP.MODELS)
+                and (names[7:] == [APP.SC2_MODEL]) == sc2_loader.is_available(),
+                f"{names}")
+
+    # the BarScript choice must reach sc2_loader.load and never load_hf
+    calls = []
+    saved_load, saved_load_hf = sc2_loader.load, APP.load_hf
+    saved_sc2 = dict(APP._SC2)
+    APP._SC2.clear()
+
+    class FakeSC2:
+        package_dir = "/tmp/nonexistent-package"
+
+    def fake_sc2_load(package_dir=None, device="cpu"):
+        calls.append(("sc2_loader.load", device))
+        return FakeSC2()
+
+    def boom(*a, **k):
+        calls.append(("load_hf", a))
+        raise AssertionError("BarScript must not reach the 1.x Hub loader")
+
+    saved_check = sc2_loader.check_preprocessing
+    sc2_loader.check_preprocessing = lambda *a, **k: calls.append(("check_preprocessing", k))
+    APP.sc2_loader.load = fake_sc2_load
+    APP.load_hf = boom
+    try:
+        APP.get_sc2()
+        ok &= check("routing: BarScript -> sc2_loader, front end checked",
+                    [c[0] for c in calls] == ["sc2_loader.load", "check_preprocessing"],
+                    f"{[c[0] for c in calls]}")
+        front_end = dict(calls[1][1])
+        ok &= check("routing: BarScript front-end check uses the app's own mel settings",
+                    front_end == {"sr": APP.SR, "n_fft": APP.N_FFT,
+                                  "hop_length": APP.HOP, "n_mels": APP.N_MELS,
+                                  "fmin": 20.0, "fmax": APP.SR / 2},
+                    f"{front_end}")
+    finally:
+        sc2_loader.load = saved_load
+        APP.sc2_loader.load = saved_load
+        sc2_loader.check_preprocessing = saved_check
+        APP.load_hf = saved_load_hf
+        APP._SC2.clear()
+        APP._SC2.update(saved_sc2)
+    return ok
+
+
+# ---------------------------------------------------------------------------
+# 2. difficulties
+
+
+def test_courses():
+    ok = True
+    for course in ALL_COURSES:
+        try:
+            APP._validate_controls(course, 9, 0, 0.8, 0.9, 0.8)
+            passed = True
+        except Exception as exc:  # noqa: BLE001
+            passed = False
+            detail = str(exc)
+        ok &= check(f"difficulty: {course} passes validation", passed,
+                    "" if passed else detail)
+    try:
+        APP._validate_controls("ex", 9, 0, 0.8, 0.9, 0.8)
+        rejected = False
+    except ValueError:
+        rejected = True
+    ok &= check("difficulty: an unknown course is still rejected", rejected)
+
+    ok &= check("difficulty: density map covers all five courses",
+                sorted(APP.COURSE_DENS) == sorted(ALL_COURSES),
+                f"{APP.COURSE_DENS}")
+    ok &= check("difficulty: the four shipped buckets are unchanged",
+                {c: APP.COURSE_DENS[c] for c in LEGACY_COURSES}
+                == {"easy": 1, "normal": 2, "hard": 4, "oni": 7})
+    ok &= check("difficulty: ura is denser than oni", APP.COURSE_DENS["ura"] > APP.COURSE_DENS["oni"],
+                f"ura={APP.COURSE_DENS['ura']} oni={APP.COURSE_DENS['oni']}")
+
+    # the rest of the repo already fixed this map; the app was the outlier
+    canonical = {}
+    with open(os.path.join(os.path.dirname(SPACES), "scripts", "infer.py"),
+              encoding="utf-8") as f:
+        for line in f:
+            if line.startswith("COURSE_DENS = "):
+                canonical = eval(line.split("=", 1)[1].strip())  # noqa: S307
+                break
+    ok &= check("difficulty: density map agrees with scripts/infer.py",
+                canonical == APP.COURSE_DENS, f"{canonical}")
+    return ok
+
+
+def test_course_header():
+    ok = check("TJA header: app map matches the loader's measured map",
+               APP.TJA_COURSE_HEADER == sc2_loader.TJA_COURSE_HEADER,
+               f"{APP.TJA_COURSE_HEADER}")
+    ok &= check("TJA header: ura is Edit, not Ura",
+                APP.TJA_COURSE_HEADER["ura"] == "Edit")
+
+    grid = fake_grid()
+    gen = fake_generation()
+    for course in ALL_COURSES:
+        text = APP.write_tja(gen, 150.0, "song", course, 9, "song.ogg",
+                             downbeats=grid["downbeats"], grid_fit=grid)
+        want = f"COURSE:{APP.TJA_COURSE_HEADER[course]}"
+        ok &= check(f"TJA header: time writer emits {want} for {course}",
+                    want in text.splitlines())
+
+    from softchart.tja import write_tja_slots as vendored
+    for course in ALL_COURSES:
+        raw = vendored(gen, grid, "song", course, 9, "song.ogg")
+        fixed = APP.normalize_course_header(raw, course)
+        want = f"COURSE:{APP.TJA_COURSE_HEADER[course]}"
+        ok &= check(f"TJA header: slot writer normalized to {want} for {course}",
+                    want in fixed.splitlines())
+        if course in LEGACY_COURSES:
+            # normalization must be a no-op on everything that shipped before
+            ok &= check(f"TJA header: normalization is a no-op for {course}",
+                        fixed == raw)
+        else:
+            ok &= check("TJA header: the vendored writer really did emit COURSE:Ura",
+                        "COURSE:Ura" in raw.splitlines(),
+                        "so the rewrite is load-bearing")
+    return ok
+
+
+# ---------------------------------------------------------------------------
+# 3. legacy regression: before/after call trace
+
+
+def scenario_name(model, scenario):
+    return (f"{model} / {scenario['course']}"
+            f"{' / bpm' if scenario.get('bpm_override') else ''}"
+            f"{' / sampling' if scenario.get('sampling') else ''}"
+            f"{' / no-beat' if scenario.get('use_beat') is False else ''}"
+            f"{' / heuristic-plan' if scenario.get('auto_plan_on') and not scenario.get('use_planner') else ''}")
+
+
+def scrub(value, replacements):
+    """Replace this run's temp directories with stable tokens.
+
+    The trace records the arguments verbatim -- including the generated TJA
+    text handed to the chart renderer, which is what makes this a byte-level
+    comparison of the writers -- so only the paths need normalizing.
+    """
+    if isinstance(value, str):
+        for real, token in replacements:
+            value = value.replace(real, token)
+        return value
+    if isinstance(value, list):
+        return [scrub(v, replacements) for v in value]
+    if isinstance(value, tuple):
+        return tuple(scrub(v, replacements) for v in value)
+    if isinstance(value, dict):
+        return {k: scrub(v, replacements) for k, v in value.items()}
+    return value
+
+
+def run_one(module, model, scenario):
+    """``(call trace, metrics)`` for one published-model run."""
+    calls = []
+    with tempfile.TemporaryDirectory() as audio_dir:
+        audio_path = os.path.join(audio_dir, "song.ogg")
+        with open(audio_path, "wb") as f:
+            f.write(b"OggS")
+        workdir = tempfile.mkdtemp(prefix="softchart-work-")
+        saved_mkdtemp = tempfile.mkdtemp
+        tempfile.mkdtemp = lambda *a, **k: workdir
+        try:
+            payload = harness(module, calls, model_choice=model,
+                              audio_path=audio_path, workdir=workdir,
+                              **scenario)
+        finally:
+            tempfile.mkdtemp = saved_mkdtemp
+        replacements = [(os.path.realpath(workdir), "<WORKDIR>"),
+                        (workdir, "<WORKDIR>"),
+                        (os.path.realpath(audio_dir), "<AUDIODIR>"),
+                        (audio_dir, "<AUDIODIR>")]
+    return scrub(calls, replacements), payload[-1].get("metrics", {})
+
+
+def legacy_trace(module):
+    trace = {}
+    for model in module.MODELS:
+        for scenario in LEGACY_SCENARIOS:
+            calls, metrics = run_one(module, model, scenario)
+            trace[scenario_name(model, scenario)] = {
+                "calls": calls, "metrics": metrics}
+    return trace
+
+
+def test_legacy_regression():
+    if not os.path.isfile(GOLDEN):
+        return check("regression: published models unchanged", False,
+                     f"no golden trace at {GOLDEN}")
+    with open(GOLDEN, encoding="utf-8") as f:
+        golden = json.load(f)
+    recorded = golden["scenarios"]
+
+    ok = check("regression: golden trace covers every published model "
+               "and control combination",
+               len(recorded) == len(APP.MODELS) * len(LEGACY_SCENARIOS),
+               f"{len(recorded)} scenarios")
+
+    for model in APP.MODELS:
+        for scenario in LEGACY_SCENARIOS:
+            name = scenario_name(model, scenario)
+            want = recorded.get(name)
+            if want is None:
+                ok &= check(f"regression: {name}", False, "not in the golden trace")
+                continue
+            calls, metrics = run_one(APP, model, scenario)
+            # json round-trips tuples to lists; compare in the golden's shape
+            calls = json.loads(json.dumps(calls))
+            if calls != want["calls"]:
+                diff = next((f"{b!r} != {a!r}" for a, b in
+                             zip(want["calls"], calls) if a != b),
+                            f"{len(want['calls'])} vs {len(calls)} calls")
+                ok &= check(f"regression: {name}", False, diff)
+                continue
+            # metrics gained keys; every key the old app reported must match
+            drift = {k: (v, metrics.get(k)) for k, v in want["metrics"].items()
+                     if metrics.get(k) != v}
+            ok &= check(f"regression: {name}", not drift,
+                        "" if not drift else f"{drift}")
+    return ok
+
+
+def test_planner_fallback():
+    ok = True
+    for course in LEGACY_COURSES:
+        ok &= check(f"planner: {course} uses its own trained id",
+                    APP.planner_course(course) == (course, False))
+    ok &= check("planner: ura falls back to the oni plan",
+                APP.planner_course("ura") == ("oni", True))
+
+    # the cid that actually reaches the model
+    seen = []
+
+    class FakePlanner:
+        def __call__(self, x, cid):
+            import torch
+            seen.append(int(cid[0]))
+            n = x.shape[1]
+            return torch.zeros(1, n, 8), torch.zeros(1, n, 3)
+
+    mel = np.zeros((128, int(30 * FPS)), dtype=np.float32)
+    for course in ALL_COURSES:
+        APP.learned_plan(FakePlanner(), mel, course, 150.0)
+    ok &= check("planner: course ids 0-3 for the trained courses, 3 for ura",
+                seen == [0, 1, 2, 3, 3], f"{seen}")
+
+    # and the substitution is disclosed, not silent
+    calls = []
+    with tempfile.TemporaryDirectory() as audio_dir:
+        audio_path = os.path.join(audio_dir, "song.ogg")
+        with open(audio_path, "wb") as f:
+            f.write(b"OggS")
+        workdir = tempfile.mkdtemp(prefix="softchart-work-")
+        saved = tempfile.mkdtemp
+        tempfile.mkdtemp = lambda *a, **k: workdir
+        try:
+            payload = harness(APP, calls, course="ura", use_planner=True,
+                              audio_path=audio_path, workdir=workdir)
+        finally:
+            tempfile.mkdtemp = saved
+    plan_source = payload[-1].get("metrics", {}).get("plan_source", "")
+    ok &= check("planner: the ura substitution is reported in the metrics",
+                "Oni plan reused for Ura" in plan_source, plan_source)
+
+    notices = APP.capabilities()["course_notices"]["ura"]
+    ok &= check("planner: the substitution is disclosed before generating",
+                any(n["when"] == "planner" and "Oni plan" in n["text"]
+                    for n in notices))
+    return ok
+
+
+# ---------------------------------------------------------------------------
+# 4. the BarScript route
+
+
+def test_sc2_conditions():
+    """What the app hands the BarScript loader, with the loader itself recorded."""
+    seen = {}
+
+    class FakeSC2:
+        package_dir = "/tmp/nonexistent-package"
+
+        def generate(self, mel, course, level=None, **kwargs):
+            seen["mel"] = mel.shape
+            seen["course"] = course
+            seen["level"] = level
+            seen.update(kwargs)
+            return {
+                "tja": TJA_STUB, "gen": fake_generation(),
+                "grid": {"source": "bpm", "denom": kwargs.get("grid_denom", 16),
+                         "bpm": kwargs["bpm"], "offset_sec": kwargs["offset_sec"],
+                         "triplets_representable": kwargs.get("grid_denom", 16) % 3 == 0,
+                         "profile": "default"},
+                "serving": {}, "serving_overrides": {
+                    k: kwargs[k] for k in ("greedy", "temperature", "top_p")
+                    if k in kwargs},
+                "motif": {"verified": True}, "counters": {},
+                "density_bucket": kwargs.get("density_bucket"),
+                "n_notes": 2, "n_spans": 1, "experimental": True,
+            }
+
+    saved_get_sc2, saved_beat = APP.get_sc2, APP.sc2_beat_model
+    saved_available = APP.available_models
+    APP.get_sc2 = lambda: FakeSC2()
+    APP.sc2_beat_model = lambda: Recorder("beat", [])
+    APP.available_models = lambda: list(APP.MODELS) + [APP.SC2_MODEL]
+    ok = True
+    try:
+        for course, sampling in (("oni", False), ("ura", True)):
+            seen.clear()
+            calls = []
+            with tempfile.TemporaryDirectory() as audio_dir:
+                audio_path = os.path.join(audio_dir, "song.ogg")
+                with open(audio_path, "wb") as f:
+                    f.write(b"OggS")
+                workdir = tempfile.mkdtemp(prefix="softchart-work-")
+                saved_mkdtemp = tempfile.mkdtemp
+                tempfile.mkdtemp = lambda *a, **k: workdir
+                try:
+                    payload = harness(APP, calls, course=course,
+                                      model_choice=APP.SC2_MODEL,
+                                      sampling=sampling, audio_path=audio_path,
+                                      workdir=workdir)
+                finally:
+                    tempfile.mkdtemp = saved_mkdtemp
+
+            grid = fake_grid()
+            ok &= check(f"BarScript ({course}): bpm and first-downbeat offset from the fitted grid",
+                        seen.get("bpm") == grid["bpm"]
+                        and abs(seen.get("offset_sec", -1) - float(grid["downbeats"][0])) < 1e-9,
+                        f"bpm={seen.get('bpm')} offset={seen.get('offset_sec')}")
+            ok &= check(f"BarScript ({course}): course and density bucket",
+                        seen.get("course") == course
+                        and seen.get("density_bucket") == APP.COURSE_DENS[course],
+                        f"{seen.get('course')} bucket={seen.get('density_bucket')}")
+            ok &= check(f"BarScript ({course}): ships on the /16 deployment lattice",
+                        seen.get("grid_denom") == sc2_loader.SHIP_GRID_DENOM)
+            if sampling:
+                ok &= check(f"BarScript ({course}): creative sampling is passed as an override",
+                            seen.get("greedy") is False and seen.get("temperature") == 0.8
+                            and seen.get("top_p") == 0.9, f"{seen.get('greedy')}")
+            else:
+                ok &= check(f"BarScript ({course}): package serving contract left alone",
+                            not {"greedy", "temperature", "top_p"} & set(seen),
+                            f"{sorted(set(seen) & {'greedy', 'temperature', 'top_p'})}")
+
+            metrics = payload[-1].get("metrics", {})
+            ok &= check(f"BarScript ({course}): the /16 lattice and its triplet gap are shown",
+                        "triplets impossible" in metrics.get("chart_grid", ""),
+                        metrics.get("chart_grid", ""))
+            ok &= check(f"BarScript ({course}): the plan switches are reported as unused",
+                        metrics.get("plan_source") == "Not used — BarScript takes no plan",
+                        metrics.get("plan_source", ""))
+            names = [c[0] for c in calls]
+            ok &= check(f"BarScript ({course}): the 1.x generators are never called",
+                        not {"generate_song", "generate_song_slot",
+                             "write_tja_slots", "snap_chart"} & set(names),
+                        f"{names}")
+
+        # no downbeat -> refuse, rather than anchor the bars on an arbitrary phase
+        calls = []
+        with tempfile.TemporaryDirectory() as audio_dir:
+            audio_path = os.path.join(audio_dir, "song.ogg")
+            with open(audio_path, "wb") as f:
+                f.write(b"OggS")
+            workdir = tempfile.mkdtemp(prefix="softchart-work-")
+            saved_mkdtemp = tempfile.mkdtemp
+            tempfile.mkdtemp = lambda *a, **k: workdir
+            try:
+                payload = harness(APP, calls, course="oni", use_beat=False,
+                                  model_choice=APP.SC2_MODEL, bpm_override=150.0,
+                                  audio_path=audio_path, workdir=workdir)
+            finally:
+                tempfile.mkdtemp = saved_mkdtemp
+        last = payload[-1]
+        ok &= check("BarScript: fails closed without a downbeat anchor",
+                    last.get("kind") == "error" and "downbeat" in last.get("detail", ""),
+                    last.get("detail", ""))
+    finally:
+        APP.get_sc2, APP.sc2_beat_model = saved_get_sc2, saved_beat
+        APP.available_models = saved_available
+    return ok
+
+
+def test_sc2_real_decode():
+    """One real 2.0 decode, if the release package is on disk."""
+    if not sc2_loader.is_available():
+        print("SKIP  2.0 real decode: no release package "
+              f"({sc2_loader.default_package_dir()})")
+        return True
+    sc = APP.get_sc2()
+    rng = np.random.default_rng(0)
+    mel = (rng.standard_normal((128, int(12 * FPS))).astype(np.float32) * 0.6 - 4.0)
+    ok = True
+    for course in ("oni", "ura"):
+        result = sc.generate(mel, course, level=9, bpm=150.0, offset_sec=0.25,
+                             density_bucket=APP.COURSE_DENS[course], seed=0,
+                             grid_denom=sc2_loader.SHIP_GRID_DENOM,
+                             title="probe", wave="song.ogg")
+        header = [ln for ln in result["tja"].splitlines() if ln.startswith("COURSE:")]
+        ok &= check(f"BarScript real decode ({course}): TJA header",
+                    header == [f"COURSE:{APP.TJA_COURSE_HEADER[course]}"], f"{header}")
+        ok &= check(f"BarScript real decode ({course}): motif constraint served",
+                    result["motif"]["verified"])
+        ok &= check(f"BarScript real decode ({course}): lattice reported as /16 without triplets",
+                    result["grid"]["denom"] == 16
+                    and result["grid"]["triplets_representable"] is False)
+    return ok
+
+
+class _EnvPatch:
+    """Set/clear environment variables for the duration of a block."""
+
+    def __init__(self, **values):
+        self.values = values
+        self.saved = {}
+
+    def __enter__(self):
+        for key, value in self.values.items():
+            self.saved[key] = os.environ.get(key)
+            if value is None:
+                os.environ.pop(key, None)
+            else:
+                os.environ[key] = value
+        return self
+
+    def __exit__(self, *exc):
+        for key, value in self.saved.items():
+            if value is None:
+                os.environ.pop(key, None)
+            else:
+                os.environ[key] = value
+        return False
+
+
+def _stub_package(root, name):
+    """A directory that looks like a package to the resolver (config.json)."""
+    path = os.path.join(root, name)
+    os.makedirs(path, exist_ok=True)
+    with open(os.path.join(path, "config.json"), "w", encoding="utf-8") as f:
+        json.dump({"schema_version": "stub"}, f)
+    return path
+
+
+def _real_package_copy(dest, *, weights_stub=True):
+    """The shipped package minus its weights, for gate tests.
+
+    ``verify_code`` and the schema check run before a single tensor is read, so
+    an empty ``model.safetensors`` is enough to prove they fire -- and keeps the
+    test off an 18 MB copy.  Returns ``None`` when no package is on disk.
+    """
+    src = sc2_loader.default_package_dir()
+    if not os.path.isfile(os.path.join(src, "config.json")):
+        return None
+    os.makedirs(dest, exist_ok=True)
+    for name in ("config.json", "spec.json", "vocab.json",
+                 "preprocessor_config.json"):
+        with open(os.path.join(src, name), "rb") as f:
+            payload = f.read()
+        with open(os.path.join(dest, name), "wb") as f:
+            f.write(payload)
+    if weights_stub:
+        open(os.path.join(dest, "model.safetensors"), "wb").close()
+    return dest
+
+
+def test_sc2_package_resolution():
+    """$SC2_PACKAGE_DIR -> local directory -> Hub, and the token error.
+
+    The Hub leg is mocked throughout: the point is the ORDER and what each leg
+    is handed, and a suite that downloaded 18 MB to assert a precedence rule
+    would be testing the network instead.  One real download is exercised
+    separately, outside this suite.
+    """
+    ok = True
+    workdir = tempfile.mkdtemp(prefix="sc2-resolve-")
+    env_pkg = _stub_package(workdir, "from_env")
+    local_pkg = _stub_package(workdir, "from_local")
+    hub_pkg = _stub_package(workdir, "from_hub")
+    empty = os.path.join(workdir, "empty")
+    os.makedirs(empty, exist_ok=True)
+
+    calls = []
+    real_local_candidates = sc2_loader.local_candidates
+    real_download = sc2_loader.download_from_hub
+
+    def fake_download(repo_id=None, revision=None, token=None):
+        calls.append({"repo_id": repo_id or sc2_loader.hub_repo_id(),
+                      "revision": revision, "token": token})
+        return hub_pkg
+
+    try:
+        sc2_loader.local_candidates = lambda: (local_pkg,)
+        sc2_loader.download_from_hub = fake_download
+
+        # 1. the explicit request wins over both a local package and the Hub
+        del calls[:]
+        with _EnvPatch(SC2_PACKAGE_DIR=env_pkg, SC2_NO_HUB=None):
+            got, source = sc2_loader.resolve_package_dir()
+        ok &= check("resolve: $SC2_PACKAGE_DIR outranks local and Hub",
+                    (got, source) == (env_pkg, "env") and not calls,
+                    f"{source} -> {os.path.basename(got)}, hub calls {len(calls)}")
+
+        # 2. an explicit request that cannot be satisfied is an ERROR, not a
+        #    quiet download of some other copy
+        del calls[:]
+        with _EnvPatch(SC2_PACKAGE_DIR=empty, SC2_NO_HUB=None):
+            try:
+                sc2_loader.resolve_package_dir()
+                failed, detail = False, "no error raised"
+            except sc2_loader.PackageError as exc:
+                failed = empty in str(exc) and not calls
+                detail = str(exc)[:110]
+        ok &= check("resolve: an unsatisfiable $SC2_PACKAGE_DIR raises and "
+                    "never falls through to the Hub", failed, detail)
+
+        # 3. a local package outranks the Hub
+        del calls[:]
+        with _EnvPatch(SC2_PACKAGE_DIR=None, SC2_NO_HUB=None):
+            got, source = sc2_loader.resolve_package_dir()
+        ok &= check("resolve: a local package outranks the Hub",
+                    (got, source) == (local_pkg, "local") and not calls,
+                    f"{source} -> {os.path.basename(got)}, hub calls {len(calls)}")
+
+        # 4. with neither, the Hub is used
+        del calls[:]
+        sc2_loader.local_candidates = lambda: (empty,)
+        with _EnvPatch(SC2_PACKAGE_DIR=None, SC2_NO_HUB=None):
+            got, source = sc2_loader.resolve_package_dir()
+        ok &= check("resolve: with no local package the Hub is used",
+                    (got, source) == (hub_pkg, "hub") and len(calls) == 1,
+                    f"{source} -> {os.path.basename(got)}, hub calls {len(calls)}")
+
+        # 5. SC2_NO_HUB pins the Space to disk
+        del calls[:]
+        with _EnvPatch(SC2_PACKAGE_DIR=None, SC2_NO_HUB="1"):
+            try:
+                sc2_loader.resolve_package_dir()
+                pinned, detail = False, "no error raised"
+            except sc2_loader.PackageError as exc:
+                pinned = not calls and "SC2_NO_HUB" in str(exc)
+                detail = str(exc)[:110]
+        ok &= check("resolve: $SC2_NO_HUB disables the Hub leg", pinned, detail)
+    finally:
+        sc2_loader.local_candidates = real_local_candidates
+        sc2_loader.download_from_hub = real_download
+
+    # 6. the private repo with no token names the credential, not a missing
+    #    model, and never reaches the network
+    downloads = []
+    import huggingface_hub
+
+    real_snapshot = huggingface_hub.snapshot_download
+    real_get_token = huggingface_hub.get_token
+
+    # download_from_hub checks the snapshot is a COMPLETE package before it
+    # hands the directory back, so the mocked snapshot must look like one.
+    hub_complete = _stub_package(workdir, "hub_complete")
+    for _name in sc2_loader.REQUIRED_FILES:
+        open(os.path.join(hub_complete, _name), "a").close()
+
+    def fake_snapshot(**kwargs):
+        downloads.append(kwargs)
+        return hub_complete
+
+    try:
+        huggingface_hub.snapshot_download = fake_snapshot
+        huggingface_hub.get_token = lambda: None
+        del downloads[:]
+        with _EnvPatch(HF_TOKEN=None, HUGGING_FACE_HUB_TOKEN=None):
+            try:
+                sc2_loader.download_from_hub()
+                named, detail = False, "no error raised"
+            except sc2_loader.PackageError as exc:
+                text = str(exc)
+                named = ("HF_TOKEN" in text and "PRIVATE" in text
+                         and "SC2_PACKAGE_DIR" in text and not downloads)
+                detail = text[:130]
+        ok &= check("hub: a missing token is reported as a credentials "
+                    "problem, before any request", named, detail)
+
+        # 7. the token that IS found is the one passed to the Hub, and only the
+        #    package's own files are requested
+        del downloads[:]
+        with _EnvPatch(HF_TOKEN="hf_test_token", HUGGING_FACE_HUB_TOKEN=None,
+                       SC2_REPO_REVISION=None):
+            sc2_loader.download_from_hub()
+        sent = downloads[0] if downloads else {}
+        ok &= check("hub: request carries the resolved token, the repo id and "
+                    "an allow-list of exactly the package files",
+                    len(downloads) == 1
+                    and sent.get("token") == "hf_test_token"
+                    and sent.get("repo_id") == sc2_loader.DEFAULT_HUB_REPO
+                    and sent.get("repo_type") == "model"
+                    and sorted(sent.get("allow_patterns") or []) ==
+                    sorted(sc2_loader.PACKAGE_FILES),
+                    f"repo={sent.get('repo_id')} "
+                    f"patterns={len(sent.get('allow_patterns') or [])}")
+
+        # 8. an auth failure is translated, not passed through raw. The Hub's
+        #    own exception classes take a response object, so the two shapes the
+        #    loader keys on -- the class NAME and an HTTP status in the text --
+        #    are raised directly rather than half-built.
+        class _RepoNotFound(Exception):
+            pass
+
+        _RepoNotFound.__name__ = "RepositoryNotFoundError"
+
+        for label, exc_obj in (("by exception class", _RepoNotFound("not found")),
+                               ("by HTTP status", Exception("401 Client Error"))):
+            def refusing_snapshot(_exc=exc_obj, **kwargs):
+                raise _exc
+
+            huggingface_hub.snapshot_download = refusing_snapshot
+            with _EnvPatch(HF_TOKEN="hf_wrong_token"):
+                try:
+                    sc2_loader.download_from_hub()
+                    translated, detail = False, "no error raised"
+                except sc2_loader.PackageError as exc:
+                    translated = "token" in str(exc).lower()
+                    detail = str(exc)[:130]
+            ok &= check(f"hub: a private-repo refusal ({label}) is explained "
+                        "as a token problem", translated, detail)
+    finally:
+        huggingface_hub.snapshot_download = real_snapshot
+        huggingface_hub.get_token = real_get_token
+
+    # 9. the integrity gates run on a downloaded package exactly as on a local
+    #    one: eight vendored modules by md5, then the package schema
+    staged = _real_package_copy(os.path.join(workdir, "downloaded"))
+    if staged is None:
+        ok &= check("hub: integrity gates run on the downloaded package", True,
+                    "skipped -- no package on disk to stage from")
+    else:
+        with open(os.path.join(staged, "config.json"), encoding="utf-8") as f:
+            good = json.load(f)
+        ok &= check("hub: the staged package records all eight vendored "
+                    "module md5s",
+                    len(sc2_loader.verify_code(good)) == 8,
+                    f"{len(sc2_loader.VENDORED_MODULES)} modules")
+
+        def _stage(mutate, name):
+            path = os.path.join(workdir, name)
+            _real_package_copy(path)
+            cfg = copy.deepcopy(good)
+            mutate(cfg)
+            with open(os.path.join(path, "config.json"), "w",
+                      encoding="utf-8") as f:
+                json.dump(cfg, f)
+            return path
+
+        def _drift(cfg):
+            cfg["code"]["module_md5"]["src/softchart/generate.py"] = "0" * 32
+
+        cases = [
+            ("a vendored module whose md5 drifted", _drift, "generate.py"),
+            ("an unknown package schema",
+             lambda c: c.update({"schema_version": "something_else"}),
+             "schema"),
+            ("a vocabulary that is not this checkpoint's",
+             lambda c: c["vocab"].update({"mapping_sha256": "0" * 64}),
+             "vocab"),
+        ]
+        for label, mutate, needle in cases:
+            path = _stage(mutate, "bad_" + needle.replace(".", "_"))
+            real_download = sc2_loader.download_from_hub
+            try:
+                sc2_loader.download_from_hub = lambda **kw: path
+                with _EnvPatch(SC2_PACKAGE_DIR=None, SC2_NO_HUB=None):
+                    sc2_loader.local_candidates = lambda: (empty,)
+                    try:
+                        sc2_loader.load()
+                        refused, detail = False, "loaded anyway"
+                    except sc2_loader.PackageError as exc:
+                        refused = needle in str(exc)
+                        detail = str(exc)[:110]
+            finally:
+                sc2_loader.download_from_hub = real_download
+                sc2_loader.local_candidates = real_local_candidates
+            ok &= check(f"hub: refuses a downloaded package with {label}",
+                        refused, detail)
+
+        # 10. a snapshot whose bytes disagree with its own manifest is refused
+        tampered = os.path.join(workdir, "tampered")
+        _real_package_copy(tampered)
+        with open(os.path.join(tampered, "release_manifest.json"), "w",
+                  encoding="utf-8") as f:
+            json.dump({"files": {"spec.json": {"sha256": "0" * 64,
+                                               "size_bytes": 2618}}}, f)
+        try:
+            sc2_loader.verify_manifest(tampered, source="test")
+            caught, detail = False, "accepted a mismatched manifest"
+        except sc2_loader.PackageError as exc:
+            caught, detail = "spec.json" in str(exc), str(exc)[:110]
+        ok &= check("hub: a snapshot that disagrees with its own "
+                    "release_manifest.json is refused", caught, detail)
+
+    # 11. the menu follows the same order
+    with _EnvPatch(SC2_PACKAGE_DIR=empty, SC2_NO_HUB=None):
+        unsatisfiable = sc2_loader.is_available()
+    with _EnvPatch(SC2_PACKAGE_DIR=None, SC2_NO_HUB="1"):
+        sc2_loader.local_candidates = lambda: (empty,)
+        try:
+            pinned_off = sc2_loader.is_available()
+        finally:
+            sc2_loader.local_candidates = real_local_candidates
+    ok &= check("menu: is_available reflects the same resolution order",
+                unsatisfiable is False and pinned_off is False,
+                f"unsatisfiable $SC2_PACKAGE_DIR -> {unsatisfiable}, "
+                f"no package and $SC2_NO_HUB -> {pinned_off}")
+    return ok
+
+
+def test_capabilities_payload():
+    caps = APP.capabilities()
+    values = [m["value"] for m in caps["models"]]
+    ok = check("UI: capabilities lists exactly the selectable models",
+               values == APP.available_models(), f"{values}")
+    ok &= check("UI: published models carry no limitation list",
+                all(not m["limitations"] for m in caps["models"]
+                    if not m["experimental"]))
+    experimental = [m for m in caps["models"] if m["experimental"]]
+    if sc2_loader.is_available():
+        card = [t for _, _, t in sc2_loader.KNOWN_LIMITATIONS]
+        shown = [entry["text"] for entry in experimental[0]["limitations"]]
+        ok &= check("UI: all eight model-card limitations shown verbatim",
+                    shown == card, f"{len(shown)}/{len(card)}")
+        ok &= check("UI: the BarScript entry is labelled experimental",
+                    "EXPERIMENTAL" in experimental[0]["group"],
+                    experimental[0]["group"])
+    ura = [n["text"] for n in caps["course_notices"]["ura"]]
+    ok &= check("UI: the ura training-data notice is shown for every model",
+                any("208" in t for t in ura), f"{len(ura)} notices")
+    return ok
+
+
+def record_golden(app_path):
+    """Capture the published-model trace from ``app_path`` into the fixture."""
+    app_path = os.path.abspath(app_path)
+    spec = importlib.util.spec_from_file_location("app_baseline", app_path)
+    module = importlib.util.module_from_spec(spec)
+    sys.modules["app_baseline"] = module
+    spec.loader.exec_module(module)
+    import hashlib
+
+    with open(app_path, "rb") as f:
+        digest = hashlib.sha256(f.read()).hexdigest()
+    payload = {
+        "_comment": "Call trace of the seven published models, recorded from "
+                    "spaces/app.py as it stood BEFORE the 2.0 / ura wiring "
+                    "change (source_sha256 identifies that exact text). "
+                    "Regenerating this file from a newer app defeats its "
+                    "purpose: the point is that the published models still do "
+                    "what they did here, call for call and byte for byte -- "
+                    "the arguments include the full generated TJA.",
+        "source_app": os.path.basename(app_path),
+        "source_sha256": digest,
+        "models": list(module.MODELS),
+        "scenarios": legacy_trace(module),
+    }
+    os.makedirs(os.path.dirname(GOLDEN), exist_ok=True)
+    with open(GOLDEN, "w", encoding="utf-8") as f:
+        json.dump(payload, f, indent=1, ensure_ascii=False, sort_keys=True)
+        f.write("\n")
+    print(f"recorded {len(payload['scenarios'])} scenarios -> {GOLDEN}")
+    return 0
+
+
+def main():
+    tests = [test_routing, test_courses, test_course_header,
+             test_legacy_regression, test_planner_fallback,
+             test_sc2_conditions, test_sc2_package_resolution,
+             test_capabilities_payload, test_sc2_real_decode]
+    for test in tests:
+        print(f"\n--- {test.__name__}")
+        try:
+            test()
+        except Exception as exc:  # noqa: BLE001
+            import traceback
+            traceback.print_exc()
+            check(test.__name__, False, f"{type(exc).__name__}: {exc}")
+    failed = [name for name, ok, _ in RESULTS if not ok]
+    print(f"\n{len(RESULTS) - len(failed)}/{len(RESULTS)} checks passed")
+    for name in failed:
+        print(f"  FAILED: {name}")
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--record", metavar="OLD_APP.py",
+                        help="re-record the golden legacy trace from this app")
+    args = parser.parse_args()
+    sys.exit(record_golden(args.record) if args.record else main())

scripts/test_sc2_vendor.py

@@ -0,0 +1,504 @@
+"""CPU acceptance for the vendored BarScript serving path.
+
+Three questions, and nothing that needs a GPU or the network:
+
+(a) does the BarScript model load from the release package ALONE -- with the
+    development worktree stripped out of ``sys.path`` -- and does it read
+    nothing outside that package?
+(b) does it generate a chart whose TJA the vendored parser reads back?
+(c) is the 1.x serving path the Space's seven published models use bit-for-bit
+    unaffected: the library files untouched, the public signatures unchanged,
+    the two namespaces isolated, and a real 1.x decode identical before and
+    after ``softchart.sc2`` is imported?
+
+Run:  python spaces/scripts/test_sc2_vendor.py
+      pytest spaces/scripts/test_sc2_vendor.py
+
+(c)'s decode check needs ``JacobLinCool/softchart-v17`` in the local Hub
+cache; it reports SKIP, and does not fail, when the cache is cold, because it
+never goes to the network.
+
+The order-sensitive checks run in subprocesses of this file (``--case``): a
+1.x baseline must be measured in a process where ``softchart.sc2`` was never
+imported, and the package-only load must happen before anything else can put
+the worktree back on the path.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import inspect
+import json
+import os
+import subprocess
+import sys
+import warnings
+
+SPACES_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+REPO_ROOT = os.path.dirname(SPACES_DIR)
+SC2_DIR = os.path.join(SPACES_DIR, "softchart", "sc2")
+LEGACY_REPO = os.environ.get("SC_LEGACY_TEST_REPO", "JacobLinCool/softchart-v17")
+
+if SPACES_DIR not in sys.path:
+    sys.path.insert(0, SPACES_DIR)
+
+
+# ---------------------------------------------------------------------------
+# (c) tripwires: the 1.x library as it stood when 2.0 was vendored in.
+# Vendoring added files; it changed none. A mismatch here means someone edited
+# a module the seven published checkpoints decode through, and their charts are
+# no longer the baseline the user keeps them around to compare against.
+
+LEGACY_MD5 = {
+    "__init__.py": "d41d8cd98f00b204e9800998ecf8427e",
+    "baseline.py": "875b63450c59bb62f5d50965ec7b218c",
+    "data.py": "72e19832477dd49e2bb0dc731e17b3ac",
+    "evaluate.py": "13840e66287b1113202d4ea0152f62ba",
+    "fonts.py": "a6b1c5c1d3f9ab852fb551d49031d3a2",
+    "generate.py": "b3ad2aa451ed413f1d9d64eafb072cd8",
+    "grid.py": "570ae31b7120369781cd02c20de74ffd",
+    "hf.py": "522c2257a402bd0d41727c6dee6732db",
+    "model.py": "5e8ee5091629c9ff718d7c136d5f97bc",
+    "preprocess.py": "1d765a119d909f2aa74d594f4ca6b9c0",
+    "preview_audio.py": "b03aec51b160ccb33d8d553dd936d2c4",
+    "proxies.py": "918c72c22d48e4ccb762e0fe814d0bfe",
+    "rhythm.py": "11cca7c58809b339f7384dcc0bf1fcaa",
+    "tja.py": "8e3905fc3b432026f034548ba8422b06",
+    "tja_image.py": "afa678700e033206d9b80f9510dfc844",
+    "train.py": "9993791112402dfee700e62cb860c470",
+    "vocab.py": "46c4df35768ea7eee5cebccf63cc6c9d",
+}
+
+# The 1.x entry points app.py calls. Signature text, not just presence: a
+# reordered or renamed keyword is a silent behaviour change for the app.
+LEGACY_SIGNATURES = {
+    "generate.generate_song": (
+        "(model, mel, course, level=None, density_bucket=None, device='cuda', "
+        "greedy=False, temperature=1.0, top_p=0.95, seed=0, batch_windows=8, "
+        "cfg_w=0.0, use_ctx=None, lattice=False, style=None, sync_band=None, "
+        "plan=None, on_progress=None)"),
+    "generate.generate_song_slot": (
+        "(model, mel, grid, course, level=None, density_bucket=None, "
+        "device='cuda', greedy=False, temperature=1.0, top_p=0.95, seed=0, "
+        "batch_windows=8, plan=None, on_progress=None)"),
+    "generate.load_hf": "(repo_or_dir, device='cuda')",
+    "tja.write_tja_slots": (
+        "(gen, grid, title, course, level, wave, out_path=None, "
+        "balloon_count=10, plan=None)"),
+}
+
+# 1.x constants the app reads, and the vocabulary geometry every published
+# checkpoint's embedding table is indexed by.
+LEGACY_CONSTANTS = {
+    "SR": 22050, "N_FFT": 2048, "HOP": 256, "N_MELS": 128,
+    "WINDOW": 1728, "MAX_TGT": 1536, "SLOTS": 96, "N_DENS": 16,
+    "N_LEVELS": 12, "VOCAB_SIZE": 1810,
+}
+
+MEL_FRAMES = 1500  # ~17.4 s at 86.13 fps -- one decode window plus change
+
+
+def _md5(path):
+    h = hashlib.md5()
+    with open(path, "rb") as f:
+        for chunk in iter(lambda: f.read(1 << 20), b""):
+            h.update(chunk)
+    return h.hexdigest()
+
+
+def _fake_mel(frames=MEL_FRAMES, seed=0):
+    """A deterministic stand-in for a log-mel spectrogram.
+
+    The acceptance here is mechanical -- loads, decodes, exports, re-parses --
+    so the audio need only be well-formed and reproducible. Chart quality is
+    the model card's subject, not this file's.
+    """
+    import numpy as np
+
+    rng = np.random.default_rng(seed)
+    t = np.arange(frames) / 86.1328125
+    pulse = np.sin(2 * np.pi * 2.0 * t)[None, :] + 1.0
+    return np.log(1e-5 + np.abs(rng.standard_normal((128, frames)) * 0.1
+                                + pulse)).astype(np.float32)
+
+
+def _run_case(case):
+    """Run one order-sensitive case in a clean subprocess of this file."""
+    env = dict(os.environ, HF_HUB_OFFLINE="1", PYTHONWARNINGS="ignore")
+    proc = subprocess.run([sys.executable, os.path.abspath(__file__),
+                           "--case", case],
+                          capture_output=True, text=True, env=env,
+                          cwd=REPO_ROOT)
+    if proc.returncode != 0:
+        raise AssertionError(
+            f"case {case!r} failed (exit {proc.returncode}):\n"
+            f"{proc.stdout[-4000:]}\n{proc.stderr[-4000:]}")
+    for line in reversed(proc.stdout.splitlines()):
+        if line.startswith("RESULT "):
+            return json.loads(line[len("RESULT "):])
+    raise AssertionError(f"case {case!r} printed no RESULT:\n{proc.stdout}")
+
+
+# ---------------------------------------------------------------------------
+# (a) package-only load
+
+
+def case_package_only_load():
+    """Load with the development worktree removed from the path, and watch every read.
+
+    ``sys.path`` is scrubbed of anything under ``.codex/worktrees`` first, so
+    an accidental ``import softchart.<module>`` resolving to the frozen
+    shipping tree would fail rather than quietly succeed. Python-level file
+    opens are recorded; every one that lands inside this repository must be
+    inside the package directory or the vendored ``softchart/sc2`` tree.
+    """
+    import builtins
+
+    # Put the development worktree on the path first, so the scrub has something
+    # real to remove: a load that only passes because the worktree happened to
+    # be absent would prove nothing.
+    worktree_src = os.path.join(REPO_ROOT, ".codex", "worktrees", "v18-8h",
+                                "src")
+    if os.path.isdir(worktree_src):
+        sys.path.insert(0, worktree_src)
+    removed = [p for p in sys.path if ".codex" + os.sep + "worktrees" in p]
+    sys.path[:] = [p for p in sys.path if p not in removed]
+    for name in [n for n in sys.modules
+                 if n == "softchart" or n.startswith("softchart.")]:
+        del sys.modules[name]
+
+    # modules that exist only in the worktree tree must now be unreachable
+    try:
+        __import__("softchart.barscript")
+        worktree_reachable = True
+    except ImportError:
+        worktree_reachable = False
+    for name in [n for n in sys.modules
+                 if n == "softchart" or n.startswith("softchart.")]:
+        del sys.modules[name]
+
+    opened = []
+    real_open = builtins.open
+
+    def watched_open(file, *a, **kw):
+        try:
+            opened.append(os.path.abspath(os.fspath(file)))
+        except TypeError:
+            pass
+        return real_open(file, *a, **kw)
+
+    from softchart import sc2_loader as loader
+
+    package_dir = os.path.abspath(loader.default_package_dir())
+    builtins.open = watched_open
+    try:
+        model, config, spec = loader.load_release(package_dir, device="cpu")
+    finally:
+        builtins.open = real_open
+
+    stray = [p for p in opened
+             if p.startswith(REPO_ROOT + os.sep)
+             and not p.startswith(package_dir + os.sep)
+             and not p.startswith(SC2_DIR + os.sep)]
+    outside = [m.__file__ for name, m in sys.modules.items()
+               if name.startswith("softchart.sc2.")
+               and getattr(m, "__file__", None)
+               and not os.path.abspath(m.__file__).startswith(SC2_DIR + os.sep)]
+
+    return {
+        "package_dir": package_dir,
+        "worktree_entries_removed": removed,
+        "worktree_src_exists": os.path.isdir(worktree_src),
+        "worktree_reachable_after_scrub": worktree_reachable,
+        "release_name": config["release_name"],
+        "parameters": int(config["weights"]["parameters"]),
+        "vocab_size": int(config["vocab"]["size"]),
+        "has_sync_forced": bool(getattr(model, "_has_sync", False)),
+        "has_axis_knobs": bool(getattr(model, "_has_axis_knobs", False)),
+        "motif_hard": config["train_provenance"]["motif_constraint"][
+            "resolved_motif_hard"],
+        "spec_labels_version": spec.labels_version,
+        "files_opened_in_repo": len([p for p in opened
+                                     if p.startswith(REPO_ROOT + os.sep)]),
+        "stray_reads": stray,
+        "sc2_modules_outside_vendor_dir": outside,
+    }
+
+
+def test_a_package_only_load():
+    r = _run_case("package-only-load")
+    if r["worktree_src_exists"]:
+        assert r["worktree_entries_removed"], "the scrub removed nothing"
+    assert r["worktree_reachable_after_scrub"] is False, (
+        "the development worktree is still importable after the scrub; the load below "
+        "proves nothing")
+    assert not r["stray_reads"], f"read outside the package: {r['stray_reads']}"
+    assert not r["sc2_modules_outside_vendor_dir"], r[
+        "sc2_modules_outside_vendor_dir"]
+    assert r["vocab_size"] == 2071, r["vocab_size"]
+    # config.serving_prefix_fix: the recorded sync_token flag is wrong and the
+    # loader must override it, or every sync bucket decodes identically.
+    assert r["has_sync_forced"] is True
+    assert r["has_axis_knobs"] is True
+    assert r["motif_hard"] is False, "package resolved motif_hard off"
+    assert r["spec_labels_version"] == "barscript_labels_v1"
+    return r
+
+
+# ---------------------------------------------------------------------------
+# (b) generate and re-parse
+
+
+def test_b_generate_and_reparse():
+    from softchart import sc2_loader as loader
+    from softchart.sc2.trace import chart_to_trace, parse_tja
+
+    sc = loader.load(device="cpu")
+
+    # the front end app.py computes must be the one the package was trained on
+    from softchart.vocab import HOP, N_FFT, N_MELS, SR
+
+    loader.check_preprocessing(sc.package_dir, sr=SR, n_fft=N_FFT,
+                               hop_length=HOP, n_mels=N_MELS, fmin=20.0,
+                               fmax=SR / 2)
+
+    out = {}
+    for course, level, dens in (("oni", 8, 6), ("ura", 10, 6), ("easy", 3, 1)):
+        r = sc.generate(_fake_mel(), course, level=level, bpm=150.0,
+                        offset_sec=0.12, density_bucket=dens, seed=7,
+                        title="vendor-acceptance", wave="acceptance.ogg")
+        charts = parse_tja(r["tja"])
+        assert len(charts) == 1, f"{course}: expected one chart section"
+        # chart_to_trace is the real acceptance: it rebuilds bar edges, tempo
+        # and meter from the exported text and raises on anything inconsistent.
+        trace = chart_to_trace(charts[0])
+        assert len(trace.bars) == r["grid"]["n_bars"], (
+            len(trace.bars), r["grid"]["n_bars"])
+        n_notes = sum(c in "1234" for b in charts[0].bars for c in b.digits)
+        n_spans = sum(c in "567" for b in charts[0].bars for c in b.digits)
+        assert n_spans == r["n_spans"], (n_spans, r["n_spans"])
+
+        assert r["grid"]["denom"] == 16, r["grid"]
+        assert r["grid"]["triplets_representable"] is False
+        assert r["grid"]["bar_denoms"] == "supplied", r["grid"]
+        assert r["serving"]["min_onset_gap_sec"] == 0.02, r["serving"]
+        assert r["serving"]["temperature"] == 1.0 and r["serving"]["top_p"] == 0.95
+        assert r["serving_overrides"] == {}
+        assert r["motif"]["verified"] is True, r["motif"]
+        assert r["experimental"] is True
+        assert n_notes == r["n_notes"], (n_notes, r["n_notes"])
+
+        header = [ln for ln in r["tja"].splitlines() if ln.startswith("COURSE:")]
+        assert header == [f"COURSE:{loader.TJA_COURSE_HEADER[course]}"], header
+        assert charts[0].course == course, (charts[0].course, course)
+        out[course] = {"n_notes": r["n_notes"], "n_spans": r["n_spans"],
+                       "n_bars": r["grid"]["n_bars"], "header": header[0],
+                       "reparsed_notes": n_notes}
+
+    # same seed, same conditions, same chart
+    a = sc.generate(_fake_mel(), "oni", level=8, bpm=150.0, offset_sec=0.12,
+                    density_bucket=6, seed=7)["tja"]
+    b = sc.generate(_fake_mel(), "oni", level=8, bpm=150.0, offset_sec=0.12,
+                    density_bucket=6, seed=7)["tja"]
+    assert a == b, "BarScript decode is not reproducible at a fixed seed"
+    out["reproducible"] = True
+    return out
+
+
+# ---------------------------------------------------------------------------
+# (c) the 1.x path is untouched
+
+
+def test_c_legacy_library_untouched():
+    lib = os.path.join(SPACES_DIR, "softchart")
+    drift = {name: _md5(os.path.join(lib, name))
+             for name in sorted(LEGACY_MD5)
+             if _md5(os.path.join(lib, name)) != LEGACY_MD5[name]}
+    assert not drift, (
+        "1.x library modules changed since 2.0 was vendored in; the seven "
+        f"published models are no longer a fixed baseline: {drift}")
+
+    added = sorted(n for n in os.listdir(lib)
+                   if n.endswith(".py") and n not in LEGACY_MD5)
+    assert added == ["sc2_loader.py"], added
+    return {"unchanged": len(LEGACY_MD5), "added": added}
+
+
+def test_c_legacy_api_unchanged():
+    from softchart import generate as v1_generate
+    from softchart import tja as v1_tja
+    from softchart import vocab as v1_vocab
+
+    mods = {"generate": v1_generate, "tja": v1_tja}
+    bad = {}
+    for key, want in LEGACY_SIGNATURES.items():
+        mod, fn = key.split(".")
+        got = str(inspect.signature(getattr(mods[mod], fn)))
+        got = got.replace('"', "'")
+        if got != want:
+            bad[key] = got
+    assert not bad, f"1.x public signatures changed: {bad}"
+
+    got = {k: getattr(v1_vocab, k) for k in LEGACY_CONSTANTS if k != "VOCAB_SIZE"}
+    got["VOCAB_SIZE"] = v1_vocab.VOCAB.size
+    assert got == LEGACY_CONSTANTS, f"1.x constants changed: {got}"
+    return got
+
+
+def test_c_namespaces_isolated():
+    """The two token tables coexist and neither can reach the other."""
+    from softchart import vocab as v1_vocab
+    from softchart.sc2 import vocab as v2_vocab
+
+    assert v1_vocab is not v2_vocab
+    assert v1_vocab.VOCAB.size == 1810 and v2_vocab.VOCAB.size == 2071
+    assert v1_vocab.MAX_TGT == 1536 and v2_vocab.MAX_TGT == 960
+
+    # every sc2 module resolves its imports inside the sc2 package
+    from softchart.sc2 import generate as v2_generate
+    from softchart.sc2 import model as v2_model
+
+    assert os.path.abspath(v2_generate.__file__).startswith(SC2_DIR + os.sep)
+    assert v2_generate.VOCAB is v2_vocab.VOCAB
+    assert v2_model.__name__ == "softchart.sc2.model"
+
+    from softchart import generate as v1_generate
+
+    assert v1_generate.VOCAB is v1_vocab.VOCAB
+    assert v1_generate is not v2_generate
+    return {"v1_vocab": v1_vocab.VOCAB.size, "v2_vocab": v2_vocab.VOCAB.size}
+
+
+def _legacy_decode():
+    """One 1.x decode, in both the time and the slot regime app.py uses."""
+    import numpy as np
+
+    from softchart.generate import (generate_song, generate_song_slot, load_hf)
+
+    model = load_hf(LEGACY_REPO, device="cpu")
+    mel = _fake_mel()
+    dur = mel.shape[1] / 86.1328125
+    time_gen = generate_song(model, mel, "oni", level=8, density_bucket=7,
+                             device="cpu", seed=0)
+    bar = 240.0 / 150.0
+    edges = np.arange(0.0, dur + bar, bar)
+    grid = {"bpm": 150.0, "downbeats": edges[:-1], "bar": bar,
+            "measure_edges": edges}
+    slot_gen = generate_song_slot(model, mel, grid, "oni", level=8,
+                                  density_bucket=7, device="cpu", seed=0)
+    payload = {
+        "time": [time_gen["hits"], time_gen["spans"]],
+        "slot": [[list(map(_jsonable, h)) for h in slot_gen["hits_slots"]],
+                 [list(map(_jsonable, s)) for s in slot_gen["spans_slots"]]],
+    }
+    blob = json.dumps(payload, sort_keys=True, default=_jsonable)
+    return {"sha256": hashlib.sha256(blob.encode()).hexdigest(),
+            "n_time_hits": len(time_gen["hits"]),
+            "n_slot_hits": len(slot_gen["hits_slots"])}
+
+
+def _jsonable(x):
+    import numpy as np
+
+    if isinstance(x, (np.integer,)):
+        return int(x)
+    if isinstance(x, (np.floating,)):
+        return float(x)
+    return x
+
+
+def case_legacy_baseline():
+    return _legacy_decode()
+
+
+def case_legacy_after_sc2():
+    """The same 1.x decode, after 2.0 is fully imported AND loaded.
+
+    This is the regression the user's comparison depends on: 2.0 living in the
+    same process must not move a single note of a 1.x chart.
+    """
+    from softchart import sc2_loader as loader
+
+    sc = loader.load(device="cpu")
+    sc.generate(_fake_mel(600), "oni", level=8, bpm=150.0, offset_sec=0.0,
+                density_bucket=6, seed=1)
+    out = _legacy_decode()
+    out["sc2_release"] = sc.release_name
+    return out
+
+
+def _legacy_weights_cached():
+    try:
+        from huggingface_hub import try_to_load_from_cache
+    except Exception:
+        return False
+    for name in ("model.safetensors", "config.json"):
+        hit = try_to_load_from_cache(LEGACY_REPO, name)
+        if not isinstance(hit, str):
+            return False
+    return True
+
+
+def test_c_legacy_decode_unchanged():
+    if not _legacy_weights_cached():
+        return {"skipped": f"{LEGACY_REPO} is not in the local Hub cache"}
+    before = _run_case("legacy-baseline")
+    after = _run_case("legacy-after-sc2")
+    assert before["sha256"] == after["sha256"], (
+        "a 1.x decode changed once softchart.sc2 was imported: "
+        f"{before} != {after}")
+    return {"sha256": before["sha256"], "n_time_hits": before["n_time_hits"],
+            "n_slot_hits": before["n_slot_hits"],
+            "sc2_release": after["sc2_release"]}
+
+
+# ---------------------------------------------------------------------------
+
+CASES = {
+    "package-only-load": case_package_only_load,
+    "legacy-baseline": case_legacy_baseline,
+    "legacy-after-sc2": case_legacy_after_sc2,
+}
+
+CHECKS = (
+    ("(a) package-only load", test_a_package_only_load),
+    ("(b) generate + re-parse", test_b_generate_and_reparse),
+    ("(c) 1.x library untouched", test_c_legacy_library_untouched),
+    ("(c) 1.x API unchanged", test_c_legacy_api_unchanged),
+    ("(c) namespaces isolated", test_c_namespaces_isolated),
+    ("(c) 1.x decode unchanged", test_c_legacy_decode_unchanged),
+)
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--case", choices=sorted(CASES),
+                    help="internal: run one subprocess case and print RESULT")
+    args = ap.parse_args(argv)
+    warnings.filterwarnings("ignore")
+    os.environ.setdefault("HF_HUB_OFFLINE", "1")
+
+    if args.case:
+        print("RESULT " + json.dumps(CASES[args.case](), default=str))
+        return 0
+
+    failures = []
+    for label, check in CHECKS:
+        try:
+            detail = check()
+        except Exception as exc:  # noqa: BLE001 - a report, not a traceback
+            failures.append(label)
+            print(f"FAIL  {label}\n      {type(exc).__name__}: {exc}")
+        else:
+            mark = "SKIP" if isinstance(detail, dict) and "skipped" in detail \
+                else "PASS"
+            print(f"{mark}  {label}\n      "
+                  + json.dumps(detail, default=str)[:600])
+    print(("FAILED: " + ", ".join(failures)) if failures
+          else "all checks passed")
+    return 1 if failures else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/test_space_e2e.py

@@ -0,0 +1,658 @@
+"""End-to-end acceptance for the Space: real weights, real music, real app entry.
+
+The other two suites stop short of this on purpose.  ``test_sc2_vendor.py``
+proves the vendored 2.0 tree is the one the release package describes, and
+``test_app_wiring.py`` proves the routing by replaying every model as a
+recorder over a synthetic mel.  Neither one ever decodes real music, so
+neither can say whether a chart comes out of ``app.generate_chart`` intact.
+
+This file drives ``app.generate_chart`` itself -- the same generator the
+frontend consumes, with a ``gr.FileData`` upload -- on held-out songs, and
+inspects the artifacts it actually produced.
+
+Two questions:
+
+1. **Does 2.0 produce a well-formed chart on every difficulty?**  For each
+   song x {easy, normal, hard, oni, ura} the emitted TJA is re-parsed with
+   ``softchart.sc2.trace`` and checked for: a single course section whose
+   ``COURSE:`` header resolves back to the course that was asked for (this is
+   what makes ``COURSE:Edit`` -> ``ura`` a semantic check and not a string
+   comparison); a bar count consistent with the tempo and excerpt length;
+   96-digit bars whose every note sits on the shipped /16 lattice; balanced,
+   properly ordered drumroll spans; and a note total that agrees with the
+   count the UI reported.
+
+   The suite also RECORDS the difficulty ladder rather than assuming it.  The
+   model card (§2.1) says the ladder is not nested, and ura carries 208
+   training charts against ~1 100 elsewhere, so "ura is harder than oni" is
+   an open question.  ``--strict-ladder`` turns the ladder into a hard
+   failure; by default it is reported, because a monotonicity failure here is
+   a MODEL result to be published, not a wiring bug to be hidden.
+
+2. **Do the published models still behave exactly as before?**  Same songs,
+   same controls, run through both the current ``app.py`` and
+   ``fixtures/app_baseline.py`` -- the app as it stood BEFORE the 2.0/ura
+   change, pinned by sha256 to the same source the wiring suite's golden
+   trace was recorded from.  Both run with REAL Hub weights, and the
+   comparison is over the full TJA text plus the reported metrics.  The
+   wiring suite compares call arguments under mocks; this one compares
+   generated charts.
+
+Fixtures.  Preferred source is the held-out ``test`` split of
+``JacobLinCool/taiko-1000-parsed`` if it is in the local HuggingFace cache:
+those songs are real music the models never trained on, and they ship
+authored charts for all five courses, which gives the difficulty ladder a
+reference to be read against.  Otherwise pass ``--audio-dir`` with any
+directory of audio files (the ladder is then reported without a reference).
+Nothing is downloaded.
+
+    python spaces/scripts/test_space_e2e.py
+    python spaces/scripts/test_space_e2e.py --songs 2 --skip-legacy
+    python spaces/scripts/test_space_e2e.py --audio-dir ../other --json report.json
+
+Runs on CPU.  Legacy models are read from the local HuggingFace cache; export
+``HF_HUB_OFFLINE=1`` to prove no network is involved.
+"""
+
+import argparse
+import collections
+import glob
+import hashlib
+import io
+import json
+import os
+import statistics
+import sys
+import tempfile
+import time
+
+SPACES = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+if SPACES not in sys.path:
+    sys.path.insert(0, SPACES)
+
+FIXTURES = os.path.join(os.path.dirname(os.path.abspath(__file__)), "fixtures")
+BASELINE_APP = os.path.join(FIXTURES, "app_baseline.py")
+GOLDEN = os.path.join(FIXTURES, "legacy_call_trace.json")
+
+ALL_COURSES = ("easy", "normal", "hard", "oni", "ura")
+# Published models replayed against the baseline. v1.7 is the default the UI
+# recommends and v1.5 is the oldest still selectable, so between them they
+# cover both ends of the published range; both are full-size, so a difference
+# cannot hide behind a small model's noise.
+LEGACY_MODELS = ("v1.7", "v1.5")
+LEGACY_COURSES = ("hard", "oni")
+
+# The TJA export lattice: write_tja_slots emits SUB=96 digits per 4/4 bar.
+SUB = 96
+# The deployment grid (sc2_loader.SHIP_GRID_DENOM). 96/16 = 6, so a note that
+# is on the /16 lattice sits at a digit index divisible by 6.
+SHIP_DENOM = 16
+
+SPAN_OPEN = {"5", "6", "7"}   # roll, big roll, balloon
+SPAN_CLOSE = "8"
+HIT_CHARS = {"1", "2", "3", "4"}
+
+# The TJA `COURSE:` literal each course must be WRITTEN as, measured over all
+# 1 155 songs of JacobLinCool/taiko-1000-parsed: the 229 ura charts are headed
+# `Edit` (223) or the numeric `4` (6), and the string "Ura" occurs zero times
+# anywhere in the corpus.
+#
+# The literal is asserted separately from the re-parse because
+# ``softchart.sc2.trace`` is deliberately lenient -- its alias table maps
+# "ura" -> ura -- so a chart written `COURSE:Ura` round-trips through this
+# repo's own parser and would pass a semantic-only check while still being
+# the exact defect the header rewrite exists to prevent.
+CORPUS_COURSE_LITERAL = {"easy": "Easy", "normal": "Normal", "hard": "Hard",
+                         "oni": "Oni", "ura": "Edit"}
+
+RESULTS = []
+
+
+def check(name, ok, detail=""):
+    RESULTS.append((name, bool(ok), detail))
+    print(f"{'PASS' if ok else 'FAIL'}  {name}" + (f"  -- {detail}" if detail else ""))
+    return bool(ok)
+
+
+def note(name, detail):
+    """Record an observation that is reported but never fails the suite."""
+    print(f"NOTE  {name}  -- {detail}")
+
+
+# ---------------------------------------------------------------------------
+# fixtures
+
+
+def dataset_test_shards():
+    pattern = os.path.expanduser(
+        "~/.cache/huggingface/hub/datasets--JacobLinCool--taiko-1000-parsed/"
+        "snapshots/*/data/test-*.parquet")
+    return sorted(glob.glob(pattern))
+
+
+def songs_from_dataset(out_dir, n_songs, seconds):
+    """Held-out songs with authored charts, decoded from the local cache.
+
+    Only songs carrying BOTH an oni and an ura chart are taken: those are the
+    ones that can say what the authored oni->ura step actually looks like on
+    the same music the model is being asked about.
+    """
+    import soundfile as sf
+    import pyarrow.parquet as pq
+
+    picked = {}
+    seen_titles = set()
+    for shard in dataset_test_shards():
+        table = pq.read_table(shard)
+        for i in range(table.num_rows):
+            if len(picked) >= n_songs:
+                break
+            meta = table["metadata"][i].as_py()
+            title = (meta.get("TITLE") or "").strip()
+            if not title or title in seen_titles:
+                continue
+            authored = {}
+            for course in ALL_COURSES:
+                chart = table[course][i].as_py()
+                if not chart:
+                    continue
+                times = sorted(
+                    ev["timestamp"] for seg in chart["segments"]
+                    for ev in seg["notes"]
+                    if ev["note_type"] in ("Don", "Ka", "DonBig", "KaBig"))
+                span = (times[-1] - times[0]) if len(times) > 1 else 0.0
+                authored[course] = {
+                    "level": chart["level"], "n_notes": len(times),
+                    "nps": (len(times) / span) if span else 0.0}
+            if "ura" not in authored or "oni" not in authored:
+                continue
+            try:
+                bpm = float(meta["BPM"])
+            except (TypeError, ValueError, KeyError):
+                continue
+            audio = table["audio"][i].as_py()
+            wav, sr = sf.read(io.BytesIO(audio["bytes"]), dtype="float32",
+                              always_2d=True)
+            wav = wav.mean(axis=1)
+            # Start 20 % in: intros are often sparse or silent, and a chart
+            # generated over silence would test nothing.
+            start = int(len(wav) * 0.20)
+            seg = wav[start:start + int(seconds * sr)]
+            if len(seg) < int(seconds * sr * 0.9):
+                continue
+            seen_titles.add(title)
+            slug = f"song{len(picked):02d}"
+            path = os.path.join(out_dir, slug + ".wav")
+            sf.write(path, seg, sr)
+            picked[slug] = {"title": title, "path": path, "bpm": bpm,
+                            "seconds": len(seg) / sr, "authored": authored,
+                            "source": "taiko-1000-parsed/test"}
+        if len(picked) >= n_songs:
+            break
+    return picked
+
+
+def songs_from_dir(audio_dir, out_dir, n_songs, seconds):
+    """Fallback fixture: any directory of audio, with no authored reference."""
+    import librosa
+    import soundfile as sf
+
+    files = []
+    for ext in ("*.ogg", "*.mp3", "*.wav", "*.flac", "*.m4a"):
+        files.extend(glob.glob(os.path.join(audio_dir, ext)))
+    picked = {}
+    for path in sorted(files)[:n_songs]:
+        wav, sr = librosa.load(path, sr=None, mono=True)
+        start = int(len(wav) * 0.20)
+        seg = wav[start:start + int(seconds * sr)]
+        if len(seg) < int(seconds * sr * 0.5):
+            continue
+        slug = f"song{len(picked):02d}"
+        out = os.path.join(out_dir, slug + ".wav")
+        sf.write(out, seg, sr)
+        picked[slug] = {"title": os.path.basename(path), "path": out,
+                        "bpm": None, "seconds": len(seg) / sr, "authored": {},
+                        "source": os.path.abspath(audio_dir)}
+    return picked
+
+
+def build_fixtures(out_dir, n_songs, seconds, audio_dir=None):
+    if audio_dir:
+        songs = songs_from_dir(audio_dir, out_dir, n_songs, seconds)
+        origin = f"--audio-dir {audio_dir}"
+    elif dataset_test_shards():
+        songs = songs_from_dataset(out_dir, n_songs, seconds)
+        origin = "taiko-1000-parsed test split (held out)"
+    else:
+        return {}, "none"
+    return songs, origin
+
+
+# ---------------------------------------------------------------------------
+# driving the app
+
+
+def run_app(module, song, course, model, *, level=8, bpm=0.0,
+            auto_plan_on=False, use_beat=True, use_planner=False,
+            sampling=False, temperature=0.7, top_p=0.95, drum_volume=0.8):
+    """Consume ``generate_chart`` and capture the artifacts it produced.
+
+    The app deletes its working directory in a ``finally`` once the generator
+    is closed, so the TJA has to be read while it is still suspended at the
+    completion yield.
+    """
+    import gradio as gr
+
+    name = os.path.basename(song["path"])
+    payload = gr.FileData(path=song["path"], orig_name=name,
+                          mime_type="audio/wav").model_dump()
+    started = time.time()
+    stages = []
+    out = {"model": model, "course": course, "song": song["title"]}
+    for event in module.generate_chart(
+            payload, name, course, level, bpm, auto_plan_on, use_beat,
+            use_planner, sampling, temperature, top_p, drum_volume, model):
+        stages.append(event.get("stage"))
+        if event["kind"] == "error":
+            out.update(kind="error", detail=event.get("detail"))
+            break
+        if event["kind"] == "complete":
+            with open(event["files"]["tja"]["path"], encoding="utf-8") as f:
+                tja = f.read()
+            out.update(kind="complete", tja=tja, metrics=dict(event["metrics"]),
+                       files={k: os.path.basename(v["path"])
+                              for k, v in event["files"].items()})
+            break
+    out["stages"] = stages
+    out["seconds"] = round(time.time() - started, 2)
+    return out
+
+
+# ---------------------------------------------------------------------------
+# chart inspection
+
+
+def inspect_tja(text, course):
+    """Structural facts about one generated TJA, from a real re-parse."""
+    from softchart.sc2 import trace as T
+
+    facts = {}
+    charts = T.parse_tja(text)          # raises on anything it will not guess about
+    facts["n_sections"] = len(charts)
+    chart = charts[0]
+    facts["parsed_course"] = chart.course
+    facts["level"] = chart.level
+    facts["n_bars"] = len(chart.bars)
+    facts["bar_lengths"] = sorted(set(len(b.digits) for b in chart.bars))
+    header = [ln for ln in text.splitlines() if ln.startswith("COURSE:")]
+    facts["course_header"] = header[0] if header else None
+
+    # Every character that is not '0' must land on the shipped lattice.
+    off_lattice = 0
+    counts = collections.Counter()
+    for bar in chart.bars:
+        stride = len(bar.digits) // SHIP_DENOM if len(bar.digits) else 0
+        for pos, ch in enumerate(bar.digits):
+            if ch == "0":
+                continue
+            counts[ch] += 1
+            if not stride or pos % stride:
+                off_lattice += 1
+    facts["off_lattice_notes"] = off_lattice
+    facts["note_chars"] = dict(counts)
+    facts["n_hits"] = sum(v for k, v in counts.items() if k in HIT_CHARS)
+
+    # Span balance, in emission order across the whole chart.
+    depth = 0
+    bad_order = 0
+    n_spans = 0
+    for bar in chart.bars:
+        for ch in bar.digits:
+            if ch in SPAN_OPEN:
+                if depth:
+                    bad_order += 1
+                depth += 1
+                n_spans += 1
+            elif ch == SPAN_CLOSE:
+                if not depth:
+                    bad_order += 1
+                else:
+                    depth -= 1
+    facts["n_spans"] = n_spans
+    facts["unclosed_spans"] = depth
+    facts["misordered_spans"] = bad_order
+
+    # A full trace conversion is the strongest re-parse available: it resolves
+    # every note to an exact bar-local Fraction and a wall-clock time.
+    tr = T.chart_to_trace(chart)
+    times = T.trace_to_times(tr)
+    facts["n_trace_notes"] = len(times)
+    facts["bpm"] = tr.bpm
+    facts["offset"] = tr.offset
+    return facts
+
+
+def note_times(text):
+    """Wall-clock times of the hit notes, for density statistics."""
+    from softchart.sc2 import trace as T
+
+    chart = T.parse_tja(text)[0]
+    out = []
+    for entry in T.trace_to_times(T.chart_to_trace(chart)):
+        t = entry[0] if isinstance(entry, (tuple, list)) else entry.get("t")
+        kind = entry[1] if isinstance(entry, (tuple, list)) else entry.get("type")
+        if str(kind) in HIT_CHARS:
+            out.append(float(t))
+    return sorted(out)
+
+
+# ---------------------------------------------------------------------------
+# test 1: BarScript on all five difficulties
+
+
+def test_sc2_all_courses(songs, records, strict_ladder=False):
+    import app as APP
+    from softchart import sc2_loader
+
+    if not sc2_loader.is_available():
+        check("BarScript release package present", False,
+              f"no package at {sc2_loader.default_package_dir()}")
+        return
+    check("BarScript release package present", True, sc2_loader.default_package_dir())
+
+    per_song = {}
+    for slug, song in sorted(songs.items()):
+        per_song[slug] = {}
+        for course in ALL_COURSES:
+            tag = f"BarScript {song['title'][:22]} / {course}"
+            result = run_app(APP, song, course, APP.SC2_MODEL)
+            records.append({"phase": "sc2", "slug": slug, **{
+                k: v for k, v in result.items() if k != "tja"}})
+            if result["kind"] != "complete":
+                check(f"{tag}: completes", False, str(result.get("detail"))[:160])
+                continue
+
+            facts = inspect_tja(result["tja"], course)
+            records[-1]["facts"] = facts
+            metrics = result["metrics"]
+
+            ok = check(f"{tag}: completes", True,
+                       f"{metrics['notes']} notes, {result['seconds']}s")
+            ok &= check(f"{tag}: single course section",
+                        facts["n_sections"] == 1, f"{facts['n_sections']}")
+            # Two independent header checks. The literal is what a Taiko
+            # simulator reads; the re-parse is what it means.
+            ok &= check(f"{tag}: COURSE header is the literal the corpus uses",
+                        facts["course_header"]
+                        == f"COURSE:{CORPUS_COURSE_LITERAL[course]}",
+                        f"{facts['course_header']!r}")
+            ok &= check(f"{tag}: COURSE header resolves to the course asked for",
+                        facts["parsed_course"] == course,
+                        f"{facts['course_header']} -> {facts['parsed_course']}")
+            ok &= check(f"{tag}: bars are {SUB}-digit",
+                        facts["bar_lengths"] == [SUB], f"{facts['bar_lengths']}")
+            ok &= check(f"{tag}: every note on the /{SHIP_DENOM} lattice",
+                        facts["off_lattice_notes"] == 0,
+                        f"{facts['off_lattice_notes']} off-lattice")
+            ok &= check(f"{tag}: no unclosed drumroll",
+                        facts["unclosed_spans"] == 0 and
+                        facts["misordered_spans"] == 0,
+                        f"open={facts['unclosed_spans']} "
+                        f"misordered={facts['misordered_spans']}")
+            ok &= check(f"{tag}: re-parsed note count matches the UI",
+                        facts["n_hits"] == metrics["notes"],
+                        f"{facts['n_hits']} vs {metrics['notes']}")
+            ok &= check(f"{tag}: span count matches the UI",
+                        facts["n_spans"] == metrics["spans"],
+                        f"{facts['n_spans']} vs {metrics['spans']}")
+
+            # Bar count must follow from tempo and length, or the lattice is
+            # anchored wrong and every barline is in the wrong place.
+            bar_sec = 4.0 * 60.0 / float(metrics["bpm"])
+            expected = (song["seconds"] - abs(facts["offset"])) / bar_sec
+            ok &= check(f"{tag}: bar count follows tempo and length",
+                        abs(facts["n_bars"] - expected) <= 1.5,
+                        f"{facts['n_bars']} bars, expected ~{expected:.1f} "
+                        f"at {metrics['bpm']} BPM")
+            ok &= check(f"{tag}: reported as slot-exact on the /16 grid",
+                        metrics["timing"] == "slot-exact"
+                        and "/16" in metrics.get("chart_grid", ""),
+                        f"{metrics['timing']}, {metrics.get('chart_grid')}")
+
+            times = note_times(result["tja"])
+            span = (times[-1] - times[0]) if len(times) > 1 else 0.0
+            per_song[slug][course] = {
+                "n_notes": facts["n_hits"], "n_spans": facts["n_spans"],
+                "nps": (len(times) / span) if span else 0.0,
+                "min_gap_ms": (min((b - a) for a, b in zip(times, times[1:]))
+                               * 1000.0) if len(times) > 1 else None}
+
+    records.append({"phase": "sc2_ladder", "generated": per_song,
+                    "authored": {slug: song["authored"]
+                                 for slug, song in songs.items()}})
+    ladder_report(songs, per_song, strict_ladder)
+    return per_song
+
+
+def ladder_report(songs, per_song, strict_ladder):
+    """Report the generated difficulty ladder against the authored one."""
+    print("\n--- difficulty ladder (BarScript, generated vs authored)")
+    header = f"{'song':26s} " + " ".join(f"{c:>13s}" for c in ALL_COURSES)
+    print(header)
+    ura_gt_oni = []
+    monotone = []
+    for slug, song in sorted(songs.items()):
+        got = per_song.get(slug) or {}
+        if not got:
+            continue
+        row = f"{song['title'][:25]:26s} "
+        for course in ALL_COURSES:
+            entry = got.get(course)
+            row += f"{entry['n_notes']:6d}/{entry['nps']:5.2f} " if entry else f"{'--':>13s} "
+        print(row)
+        if song["authored"]:
+            arow = f"{'  (authored)':26s} "
+            for course in ALL_COURSES:
+                a = song["authored"].get(course)
+                arow += f"{a['n_notes']:6d}/{a['nps']:5.2f} " if a else f"{'--':>13s} "
+            print(arow)
+        counts = [got[c]["n_notes"] for c in ALL_COURSES if c in got]
+        if len(counts) == len(ALL_COURSES):
+            monotone.append(all(x < y for x, y in zip(counts, counts[1:])))
+            ura_gt_oni.append(got["ura"]["n_notes"] > got["oni"]["n_notes"])
+
+    if not ura_gt_oni:
+        return
+    n_ura = sum(ura_gt_oni)
+    n_mono = sum(monotone)
+    detail = (f"ura denser than oni on {n_ura}/{len(ura_gt_oni)} songs; "
+              f"fully monotone easy<normal<hard<oni<ura on "
+              f"{n_mono}/{len(monotone)}")
+    authored = [s for s in songs.values() if s["authored"].get("ura")
+                and s["authored"].get("oni")]
+    if authored:
+        a_ura = sum(1 for s in authored
+                    if s["authored"]["ura"]["n_notes"]
+                    > s["authored"]["oni"]["n_notes"])
+        detail += f"; authored reference: ura denser on {a_ura}/{len(authored)}"
+    if strict_ladder:
+        check("BarScript: ura is harder than oni on every song",
+              n_ura == len(ura_gt_oni), detail)
+        check("BarScript: the difficulty ladder is monotone on every song",
+              n_mono == len(monotone), detail)
+    else:
+        note("BarScript difficulty ladder (reported, not asserted -- MODEL_CARD "
+             "§2.1 says the ladder is not nested)", detail)
+
+
+# ---------------------------------------------------------------------------
+# test 2: published models unchanged, on real audio and real weights
+
+
+def load_baseline():
+    """Import the pre-change app, pinned to the wiring suite's provenance.
+
+    The baseline mounts its static assets from ``Path(__file__).with_name
+    ("static")``, so it is staged in a scratch directory next to a link to
+    the Space's real ``static/`` rather than imported from ``fixtures/``.
+    """
+    import importlib.util
+    import shutil
+
+    if not os.path.isfile(BASELINE_APP):
+        return None, "fixtures/app_baseline.py is missing"
+    with open(BASELINE_APP, "rb") as f:
+        digest = hashlib.sha256(f.read()).hexdigest()
+    expected = None
+    if os.path.isfile(GOLDEN):
+        with open(GOLDEN, encoding="utf-8") as f:
+            expected = json.load(f).get("source_sha256")
+    if expected and digest != expected:
+        return None, (f"baseline sha256 {digest[:16]} does not match the "
+                      f"golden trace's source {expected[:16]}")
+
+    stage = tempfile.mkdtemp(prefix="softchart-baseline-")
+    staged = os.path.join(stage, "app_baseline.py")
+    shutil.copyfile(BASELINE_APP, staged)
+    os.symlink(os.path.join(SPACES, "static"), os.path.join(stage, "static"))
+    spec = importlib.util.spec_from_file_location("app_pre_sc2", staged)
+    module = importlib.util.module_from_spec(spec)
+    sys.modules["app_pre_sc2"] = module
+    spec.loader.exec_module(module)
+    return module, digest
+
+
+def test_legacy_unchanged(songs, records):
+    import app as APP
+
+    baseline, info = load_baseline()
+    if baseline is None:
+        check("legacy baseline available", False, info)
+        return
+    check("legacy baseline available", True,
+          f"fixtures/app_baseline.py sha256 {info[:16]}, matches the golden "
+          f"trace source")
+
+    for model in LEGACY_MODELS:
+        for course in LEGACY_COURSES:
+            for slug, song in sorted(songs.items()):
+                tag = f"{model} {song['title'][:22]} / {course}"
+                try:
+                    now = run_app(APP, song, course, model)
+                    was = run_app(baseline, song, course, model)
+                except Exception as exc:  # noqa: BLE001
+                    check(f"{tag}: runs", False, f"{type(exc).__name__}: {exc}")
+                    continue
+                if now["kind"] != "complete" or was["kind"] != "complete":
+                    check(f"{tag}: both apps complete", False,
+                          f"now={now['kind']} baseline={was['kind']} "
+                          f"{now.get('detail') or was.get('detail')}")
+                    continue
+                same_tja = now["tja"] == was["tja"]
+                # The metrics panel legitimately GAINED fields in this change
+                # (the model name, the TJA course literal, the plan source).
+                # What may not change is any value the baseline already
+                # reported, so the shared keys are compared exactly and the
+                # added ones are recorded separately.
+                shared = set(was["metrics"]) & set(now["metrics"])
+                same_shared = all(was["metrics"][k] == now["metrics"][k]
+                                  for k in shared)
+                added = sorted(set(now["metrics"]) - set(was["metrics"]))
+                dropped = sorted(set(was["metrics"]) - set(now["metrics"]))
+                records.append({
+                    "phase": "legacy", "slug": slug, "model": model,
+                    "course": course, "identical_tja": same_tja,
+                    "identical_shared_metrics": same_shared,
+                    "metrics_added": added, "metrics_dropped": dropped,
+                    "notes": now["metrics"]["notes"],
+                    "sha256": hashlib.sha256(
+                        now["tja"].encode("utf-8")).hexdigest()[:16]})
+                check(f"{tag}: TJA byte-identical to the pre-change app",
+                      same_tja,
+                      f"{len(now['tja'])} bytes, sha "
+                      f"{records[-1]['sha256']}" if same_tja else
+                      _first_difference(was["tja"], now["tja"]))
+                check(f"{tag}: every metric the old app reported is unchanged",
+                      same_shared and not dropped,
+                      f"{len(shared)} shared keys equal"
+                      + (f", added {added}" if added else "")
+                      + (f", DROPPED {dropped}" if dropped else "")
+                      if same_shared and not dropped else
+                      _dict_difference(was["metrics"], now["metrics"]))
+
+
+def _first_difference(a, b):
+    for i, (x, y) in enumerate(zip(a, b)):
+        if x != y:
+            return f"first differs at byte {i}: {a[i:i+40]!r} vs {b[i:i+40]!r}"
+    return f"lengths {len(a)} vs {len(b)}"
+
+
+def _dict_difference(a, b):
+    keys = sorted(set(a) | set(b))
+    return "; ".join(f"{k}: {a.get(k)!r} -> {b.get(k)!r}"
+                     for k in keys if a.get(k) != b.get(k))
+
+
+# ---------------------------------------------------------------------------
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("--songs", type=int, default=6,
+                        help="number of songs (default 6)")
+    parser.add_argument("--seconds", type=float, default=60.0,
+                        help="excerpt length in seconds (default 60)")
+    parser.add_argument("--audio-dir",
+                        help="use this directory of audio instead of the "
+                             "held-out dataset split")
+    parser.add_argument("--skip-legacy", action="store_true",
+                        help="skip the published-model regression")
+    parser.add_argument("--skip-sc2", action="store_true",
+                        help="skip the BarScript five-difficulty run")
+    parser.add_argument("--strict-ladder", action="store_true",
+                        help="fail if the generated ladder is not monotone")
+    parser.add_argument("--json", help="write a machine-readable report here")
+    args = parser.parse_args()
+
+    workdir = tempfile.mkdtemp(prefix="softchart-e2e-")
+    records = []
+    try:
+        songs, origin = build_fixtures(workdir, args.songs, args.seconds,
+                                       args.audio_dir)
+        if not songs:
+            print("no audio fixtures available: the taiko-1000-parsed test "
+                  "split is not in the local HuggingFace cache and no "
+                  "--audio-dir was given")
+            return 2
+        print(f"fixtures: {len(songs)} songs x {args.seconds:.0f}s from {origin}")
+        for slug, song in sorted(songs.items()):
+            print(f"  {slug}  {song['title'][:40]:42s} {song['bpm'] or '?'} BPM")
+
+        if not args.skip_sc2:
+            print("\n--- test_sc2_all_courses")
+            test_sc2_all_courses(songs, records, args.strict_ladder)
+        if not args.skip_legacy:
+            print("\n--- test_legacy_unchanged")
+            test_legacy_unchanged(songs, records)
+    finally:
+        import shutil
+        shutil.rmtree(workdir, ignore_errors=True)
+
+    failed = [n for n, ok, _ in RESULTS if not ok]
+    print(f"\n{len(RESULTS) - len(failed)}/{len(RESULTS)} checks passed")
+    for name in failed:
+        print(f"  FAILED: {name}")
+    if args.json:
+        with open(args.json, "w", encoding="utf-8") as f:
+            json.dump({"checks": [{"name": n, "ok": ok, "detail": d}
+                                  for n, ok, d in RESULTS],
+                       "records": records}, f, indent=1, ensure_ascii=False)
+        print(f"report -> {args.json}")
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

softchart/baseline.py

@@ -0,0 +1,43 @@
+"""Onset-detection baseline: spectral-flux peak picking on the cached log-mel
++ band-energy don/ka heuristic.
+
+This is the "chart generation is just onset detection" strawman the model must beat.
+Operates on the same cached mel features as the model, so no raw audio is needed.
+"""
+
+import numpy as np
+from scipy.signal import find_peaks
+
+from .vocab import FPS
+
+
+def baseline_chart(mel, target_nps):
+    """mel: (n_mels, T) log-mel (natural log). Returns list of (time_sec, class)."""
+    m = mel.astype(np.float32)
+    # spectral flux onset envelope
+    flux = np.maximum(0.0, np.diff(m, axis=1)).sum(axis=0)
+    flux = np.concatenate([[0.0], flux])
+    # smooth lightly
+    k = np.hanning(5)
+    k /= k.sum()
+    env = np.convolve(flux, k, mode="same")
+
+    dur = m.shape[1] / FPS
+    want = max(1, int(round(target_nps * dur)))
+    peaks, props = find_peaks(env, distance=int(0.09 * FPS), height=0.0)
+    if len(peaks) == 0:
+        return []
+    if len(peaks) > want:
+        strongest = np.argsort(props["peak_heights"])[-want:]
+        peaks = np.sort(peaks[strongest])
+
+    # don/ka: low vs high mel-band energy at the onset (linear power domain)
+    p = np.exp(m)
+    low = p[:40].sum(axis=0)   # ~ <500 Hz
+    high = p[64:].sum(axis=0)  # upper bands
+    events = []
+    for fr in peaks:
+        fr2 = min(fr + 1, m.shape[1] - 1)
+        ratio = high[fr2] / (low[fr2] + 1e-9)
+        events.append((float(fr / FPS), "ka" if ratio > 0.35 else "don"))
+    return events

softchart/data.py

@@ -0,0 +1,590 @@
+"""PyTorch dataset: (mel window, token sequence) pairs from the preprocessed cache."""
+
+import json
+import os
+import random
+
+import numpy as np
+import torch
+from torch.utils.data import Dataset
+
+from .vocab import (FPS, MAX_TGT, MEAS_MAX, NOTE_CLASSES, SLOTS, VOCAB, WINDOW,
+                    encode_window)
+
+G_CHUNK = 172  # ~2s per whole-song summary chunk
+G_LEN = 100  # fixed number of summary chunks (covers ~200s songs)
+
+
+def song_summary(mel):
+    """(n_mels, T) log-mel -> (G_LEN, n_mels) coarse whole-song summary."""
+    n_mels, T = mel.shape
+    n = min(G_LEN, max(1, T // G_CHUNK))
+    out = np.full((G_LEN, n_mels), np.log(1e-5), dtype=np.float32)
+    for k in range(n):
+        out[k] = mel[:, k * G_CHUNK : (k + 1) * G_CHUNK].astype(np.float32).mean(axis=1)
+    return out
+
+
+class ChartWindowDataset(Dataset):
+    """One item = one (song, course) chart; a random window is cropped per access."""
+
+    def __init__(
+        self,
+        cache_dir,
+        song_ids,
+        train=True,
+        cond_drop=0.15,
+        spec_augment=True,
+        windows_per_chart=8,
+        use_ctx=False,
+        aux=False,
+        global_ctx=False,
+        importance_sampling=False,
+        tempo_aug=False,
+        sibling=False,
+        style=False,
+        beat_head=False,
+        sync_token=False,
+        plan=False,
+        mask_infill=0.0,
+        complexity=False,
+        slot=False,
+        dual=False,
+        dual_slot_p=0.65,
+        beat_hires=False,
+    ):
+        self.cache = cache_dir
+        self.train = train
+        self.slot = slot
+        self.dual = dual  # mixed slot/time batches with an explicit mode token
+        self.dual_slot_p = dual_slot_p
+        self._meas_cache = {}
+        self.cond_drop = cond_drop if train else 0.0
+        self.spec_augment = spec_augment and train
+        self.use_ctx = use_ctx
+        self.aux = aux
+        self.global_ctx = global_ctx
+        self.importance = importance_sampling and train
+        self.tempo_aug = tempo_aug and train
+        self.sibling = sibling
+        self.beat_head = beat_head
+        self.beat_hires = beat_hires
+        self.sync_token = sync_token
+        self.mask_infill = mask_infill if train else 0.0
+        self.complexity = complexity
+        self.plans = None
+        if plan:
+            with open(os.path.join(cache_dir, "plans.json")) as f:
+                self.plans = json.load(f)
+        self.styles = None
+        if style:
+            with open(os.path.join(cache_dir, "styles.json")) as f:
+                self.styles = json.load(f)["styles"]
+        with open(os.path.join(cache_dir, "index.json")) as f:
+            index = {e["id"]: e for e in json.load(f)}
+        self.items = []  # (sid, course, level, n_frames)
+        for sid in song_ids:
+            e = index.get(sid)
+            if e is None:
+                continue
+            for c, info in e["courses"].items():
+                self.items.append((sid, c, info["level"], e["n_frames"]))
+        if self.slot and not self.dual:  # pure slot mode needs a grid per song
+            keep = []
+            for it in self.items:
+                src = it[0].split("x")[0] if "x" in it[0] else it[0]
+                if os.path.exists(os.path.join(cache_dir, f"{src}.beats.npz")):
+                    keep.append(it)
+            self.items = keep
+        # virtual epoch length: several windows per chart
+        self.mult = windows_per_chart if train else 1
+        self._rng = random.Random(int(os.environ.get('SC_DATA_SEED', 1234)))
+
+    def __len__(self):
+        return len(self.items) * self.mult
+
+    def _load(self, sid):
+        mel = np.load(os.path.join(self.cache, f"{sid}.mel.npy"), mmap_mode="r")
+        notes = np.load(os.path.join(self.cache, f"{sid}.notes.npz"))
+        return mel, notes
+
+    def _beats(self, sid):
+        try:
+            z = np.load(os.path.join(self.cache, f"{sid}.beats.npz"))
+            return z["beats"], z["downbeats"]
+        except Exception:
+            return None, None
+
+    def _measures(self, sid):
+        """Slot mode: (downbeat_times, beats_per_measure, ok_mask) — measure m
+        spans [db[m], db[m+1]). ok = sane meter (2..6 beats) and duration.
+        4/4-only models pass meter_filter=4 at window time."""
+        if sid in self._meas_cache:
+            return self._meas_cache[sid]
+        bts, dbs = self._beats(sid)
+        out = None
+        if dbs is not None and len(dbs) >= 3:
+            db = np.asarray(dbs, np.float64)
+            durs = np.diff(db)
+            if bts is not None and len(bts):
+                nb = np.searchsorted(bts, db[1:] - 1e-4) - np.searchsorted(bts, db[:-1] - 1e-4)
+            else:
+                nb = np.full(len(durs), 4)
+            ok = (nb >= 2) & (nb <= 6) & (durs > 0.5) & (durs < 8.0)
+            if ok.any():
+                out = (db, nb.astype(np.int64), ok)
+        self._meas_cache[sid] = out
+        return out
+
+    def _getitem_slot(self, i):
+        """Slot-token window: measure-aligned crop; note positions are exact
+        TJA lattice indices instead of audio frames. Lattice (v3) is
+        SLOTS_PER_BEAT per beat, so any meter is exact (4/4 -> 96/measure,
+        3/4 -> 72). Slot labels are tempo-INVARIANT, so tempo augmentation
+        only touches the mel. Two phase channels (measure/beat sawtooth) are
+        appended so the decoder reads the grid instead of inferring it."""
+        from .vocab import BEATS_MAX, SLOTS_PER_BEAT
+
+        sid, course, level, n_frames = self.items[i % len(self.items)]
+        src_sid = sid.split("x")[0] if "x" in sid else sid
+        tab = self._measures(src_sid)
+        if tab is None:  # unusable grid: deterministic redirect to another item
+            return self._getitem_slot((i + 9973) % len(self.items))
+        db, nbm, okm = tab
+        mel, notes = self._load(sid)
+        times = notes[f"{course}_t"]
+        classes = notes[f"{course}_c"]
+
+        rng0 = random.Random(i * 31337 + (self._rng.randint(0, 1 << 30) if self.train else 0))
+        rng = random.Random(i * 7919 + (self._rng.randint(0, 1 << 30) if self.train else 0))
+        r = 1.0
+        if self.tempo_aug and rng0.random() < 0.5:
+            r = rng0.uniform(0.9, 1.111)
+
+        # start measure: importance-weighted among valid measures
+        cand = np.nonzero(okm)[0]
+        cand = cand[db[cand] * FPS < max(n_frames - 1, 1)]
+        if len(cand) == 0:
+            return self._getitem_slot((i + 9973) % len(self.items))
+        if self.train:
+            if self.importance:
+                lo = np.searchsorted(times, db[cand])
+                hi = np.searchsorted(times, db[cand] + WINDOW / FPS * r)
+                w = (hi - lo).astype(np.float64) + 3.0
+                j = int(rng.choices(cand.tolist(), weights=w.tolist())[0])
+            else:
+                j = int(rng.choice(cand.tolist()))
+        else:
+            j = int(cand[(i * 2654435761) % len(cand)])
+
+        # extend K whole measures while valid, inside audio, and within both
+        # the frame window and the 72-beat token budget
+        K = 0
+        nbeats = 0
+        while (j + K < len(okm) and okm[j + K]
+               and nbeats + nbm[j + K] <= BEATS_MAX
+               and (db[j + K + 1] - db[j]) * FPS / r <= WINDOW
+               and db[j + K + 1] * FPS <= n_frames + FPS):
+            nbeats += nbm[j + K]
+            K += 1
+        if K == 0:
+            return self._getitem_slot((i + 9973) % len(self.items))
+        t0, t_end = db[j], db[j + K]
+        boff = np.concatenate([[0], np.cumsum(nbm[j : j + K])])  # beat offsets
+
+        def to_slot(t, m):
+            a, b = db[j + m], db[j + m + 1]
+            s = int(np.floor(SLOTS_PER_BEAT * nbm[j + m] * (t - a) / (b - a) + 0.5))
+            return min(boff[m] * SLOTS_PER_BEAT + s, WINDOW - 1)
+
+        # notes -> exact lattice indices; keep frame positions for the aux head
+        pairs, aux_frames = [], []
+        seen = set()
+        for m in range(K):
+            a, b = db[j + m], db[j + m + 1]
+            sel = (times >= a - 1e-4) & (times < b - 1e-4)
+            for t, c in zip(times[sel], classes[sel]):
+                g = to_slot(t, m)
+                if (g, int(c)) not in seen:
+                    seen.add((g, int(c)))
+                    pairs.append((g, int(c)))
+                    aux_frames.append((t - t0) / r * FPS)
+        pairs_sorted = sorted(range(len(pairs)), key=lambda k: pairs[k][0])
+        pairs = [pairs[k] for k in pairs_sorted]
+        aux_frames = [aux_frames[k] for k in pairs_sorted]
+
+        sib = None
+        if self.sibling:
+            order = ["easy", "normal", "hard", "oni", "ura"]
+            sib_pairs = []
+            if not (self.train and rng0.random() < 0.2):
+                for c2 in reversed(order[: order.index(course)]):
+                    if f"{c2}_t" in notes:
+                        t2, c2c = notes[f"{c2}_t"], notes[f"{c2}_c"]
+                        ev = []
+                        for m in range(K):
+                            a, b = db[j + m], db[j + m + 1]
+                            s2 = (t2 >= a - 1e-4) & (t2 < b - 1e-4) & (c2c <= 3)
+                            for t, c in zip(t2[s2], c2c[s2]):
+                                ev.append((to_slot(t, m), int(c)))
+                        if len(ev) > 12:
+                            idx2 = np.linspace(0, len(ev) - 1, 12).astype(int)
+                            ev = [ev[k] for k in idx2]
+                        sib_pairs = ev
+                        break
+            sib = sib_pairs
+
+        style = None
+        if self.styles is not None:
+            style = self.styles.get(f"{sid}:{course}", -1)
+
+        plan_slice = None
+        if self.plans is not None:
+            blocks = self.plans.get(f"{sid}:{course}") or []
+            plan_slice = [(b[2], b[3]) for b in blocks if b[1] > t0 and b[0] < t_end]
+
+        seq, prefix_len = encode_window(
+            VOCAB, course, level, pairs, cond_drop=self.cond_drop, rng=rng,
+            sib_pairs=sib, style=style, plan_slice=plan_slice,
+            mode=("slot" if self.dual else None),
+        )
+        if len(seq) > MAX_TGT:
+            seq = seq[: MAX_TGT - 1] + [VOCAB.eos]
+            if VOCAB.is_time(seq[-2]):
+                seq = seq[:-2] + [VOCAB.eos]
+
+        # mel crop [t0, t_end) -> stretch by r -> pad to WINDOW
+        f0 = int(round(t0 * FPS))
+        src_len = max(1, int(round((t_end - t0) * FPS)))
+        x = mel[:, f0 : f0 + src_len].astype(np.float32)
+        if x.shape[1] < src_len:
+            x = np.pad(x, ((0, 0), (0, src_len - x.shape[1])), constant_values=np.log(1e-5))
+        x = torch.from_numpy(x)
+        tgt_len = min(WINDOW, max(1, int(round(src_len / r))))
+        if tgt_len != src_len:
+            x = torch.nn.functional.interpolate(
+                x[None], size=tgt_len, mode="linear", align_corners=False)[0]
+        if x.shape[1] < WINDOW:
+            x = torch.nn.functional.pad(x, (0, WINDOW - x.shape[1]),
+                                        value=float(np.log(1e-5)))
+        if self.tempo_aug and rng0.random() < 0.3:
+            k = rng0.randint(-4, 4)
+            if k:
+                x = torch.roll(x, k, dims=0)
+                if k > 0:
+                    x[:k] = x.min()
+                else:
+                    x[k:] = x.min()
+        if self.spec_augment:
+            for _ in range(2):
+                w = rng.randint(0, 12)
+                fq = rng.randint(0, x.shape[0] - w) if w else 0
+                if w:
+                    x[fq : fq + w] = x.mean()
+            for _ in range(2):
+                w = rng.randint(0, 24)
+                fq = rng.randint(0, x.shape[1] - w) if w else 0
+                if w:
+                    x[:, fq : fq + w] = x.mean()
+            x = x + (rng.random() * 1.38 - 0.69)
+        # phase channels AFTER augmentation (the grid is clean metadata):
+        # ch0 = measure phase 0..1, ch1 = beat phase 0..1; -1 marks padding
+        ph = torch.full((2, WINDOW), -1.0)
+        edges = (db[j : j + K + 1] - t0) / r * FPS
+        for m in range(K):
+            a, b = edges[m], edges[m + 1]
+            i0, i1 = int(np.ceil(a - 1e-6)), min(int(np.ceil(b - 1e-6)), WINDOW)
+            if i1 <= i0:
+                continue
+            frac = (torch.arange(i0, i1, dtype=torch.float32) - a) / max(b - a, 1e-6)
+            ph[0, i0:i1] = frac
+            ph[1, i0:i1] = (frac * float(nbm[j + m])) % 1.0
+        x = torch.cat([x, ph], dim=0)
+
+        aux_t = torch.zeros(0)
+        if self.aux:
+            aux_t = torch.zeros(WINDOW // 4)
+            for f, (g, c) in zip(aux_frames, pairs):
+                if NOTE_CLASSES[c] != "end":
+                    aux_t[min(max(int(f), 0) // 4, WINDOW // 4 - 1)] = 1.0
+            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, 1))
+            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, -1))
+
+        beat_t = torch.zeros(0)
+        if self.beat_head:
+            # measured LEAK (v15a): slot windows carry grid-phase INPUT
+            # channels, so a beat head trained here just copies the channels
+            # and collapses at inference (p50 723 ms). Mask beat supervision
+            # (-1) on slot windows; the head learns from gridless time-mode
+            # windows in dual training instead.
+            res = WINDOW if self.beat_hires else WINDOW // 4
+            beat_t = torch.full((res, 2), -1.0)
+
+        in_t = torch.tensor(seq, dtype=torch.long)
+        return (x, torch.tensor(seq, dtype=torch.long), prefix_len, aux_t,
+                torch.zeros(0), 0, beat_t, in_t)
+
+    def __getitem__(self, i):
+        if self.dual:
+            sid = self.items[i % len(self.items)][0]
+            src = sid.split("x")[0] if "x" in sid else sid
+            rngm = random.Random(i * 104729 + (self._rng.randint(0, 1 << 30) if self.train else 0))
+            if self._measures(src) is not None and (not self.train or rngm.random() < self.dual_slot_p):
+                return self._getitem_slot(i)
+            # else: fall through to the time path (mode token + blank grid channels)
+        elif self.slot:
+            return self._getitem_slot(i)
+        sid, course, level, n_frames = self.items[i % len(self.items)]
+        mel, notes = self._load(sid)
+        times = notes[f"{course}_t"]
+        classes = notes[f"{course}_c"]
+
+        # rhythm-preserving tempo augmentation: crop a longer/shorter source
+        # window and resample it to WINDOW (mel-domain time stretch); note
+        # times scale by the same factor, so all rhythmic relations survive.
+        rng0 = random.Random(i * 31337 + (self._rng.randint(0, 1 << 30) if self.train else 0))
+        r = 1.0
+        if self.tempo_aug and rng0.random() < 0.5:
+            r = rng0.uniform(0.9, 1.111)
+        src_len = int(round(WINDOW * r))
+
+        max_start = max(0, n_frames - src_len)
+        rng = random.Random(i * 7919 + (self._rng.randint(0, 1 << 30) if self.train else 0))
+        if self.train and max_start > 0:
+            if self.importance:
+                # importance sampling: bias window starts toward note-dense /
+                # informative regions (uniform floor keeps sparse regions seen)
+                cands = np.arange(0, max_start + 1, int(FPS))  # 1s stride
+                lo = np.searchsorted(times, cands / FPS)
+                hi = np.searchsorted(times, cands / FPS + WINDOW / FPS)
+                w = (hi - lo).astype(np.float64) + 3.0
+                start = int(rng.choices(cands.tolist(), weights=w.tolist())[0])
+                start = min(max_start, start + rng.randint(0, int(FPS)))  # sub-second jitter
+            else:
+                start = rng.randint(0, max_start)
+        else:  # deterministic pseudo-random window per item (not always the intro)
+            start = (i * 2654435761) % (max_start + 1) if max_start > 0 else 0
+
+        t0 = start / FPS
+        t1 = (start + src_len) / FPS
+        sel = (times >= t0) & (times < t1)
+        frames = np.round((times[sel] - t0) * FPS / r).astype(np.int64)
+        frames = np.clip(frames, 0, WINDOW - 1)
+        pairs = list(zip(frames.tolist(), classes[sel].tolist()))
+
+        sib = None
+        if self.sibling:  # skeleton hint from the nearest easier course
+            order = ["easy", "normal", "hard", "oni", "ura"]
+            sib_pairs = []
+            if not (self.train and rng0.random() < 0.2):  # p=0.2: train without hint
+                for c2 in reversed(order[: order.index(course)]):
+                    if f"{c2}_t" in notes:
+                        t2, c2c = notes[f"{c2}_t"], notes[f"{c2}_c"]
+                        s2 = (t2 >= t0) & (t2 < t1) & (c2c <= 3)  # hits only
+                        f2 = np.clip(np.round((t2[s2] - t0) * FPS / r), 0, WINDOW - 1)
+                        ev = list(zip(f2.astype(np.int64).tolist(), c2c[s2].tolist()))
+                        if len(ev) > 12:  # even subsample to 12 slots
+                            idx2 = np.linspace(0, len(ev) - 1, 12).astype(int)
+                            ev = [ev[k] for k in idx2]
+                        sib_pairs = ev
+                        break
+            sib = sib_pairs
+
+        ctx = None
+        if self.use_ctx:
+            from .generate import CTX_LEN, HIT_CLASSES
+
+            hit_ids = [NOTE_CLASSES.index(c) for c in HIT_CLASSES]
+            prev = (times >= t0 - 3.0) & (times < t0)
+            tail = [int(c) for c in classes[prev] if int(c) in hit_ids][-CTX_LEN:]
+            if self.train and rng.random() < 0.1:
+                tail = []  # simulate missing context (first window)
+            elif self.train:
+                tail = [c if rng.random() > 0.1 else rng.choice(hit_ids) for c in tail]
+            ctx = [-1] * (CTX_LEN - len(tail)) + tail
+
+        style = None
+        if self.styles is not None:
+            style = self.styles.get(f"{sid}:{course}", -1)
+
+        sync_band = None
+        beats_w = db_w = None
+        if self.sync_token or self.beat_head:
+            src_sid = sid.split("x")[0] if "x" in sid else sid
+            bts, dbs = self._beats(src_sid)
+            if bts is not None:
+                # window-local, tempo-scaled beat grid
+                bsel = (bts >= t0) & (bts < t1)
+                beats_w = (bts[bsel] - t0) / r
+                dsel = (dbs >= t0) & (dbs < t1)
+                db_w = (dbs[dsel] - t0) / r
+        if self.sync_token:
+            sync_band = lhl_band(
+                np.array([f / FPS for f, c in pairs if NOTE_CLASSES[c] != "end"]),
+                beats_w, db_w)
+
+        plan_slice = None
+        if self.plans is not None:
+            blocks = self.plans.get(f"{sid}:{course}") or []
+            # blocks overlapping this window; times already in source timescale,
+            # window is [t0, t1) in the same scale (r rescales note frames only,
+            # block membership is decided in source time)
+            plan_slice = [(b[2], b[3]) for b in blocks if b[1] > t0 and b[0] < t1]
+
+        cplx = None
+        if self.complexity:
+            from .vocab import complexity_band
+            cplx = complexity_band(pairs)
+
+        seq, prefix_len = encode_window(
+            VOCAB, course, level, pairs, cond_drop=self.cond_drop, rng=rng,
+            ctx_types=ctx, sib_pairs=sib, style=style, sync_band=sync_band,
+            plan_slice=plan_slice, complexity=cplx,
+            mode=("time" if self.dual else None),
+        )
+        in_seq = None
+        if self.mask_infill > 0 and rng.random() < self.mask_infill:
+            # skeleton->color curriculum: decoder INPUT sees MASK where note
+            # types were; gold targets keep the true types
+            note_ids = set(VOCAB.note.values())
+            in_seq = [VOCAB.mask if (i >= prefix_len and tok in note_ids) else tok
+                      for i, tok in enumerate(seq)]
+        if len(seq) > MAX_TGT:  # truncate overly dense windows, keep EOS
+            seq = seq[: MAX_TGT - 1] + [VOCAB.eos]
+            if VOCAB.is_time(seq[-2]):  # do not end on a dangling TIME token
+                seq = seq[:-2] + [VOCAB.eos]
+
+        x = mel[:, start : start + src_len].astype(np.float32)
+        if x.shape[1] < src_len:
+            x = np.pad(x, ((0, 0), (0, src_len - x.shape[1])), constant_values=np.log(1e-5))
+        x = torch.from_numpy(x)
+        if src_len != WINDOW:  # mel-domain time stretch to the fixed window size
+            x = torch.nn.functional.interpolate(
+                x[None], size=WINDOW, mode="linear", align_corners=False
+            )[0]
+        if self.tempo_aug and rng0.random() < 0.3:  # mild pitch shift (mel-bin roll)
+            k = rng0.randint(-4, 4)
+            if k:
+                x = torch.roll(x, k, dims=0)
+                if k > 0:
+                    x[:k] = x.min()
+                else:
+                    x[k:] = x.min()
+        if self.spec_augment:
+            for _ in range(2):  # freq masks
+                w = rng.randint(0, 12)
+                f0 = rng.randint(0, x.shape[0] - w) if w else 0
+                if w:
+                    x[f0 : f0 + w] = x.mean()
+            for _ in range(2):  # time masks
+                w = rng.randint(0, 24)
+                f0 = rng.randint(0, x.shape[1] - w) if w else 0
+                if w:
+                    x[:, f0 : f0 + w] = x.mean()
+            x = x + (rng.random() * 1.38 - 0.69)  # gain +-6 dB in log space
+        if self.dual:  # time mode carries no grid: phase channels = -1
+            x = torch.cat([x, torch.full((2, WINDOW), -1.0)], dim=0)
+
+        aux_t = torch.zeros(0)
+        if self.aux:  # onset heatmap at encoder resolution (WINDOW // 4)
+            aux_t = torch.zeros(WINDOW // 4)
+            for f, c in pairs:
+                if NOTE_CLASSES[c] != "end":
+                    aux_t[min(f // 4, WINDOW // 4 - 1)] = 1.0
+            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, 1))
+            aux_t = torch.max(aux_t, 0.5 * torch.roll(aux_t, -1))
+
+        beat_t = torch.zeros(0)
+        if self.beat_head:
+            res = WINDOW if self.beat_hires else WINDOW // 4
+            div = 1 if self.beat_hires else 4
+            beat_t = torch.zeros(res, 2)
+            for arr, ch in ((beats_w, 0), (db_w, 1)):
+                if arr is None:
+                    continue
+                for bt in arr:
+                    k = int(bt * FPS) // div
+                    if 0 <= k < res:
+                        beat_t[k, ch] = 1.0
+            for ch in range(2):
+                col = beat_t[:, ch]
+                beat_t[:, ch] = torch.max(col, 0.5 * torch.roll(col, 1))
+                beat_t[:, ch] = torch.max(beat_t[:, ch], 0.5 * torch.roll(col, -1))
+
+        gsum = torch.zeros(0)
+        posb = 0
+        if self.global_ctx:
+            gsum = torch.from_numpy(song_summary(np.asarray(mel)))
+            posb = min(15, int(16 * start / max(n_frames, 1)))
+
+        in_t = torch.tensor(in_seq if in_seq is not None else seq, dtype=torch.long)
+        return (x, torch.tensor(seq, dtype=torch.long), prefix_len, aux_t, gsum, posb,
+                beat_t, in_t)
+
+
+def lhl_band(note_times, beats, downbeats, band_width=0.6):
+    """Longuet-Higgins & Lee (1984) style syncopation, simplified to the
+    eighth-note grid, normalized per bar, bucketed into 6 bands."""
+    if beats is None or len(beats) < 4 or len(note_times) < 2:
+        return -1
+    beat_len = float(np.median(np.diff(beats))) if len(beats) > 1 else 0.5
+    grid = []  # (time, metrical level)
+    dbset = set(np.round(downbeats, 3)) if downbeats is not None else set()
+    for b in beats:
+        lvl = 3 if round(float(b), 3) in dbset else 2
+        grid.append((float(b), lvl))
+        grid.append((float(b) + beat_len / 2, 1))  # eighth positions
+    grid.sort()
+    tol = 0.04
+    score = 0.0
+    for i, (gt_pos, lvl) in enumerate(grid[:-1]):
+        has_note = np.any(np.abs(note_times - gt_pos) < tol)
+        nxt_pos, nxt_lvl = grid[i + 1]
+        nxt_note = np.any(np.abs(note_times - nxt_pos) < tol)
+        if has_note and not nxt_note and nxt_lvl > lvl:
+            score += nxt_lvl - lvl
+    n_bars = max(1.0, len([g for g in grid if g[1] == 3]))
+    return min(5, int(score / n_bars / band_width))
+
+
+def collate(batch):
+    xs, seqs, plens, auxs, gsums, posbs, beats, inseqs = zip(*batch)
+    x = torch.stack(xs)
+    maxlen = max(len(s) for s in seqs)
+    tgt = torch.full((len(seqs), maxlen), VOCAB.pad, dtype=torch.long)
+    loss_mask = torch.zeros((len(seqs), maxlen), dtype=torch.bool)
+    for i, (s, pl) in enumerate(zip(seqs, plens)):
+        tgt[i, : len(s)] = s
+        loss_mask[i, pl : len(s)] = True  # loss on events + EOS only
+    aux = torch.stack(auxs) if auxs[0].numel() else None
+    gsum = torch.stack(gsums) if gsums[0].numel() else None
+    posb = torch.tensor(posbs, dtype=torch.long)
+    beat = torch.stack(beats) if beats[0].numel() else None
+    in_tgt = torch.full((len(inseqs), maxlen), VOCAB.pad, dtype=torch.long)
+    for i, sq in enumerate(inseqs):
+        in_tgt[i, : len(sq)] = sq
+    return x, tgt, loss_mask, aux, gsum, posb, beat, in_tgt
+
+
+def load_split_ids(cache_dir, val_songs=40, seed=42, use_aug=False):
+    """Song-level split: train / val (carved from train) / test (provided).
+
+    Augmented variants (speed-aug entries with "aug": true) are never used for
+    val, and variants whose source song is in val are excluded from train
+    (no leakage). The base split is stable regardless of augmentation.
+    """
+    with open(os.path.join(cache_dir, "index.json")) as f:
+        index = json.load(f)
+    base_train = sorted(e["id"] for e in index
+                        if e["split"] == "train" and not e.get("aug"))
+    test_ids = sorted(e["id"] for e in index if e["split"] == "test")
+    rng = random.Random(seed)
+    rng.shuffle(base_train)
+    val_ids = base_train[:val_songs]
+    train_ids = base_train[val_songs:]
+    if use_aug:
+        val_set = set(val_ids)
+        train_ids = train_ids + sorted(
+            e["id"] for e in index
+            if e.get("aug") and e.get("src") not in val_set)
+    return train_ids, val_ids, test_ids

softchart/evaluate.py

@@ -0,0 +1,151 @@
+"""Objective evaluation metrics for generated charts."""
+
+import numpy as np
+
+HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
+
+
+def match_onsets(ref_times, est_times, tol=0.05):
+    """Greedy bipartite matching (mir_eval style). Returns list of (ri, ei)."""
+    matches = []
+    ri, ei = 0, 0
+    used_r = set()
+    # standard approach: for each est, match nearest unmatched ref within tol
+    ref = np.asarray(ref_times)
+    est = np.asarray(est_times)
+    order = np.argsort(est)
+    ref_order = np.argsort(ref)
+    ref_sorted = ref[ref_order]
+    taken = np.zeros(len(ref), dtype=bool)
+    for e_i in order:
+        t = est[e_i]
+        lo = np.searchsorted(ref_sorted, t - tol)
+        hi = np.searchsorted(ref_sorted, t + tol)
+        best, best_d = -1, tol + 1
+        for k in range(lo, hi):
+            rid = ref_order[k]
+            if taken[rid]:
+                continue
+            d = abs(ref_sorted[k] - t)
+            if d < best_d:
+                best, best_d = rid, d
+        if best >= 0:
+            taken[best] = True
+            matches.append((best, e_i))
+    return matches
+
+
+def onset_prf(ref_times, est_times, tol=0.05):
+    if len(ref_times) == 0 and len(est_times) == 0:
+        return 1.0, 1.0, 1.0
+    if len(ref_times) == 0 or len(est_times) == 0:
+        return 0.0, 0.0, 0.0
+    m = len(match_onsets(ref_times, est_times, tol))
+    p = m / len(est_times)
+    r = m / len(ref_times)
+    f = 2 * p * r / (p + r) if p + r > 0 else 0.0
+    return p, r, f
+
+
+def type_accuracy(ref, est, tol=0.05):
+    """ref/est: list of (time, class). Accuracy + per-class stats on matched onsets."""
+    if not ref or not est:
+        return None
+    rt = [t for t, _ in ref]
+    et = [t for t, _ in est]
+    matches = match_onsets(rt, et, tol)
+    if not matches:
+        return None
+    correct = sum(1 for ri, ei in matches if ref[ri][1] == est[ei][1])
+    per_class = {}
+    for ri, ei in matches:
+        c = ref[ri][1]
+        d = per_class.setdefault(c, [0, 0])
+        d[1] += 1
+        if est[ei][1] == c:
+            d[0] += 1
+    # binary don/ka accuracy (big variants folded in)
+    fold = lambda c: "don" if "don" in c else ("ka" if "ka" in c else c)
+    correct2 = sum(1 for ri, ei in matches if fold(ref[ri][1]) == fold(est[ei][1]))
+    return {
+        "acc": correct / len(matches),
+        "acc_donka": correct2 / len(matches),
+        "n_matched": len(matches),
+        "per_class": {c: v[0] / v[1] for c, v in per_class.items()},
+    }
+
+
+def density(events, span=None):
+    times = [t for t, c in events if c in HIT_CLASSES]
+    if len(times) < 2:
+        return 0.0
+    span = span or (max(times) - min(times))
+    return len(times) / max(span, 1e-6) if span > 5 else 0.0
+
+
+def ioi_hist(events, bpm, bins=None):
+    """Inter-onset intervals in beat units, snapped to musical fractions."""
+    if bins is None:
+        bins = [0.125, 1 / 6, 0.25, 1 / 3, 0.5, 2 / 3, 0.75, 1.0, 1.5, 2.0, 3.0, 4.0]
+    times = sorted(t for t, c in events if c in HIT_CLASSES)
+    if len(times) < 3 or not bpm or bpm <= 0:
+        return None
+    ioi = np.diff(times) * bpm / 60.0  # beats
+    hist = np.zeros(len(bins) + 1)
+    for x in ioi:
+        d = [abs(x - b) / b for b in bins]
+        j = int(np.argmin(d))
+        if d[j] < 0.2:
+            hist[j] += 1
+        else:
+            hist[-1] += 1  # off-grid
+    s = hist.sum()
+    return hist / s if s > 0 else None
+
+
+def js_divergence(p, q, eps=1e-9):
+    p = np.asarray(p) + eps
+    q = np.asarray(q) + eps
+    p, q = p / p.sum(), q / q.sum()
+    m = (p + q) / 2
+    kl = lambda a, b: float(np.sum(a * np.log(a / b)))
+    return 0.5 * kl(p, m) + 0.5 * kl(q, m)
+
+
+def ngram_dist(events, n=2):
+    """Distribution over note-class n-grams (hits only)."""
+    seq = [c for _, c in sorted(events) if c in HIT_CLASSES]
+    if len(seq) < n + 1:
+        return {}
+    counts = {}
+    for i in range(len(seq) - n + 1):
+        g = tuple(seq[i : i + n])
+        counts[g] = counts.get(g, 0) + 1
+    tot = sum(counts.values())
+    return {g: c / tot for g, c in counts.items()}
+
+
+def ngram_js(ref_events, est_events, n=2):
+    pr = ngram_dist(ref_events, n)
+    pe = ngram_dist(est_events, n)
+    if not pr or not pe:
+        return None
+    keys = sorted(set(pr) | set(pe))
+    return js_divergence([pr.get(k, 0) for k in keys], [pe.get(k, 0) for k in keys])
+
+
+def distinct_n(events, n=3):
+    seq = [c for _, c in sorted(events) if c in HIT_CLASSES]
+    if len(seq) < n:
+        return None
+    grams = [tuple(seq[i : i + n]) for i in range(len(seq) - n + 1)]
+    return len(set(grams)) / len(grams)
+
+
+def spearman(x, y):
+    from scipy.stats import spearmanr
+
+    if len(x) < 2:
+        return None
+    r = spearmanr(x, y).statistic
+    return None if np.isnan(r) else float(r)

softchart/generate.py

@@ -12,26 +12,9 @@ Features:
 import numpy as np
 import torch
 
-from .checkpoint import (attach_generator_capabilities,
-                         parse_generator_checkpoint)
-from .meter import resolve_meter_layout
 from .model import ChartModel
 from .vocab import FPS, MAX_TGT, N_LEVELS, NOTE_CLASSES, VOCAB, WINDOW
 
-G_LEN = 100
-
-
-def song_summary(mel):
-    """Summarize a complete song into fixed relative-time sections."""
-    n_mels, frames = mel.shape
-    if frames < G_LEN:
-        raise ValueError(f"song has only {frames} mel frames; need at least {G_LEN}")
-    edges = np.linspace(0, frames, G_LEN + 1, dtype=np.int64)
-    summary = np.empty((G_LEN, n_mels), dtype=np.float32)
-    for index, (start, end) in enumerate(zip(edges[:-1], edges[1:])):
-        summary[index] = mel[:, start:end].astype(np.float32).mean(axis=1)
-    return summary
-
 HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
 SPAN_CLASSES = ("roll", "roll_big", "balloon")
 
@@ -50,15 +33,73 @@ def _autocast(device):
 CTX_LEN = 12  # fixed-length previous-window context (note-type tokens)
 MIN_GAP_FRAMES = 4  # ~46 ms minimum inter-note gap enforced during decoding
 LATTICE_RATIOS = (1 / 3, 0.5, 2 / 3, 1.0, 4 / 3, 1.5, 2.0, 3.0)  # allowed IOI ratios
+# GT span-length p99 per class (train stats); generated spans beyond this are
+# truncated — a visual audit found generated balloons up to ~10s vs GT p50 ~1s
+SPAN_MAX = {"roll": 3.5, "roll_big": 4.0, "balloon": 6.5}
 
 
 def load_model(ckpt_path, device="cuda"):
-    checkpoint = torch.load(ckpt_path, map_location=device, weights_only=False)
-    state, _args, config, capabilities = parse_generator_checkpoint(checkpoint)
-    model = ChartModel(**config).to(device)
-    model.load_state_dict(state, strict=True)
+    ck = torch.load(ckpt_path, map_location=device)
+    sd = ck["model"]
+    vocab_size = sd["tok_emb.weight"].shape[0]
+    aux = any(k.startswith("aux") for k in sd)
+    gctx = any(k.startswith("gsum_proj") for k in sd)
+    a = ck.get("args", {}) or {}
+    front_ch = sd["frontend.0.weight"].shape[1]  # 128 (clean) or 130 (legacy slot)
+    emb_w = sd["tok_emb.weight"].shape[1]
+    d_model = a.get("d_model", 512)
+    # clean_phase (v1.6): encoder frontend is 128ch and a phase_proj is present.
+    clean_phase = bool(a.get("clean_phase", False)) or ("phase_proj.0.weight" in sd)
+    # legacy slot models fed phase to the encoder (in_ch=130); clean models keep
+    # in_ch=130 at the data level but only 128 reach the frontend.
+    in_ch = 130 if (clean_phase or front_ch == 130) else front_ch
+    # v1.7 arch flags: prefer saved args; fall back to state-dict keys so
+    # args-less checkpoints of the new arch still load
+    adapter_rank_ffn = int(a.get("adapter_rank_ffn", 0) or 0)
+    if not adapter_rank_ffn and "decoder.ffn_adapters.0.down.weight" in sd:
+        adapter_rank_ffn = sd["decoder.ffn_adapters.0.down.weight"].shape[0]
+    depth_emb = bool(a.get("depth_emb", False)) or ("decoder.depth_emb" in sd)
+    model = ChartModel(
+        d_model=d_model, nhead=a.get("nhead", 8),
+        enc_layers=a.get("enc_layers", 6),
+        dec_layers=a.get("dec_layers", 6), ffn=a.get("ffn", 2048),
+        vocab_size=vocab_size, aux=aux, global_ctx=gctx,
+        func_time=a.get("func_time", False), in_ch=in_ch,
+        emb_factor=emb_w if emb_w != d_model else None,
+        clean_phase=clean_phase,
+        enc_share=a.get("enc_share") or None,
+        dec_share=a.get("dec_share") or None,
+        adapter_rank=int(a.get("adapter_rank", 0) or 0),
+        unique_layernorm=bool(a.get("unique_layernorm", False)),
+        adapter_rank_ffn=adapter_rank_ffn, depth_emb=depth_emb,
+        unshare_last_dec=bool(a.get("unshare_last_dec", False)),
+    ).to(device)
+    if any(k.startswith("ptr") for k in sd):
+        model.enable_ptr()
+    if any(k.startswith("beat") for k in sd):
+        model.enable_beat(hires=any(k.startswith("beat.up") for k in sd))
+    model.to(device)  # newly enabled heads default to CPU
+    model.load_state_dict(sd)
     model.eval()
-    attach_generator_capabilities(model, capabilities)
+    # capability detection: prefer explicit training args (vocab-size inference
+    # wrongly added prefix slots to models that never trained with them)
+    if a:
+        model._has_ctx = bool(a.get("ctx", False))
+        model._has_sib = bool(a.get("sibling", False))
+        model._has_style = bool(a.get("style", False))
+        model._has_sync = bool(a.get("sync_token", False))
+        model._has_plan = bool(a.get("plan", False))
+        model._dual = bool(a.get("dual", False))
+        model._slot = bool(a.get("slot", False)) or model._dual
+    else:  # legacy fallback
+        model._has_ctx = vocab_size > VOCAB.sep
+        model._has_sib = vocab_size > VOCAB.sib
+        model._has_style = False
+        model._has_sync = False
+        model._has_plan = False
+        model._slot = False
+        model._dual = False
+    model._has_gctx = gctx
     return model
 
 
@@ -69,23 +110,20 @@ def load_hf(repo_or_dir, device="cuda"):
 
     hf = SoftChartGenerator.from_pretrained(repo_or_dir).to(device).eval()
     m = hf.net
-    caps = hf.capabilities
-    if not isinstance(caps, dict):
-        raise ValueError("HF generator capabilities must be an explicit mapping")
-    attach_generator_capabilities(m, caps)
-    m._softchart_capabilities = dict(caps)
-    if (m.section_encoder is not None) != m._hierarchical_ctx:
-        raise ValueError("HF hierarchy capability disagrees with its architecture")
-    if bool(m.gsum_proj is not None) != bool(caps.get("global_ctx", False)):
-        # New V1.8 artifacts include global_ctx explicitly.  Requiring it here
-        # prevents a semantically different flat-global model from loading as
-        # a local-only generator.
-        raise ValueError("HF global-context capability disagrees with architecture")
+    caps = hf.capabilities or {}
+    m._has_ctx = bool(caps.get("ctx", False))
+    m._has_sib = bool(caps.get("sibling", False))
+    m._has_style = bool(caps.get("style", False))
+    m._has_sync = bool(caps.get("sync_token", False))
+    m._has_plan = bool(caps.get("plan", False))
+    m._dual = bool(caps.get("dual", False))
+    m._slot = bool(caps.get("slot", False)) or m._dual
+    m._has_gctx = m.gsum_proj is not None
     return m
 
 
 def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
-                 sib_pairs=None, style=None, sync_band=None,
+                 uncond=False, sib_pairs=None, style=None, sync_band=None,
                  plan_slice=None, mode=None):
     v = VOCAB
     seq = [v.bos]
@@ -104,11 +142,14 @@ def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
             cls = c if isinstance(c, int) else NOTE_CLASSES.index(c)
             seq += [v.time(int(f)), v.note[NOTE_CLASSES[cls]]]
         seq += [v.unk_cond] * (2 * (SIB_EVENTS - len(pairs)))
-    seq += [
-        v.course[course] if course else v.unk_cond,
-        v.level[max(1, min(N_LEVELS, level))] if level else v.unk_cond,
-        v.dens[density_bucket] if density_bucket is not None else v.unk_cond,
-    ]
+    if uncond:
+        seq += [v.unk_cond] * 3
+    else:
+        seq += [
+            v.course[course] if course else v.unk_cond,
+            v.level[max(1, min(N_LEVELS, level))] if level else v.unk_cond,
+            v.dens[density_bucket] if density_bucket is not None else v.unk_cond,
+        ]
     if style is not None:  # -1 = style-capable model, no specific style requested
         seq.append(v.style[style] if style >= 0 else v.unk_cond)
     if sync_band is not None:
@@ -124,38 +165,10 @@ def build_prefix(course=None, level=None, density_bucket=None, ctx_types=None,
     return seq
 
 
-def _conditioning_token_ids():
-    """Token ids whose *values* CFG is allowed to remove.
-
-    Structural markers (BOS/SEP/SIB/PLAN/mode), prior-window note context, and
-    sibling-chart events deliberately stay intact.  Positional masking such as
-    ``prefix[:, -n:]`` is wrong as soon as a variable-size plan or another
-    field follows the basic course/level/density tuple.
-    """
-    fields = (
-        VOCAB.course, VOCAB.level, VOCAB.dens, VOCAB.style, VOCAB.sync,
-        VOCAB.cplx, VOCAB.pdens, VOCAB.pflag,
-    )
-    return frozenset(tok for field in fields for tok in field.values())
-
-
-_CONDITIONING_IDS = _conditioning_token_ids()
-
-
-def _make_unconditional_prefixes(prefixes):
-    """Return CFG prefixes with named condition values replaced by UNK."""
-    unc = prefixes.clone()
-    is_condition = torch.zeros_like(unc, dtype=torch.bool)
-    for tok in _CONDITIONING_IDS:
-        is_condition |= unc == tok
-    return unc.masked_fill(is_condition, VOCAB.unk_cond)
-
-
 @torch.no_grad()
 def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=0.95,
                    greedy=False, seed=0, cfg_w=0.0, gsum=None, pos_buckets=None,
-                   lattice=False, min_gap=MIN_GAP_FRAMES,
-                   position_limits=None):
+                   lattice=False, n_cond=3, min_gap=MIN_GAP_FRAMES):
     """mels: (B, n_mels, WINDOW); prefixes: list of B equal-length token lists.
     Returns list of lists of (frame, note_class)."""
     v = VOCAB
@@ -180,21 +193,11 @@ def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=
 
     seqs = torch.tensor(prefixes, dtype=torch.long, device=device)
     if use_cfg:  # rows B..2B: same ctx, conditions replaced by UNK
-        unc = _make_unconditional_prefixes(seqs)
+        unc = seqs.clone()
+        unc[:, -n_cond:] = v.unk_cond
         seqs = torch.cat([seqs, unc], dim=0)
         memory = torch.cat([memory, memory], dim=0)
 
-    if position_limits is None:
-        position_limits = torch.full((B,), WINDOW, dtype=torch.long, device=device)
-    else:
-        position_limits = torch.as_tensor(position_limits, dtype=torch.long,
-                                          device=device)
-        if position_limits.shape != (B,):
-            raise ValueError(f"position_limits must have shape ({B},), got "
-                             f"{tuple(position_limits.shape)}")
-        if ((position_limits < 1) | (position_limits > WINDOW)).any():
-            raise ValueError(f"position_limits must be within [1, {WINDOW}]")
-
     done = torch.zeros(B, dtype=torch.bool, device=device)
     last_time = torch.full((B,), -min_gap, dtype=torch.long, device=device)
     expect_note = torch.zeros(B, dtype=torch.bool, device=device)
@@ -220,8 +223,7 @@ def decode_windows(model, mels, prefixes, device="cuda", temperature=1.0, top_p=
             time_pos = torch.arange(WINDOW, device=device).unsqueeze(0)
             sub = torch.full((len(idx), logits.shape[1]), float("-inf"), device=device)
             sub[:, v.eos] = 0.0
-            ok = ((time_pos >= (last_time[idx] + min_gap).unsqueeze(1))
-                  & (time_pos < position_limits[idx].unsqueeze(1)))
+            ok = time_pos >= (last_time[idx] + min_gap).unsqueeze(1)
             tmask = torch.where(
                 ok, torch.zeros_like(sub[:, :WINDOW]),
                 torch.full_like(sub[:, :WINDOW], float("-inf")),
@@ -315,19 +317,13 @@ def _decode_pass(model, wins, starts, course, level, density_bucket, ctx_lists,
         g = pb = None
         if gsum is not None:
             g = gsum.unsqueeze(0).expand(chunk.shape[0], -1, -1)
-            n_pos = (100 if getattr(model, "_hierarchical_ctx", False)
-                     else getattr(getattr(model, "pos_emb", None),
-                                  "num_embeddings", 16))
             pb = torch.tensor(
-                [min(n_pos - 1, int(n_pos * starts[i + j] / max(T, 1)))
-                 for j in range(chunk.shape[0])],
+                [min(15, int(16 * starts[i + j] / max(T, 1))) for j in range(chunk.shape[0])],
                 dtype=torch.long)
-        limits = [min(WINDOW, max(1, int(T) - starts[i + j]))
-                  for j in range(chunk.shape[0])]
         evs = decode_windows(model, chunk, prefixes, device=device, greedy=greedy,
                              temperature=temperature, top_p=top_p, seed=seed + i,
                              cfg_w=cfg_w, gsum=g, pos_buckets=pb, lattice=lattice,
-                             min_gap=MIN_GAP_FRAMES, position_limits=limits)
+                             n_cond=4 if style is not None else 3)
         for j, events in enumerate(evs):
             per_window.append(events)
             t_off = starts[i + j] / FPS
@@ -347,7 +343,7 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     if isinstance(mel, np.ndarray):
         mel = torch.from_numpy(mel.astype(np.float32))
     T = mel.shape[1]
-    starts = list(range(0, max(T, 1), WINDOW))
+    starts = list(range(0, max(T - 1, 1), WINDOW))
     dual = getattr(model, "_dual", False)
     wins = []
     for s in starts:
@@ -371,7 +367,9 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     plan_default = getattr(model, "_has_plan", False) and plan is None
     gsum = None
     if getattr(model, "_has_gctx", False):
-        gsum = torch.from_numpy(song_summary(mel.detach().cpu().numpy()))
+        from .data import song_summary
+
+        gsum = torch.from_numpy(song_summary(mel.numpy()))
 
     all_events, per_window = _decode_pass(
         model, wins, starts, course, level, density_bucket,
@@ -394,6 +392,29 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
             sync_band=sync_val, plan_blocks=plan_blocks, plan_default=plan_default,
             mode=("time" if dual else None))
 
+    if True:
+        # rescue decoding: decoding occasionally EOSes a whole window early; if a
+        # window is near-empty while its audio is musically active, redo it with
+        # the opposite mode (sampling was empty -> greedy; greedy was empty ->
+        # sampled retry, since a greedy redo would reproduce the same output)
+        flux = [float(np.maximum(0, np.diff(w.numpy(), axis=1)).sum()) for w in wins]
+        med = float(np.median(flux)) if flux else 0.0
+        med_hits = float(np.median([len(ev) for ev in per_window])) if per_window else 0.0
+        retry = [i for i, ev in enumerate(per_window)
+                 if (len(ev) < 4 and flux[i] > 0.3 * med)
+                 or (len(ev) < 0.4 * med_hits and flux[i] > 0.7 * med)]
+        if retry:
+            r_evs, _ = _decode_pass(
+                model, [wins[i] for i in retry], [starts[i] for i in retry],
+                course, level, density_bucket, None, device, not greedy, 0.9,
+                top_p, seed + 7, 0.0, batch_windows, gsum=gsum, T=T,
+                lattice=lattice, sib_default=sib_default, style=style_val,
+                sync_band=sync_val, plan_blocks=plan_blocks, plan_default=plan_default,
+                mode=("time" if dual else None))
+            all_events = [e for i, w_ev in enumerate(per_window) if i not in retry
+                          for e in [(starts[i] / FPS + f / FPS, c) for f, c in w_ev]]
+            all_events += r_evs
+
     all_events.sort(key=lambda e: e[0])
     hits, spans = [], []
     open_span = None
@@ -401,10 +422,15 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
     for t, cls in all_events:
         if cls in HIT_CLASSES:
             if open_span is not None:
-                # Missing END is malformed decoder output.  Drop the orphaned
-                # span and preserve the authored hit; inventing an endpoint or
-                # swallowing hits would silently rewrite the sequence.
-                open_span = None
+                # a span whose end never arrived would swallow every following
+                # hit (visual audit: 20 s of silence) — force-close at SPAN_MAX
+                if t - open_span[0] > SPAN_MAX.get(open_span[1], 6.5):
+                    spans.append({"t0": round(open_span[0], 4),
+                                  "t1": round(open_span[0] + SPAN_MAX.get(open_span[1], 6.5), 4),
+                                  "type": open_span[1]})
+                    open_span = None
+                else:
+                    continue  # no hits inside an open span
             if t - last_hit_t < 0.025:
                 continue
             hits.append({"t": round(t, 4), "type": cls})
@@ -413,102 +439,52 @@ def generate_song(model, mel, course, level=None, density_bucket=None, device="c
             if open_span is None:
                 open_span = (t, cls)
         elif cls == "end":
-            if open_span is not None:
-                if t - open_span[0] > 0.05:
-                    spans.append({"t0": round(open_span[0], 4), "t1": round(t, 4),
-                                  "type": open_span[1]})
-                # END always consumes the open span.  A too-short span is
-                # malformed and dropped; keeping it open would let a later END
-                # close the wrong start and swallow subsequent span starts.
+            if open_span is not None and t - open_span[0] > 0.05:
+                t1_span = min(t, open_span[0] + SPAN_MAX.get(open_span[1], 6.5))
+                spans.append({"t0": round(open_span[0], 4), "t1": round(t1_span, 4),
+                              "type": open_span[1]})
                 open_span = None
     return {"hits": hits, "spans": spans, "course": course, "level": level,
             "density_bucket": density_bucket}
 
 
-def _grid_edges(grid, downbeats):
-    """Return explicitly supplied measure edges without inventing a tail."""
-    if len(downbeats) == 0:
-        raise ValueError("slot generation requires at least one downbeat")
-    if not np.all(np.isfinite(downbeats)) or (np.diff(downbeats) <= 0).any():
-        raise ValueError("grid downbeats must be finite and strictly increasing")
-    if "measure_edges" not in grid:
-        raise ValueError(
-            "slot generation requires explicit measure_edges (starts plus a "
-            "trusted terminal edge); terminal duration is never extrapolated"
-        )
-    edges = np.asarray(grid["measure_edges"], dtype=np.float64)
-    if (len(edges) != len(downbeats) + 1
-            or not np.allclose(edges[:-1], downbeats, rtol=0, atol=1e-6)
-            or not np.all(np.isfinite(edges))
-            or (np.diff(edges) <= 0).any()):
-        raise ValueError("grid measure_edges must be valid starts plus one end")
-    return edges
-
-
 @torch.no_grad()
 def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None,
                        device="cuda", greedy=False, temperature=1.0, top_p=0.95,
                        seed=0, batch_windows=8, plan=None, on_progress=None):
-    """Meter-aware generation on the exact 96-slots-per-4/4 TJA lattice.
+    """Slot-mode generation: windows are anchored at the fitted barlines and the
+    decoder emits exact TJA lattice indices (measure*96 + slot). No
+    quantization step exists — 'hits_slots' ARE the chart.
 
-    Decoder positions are slot-linear across complete measures. Rational
-    meters such as 1/16, 5/8, 7/8, and 8/4 are represented by
-    ``96 * numerator / denominator`` slots. No quantization step exists:
-    ``hits_slots`` are exact per-measure lattice positions.
-
-    mel: (n_mels, T); grid: trusted rational layout with explicit
-    ``measure_edges`` including its terminal edge.
+    mel: (n_mels, T); grid: fit_grid() result (must be trustworthy).
     Returns {hits, spans (seconds, for rendering/metrics),
              hits_slots [(measure, slot, cls)], spans_slots, n_measures}.
     """
-    from .vocab import SLOTS
+    from .vocab import MEAS_MAX, SLOTS
 
     if isinstance(mel, np.ndarray):
         mel = torch.from_numpy(mel.astype(np.float32))
     T = mel.shape[1]
+    dur = T / FPS
     db = np.asarray(grid["downbeats"], np.float64)
-    measure_slots, measure_num, measure_den, quarter_beats = \
-        resolve_meter_layout(grid, len(db))
-    if "measure_slot_safe" in grid:
-        slot_safe = np.asarray(grid["measure_slot_safe"])
-        if (slot_safe.dtype != np.bool_ or slot_safe.ndim != 1
-                or len(slot_safe) != len(db)):
-            raise ValueError(
-                "measure_slot_safe must be one boolean per measure")
-        unsafe = np.flatnonzero(~slot_safe)
-        if len(unsafe):
-            raise ValueError(
-                "slot generation refuses unsafe BPM/DELAY/off-lattice "
-                f"measures: {unsafe.astype(int).tolist()}")
-    edges = _grid_edges(grid, db)
-
-    # meta = (first_measure_idx, measure_count, cumulative local slot offsets)
-    wins, metas = [], []
+    bar = float(grid["bar"])
+    edges = np.append(db, db[-1] + bar)  # measure m spans [edges[m], edges[m+1])
+
+    wins, metas = [], []  # meta = (first_measure_idx, K)
     j = 0
     while j < len(db):
         K = 0
-        nslots = 0
-        while (j + K < len(db)
-               and nslots + int(measure_slots[j + K]) <= WINDOW
+        while (j + K < len(db) and K < MEAS_MAX
                and (edges[j + K + 1] - edges[j]) * FPS <= WINDOW):
-            nslots += int(measure_slots[j + K])
             K += 1
         if K == 0:
-            raise ValueError(
-                f"measure {j} does not fit the {WINDOW}-frame/{WINDOW}-slot "
-                "slot window"
-            )
+            break
         t0 = edges[j]
         f0 = int(round(t0 * FPS))
         src = max(1, int(round((edges[j + K] - t0) * FPS)))
-        # Negative chart offsets are real; pad the pre-audio region instead of
-        # letting Python's negative slice wrap around to the song tail.
-        x = torch.full((mel.shape[0], src), float(np.log(1e-5)),
-                       dtype=mel.dtype, device=mel.device)
-        src0, src1 = max(0, f0), min(T, f0 + src)
-        if src1 > src0:
-            dst0 = src0 - f0
-            x[:, dst0:dst0 + (src1 - src0)] = mel[:, src0:src1]
+        x = mel[:, f0 : f0 + src]
+        if x.shape[1] < src:
+            x = torch.nn.functional.pad(x, (0, src - x.shape[1]), value=float(np.log(1e-5)))
         if x.shape[1] < WINDOW:
             x = torch.nn.functional.pad(x, (0, WINDOW - x.shape[1]), value=float(np.log(1e-5)))
         ph = torch.full((2, WINDOW), -1.0)
@@ -520,29 +496,22 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
                 continue
             frac = (torch.arange(i0, i1, dtype=torch.float32) - a) / max(b - a, 1e-6)
             ph[0, i0:i1] = frac
-            ph[1, i0:i1] = (frac * float(quarter_beats[j + m])) % 1.0
+            ph[1, i0:i1] = (frac * 4) % 1.0
         wins.append(torch.cat([x, ph], dim=0))
-        local_offsets = np.concatenate([
-            np.array([0], dtype=np.int64),
-            np.cumsum(measure_slots[j:j + K], dtype=np.int64),
-        ])
-        metas.append((j, K, local_offsets))
+        metas.append((j, K))
         j += K
 
     sib_default = getattr(model, "_has_sib", False)
     style_val = -1 if getattr(model, "_has_style", False) else None
     plan_blocks = plan if getattr(model, "_has_plan", False) else None
     plan_default = getattr(model, "_has_plan", False) and plan is None
-    gsum = None
-    if getattr(model, "_has_gctx", False):
-        gsum = torch.from_numpy(song_summary(mel.detach().cpu().numpy()))
 
     events = []  # (measure_global, slot, cls) in decode order
     for i0 in range(0, len(wins), batch_windows):
         chunk = torch.stack(wins[i0 : i0 + batch_windows])
         prefixes = []
         for b_ in range(chunk.shape[0]):
-            jj, KK, _ = metas[i0 + b_]
+            jj, KK = metas[i0 + b_]
             t0, t1 = edges[jj], edges[jj + KK]
             prefixes.append(build_prefix(
                 course, level, density_bucket,
@@ -551,36 +520,20 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
                 plan_slice=([(b[2], b[3]) for b in plan_blocks
                              if b[1] > t0 and b[0] < t1] if plan_blocks is not None
                             else ([] if plan_default else None))))
-        limits = [int(metas[i0 + b_][2][-1])
-                  for b_ in range(chunk.shape[0])]
-        g = pb = None
-        if gsum is not None:
-            g = gsum.unsqueeze(0).expand(chunk.shape[0], -1, -1)
-            n_pos = (100 if getattr(model, "_hierarchical_ctx", False)
-                     else getattr(getattr(model, "pos_emb", None),
-                                  "num_embeddings", 16))
-            pb = torch.tensor([
-                min(n_pos - 1, max(0, int(
-                    n_pos * edges[metas[i0 + b_][0]] * FPS / max(T, 1))))
-                for b_ in range(chunk.shape[0])
-            ], dtype=torch.long)
         evs = decode_windows(model, chunk, prefixes, device=device, greedy=greedy,
                              temperature=temperature, top_p=top_p, seed=seed + i0,
-                             min_gap=2, position_limits=limits,
-                             gsum=g, pos_buckets=pb)
+                             n_cond=4 if style_val is not None else 3, min_gap=2)
         for b_, w_ev in enumerate(evs):
-            jj, KK, offsets = metas[i0 + b_]
+            jj, KK = metas[i0 + b_]
             for g, cls in w_ev:
-                if g < offsets[-1]:
-                    local_m = int(np.searchsorted(offsets[1:], g, side="right"))
-                    events.append((jj + local_m, int(g - offsets[local_m]), cls))
+                if g < KK * SLOTS:
+                    events.append((jj + g // SLOTS, g % SLOTS, cls))
         if on_progress:
             on_progress(min(i0 + batch_windows, len(wins)), len(wins))
 
     events.sort(key=lambda e: (e[0], e[1]))
     def _t(me, sl):
-        slots_in_measure = int(measure_slots[me])
-        return float(edges[me] + sl / slots_in_measure * (edges[me + 1] - edges[me])) \
+        return float(edges[me] + sl / SLOTS * (edges[me + 1] - edges[me])) \
             if me < len(edges) - 1 else float(edges[-1])
 
     hits, spans, hits_slots, spans_slots = [], [], [], []
@@ -589,11 +542,15 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
     for me, sl, cls in events:
         t = _t(me, sl)
         if cls in HIT_CLASSES:
-            # A hit is never a valid implicit span terminator.  Keeping the
-            # authored hit and dropping the malformed, unclosed span avoids
-            # inventing an end position that the decoder did not emit.
             if open_span is not None:
-                open_span = None
+                if t - open_span[2] > SPAN_MAX.get(open_span[3], 6.5):
+                    t1s = open_span[2] + SPAN_MAX.get(open_span[3], 6.5)
+                    spans.append({"t0": round(open_span[2], 4), "t1": round(t1s, 4),
+                                  "type": open_span[3]})
+                    spans_slots.append((open_span[0], open_span[1], me, sl, open_span[3]))
+                    open_span = None
+                else:
+                    continue
             if (me, sl) == last_key:
                 continue
             hits.append({"t": round(t, 4), "type": cls})
@@ -603,21 +560,13 @@ def generate_song_slot(model, mel, grid, course, level=None, density_bucket=None
             if open_span is None:
                 open_span = (me, sl, t, cls)
         elif cls == "end":
-            if open_span is not None:
-                if t - open_span[2] > 0.05:
-                    # The explicit END slot is authoritative.  Derive both
-                    # preview seconds and TJA coordinates from that same slot;
-                    # duration caps must not move either representation.
-                    spans.append({"t0": round(open_span[2], 4), "t1": round(t, 4),
-                                  "type": open_span[3]})
-                    spans_slots.append(
-                        (open_span[0], open_span[1], me, sl, open_span[3]))
+            if open_span is not None and t - open_span[2] > 0.05:
+                t1s = min(t, open_span[2] + SPAN_MAX.get(open_span[3], 6.5))
+                spans.append({"t0": round(open_span[2], 4), "t1": round(t1s, 4),
+                              "type": open_span[3]})
+                spans_slots.append((open_span[0], open_span[1], me, sl, open_span[3]))
                 open_span = None
     n_meas = (metas[-1][0] + metas[-1][1]) if metas else 0
     return {"hits": hits, "spans": spans, "hits_slots": hits_slots,
             "spans_slots": spans_slots, "n_measures": n_meas,
-            "measure_slots": measure_slots[:n_meas].tolist(),
-            "measure_num": measure_num[:n_meas].tolist(),
-            "measure_den": measure_den[:n_meas].tolist(),
-            "slots_per_4_4": SLOTS,
             "course": course, "level": level, "density_bucket": density_bucket}

softchart/grid.py

@@ -59,9 +59,7 @@ def beat_activations(model, mel, device="cuda", hop=WINDOW // 2):
         if w.shape[0] < model.in_ch:  # dual/slot models: blank grid channels
             w = torch.cat([w, torch.full((model.in_ch - w.shape[0], WINDOW), -1.0)])
         with _autocast(device):
-            # Beat/downbeat estimation must remain local and phase-free even
-            # when the chart decoder also owns a whole-song section encoder.
-            mem = model.encode_local(w[None].to(device))
+            mem = model.encode(w[None].to(device))
             pr = torch.sigmoid(model.beat(mem[:, -L:]).float())[0].cpu().numpy().T
         if acc is None:
             factor = WINDOW // pr.shape[1]

softchart/hf.py

@@ -1,17 +1,22 @@
 """Hugging Face Hub-compatible wrappers (safetensors + from_pretrained/push_to_hub).
 
-Space runtime wrapper for the unified V1.8 generator.
+Three model types, all <=16M params, MIT-licensed:
+  SoftChartGenerator  audio log-mel  -> chart event tokens (the main model; pr12)
+  SoftChartBeat       audio log-mel  -> beat/downbeat activations (barline anchor)
+  SoftChartPlanner    block features -> per-block plan tokens (auto-planning)
 
 Usage:
     from softchart.hf import SoftChartGenerator
     gen = SoftChartGenerator.from_pretrained("JacobLinCool/softchart-generator")
 """
 
+import torch
 import torch.nn as nn
 from huggingface_hub import PyTorchModelHubMixin
 
-from .model import ChartModel
-from .vocab import VOCAB
+from .model import ChartModel, sinusoidal
+
+_CARD = "See https://github.com/JacobLinCool/SoftChart — MIT licensed."
 
 
 class SoftChartGenerator(
@@ -24,21 +29,27 @@ class SoftChartGenerator(
     """Encoder-decoder chart generator. Config is stored as config.json and the
     full architecture (condition tokens, heads) is reconstructed on load."""
 
-    def __init__(self, d_model=256, nhead=8, enc_layers=4, dec_layers=4, ffn=1024,
-                 dropout=0.1,
-                 vocab_size=VOCAB.size, aux=True, global_ctx=False, func_time=False,
+    def __init__(self, d_model=256, enc_layers=4, dec_layers=4, ffn=1024,
+                 vocab_size=1802, aux=True, global_ctx=False, func_time=False,
                  ptr=False, beat=False, in_ch=None, emb_factor=None,
-                 hierarchical_ctx=False, section_layers=2, section_ffn=None,
-                 clean_phase=False,
-                 capabilities=None):
+                 capabilities=None, nhead=8, clean_phase=False,
+                 enc_share=None, dec_share=None, adapter_rank=0,
+                 unique_layernorm=False, adapter_rank_ffn=0, depth_emb=False,
+                 unshare_last_dec=False):
         super().__init__()
+        # nhead and the v1.6/v1.7 sharing flags (enc_share/dec_share/adapter_rank/
+        # unique_layernorm/adapter_rank_ffn/depth_emb/unshare_last_dec) and
+        # clean_phase must be persisted in config.json: from_pretrained rebuilds
+        # the architecture from these kwargs alone, before any weights are loaded.
+        # Defaults reproduce the pre-v1.6 unshared architecture for older repos.
         self.net = ChartModel(
-            d_model=d_model, nhead=nhead, enc_layers=enc_layers,
-            dec_layers=dec_layers, ffn=ffn, dropout=dropout,
-            vocab_size=vocab_size, aux=aux, global_ctx=global_ctx, func_time=func_time,
-            ptr=ptr, beat=beat, in_ch=in_ch, emb_factor=emb_factor,
-            hierarchical_ctx=hierarchical_ctx, section_layers=section_layers,
-            section_ffn=section_ffn, clean_phase=clean_phase,
+            d_model=d_model, nhead=nhead, enc_layers=enc_layers, dec_layers=dec_layers,
+            ffn=ffn, vocab_size=vocab_size, aux=aux, global_ctx=global_ctx,
+            func_time=func_time, ptr=ptr, beat=beat, in_ch=in_ch, emb_factor=emb_factor,
+            clean_phase=clean_phase, enc_share=enc_share, dec_share=dec_share,
+            adapter_rank=adapter_rank, unique_layernorm=unique_layernorm,
+            adapter_rank_ffn=adapter_rank_ffn, depth_emb=depth_emb,
+            unshare_last_dec=unshare_last_dec,
         )
         # capabilities: which condition axes this checkpoint was trained with,
         # so inference knows which prefix tokens to emit
@@ -65,3 +76,29 @@ class SoftChartGenerator(
     @property
     def tok_emb(self):
         return self.net.tok_emb
+
+
+class SoftChartPlanner(
+    nn.Module,
+    PyTorchModelHubMixin,
+    repo_url="https://github.com/JacobLinCool/SoftChart",
+    license="mit",
+    tags=["taiko", "rhythm-game", "planning", "music"],
+):
+    """Song-level plan generator: block audio summaries -> (density, flag) tokens."""
+
+    def __init__(self, feat_dim=261, d=192, layers=4, n_dens=8, n_flag=3):
+        super().__init__()
+        self.inp = nn.Linear(feat_dim, d)
+        self.course = nn.Embedding(5, d)
+        self.register_buffer("pos", sinusoidal(128, d), persistent=False)
+        enc = nn.TransformerEncoderLayer(d, 6, d * 4, 0.1, activation="gelu",
+                                         batch_first=True, norm_first=True)
+        self.enc = nn.TransformerEncoder(enc, layers, nn.LayerNorm(d))
+        self.h_dens = nn.Linear(d, n_dens)
+        self.h_flag = nn.Linear(d, n_flag)
+
+    def forward(self, x, cid):
+        h = self.inp(x) + self.course(cid)[:, None] + self.pos[: x.shape[1]]
+        h = self.enc(h)
+        return self.h_dens(h), self.h_flag(h)

softchart/model.py

@@ -33,42 +33,6 @@ class BeatHiRes(nn.Module):
         return self.out(h).transpose(1, 2)
 
 
-class HierarchicalSectionEncoder(nn.Module):
-    """Encode a fixed relative-time summary of the complete song.
-
-    The local encoder remains dedicated to the current 20-second window. This
-    separate branch models section-to-section structure, marks the section that
-    contains the current window, and exposes its states directly to decoder
-    cross-attention. It is trained from scratch with the chart objective; no
-    external audio representation is used.
-    """
-
-    def __init__(self, d, nhead, ffn, dropout, layers=2, sections=100):
-        super().__init__()
-        self.sections = sections
-        self.inp = nn.Linear(N_MELS, d)
-        self.pos = nn.Parameter(torch.randn(sections, d) * 0.02)
-        self.current = nn.Parameter(torch.randn(d) * 0.02)
-        layer = nn.TransformerEncoderLayer(
-            d, nhead, ffn, dropout, activation="gelu",
-            batch_first=True, norm_first=True,
-        )
-        self.encoder = nn.TransformerEncoder(
-            layer, layers, nn.LayerNorm(d), enable_nested_tensor=False)
-
-    def forward(self, summary, position):
-        if summary.ndim != 3 or summary.shape[1:] != (self.sections, N_MELS):
-            raise ValueError(
-                f"expected song summary (B,{self.sections},{N_MELS}), "
-                f"got {tuple(summary.shape)}"
-            )
-        position = position.to(summary.device).long().clamp(0, self.sections - 1)
-        h = self.inp(summary) + self.pos
-        marker = torch.zeros_like(h)
-        marker[torch.arange(h.shape[0], device=h.device), position] = self.current
-        return self.encoder(h + marker)
-
-
 def _parse_pattern(spec, n):
     """Layer-sharing schedule. spec is either an int (number of UNIQUE
     physical layers, cycled to length n) or a comma string like "E0,E1,E0,E1"
@@ -91,18 +55,17 @@ def _parse_pattern(spec, n):
 
 class Adapter(nn.Module):
     """Rank-r residual adapter applied after a (possibly reused) transformer
-    block: x + gate * up(gelu(down(x))). Both projections and the gate start on
-    an active gradient path. The up projection is initialized very small so the
-    initial adapter is a near-identity perturbation without the dead-gradient
-    state caused by zero-initializing both ``up`` and ``gate``."""
+    block: x + gate * up(gelu(down(x))). gate init 0 so the whole stack starts
+    identical to hard weight sharing; the adapter only differentiates a reused
+    physical layer as training moves the gate off zero."""
 
     def __init__(self, d, rank):
         super().__init__()
         self.down = nn.Linear(d, rank, bias=False)
         self.up = nn.Linear(rank, d, bias=False)
         nn.init.normal_(self.down.weight, std=0.02)
-        nn.init.normal_(self.up.weight, std=1e-3)
-        self.gate = nn.Parameter(torch.ones(1))
+        nn.init.zeros_(self.up.weight)
+        self.gate = nn.Parameter(torch.zeros(1))
 
     def forward(self, x):
         return x + self.gate * self.up(nn.functional.gelu(self.down(x)))
@@ -111,17 +74,17 @@ class Adapter(nn.Module):
 class SharedEncoder(nn.Module):
     """Encoder whose physical layers are reused according to a sharing pattern.
     Each POSITION in the depth keeps its own (a) rank-r adapter and (b) final
-    LayerNorm, so every logical depth can specialize while reusing the block.
+    LayerNorm — so ΔW starts at 0 (hard sharing) but every position can drift.
     With pattern=None this is exactly n unique layers + a norm, matching the
     stock nn.TransformerEncoder param count (adapters/pos-norms are opt-in).
 
-    v1.7 depth-diversity options (all default-off):
+    v1.7 depth-diversity options (all default-off, init == v1.6 behaviour):
       depth_emb: per-depth learned d-vector ADDED to the block input at each
         reuse (params = depth x d, zero-init so init == hard sharing).
       adapter_rank_ffn > 0: SPLIT adapters — instead of one post-block adapter,
         each depth owns a rank-`adapter_rank` adapter on the attention sublayer
         output and a rank-`adapter_rank_ffn` adapter on the FFN sublayer output
-        (patterning lives in the FFN).
+        (patterning lives in the FFN). Gate init 0 keeps init == hard sharing.
     """
 
     def __init__(self, d, nhead, ffn, dropout, pattern, adapter_rank=0,
@@ -160,7 +123,7 @@ class SharedEncoder(nn.Module):
             if self.split:
                 # norm_first decomposition of nn.TransformerEncoderLayer with
                 # per-depth adapters applied to each sublayer OUTPUT (Houlsby
-                # placement)
+                # placement); at gate=0 this is bit-equal to the stock layer
                 x = x + self.attn_adapters[k](
                     layer._sa_block(layer.norm1(x), None, src_key_padding_mask))
                 x = x + self.ffn_adapters[k](layer._ff_block(layer.norm2(x)))
@@ -241,9 +204,6 @@ class ChartModel(nn.Module):
         max_tgt=None,
         aux=False,
         global_ctx=False,
-        hierarchical_ctx=False,
-        section_layers=2,
-        section_ffn=None,
         func_time=False,
         ptr=False,
         beat=False,
@@ -264,10 +224,6 @@ class ChartModel(nn.Module):
         vocab_size = vocab_size or VOCAB.size
         max_tgt = max_tgt or MAX_TGT
         self.d_model = d_model
-        if global_ctx and hierarchical_ctx:
-            raise ValueError("global_ctx and hierarchical_ctx are mutually exclusive")
-        self.hierarchical_ctx = bool(hierarchical_ctx)
-        self._hierarchical_ctx = self.hierarchical_ctx
         # clean_phase (v1.6): the encoder never sees the 2 grid-phase channels
         # (in_ch stays N_MELS), and phase is injected into the DECODER memory
         # only, via a small projection. This removes the condition leak that
@@ -311,9 +267,8 @@ class ChartModel(nn.Module):
             enc_layer = nn.TransformerEncoderLayer(
                 d_model, nhead, ffn, dropout, activation="gelu",
                 batch_first=True, norm_first=True)
-            self.encoder = nn.TransformerEncoder(
-                enc_layer, enc_layers, nn.LayerNorm(d_model),
-                enable_nested_tensor=False)
+            self.encoder = nn.TransformerEncoder(enc_layer, enc_layers,
+                                                 nn.LayerNorm(d_model))
             dec_layer = nn.TransformerDecoderLayer(
                 d_model, nhead, ffn, dropout, activation="gelu",
                 batch_first=True, norm_first=True)
@@ -351,17 +306,8 @@ class ChartModel(nn.Module):
             self.out = nn.Linear(d_model, vocab_size, bias=False)
             self.out.weight = self.tok_emb.weight  # weight tying
         self.dropout = nn.Dropout(dropout)
-        # Auxiliary local rhythmic skeleton. In the hierarchical model its
-        # predicted occupancy is also projected back into decoder memory, so
-        # the event realizer is explicitly conditioned on a coarse audio-led
-        # skeleton rather than learning placement and pattern in one flat pass.
-        if hierarchical_ctx and not aux:
-            raise ValueError("hierarchical_ctx requires the auxiliary skeleton head")
+        # auxiliary per-position onset-heatmap head on the encoder (v2)
         self.aux = nn.Linear(d_model, 1) if aux else None
-        self.skeleton_proj = nn.Linear(1, d_model) if hierarchical_ctx else None
-        if self.skeleton_proj is not None:
-            nn.init.zeros_(self.skeleton_proj.weight)
-            nn.init.zeros_(self.skeleton_proj.bias)
         # pointer alignment head: each generated note must point to its audio
         # frame (differentiable provenance; explainability that trains alignment)
         self.ptr = nn.Linear(d_model, d_model) if ptr else None
@@ -377,18 +323,10 @@ class ChartModel(nn.Module):
         if global_ctx:
             self.gsum_proj = nn.Linear(N_MELS, d_model)
             self.seg_emb = nn.Parameter(torch.zeros(2, d_model))
-            self.pos_emb = nn.Embedding(100, d_model)  # relative song position
+            self.pos_emb = nn.Embedding(16, d_model)  # window position bucket
             self.gpos = nn.Parameter(torch.randn(128, d_model) * 0.02)
         else:
             self.gsum_proj = None
-        if hierarchical_ctx:
-            self.section_encoder = HierarchicalSectionEncoder(
-                d_model, nhead, section_ffn or (2 * d_model), dropout,
-                layers=section_layers,
-            )
-            self.hier_seg_emb = nn.Parameter(torch.zeros(2, d_model))
-        else:
-            self.section_encoder = None
         # functional time embeddings: the 1728 TIME tokens share a sinusoidal
         # basis + small projection instead of free embeddings (fewer params,
         # neighbouring times get similar representations)
@@ -401,31 +339,15 @@ class ChartModel(nn.Module):
         else:
             self.time_proj = None
 
-    def _local_input(self, mel):
-        audio = mel[:, :N_MELS] if self.clean_phase else mel
-        h = self.frontend(audio).transpose(1, 2)  # (B, T/4, d)
-        return h + self.enc_pos[: h.shape[1]]
-
-    def encode_local(self, mel):
-        """Encode only the current window for beat/downbeat estimation."""
-        h = self._local_input(mel)
-        if self.section_encoder is not None:
-            h = h + self.hier_seg_emb[0]
-        return self.encoder(self.dropout(h))
-
     def encode(self, mel, gsum=None, pos_bucket=None):
         # mel: (B, C, T). clean_phase: C may be N_MELS (audio only) or N_MELS+2
         # (audio + 2 grid-phase channels); only the audio channels reach the
         # encoder, so the returned memory is CLEAN (phase-free) and safe for the
-        # beat head. gsum: (B, 100, n_mels) complete-song section summaries;
-        # pos_bucket: (B,) current relative song section in [0, 100).
-        h = self._local_input(mel)
-        if self.section_encoder is not None:
-            if gsum is None or pos_bucket is None:
-                raise ValueError("hierarchical_ctx requires whole-song summary and position")
-            local = self.encoder(self.dropout(h + self.hier_seg_emb[0]))
-            sections = self.section_encoder(gsum, pos_bucket) + self.hier_seg_emb[1]
-            return torch.cat([sections, local], dim=1)
+        # beat head. gsum: (B, G, n_mels) whole-song summary chunks;
+        # pos_bucket: (B,) window-position bucket in [0, 16).
+        audio = mel[:, :N_MELS] if self.clean_phase else mel
+        h = self.frontend(audio).transpose(1, 2)  # (B, T/4, d)
+        h = h + self.enc_pos[: h.shape[1]]
         if self.gsum_proj is not None and gsum is not None:
             g = self.gsum_proj(gsum) + self.seg_emb[1] + self.gpos[: gsum.shape[1]]
             h = h + self.seg_emb[0]
@@ -438,25 +360,17 @@ class ChartModel(nn.Module):
         Adds a projection of the grid-phase channels to the audio-frame slice
         of memory; the beat head keeps reading the clean `memory`. No-op unless
         clean_phase is on and phase channels are present. Time-mode windows
-        (raw phase = -1) get valid=0 and exactly zero phase conditioning."""
-        out = memory.clone()
-        L = WINDOW // 4
+        (raw phase = -1) get valid=0 and zeroed phase -> ~zero conditioning."""
         nch = N_MELS
-        if self.clean_phase and mel.shape[1] >= nch + 2:
-            ph = mel[:, nch:nch + 2]  # [0,1); -1 marks no-grid frames
-            valid = (ph[:, :1] >= 0).float()
-            ph_in = torch.cat([ph.clamp(min=0.0), valid], dim=1)
-            p = self.phase_proj(ph_in)
-            # Mask *after* both biased convolutions. Masking only their input
-            # leaves a learned bias residual in gridless time-mode windows.
-            valid_out = nn.functional.adaptive_max_pool1d(valid, p.shape[-1])
-            p = (p * valid_out).transpose(1, 2)
-            L = p.shape[1]
-            out[:, -L:] = out[:, -L:] + p
-        if self.skeleton_proj is not None:
-            clean_local = memory[:, -L:]
-            skeleton = torch.sigmoid(self.aux(clean_local))
-            out[:, -L:] = out[:, -L:] + self.skeleton_proj(skeleton)
+        if not self.clean_phase or mel.shape[1] < nch + 2:
+            return memory
+        ph = mel[:, nch:nch + 2]  # (B, 2, T) in [0,1); -1 marks no-grid frames
+        valid = (ph[:, :1] >= 0).float()  # 1 where a grid exists, else 0
+        ph_in = torch.cat([ph.clamp(min=0.0), valid], dim=1)  # (B, 3, T)
+        p = self.phase_proj(ph_in).transpose(1, 2)  # (B, T/4, d)
+        L = p.shape[1]
+        out = memory.clone()
+        out[:, -L:] = out[:, -L:] + p  # audio frames are the last L memory slots
         return out
 
     def emb_matrix(self):
@@ -474,7 +388,7 @@ class ChartModel(nn.Module):
         L = tgt_in.shape[1]
         E = self.emb_matrix()
         h = nn.functional.embedding(tgt_in, E) * math.sqrt(self.d_model) + self.dec_pos[:L]
-        mask = torch.ones((L, L), dtype=torch.bool, device=tgt_in.device).triu(1)
+        mask = nn.Transformer.generate_square_subsequent_mask(L, device=tgt_in.device)
         pad_mask = tgt_in == VOCAB.pad
         h = self.decoder(
             self.dropout(h),
@@ -502,7 +416,7 @@ class ChartModel(nn.Module):
         E = self.emb_matrix()
         import math as _m
         h = nn.functional.embedding(dec_in, E) * _m.sqrt(self.d_model) + self.dec_pos[:L]
-        dm = torch.ones((L, L), dtype=torch.bool, device=tgt.device).triu(1)
+        dm = nn.Transformer.generate_square_subsequent_mask(L, device=tgt.device)
         h = self.decoder(self.dropout(h), dec_mem, tgt_mask=dm,
                          tgt_key_padding_mask=dec_in == VOCAB.pad, tgt_is_causal=True)
         logits = h @ E.T

softchart/preprocess.py

@@ -0,0 +1,156 @@
+"""Convert the HF dataset into per-song cache files: log-mel (.npy) + notes (.npz).
+
+Usage (on the training machine):
+    python -m softchart.preprocess --out /workspace/softchart/cache [--split train]
+"""
+
+import argparse
+import json
+import os
+from collections import Counter
+
+import numpy as np
+
+from .vocab import HOP, N_FFT, N_MELS, NOTE_CLASSES, NOTE_TYPE_MAP, SR, COURSES
+
+
+def to_mono_22k(audio):
+    """Handle both datasets<=3 dict audio and datasets>=4 torchcodec AudioDecoder."""
+    import torch
+
+    if isinstance(audio, dict):
+        arr = np.asarray(audio["array"], dtype=np.float32)
+        if arr.ndim == 2:
+            arr = arr.mean(axis=0)
+        sr = audio["sampling_rate"]
+    else:  # torchcodec AudioDecoder
+        samples = audio.get_all_samples()
+        arr = samples.data.to(torch.float32).mean(dim=0).numpy()
+        sr = samples.sample_rate
+    if sr != SR:
+        import librosa
+
+        arr = librosa.resample(arr, orig_sr=sr, target_sr=SR, res_type="soxr_hq")
+    return torch.from_numpy(np.ascontiguousarray(arr))[None]  # (1, T)
+
+
+_mel_fb = None
+_window = None
+
+
+def logmel(wav):
+    """(1, T) float32 -> (n_mels, frames) float16 log-mel (torch.stft + librosa fb)."""
+    global _mel_fb, _window
+    import torch
+
+    if _mel_fb is None:
+        import librosa
+
+        fb = librosa.filters.mel(sr=SR, n_fft=N_FFT, n_mels=N_MELS, fmin=20.0, fmax=SR / 2)
+        _mel_fb = torch.from_numpy(fb)  # (n_mels, n_fft//2+1)
+        _window = torch.hann_window(N_FFT)
+    with torch.no_grad():
+        spec = torch.stft(
+            wav, N_FFT, hop_length=HOP, window=_window, center=True, return_complex=True
+        )[0]
+        power = spec.abs().pow(2)  # (freq, frames)
+        m = torch.log(_mel_fb @ power + 1e-5)
+    return m.numpy().astype(np.float16)
+
+
+def extract_notes(course_struct):
+    """course struct -> (times, classes, level) or None."""
+    if course_struct is None or not course_struct.get("segments"):
+        return None
+    times, classes, bpms = [], [], []
+    unknown = Counter()
+    for seg in course_struct["segments"]:
+        for n in seg.get("notes") or []:
+            nt = n["note_type"]
+            canon = NOTE_TYPE_MAP.get(nt) or NOTE_TYPE_MAP.get(nt.lower())
+            if canon is None:
+                unknown[nt] += 1
+                continue
+            times.append(n["timestamp"])
+            classes.append(NOTE_CLASSES.index(canon))
+            if n.get("bpm"):
+                bpms.append(n["bpm"])
+    if not times:
+        return None
+    order = np.argsort(times, kind="stable")
+    return (
+        np.asarray(times, np.float64)[order],
+        np.asarray(classes, np.int8)[order],
+        int(course_struct.get("level") or 0),
+        float(np.median(bpms)) if bpms else 0.0,
+        unknown,
+    )
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", default="cache")
+    ap.add_argument("--splits", nargs="+", default=["train", "test"])
+    args = ap.parse_args()
+
+    from datasets import Audio, load_dataset
+
+    ds = load_dataset("JacobLinCool/taiko-1000-parsed")
+    ds = ds.cast_column("audio", Audio(sampling_rate=SR))
+    os.makedirs(args.out, exist_ok=True)
+    index = []
+    unknown_total = Counter()
+
+    for split in args.splits:
+        d = ds[split]
+        for i in range(len(d)):
+            row = d[i]
+            sid = f"{split}_{i:05d}"
+            mel_path = os.path.join(args.out, f"{sid}.mel.npy")
+            notes_path = os.path.join(args.out, f"{sid}.notes.npz")
+            entry = {
+                "id": sid,
+                "split": split,
+                "title": (row.get("metadata") or {}).get("TITLE", ""),
+                "genre": (row.get("metadata") or {}).get("GENRE", ""),
+                "courses": {},
+            }
+            arrays = {}
+            for c in COURSES:
+                ex = extract_notes(row.get(c))
+                if ex is None:
+                    continue
+                times, classes, level, bpm, unknown = ex
+                unknown_total.update(unknown)
+                arrays[f"{c}_t"] = times
+                arrays[f"{c}_c"] = classes
+                entry["courses"][c] = {
+                    "level": level,
+                    "n_notes": int(len(times)),
+                    "bpm": bpm,
+                }
+            if not arrays:
+                print(f"skip {sid}: no charts")
+                continue
+            if not (os.path.exists(mel_path) and os.path.exists(notes_path)):
+                try:
+                    mel = logmel(to_mono_22k(row["audio"]))
+                except Exception as e:
+                    raise RuntimeError(f"{sid}: target-rate audio decode failed") from e
+                np.save(mel_path, mel)
+                np.savez(notes_path, **arrays)
+                entry["n_frames"] = int(mel.shape[1])
+            else:
+                entry["n_frames"] = int(np.load(mel_path, mmap_mode="r").shape[1])
+            index.append(entry)
+            if i % 50 == 0:
+                print(f"{split} {i}/{len(d)}", flush=True)
+
+    with open(os.path.join(args.out, "index.json"), "w") as f:
+        json.dump(index, f)
+    print("unknown note types:", dict(unknown_total))
+    print(f"done: {len(index)} songs")
+
+
+if __name__ == "__main__":
+    main()

softchart/proxies.py

@@ -0,0 +1,238 @@
+"""Automated proxies for playability, groove, and pattern structure.
+
+Every proxy is computable without human input, from (events, bpm[, mel]).
+Together with rhythm.py metrics they form the scorecard that closes the
+evaluate -> diagnose -> improve loop.
+
+Proxy design notes:
+- playability = physical feasibility on the drum (min gaps, stream length,
+  big-note crowding) + learnable structure (motif reuse, compressibility)
+- groove = alignment with the metrical grid (strong-beat rate, accent placement)
+  + audio grounding (onset-envelope energy at note times)
+Reference values come from ground-truth charts of the same course (see scorecard).
+"""
+
+import zlib
+
+import numpy as np
+
+HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
+BIG_CLASSES = ("don_big", "ka_big")
+
+
+def _hits(events):
+    return sorted((t, c) for t, c in events if c in HIT_CLASSES)
+
+
+def min_gap_violation_rate(events, min_gap=0.045):
+    """Fraction of consecutive hits closer than a physically playable gap."""
+    h = _hits(events)
+    if len(h) < 2:
+        return 0.0
+    ioi = np.diff([t for t, _ in h])
+    return float(np.mean(ioi < min_gap))
+
+
+def stream_p99_nps(events, win=2.0):
+    """99th percentile of local density (notes/sec in rolling windows) — stamina proxy."""
+    t = np.array([x for x, _ in _hits(events)])
+    if len(t) < 4:
+        return 0.0
+    counts = [np.sum((t >= x) & (t < x + win)) for x in np.arange(t[0], t[-1], 0.5)]
+    return float(np.percentile(counts, 99) / win) if counts else 0.0
+
+
+def big_crowding_rate(events, bpm):
+    """Fraction of big notes followed by another hit within 0.45 beat.
+
+    Big notes are played with both hands; crowding them against the next note
+    is physically awkward and rarely done in official charts.
+    """
+    h = _hits(events)
+    bigs = [(i, t) for i, (t, c) in enumerate(h) if c in BIG_CLASSES]
+    if not bigs or not bpm or bpm <= 0:
+        return None
+    thresh = 0.45 * 60.0 / bpm
+    crowded = sum(1 for i, t in bigs if i + 1 < len(h) and h[i + 1][0] - t < thresh)
+    return crowded / len(bigs)
+
+
+def big_note_rate(events):
+    h = _hits(events)
+    if not h:
+        return 0.0
+    return sum(1 for _, c in h if c in BIG_CLASSES) / len(h)
+
+
+def _symbol_stream(events, bpm):
+    """Quantized (ioi-class, note-type) symbol stream for structure metrics."""
+    h = _hits(events)
+    if len(h) < 3 or not bpm or bpm <= 0:
+        return []
+    beat = 60.0 / bpm
+    syms = []
+    for i in range(1, len(h)):
+        ioi = h[i][0] - h[i - 1][0]
+        frac = ioi / beat
+        # classify IOI into musical classes
+        classes = [0.25, 1 / 3, 0.5, 2 / 3, 0.75, 1.0, 1.5, 2.0]
+        j = int(np.argmin([abs(frac - c) / c for c in classes]))
+        ioi_cls = j if abs(frac - classes[j]) / classes[j] < 0.25 else len(classes)
+        syms.append((ioi_cls, h[i][1]))
+    return syms
+
+
+def motif_reuse(events, bpm, n=4):
+    """Fraction of n-grams (rhythm+type) that occur at least twice in the chart.
+
+    Human charts repeat and transform motifs; random note salads do not.
+    """
+    syms = _symbol_stream(events, bpm)
+    if len(syms) < n + 2:
+        return None
+    grams = [tuple(syms[i : i + n]) for i in range(len(syms) - n + 1)]
+    from collections import Counter
+
+    counts = Counter(grams)
+    return float(sum(1 for g in grams if counts[g] >= 2) / len(grams))
+
+
+def compression_ratio(events, bpm):
+    """zlib-compressed size / raw size of the symbol stream. Lower = more structured."""
+    syms = _symbol_stream(events, bpm)
+    if len(syms) < 16:
+        return None
+    raw = bytes(b for s in syms for b in (s[0], HIT_CLASSES.index(s[1])))
+    return float(len(zlib.compress(raw, 9)) / len(raw))
+
+
+def strong_beat_rate(events, bpm, phase=None, tol=0.03):
+    """Fraction of hits on integer beats (downbeat proxy needs measures; beat is enough)."""
+    from .rhythm import estimate_phase
+
+    t = np.array([x for x, _ in _hits(events)])
+    if len(t) < 5 or not bpm or bpm <= 0:
+        return None
+    beat = 60.0 / bpm
+    if phase is None:
+        phase = estimate_phase(t, beat)
+    rel = (t - phase) / beat
+    return float(np.mean(np.abs(rel - np.round(rel)) * beat < tol))
+
+
+def accent_on_strong(events, bpm, phase=None, tol=0.03):
+    """Fraction of BIG notes that fall on integer beats — accent placement proxy."""
+    from .rhythm import estimate_phase
+
+    h = _hits(events)
+    tb = np.array([t for t, c in h if c in BIG_CLASSES])
+    if len(tb) < 3 or not bpm or bpm <= 0:
+        return None
+    beat = 60.0 / bpm
+    if phase is None:
+        phase = estimate_phase(np.array([t for t, _ in h]), beat)
+    rel = (tb - phase) / beat
+    return float(np.mean(np.abs(rel - np.round(rel)) * beat < tol))
+
+
+def audio_energy_alignment(events, mel, fps):
+    """Mean z-scored spectral-flux strength at hit times (GT-free audio grounding)."""
+    t = np.array([x for x, _ in _hits(events)])
+    if len(t) < 5:
+        return None
+    flux = np.maximum(0.0, np.diff(mel.astype(np.float32), axis=1)).sum(axis=0)
+    flux = (flux - flux.mean()) / (flux.std() + 1e-9)
+    fr = np.clip(np.round(t * fps).astype(int), 1, len(flux) - 1)
+    return float(np.mean(np.maximum.reduce([flux[fr - 1], flux[fr], flux[np.minimum(fr + 1, len(flux) - 1)]])))
+
+
+def _pair_spans(events):
+    """Pair span-start/end events into (t0, t1) tuples (for GT event streams)."""
+    out = []
+    open_t = None
+    for t, c in sorted(events):
+        if c in ("roll", "roll_big", "balloon") and open_t is None:
+            open_t = t
+        elif c == "end" and open_t is not None:
+            out.append((open_t, t))
+            open_t = None
+    return out
+
+
+def span_stats(events, spans=None):
+    """Span-length distribution + rate. Added after a visual audit found
+    generated balloons ~10s vs GT p50 ~1s (a metric blind spot)."""
+    if spans is None:
+        pairs = _pair_spans(events)
+    else:
+        pairs = [(s["t0"], s["t1"]) if isinstance(s, dict) else tuple(s) for s in spans]
+    h = _hits(events)
+    dur_min = max((h[-1][0] - h[0][0]) / 60.0, 1e-6) if len(h) > 1 else None
+    if not pairs or dur_min is None:
+        return {"span_p90_len": None, "span_per_min": 0.0 if dur_min else None}
+    lens = np.array([b - a for a, b in pairs])
+    return {"span_p90_len": float(np.percentile(lens, 90)),
+            "span_per_min": float(len(pairs) / dur_min)}
+
+
+def compute_proxies(events, bpm, mel=None, fps=None, spans=None):
+    """All proxies for one chart. Returns dict (None where not computable)."""
+    from .rhythm import grid_consistency, tuplet_evenness
+
+    t = [x for x, _ in _hits(events)]
+    out = {
+        "nps": len(t) / max(t[-1] - t[0], 1e-6) if len(t) > 1 else 0.0,
+        "min_gap_violation": min_gap_violation_rate(events),
+        "stream_p99_nps": stream_p99_nps(events),
+        "big_rate": big_note_rate(events),
+        "big_crowding": big_crowding_rate(events, bpm),
+        "motif_reuse": motif_reuse(events, bpm),
+        "compression": compression_ratio(events, bpm),
+        "strong_beat": strong_beat_rate(events, bpm),
+        "accent_on_strong": accent_on_strong(events, bpm),
+    }
+    g = grid_consistency(t, bpm)
+    if g:
+        out["offgrid_rate"] = 1.0 - g["on_grid_frac"]
+        out["grid_dev_ms"] = g["mean_dev_ms"]
+    tp = tuplet_evenness(t, bpm)
+    if tp:
+        out["tuplet_cv"] = tp["cv_mean"]
+    if mel is not None and fps:
+        out["energy_align"] = audio_energy_alignment(events, mel, fps)
+    out.update(span_stats(events, spans))
+    return out
+
+
+# direction: +1 higher is better, -1 lower is better, 0 match GT reference
+PROXY_DIRECTION = {
+    "nps": 0, "min_gap_violation": -1, "stream_p99_nps": 0, "big_rate": 0,
+    "big_crowding": -1, "motif_reuse": 0, "compression": 0, "strong_beat": 0,
+    "accent_on_strong": 0, "offgrid_rate": -1, "grid_dev_ms": -1,
+    "tuplet_cv": -1, "energy_align": 1, "span_p90_len": 0, "span_per_min": 0,
+}
+
+
+def score_against_reference(proxy_row, ref_stats):
+    """Normalized deviation of each proxy vs the GT reference distribution.
+
+    ref_stats: {proxy: {"p25": .., "p50": .., "p75": ..}} per course.
+    Returns {proxy: score} where score = robust z of |dev| (0 = at GT median;
+    for directional proxies, only penalize the bad direction).
+    """
+    scores = {}
+    for k, v in proxy_row.items():
+        if v is None or k not in ref_stats:
+            continue
+        r = ref_stats[k]
+        iqr = max(r["p75"] - r["p25"], 1e-6)
+        d = PROXY_DIRECTION.get(k, 0)
+        dev = (v - r["p50"]) / iqr
+        if d == -1:
+            dev = max(0.0, dev)  # only worse-than-GT (higher) penalized
+        elif d == 1:
+            dev = max(0.0, -dev)
+        else:
+            dev = abs(dev)
+        scores[k] = float(dev)
+    return scores

softchart/sc2/__init__.py

@@ -0,0 +1,36 @@
+"""SoftChart 2.0 serving code, vendored verbatim into its own namespace.
+
+The Space already ships a SoftChart 1.x library at ``softchart.*``.  The 2.0
+modules carry the same file names but an incompatible token table
+(``VOCAB.size`` 2071 vs 1.x, ``MAX_TGT`` 960 vs 1536), so they cannot share a
+package: overwriting ``softchart/vocab.py`` or ``softchart/model.py`` would
+break every 1.x checkpoint the Space serves.  They live here instead, as a
+sibling subpackage.  Both trees are importable in one process and neither can
+see the other -- every module below uses package-relative imports only, so
+``from .vocab import VOCAB`` inside ``softchart.sc2.generate`` resolves to
+``softchart.sc2.vocab`` and never to ``softchart.vocab``.
+
+The eight modules are BYTE-IDENTICAL copies of the frozen shipping tree at
+``.codex/worktrees/v18-8h/src/softchart/``.  Their md5s are the ones the
+release package recorded in ``config.json:code.module_md5``:
+
+    barscript.py   9afc715a56ac28e9ec5675d64cd0536c
+    checkpoint.py  10e215e86ac77ada9046a164bffa0644
+    generate.py    73b6e49b00fb60f3229c8539bc7a1029
+    meter.py       18ccd6a003086f81276f2a916ada4c64
+    model.py       eace5d89abeebf0edcf28d10b5ad3e68
+    tja.py         360b95e19dabe925bf8eab03fa3fd110
+    trace.py       e707f7204f0d86deacca6b6325f031f1
+    vocab.py       0f948a0efc52f5ce60e8219fcaeb136c
+
+Verbatim is the point: ``softchart/sc2_loader.py`` checks these md5s against
+the package's own record at load time, so serving code that has drifted from
+the code the checkpoint was evaluated under is refused rather than served.
+Any behaviour change belongs upstream in the shipping tree, not here.
+
+Not copied, because the BarScript serving path never reaches them:
+``data.py`` (``generate.generate_song`` imports ``song_summary`` lazily; that is
+the 1.x time-mode decoder), ``hf.py`` (``generate.load_hf``; this Space loads
+2.0 from a local directory and must not need a Hub client), and the training,
+evaluation and grid-fitting modules.
+"""

softchart/sc2/barscript.py

@@ -0,0 +1,1153 @@
+"""BarScript encoding layer: SoftChart 2.0 bar-scoped multi-stage stream.
+
+Implements the content layer of SOFTCHART2_DESIGN.md on top of the slot+ S0
+command trace (softchart.trace): the trace -> token reordering of §1.1, the
+deterministic stage-label derivation of §3 (versioned ``barscript_labels_v1``),
+the grammar FSM v2 with the §8.1 plan->decode hard constraints, and the
+deterministic re-zip inverse that the UL v2 symbolizer consumes (§1.2).
+
+Per-bar token layout (§1.1)
+---------------------------
+::
+
+    [BAR] BAR_DENOM_D                                  <- header stage
+    DENS_D8_d SPAN_BUDGET_s BAR_SYNC_y MOTIF_m          <- plan stage (4 slots)
+    ( SPAN_START? SPAN_END? POS_i )*                    <- skeleton stage
+    [REAL] ( TYPE (SPAN_DUR_b)? )*                      <- realization stage
+
+followed by the next ``[BAR]`` or ``EOS``.  POSITION tokens reuse the
+``vocab.time0`` family as BAR-LOCAL lattice indices ``i`` with exact position
+``i / D`` (v3 slot-mode precedent; keeps hit token counts identical to the
+flat interleaved encoding, §1.1 "hit 的 token 總數不變").  ``D`` is the bar's
+rational-lattice denominator = LCM of the reduced denominators of the bar's
+note positions (SLOTPLUS_DESIGN §2.1 "authored 分割的最小公倍細分"), which is
+canonical under authored zero-padding; ``D = 1`` for empty bars; bars needing
+``D > MAX_BAR_DENOM`` fail closed (OffLatticeError), never snapped.
+
+Realization emits exactly one TYPE token per skeleton position, in position
+order.  A skeleton position preceded by ``SPAN_START`` realizes as a span
+class (roll / roll_big / balloon) followed by its duration bucket token
+(§1.3 span duration head supervision); a position preceded by ``SPAN_END``
+realizes as the ``end`` class; unmarked positions realize as hit classes.
+
+Label derivation (``barscript_labels_v1``, all pure functions of the trace
+and a frozen BarscriptSpec; §3)
+-------------------------------
+- DENS_d8: count of non-``end`` onsets in the bar, binned by
+  ``spec.dens_edges`` (bucket = number of edges <= count; the same edges give
+  the FSM's [L, U] legal note-count interval, §8.1).
+- SPAN_BUDGET: number of span starts in the bar, capped at 3+ .
+- Span duration: exact length in quarter-note beats (meter-state weighted
+  across bars, tempo-independent), binned on the ``spec.span_dur_edges`` log
+  grid.
+- SYNC: Longuet-Higgins & Lee note-rest syncopation banded by
+  ``spec.sync_edges``.  Deterministic variant frozen here: metric weight
+  w(0) = 0, w(beat) = -1, w(other) = -(1 + ceil(log2(q))) with q the reduced
+  within-beat denominator; onsets are the bar's non-``end`` notes; rest
+  candidates live on the eighth-note grid (precedent: data.py lhl_band
+  "simplified to the eighth-note grid") plus the next bar's downbeat when
+  that downbeat carries no onset; each onset scores max(w(rest)) - w(onset)
+  over rests strictly before the next onset, when positive.
+- MOTIF_REF: bar occupancy signature = ordered tuple of exact positions of
+  non-``end`` onsets (type-agnostic: occupancy is a placement decision; the
+  realizer owns types).  Exact match against bar b-δ for δ in (1, 2, 4, 8)
+  ascending -> REPEAT_δ; else normalized Levenshtein distance (over position
+  sequences, / max length) minimized over available δ (ties -> smaller δ)
+  <= spec.motif_tau -> VAR_δ; else NEW.  Bars with no δ-predecessor are NEW.
+
+Grammar FSM v2 (§1.2, §8.1)
+---------------------------
+``BarscriptFSM`` returns a per-step legal-token mask over the vocab.  Hard
+constraints: header -> plan (4 fixed slots; REPEAT_δ/VAR_δ masked when fewer
+than δ bars precede) -> skeleton (positions strictly increasing on the bar's
+D grid; DENS bucket bounds the count of non-end positions to [L, U]: [REAL]
+is masked below L, new countable positions are masked at U; SPAN_BUDGET is a
+consumed budget masking SPAN_START at zero, 3+ = unbounded upper bucket) ->
+[REAL] -> exactly N_onset type tokens role-constrained by the skeleton.
+Cross-bar open-span carry: while a span is open, SPAN_START and plain
+positions are masked (a span may not swallow hits), SPAN_END is the only way
+to place content, EOS is masked (no orphan starts), and the state survives
+[BAR] boundaries (multi-bar rolls/balloons).  Position masks reserve room so
+the DENS lower bound stays reachable; as a documented fail-open escape hatch
+(generation robustness, never exercised by gold sequences) [REAL] becomes
+legal when no skeleton token could otherwise be emitted.
+
+MOTIF_REF hard constraint (``motif_hard=True``, the default)
+------------------------------------------------------------
+A committed ``REPEAT_δ`` plan token compiles into the skeleton exactly the
+way DENS and SPAN_BUDGET already do: the bar's countable (non-``end``)
+positions are forced to reproduce, index for index, the occupancy signature
+of bar ``b-δ`` — the same signature the label derivation compares
+(``occupancy_signature`` / ``motif_ref``), so a gold ``REPEAT_δ`` bar
+satisfies it by construction and teacher forcing is untouched.  ``VAR_δ``
+(normalized edit distance <= τ, not an equality) and ``NEW`` stay soft.
+
+The signature is a tuple of EXACT rational positions, so it is
+denominator-free: a bar whose ``BAR_DENOM`` differs from its δ-predecessor's
+still counts as a REPEAT whenever every predecessor position lands on this
+bar's lattice (``bar_labels`` never looks at ``D``; the FSM matches that
+reading and only requires representability).
+
+Conflicts are resolved by PREVENTION at the plan stage, which is where the
+priority order lives.  ``BAR_DENOM`` (header), ``DENS`` (plan0) and
+``SPAN_BUDGET`` (plan1) are all committed before ``MOTIF`` (plan3), so at
+plan3 the FSM masks every ``REPEAT_δ`` it could not honour: signature not
+representable on this bar's ``D``, ``len(signature)`` outside the DENS
+``[L, U]`` interval, or a carried open span that would have to close before
+a first target position sitting at index 0.  ``NEW`` is always legal, so the
+plan mask is never empty and the skeleton stage can never dead-end.  Within
+the skeleton the carry rule keeps absolute priority (a span still may not
+swallow hits): while a span is open only ``SPAN_END`` places content, and its
+position is additionally bounded to stay strictly before the next forced
+target.  ``SPAN_START`` is offered only where the following target leaves a
+slot for the close, so committing to a span can never strand the FSM.
+Redundant references are also masked: ``REPEAT_δ`` is illegal when a smaller
+available δ' has the identical signature, because the realized bar would
+relabel as ``REPEAT_δ'`` — gold always emits the minimal δ, so this is a
+pure canonicalization.  ``motif_hard=False`` restores the pre-v18 soft
+behaviour for ablations.
+
+Scope rulings (documented, with authority)
+------------------------------------------
+- slot+ header state tokens (TEMPO/MEASURE/DELAY/k families) are the time
+  layer's vocabulary (SLOTPLUS_DESIGN §3.2 puts their values on regression /
+  classification heads; §9.2 defers their token families).  This content
+  layer anchors them at the ``[BAR]`` header insertion point but does not
+  invent them: TEMPO/MEASURE/DELAY events pass through encoding untouched
+  (they move no lattice position, so the (pos, type) re-zip is exact without
+  them) and labels that need meter (beats, LHL) read it from the trace.
+- ``barscript_to_events`` restores the (bar, pos, type-class) event stream
+  only (the task's re-zip inverse); '9' kusudama collapses into the balloon
+  class exactly as in vocab.NOTE_TYPE_MAP, so equality checks against the
+  flat encoding and against the trace are at note-class level.
+- Span-semantics normalization (coordinator ruling 2026-07-25, full-corpus
+  diagnostic ``span_diag.json``: 328 fail-closed charts, 98.2% one pattern;
+  BALLOON header counts are the metadata evidence that the game engine treats
+  these as decorations of the OPEN span, not new spans — implicit
+  close+reopen would break the balloon counts): while a span is open, a
+  SAME-FAMILY span-start event is dropped and counted (balloon family:
+  balloon/kusudama chars are interchangeable, so any balloon-class start
+  inside an open balloon; roll family: exactly roll->roll). A span-end event
+  with no open span is likewise dropped and counted (engine ignores it).
+  The dropped events vanish from the encoding entirely: lattice denominators,
+  DENS counts, occupancy signatures and the re-zip reference stream are all
+  computed on the NORMALIZED stream (``normalized_events``); the flat B0
+  encoding applies the identical normalization so both arms share one data
+  population. Counts return via ``return_stats=True`` /
+  ``normalized_events``. Everything else stays fail-closed: cross-family
+  restarts (e.g. roll open + balloon start), roll_big->roll_big (unobserved,
+  not sanctioned by the ruling — pending adjudication if it ever appears),
+  hits strictly inside an open span, and spans left open at end of chart.
+  trace.py stays untouched (S0 verbatim contract): normalization lives
+  strictly in this encoding layer.
+"""
+
+import json
+import math
+from dataclasses import dataclass, field
+from fractions import Fraction
+
+import numpy as np
+
+from .trace import NoteEvent, note_class
+from .vocab import (MAX_BAR_DENOM, MOTIF_DELTAS, MOTIF_REF_NAMES, N_BAR_SYNC,
+                    N_DENS_D8, N_SPAN_BUDGET, N_SPAN_DUR_BUCKETS,
+                    NOTE_CLASSES, STAGE_HEADER, STAGE_PLAN, STAGE_REALIZATION,
+                    STAGE_SKELETON, VOCAB, WINDOW)
+
+HIT_CLASSES = ("don", "ka", "don_big", "ka_big")
+SPAN_CLASSES = ("roll", "roll_big", "balloon")
+
+LABELS_VERSION = "barscript_labels_v1"
+
+
+class BarscriptError(ValueError):
+    """A construct the BarScript layer refuses to guess about."""
+
+
+class OffLatticeError(BarscriptError):
+    """Bar needs a lattice denominator beyond MAX_BAR_DENOM (fail-closed)."""
+
+
+class SpanStructureError(BarscriptError):
+    """Authored span structure is inconsistent (orphan start/end, swallowed
+    hit inside an open span). Fail-closed and counted, never repaired."""
+
+
+class IllegalTokenError(BarscriptError):
+    """A token outside the FSM's legal mask was stepped."""
+
+
+# ---------------------------------------------------------------------------
+# spec
+
+
+def _as_tuple(x):
+    return tuple(x) if isinstance(x, (list, tuple)) else x
+
+
+@dataclass(frozen=True)
+class BarscriptSpec:
+    """Frozen label-derivation parameters (§3: bucket edges and τ are NOT
+    hard-coded; they load from the spec artifact built on the train split).
+
+    Defaults are DRAFT values for local development; the frozen artifact from
+    scripts/build_barscript_spec.py supersedes them before training.
+    """
+
+    labels_version: str = LABELS_VERSION
+    # 7 strictly increasing positive ints -> 8 DENS_d8 buckets over per-bar
+    # non-end onset counts; bucket b = number of edges <= count, so bucket b
+    # spans [edges[b-1], edges[b]-1] (b=0: [0, edges[0]-1]; top: unbounded).
+    dens_edges: tuple = (1, 3, 5, 8, 11, 14, 18)
+    # 15 strictly increasing floats (quarter-note beats, log-ish grid) -> 16
+    # span-duration buckets; frozen grid anchors on authored p50/p90/p99.
+    span_dur_edges: tuple = (0.25, 0.375, 0.5, 0.75, 1.0, 1.5, 2.0, 3.0, 4.0,
+                             6.0, 8.0, 12.0, 16.0, 24.0, 32.0)
+    # 5 strictly increasing positive ints -> 6 LHL bands over per-bar scores.
+    sync_edges: tuple = (1, 2, 3, 5, 8)
+    # normalized-edit-distance threshold for VAR (frozen after the τ sweep).
+    motif_tau: float = 0.25
+    motif_deltas: tuple = MOTIF_DELTAS
+    provenance: dict = field(default_factory=dict, compare=False)
+
+    def __post_init__(self):
+        object.__setattr__(self, "dens_edges", _as_tuple(self.dens_edges))
+        object.__setattr__(self, "span_dur_edges",
+                           _as_tuple(self.span_dur_edges))
+        object.__setattr__(self, "sync_edges", _as_tuple(self.sync_edges))
+        object.__setattr__(self, "motif_deltas",
+                           _as_tuple(self.motif_deltas))
+        if self.labels_version != LABELS_VERSION:
+            raise BarscriptError(
+                f"unsupported labels_version {self.labels_version!r} "
+                f"(this code implements {LABELS_VERSION!r})")
+        for name, edges, n in (("dens_edges", self.dens_edges, N_DENS_D8),
+                               ("span_dur_edges", self.span_dur_edges,
+                                N_SPAN_DUR_BUCKETS),
+                               ("sync_edges", self.sync_edges, N_BAR_SYNC)):
+            if len(edges) != n - 1:
+                raise BarscriptError(
+                    f"{name} must have {n - 1} edges, got {len(edges)}")
+            if any(b <= a for a, b in zip(edges, edges[1:])):
+                raise BarscriptError(f"{name} must be strictly increasing")
+        if self.dens_edges[0] < 1 or any(
+                int(e) != e for e in self.dens_edges):
+            raise BarscriptError("dens_edges must be positive integers")
+        if self.motif_deltas != MOTIF_DELTAS:
+            raise BarscriptError(
+                f"motif_deltas is frozen to {MOTIF_DELTAS} by the vocab "
+                "MOTIF_REF token family")
+        if not (0.0 <= self.motif_tau <= 1.0):
+            raise BarscriptError("motif_tau must be in [0, 1]")
+
+    # --- deterministic binning -------------------------------------------
+    def dens_bucket(self, count):
+        return sum(1 for e in self.dens_edges if e <= count)
+
+    def dens_bounds(self, bucket):
+        """Legal non-end onset count interval [L, U] for a DENS bucket
+        (U is None for the unbounded top bucket). §8.1 hard constraint."""
+        lo = 0 if bucket == 0 else int(self.dens_edges[bucket - 1])
+        hi = (None if bucket >= len(self.dens_edges)
+              else int(self.dens_edges[bucket]) - 1)
+        return lo, hi
+
+    def span_budget_bucket(self, count):
+        return min(count, N_SPAN_BUDGET - 1)
+
+    def span_dur_bucket(self, beats):
+        return sum(1 for e in self.span_dur_edges if e <= float(beats))
+
+    def sync_band(self, score):
+        return min(sum(1 for e in self.sync_edges if e <= score),
+                   N_BAR_SYNC - 1)
+
+    # --- (de)serialization ------------------------------------------------
+    def to_dict(self):
+        return {"labels_version": self.labels_version,
+                "dens_edges": list(self.dens_edges),
+                "span_dur_edges": list(self.span_dur_edges),
+                "sync_edges": list(self.sync_edges),
+                "motif_tau": self.motif_tau,
+                "motif_deltas": list(self.motif_deltas),
+                "provenance": dict(self.provenance)}
+
+    @classmethod
+    def from_dict(cls, d):
+        if "spec" in d and "labels_version" not in d:
+            d = d["spec"]  # accept the build_barscript_spec.py artifact
+        known = {"labels_version", "dens_edges", "span_dur_edges",
+                 "sync_edges", "motif_tau", "motif_deltas", "provenance"}
+        return cls(**{k: v for k, v in d.items() if k in known})
+
+    @classmethod
+    def load(cls, path):
+        with open(path, encoding="utf-8") as f:
+            return cls.from_dict(json.load(f))
+
+    def save(self, path):
+        with open(path, "w", encoding="utf-8") as f:
+            json.dump(self.to_dict(), f, indent=2)
+            f.write("\n")
+
+
+# ---------------------------------------------------------------------------
+# trace -> per-bar content
+
+
+def _bar_notes(trace):
+    """Per-bar ordered [(pos: Fraction, class: str)] over trace.bars."""
+    bars = [[] for _ in trace.bars]
+    for ev in trace.events:
+        if isinstance(ev, NoteEvent):
+            cls = note_class(ev.type)
+            if cls is None:
+                raise BarscriptError(
+                    f"bar {ev.bar}: unknown note char {ev.type!r}")
+            bars[ev.bar].append((ev.pos, cls))
+    return bars
+
+
+def bar_lattice_denom(notes):
+    """Canonical lattice denominator: LCM of reduced position denominators
+    (1 for an empty bar). Fail-closed above MAX_BAR_DENOM (never snapped)."""
+    d = 1
+    for pos, _cls in notes:
+        d = d * pos.denominator // math.gcd(d, pos.denominator)
+        if d > MAX_BAR_DENOM:
+            raise OffLatticeError(
+                f"bar lattice denominator {d} exceeds support "
+                f"{MAX_BAR_DENOM} (fail-closed)")
+    return d
+
+
+def _same_family_restart(open_cls, cls):
+    """Sanctioned decoration pattern (ruling 2026-07-25): balloon-family
+    starts are interchangeable inside an open balloon (7/9 chars both map to
+    the balloon class); in the roll family exactly roll->roll is sanctioned.
+    roll_big->roll_big and all cross-family restarts stay fail-closed."""
+    return ((open_cls == "balloon" and cls == "balloon")
+            or (open_cls == "roll" and cls == "roll"))
+
+
+def _normalize_spans(trace, bars_notes):
+    """Span-semantics normalization + global span pairing (fail-closed for
+    everything the ruling does not sanction; module docstring).
+
+    Returns (norm_bars, spans, start_index, stats): norm_bars mirrors
+    bars_notes with ignored events dropped; spans carry exact (bar, pos)
+    endpoints and duration in quarter-note beats; start_index maps each kept
+    span start's (bar, pos) to its span dict; stats counts
+    {"ignored_start", "ignored_end"}.
+    """
+    beats = [Fraction(4 * b.num, b.den) for b in trace.bars]
+    norm_bars = [[] for _ in bars_notes]
+    spans = []
+    start_index = {}
+    stats = {"ignored_start": 0, "ignored_end": 0}
+    open_span = None
+    for bar_i, notes in enumerate(bars_notes):
+        for pos, cls in notes:
+            if cls in SPAN_CLASSES:
+                if open_span is None:
+                    open_span = {"bar": bar_i, "pos": pos, "class": cls}
+                    norm_bars[bar_i].append((pos, cls))
+                elif _same_family_restart(open_span["class"], cls):
+                    stats["ignored_start"] += 1  # decoration; engine ignores
+                else:
+                    raise SpanStructureError(
+                        f"bar {bar_i} pos {pos}: {cls} span start while the "
+                        f"{open_span['class']} span opened at bar "
+                        f"{open_span['bar']} is open (not a sanctioned "
+                        "same-family decoration)")
+            elif cls == "end":
+                if open_span is None:
+                    stats["ignored_end"] += 1  # orphan end; engine ignores
+                    continue
+                dur = Fraction(0)
+                for b in range(open_span["bar"], bar_i + 1):
+                    lo = open_span["pos"] if b == open_span["bar"] else Fraction(0)
+                    hi = pos if b == bar_i else Fraction(1)
+                    dur += (hi - lo) * beats[b]
+                span = dict(open_span, end_bar=bar_i, end_pos=pos,
+                            dur_beats=dur)
+                spans.append(span)
+                start_index[(open_span["bar"], open_span["pos"])] = span
+                open_span = None
+                norm_bars[bar_i].append((pos, cls))
+            else:
+                if open_span is not None:
+                    raise SpanStructureError(
+                        f"bar {bar_i} pos {pos}: hit inside the span opened "
+                        f"at bar {open_span['bar']} (swallowed hit)")
+                norm_bars[bar_i].append((pos, cls))
+    if open_span is not None:
+        raise SpanStructureError(
+            f"span opened at bar {open_span['bar']} pos {open_span['pos']} "
+            "never closes (orphan start)")
+    return norm_bars, spans, start_index, stats
+
+
+def normalized_events(trace):
+    """Post-normalization (bar, pos, class) stream + normalization stats.
+
+    This is the re-zip REFERENCE stream: both the flat and the BarScript
+    encodings reproduce exactly these events. Equal to trace_events(trace)
+    with stats == zeros for charts that need no normalization."""
+    norm_bars, _spans, _idx, stats = _normalize_spans(trace, _bar_notes(trace))
+    events = [(b, pos, cls) for b, notes in enumerate(norm_bars)
+              for pos, cls in notes]
+    return events, stats
+
+
+# --- LHL syncopation (barscript_labels_v1 frozen variant) ------------------
+
+
+def metric_weight(pos, num):
+    """LHL metric weight of an in-bar position (Fraction in [0,1)); num =
+    metrical beats per bar. 0 = downbeat, -1 = beat, deeper = weaker."""
+    if pos == 0:
+        return 0
+    beat = pos * num
+    if beat.denominator == 1:
+        return -1
+    frac = beat - (beat.numerator // beat.denominator)
+    q = frac.denominator  # >= 2
+    return -(1 + (q - 1).bit_length())  # 1 + ceil(log2(q))
+
+
+def lhl_score(onsets, num, next_downbeat_has_onset):
+    """Bar-level LHL note-rest syncopation score (non-negative int)."""
+    ons = sorted(set(onsets))
+    if not ons:
+        return 0
+    onset_set = set(ons)
+    grid = [Fraction(k, 2 * num) for k in range(1, 2 * num)]
+    score = 0
+    for i, p in enumerate(ons):
+        bound = ons[i + 1] if i + 1 < len(ons) else Fraction(1)
+        best = None
+        for q in grid:
+            if p < q < bound and q not in onset_set:
+                w = metric_weight(q, num)
+                if best is None or w > best:
+                    best = w
+        if i == len(ons) - 1 and not next_downbeat_has_onset:
+            best = 0 if best is None else max(best, 0)
+        wp = metric_weight(p, num)
+        if best is not None and best > wp:
+            score += best - wp
+    return score
+
+
+# --- motif reference -------------------------------------------------------
+
+
+def occupancy_signature(notes):
+    """Ordered tuple of exact positions of the bar's non-end onsets."""
+    return tuple(pos for pos, cls in notes if cls != "end")
+
+
+def norm_edit_distance(a, b):
+    """Levenshtein over position sequences, normalized by max length
+    (0.0 for two empty sequences)."""
+    if not a and not b:
+        return 0.0
+    if not a or not b:
+        return 1.0
+    prev = list(range(len(b) + 1))
+    for i, x in enumerate(a, 1):
+        cur = [i]
+        for j, y in enumerate(b, 1):
+            cur.append(min(prev[j] + 1, cur[-1] + 1,
+                           prev[j - 1] + (0 if x == y else 1)))
+        prev = cur
+    return prev[-1] / max(len(a), len(b))
+
+
+def motif_ref(signatures, bar_i, tau, deltas=MOTIF_DELTAS):
+    """(name, distance) for bar bar_i against its δ-predecessors.
+
+    Exact match at the smallest available δ wins REPEAT_δ; otherwise the
+    minimal normalized edit distance over available δ (ties -> smaller δ)
+    yields VAR_δ when <= τ; otherwise NEW. distance is None for NEW-by-
+    no-predecessor, 0.0 for REPEAT, the winning distance otherwise.
+    """
+    sig = signatures[bar_i]
+    avail = [d for d in deltas if bar_i - d >= 0]
+    if not avail:
+        return "new", None
+    for d in avail:
+        if signatures[bar_i - d] == sig:
+            return f"repeat_{d}", 0.0
+    best_d, best = None, None
+    for d in avail:
+        dist = norm_edit_distance(sig, signatures[bar_i - d])
+        if best is None or dist < best:
+            best_d, best = d, dist
+    if best is not None and best <= tau:
+        return f"var_{best_d}", best
+    return "new", best
+
+
+# ---------------------------------------------------------------------------
+# labels
+
+
+def _bar_labels_impl(trace, spec):
+    """(labels, normalization stats); labels are computed on the NORMALIZED
+    note stream (ignored decorations do not shape lattice denominators, DENS
+    counts, occupancy signatures or LHL scores)."""
+    norm_bars, _spans, start_index, stats = _normalize_spans(
+        trace, _bar_notes(trace))
+    signatures = [occupancy_signature(n) for n in norm_bars]
+    out = []
+    for i, notes in enumerate(norm_bars):
+        denom = bar_lattice_denom(notes)
+        onset_pos = [pos for pos, cls in notes if cls != "end"]
+        n_onsets = len(onset_pos)
+        next_db = (i + 1 < len(norm_bars)
+                   and any(pos == 0 and cls != "end"
+                           for pos, cls in norm_bars[i + 1]))
+        score = lhl_score(onset_pos, trace.bars[i].num, next_db)
+        ref, dist = motif_ref(signatures, i, spec.motif_tau,
+                              spec.motif_deltas)
+        spans_here = []
+        for pos, cls in notes:
+            if cls in SPAN_CLASSES:
+                sp = start_index[(i, pos)]
+                spans_here.append({
+                    "pos": pos, "class": cls,
+                    "dur_beats": sp["dur_beats"],
+                    "dur_bucket": spec.span_dur_bucket(sp["dur_beats"]),
+                    "end_bar": sp["end_bar"], "end_pos": sp["end_pos"]})
+        out.append({
+            "bar": i, "denom": denom, "notes": notes,
+            "n_onsets": n_onsets,
+            "dens_bucket": spec.dens_bucket(n_onsets),
+            "n_span_starts": len(spans_here),
+            "span_budget_bucket": spec.span_budget_bucket(len(spans_here)),
+            "sync_score": score, "sync_band": spec.sync_band(score),
+            "motif_ref": ref, "motif_distance": dist,
+            "spans": spans_here,
+        })
+    return out, stats
+
+
+def bar_labels(trace, spec):
+    """Deterministic per-bar labels (barscript_labels_v1). Pure function of
+    (trace, spec); §3. Returns a list with one dict per trace bar (computed
+    on the normalized note stream; use normalized_events for the stats)."""
+    return _bar_labels_impl(trace, spec)[0]
+
+
+# ---------------------------------------------------------------------------
+# encoding
+
+
+def _pos_token(pos, denom):
+    idx = pos * denom
+    if idx.denominator != 1:
+        raise OffLatticeError(
+            f"position {pos} does not sit on the /{denom} lattice")
+    idx = int(idx)
+    if not 0 <= idx < min(denom, WINDOW):
+        raise BarscriptError(f"position index {idx} out of [0, {denom})")
+    return VOCAB.time(idx)
+
+
+def trace_to_barscript(trace, spec, return_stats=False):
+    """Encode a trace as the BarScript stream. Returns (token_ids, stage_ids)
+    (parallel lists; stages use vocab.STAGE_*), or (token_ids, stage_ids,
+    stats) with ``return_stats=True`` where stats carries the span
+    normalization counters. Ends with EOS; the window prefix ([BOS],
+    conditions) is the train-side builder's job."""
+    labels, stats = _bar_labels_impl(trace, spec)
+    v = VOCAB
+    tokens, stages = [], []
+
+    def put(tok, stage):
+        tokens.append(tok)
+        stages.append(stage)
+
+    for lab in labels:
+        denom = lab["denom"]
+        put(v.bar, STAGE_HEADER)
+        put(v.bar_denom[denom], STAGE_HEADER)
+        put(v.bar_dens[lab["dens_bucket"]], STAGE_PLAN)
+        put(v.span_budget[lab["span_budget_bucket"]], STAGE_PLAN)
+        put(v.bar_sync[lab["sync_band"]], STAGE_PLAN)
+        put(v.motif[lab["motif_ref"]], STAGE_PLAN)
+        for pos, cls in lab["notes"]:
+            if cls in SPAN_CLASSES:
+                put(v.span_start, STAGE_SKELETON)
+            elif cls == "end":
+                put(v.span_end, STAGE_SKELETON)
+            put(_pos_token(pos, denom), STAGE_SKELETON)
+        put(v.real, STAGE_REALIZATION)
+        span_iter = iter(lab["spans"])
+        for pos, cls in lab["notes"]:
+            put(v.note[cls], STAGE_REALIZATION)
+            if cls in SPAN_CLASSES:
+                put(v.span_dur[next(span_iter)["dur_bucket"]],
+                    STAGE_REALIZATION)
+    put(v.eos, STAGE_HEADER)
+    if return_stats:
+        return tokens, stages, stats
+    return tokens, stages
+
+
+def trace_to_flat(trace, return_stats=False):
+    """Flat interleaved encoding (the B0 control arm's ordering): per bar
+    [BAR] BAR_DENOM_D then (POS, TYPE) pairs in stream order. Same position
+    and type token families as BarScript; no plan/skeleton/duration tokens.
+    Applies the IDENTICAL span normalization as trace_to_barscript so both
+    arms train on one data population."""
+    norm_bars, _spans, _idx, stats = _normalize_spans(trace, _bar_notes(trace))
+    v = VOCAB
+    tokens = []
+    for notes in norm_bars:
+        denom = bar_lattice_denom(notes)
+        tokens += [v.bar, v.bar_denom[denom]]
+        for pos, cls in notes:
+            tokens += [_pos_token(pos, denom), v.note[cls]]
+    tokens.append(v.eos)
+    if return_stats:
+        return tokens, stats
+    return tokens
+
+
+# ---------------------------------------------------------------------------
+# re-zip (decoding back to events)
+
+
+def _tok_family(tok):
+    v = VOCAB
+    if tok == v.bar:
+        return "bar"
+    if tok == v.real:
+        return "real"
+    if tok == v.eos:
+        return "eos"
+    if tok == v.span_start:
+        return "span_start"
+    if tok == v.span_end:
+        return "span_end"
+    if v.time0 <= tok < v.time0 + WINDOW:
+        return "pos"
+    if tok in v.id2note:
+        return "note"
+    for fam in ("bar_denom", "bar_dens", "span_budget", "bar_sync", "motif",
+                "span_dur"):
+        if tok in getattr(v, fam).values():
+            return fam
+    return "other"
+
+
+def barscript_to_events(token_ids):
+    """Deterministic re-zip inverse: BarScript tokens -> ordered
+    [(bar_index, pos: Fraction, note_class)] (§1.2 "skeleton+realization
+    確定性 re-zip 回 (pos,type) 事件流"). Plan tokens are skipped; skeleton
+    and realization are zipped 1:1 in position order. Raises BarscriptError
+    on malformed structure."""
+    v = VOCAB
+    events = []
+    i, n = 0, len(token_ids)
+    bar_i = -1
+    denom_rev = {t: d for d, t in v.bar_denom.items()}
+    while i < n:
+        tok = token_ids[i]
+        if tok == v.eos:
+            return events
+        if tok != v.bar:
+            raise BarscriptError(f"token {i}: expected [BAR] or EOS")
+        bar_i += 1
+        i += 1
+        if i >= n or token_ids[i] not in denom_rev:
+            raise BarscriptError(f"token {i}: expected BAR_DENOM after [BAR]")
+        denom = denom_rev[token_ids[i]]
+        i += 1
+        # plan stage: skip plan-family tokens
+        while i < n and _tok_family(token_ids[i]) in (
+                "bar_dens", "span_budget", "bar_sync", "motif"):
+            i += 1
+        # skeleton stage
+        skeleton = []  # (idx, role)
+        while i < n:
+            fam = _tok_family(token_ids[i])
+            if fam == "real":
+                i += 1
+                break
+            role = "hit"
+            if fam in ("span_start", "span_end"):
+                role = fam
+                i += 1
+                fam = _tok_family(token_ids[i]) if i < n else None
+            if fam != "pos":
+                raise BarscriptError(
+                    f"token {i}: expected a POSITION token in skeleton")
+            idx = token_ids[i] - v.time0
+            if idx >= denom or (skeleton and idx <= skeleton[-1][0]):
+                raise BarscriptError(
+                    f"token {i}: position {idx} not strictly increasing "
+                    f"on the /{denom} grid")
+            skeleton.append((idx, role))
+            i += 1
+        else:
+            raise BarscriptError("missing [REAL] before end of stream")
+        # realization stage: exactly one type per skeleton entry
+        for idx, role in skeleton:
+            if i >= n or token_ids[i] not in v.id2note:
+                raise BarscriptError(
+                    f"token {i}: expected a TYPE token in realization")
+            cls = v.id2note[token_ids[i]]
+            i += 1
+            if role == "span_start":
+                if cls not in SPAN_CLASSES:
+                    raise BarscriptError(
+                        f"token {i - 1}: SPAN_START position realized as "
+                        f"{cls!r}")
+                if i < n and _tok_family(token_ids[i]) == "span_dur":
+                    i += 1
+            elif role == "span_end":
+                if cls != "end":
+                    raise BarscriptError(
+                        f"token {i - 1}: SPAN_END position realized as "
+                        f"{cls!r}")
+            elif cls not in HIT_CLASSES:
+                raise BarscriptError(
+                    f"token {i - 1}: unmarked position realized as {cls!r}")
+            events.append((bar_i, Fraction(idx, denom), cls))
+    raise BarscriptError("stream ended without EOS")
+
+
+def flat_to_events(token_ids):
+    """Flat-encoding inverse: [(bar_index, pos: Fraction, note_class)]."""
+    v = VOCAB
+    denom_rev = {t: d for d, t in v.bar_denom.items()}
+    events = []
+    bar_i, denom = -1, None
+    i, n = 0, len(token_ids)
+    while i < n:
+        tok = token_ids[i]
+        if tok == v.eos:
+            return events
+        if tok == v.bar:
+            bar_i += 1
+            i += 1
+            if i >= n or token_ids[i] not in denom_rev:
+                raise BarscriptError(
+                    f"token {i}: expected BAR_DENOM after [BAR]")
+            denom = denom_rev[token_ids[i]]
+            i += 1
+            continue
+        if _tok_family(tok) != "pos" or denom is None:
+            raise BarscriptError(f"token {i}: expected [BAR]/POS/EOS")
+        idx = tok - v.time0
+        i += 1
+        if i >= n or token_ids[i] not in v.id2note:
+            raise BarscriptError(f"token {i}: expected TYPE after POSITION")
+        events.append((bar_i, Fraction(idx, denom), v.id2note[token_ids[i]]))
+        i += 1
+    raise BarscriptError("stream ended without EOS")
+
+
+def trace_events(trace):
+    """RAW [(bar, pos, class)] stream straight from the trace (no span
+    normalization; the re-zip reference is normalized_events)."""
+    return [(ev.bar, ev.pos, note_class(ev.type))
+            for ev in trace.events if isinstance(ev, NoteEvent)]
+
+
+# ---------------------------------------------------------------------------
+# UL v2 symbolizer (re-zip consumer)
+
+
+def ul_symbols(events):
+    """Interleaved (IOI, class) symbol stream over a (bar, pos, class) event
+    stream — the same normalization as train.py _ngram_symbols (class ids
+    0..7; IOI = delta to the previous event's global position; the first
+    event carries no IOI symbol), lifted to the exact rational lattice.
+    Global position is bar + pos in bar units; any deterministic
+    symbolization gives identical statistics for identical event streams."""
+    syms = []
+    prev = None
+    for bar, pos, cls in events:
+        g = Fraction(bar) + pos
+        if prev is not None:
+            syms.append(("ioi", g - prev))
+        syms.append(("cls", NOTE_CLASSES.index(cls)))
+        prev = g
+    return syms
+
+
+def ul_4gram_counts(events):
+    """Counter of 4-grams over the UL symbol stream (the statistic the
+    unlikelihood objective conditions on)."""
+    from collections import Counter
+    syms = ul_symbols(events)
+    return Counter(tuple(syms[i:i + 4]) for i in range(len(syms) - 3))
+
+
+# ---------------------------------------------------------------------------
+# grammar FSM v2
+
+
+class BarscriptFSM:
+    """Stepwise legal-next-token mask for the BarScript grammar (§1.2 FSM v2
+    row + §8.1 plan->decode hard constraints; cross-bar open-span carry).
+
+    Usage: ``mask = fsm.legal_mask(); fsm.step(tok)``. ``step`` raises
+    IllegalTokenError (with a state summary) when tok is outside the mask.
+    ``gold_masks`` below batch-precomputes masks for training.
+    """
+
+    def __init__(self, spec, vocab=VOCAB, motif_hard=True):
+        self.spec = spec
+        self.v = vocab
+        self.motif_hard = bool(motif_hard)
+        self._denom_rev = {t: d for d, t in vocab.bar_denom.items()}
+        self._motif_rev = {t: m for m, t in vocab.motif.items()}
+        self.reset()
+
+    def reset(self):
+        self.phase = "start"  # start|denom|plan0..plan3|skel|skel_pos|real|real_dur
+        self.bar_i = -1
+        self.denom = None
+        self.dens_lo = self.dens_hi = None
+        self.budget = None  # remaining SPAN_START budget (None = unbounded)
+        self.open_span = False  # cross-bar carry state
+        self.last_pos = -1
+        self.n_count = 0  # non-end skeleton positions this bar
+        self.pending = None  # marker awaiting its position ("span_start"/"span_end")
+        self.skeleton = []  # (idx, role) for realization typing
+        self.real_i = 0
+        self.done = False
+        # MOTIF_REF hard constraint state
+        self.sigs = []  # per completed bar: occupancy signature (Fractions)
+        self.motif_ref = None  # committed MOTIF_REF name for the current bar
+        self.repeat_target = None  # forced countable indices (None = soft bar)
+        self.repeat_i = 0  # how many forced targets are already placed
+
+    # -- MOTIF_REF occupancy bookkeeping -----------------------------------
+    def _record_bar_sig(self):
+        """Append the completed bar's occupancy signature (exact rational
+        positions of countable skeleton entries).  Mirrors
+        ``occupancy_signature`` on the normalized note stream: span starts
+        count (they are onsets), span ends do not."""
+        sig = tuple(Fraction(idx, self.denom)
+                    for idx, role in self.skeleton if role != "span_end")
+        while len(self.sigs) < self.bar_i:  # defensive (truncated stream)
+            self.sigs.append(())
+        if len(self.sigs) == self.bar_i:
+            self.sigs.append(sig)
+        else:
+            self.sigs[self.bar_i] = sig
+
+    def _repeat_indices(self, delta):
+        """Forced countable indices for REPEAT_δ on this bar's lattice, or
+        None when the reference cannot be honoured under the already
+        committed BAR_DENOM / DENS / open-span state (plan-stage
+        prevention)."""
+        j = self.bar_i - delta
+        if j < 0 or j >= len(self.sigs) or self.denom is None:
+            return None
+        sig = self.sigs[j]
+        n = len(sig)
+        if n < self.dens_lo or (self.dens_hi is not None and n > self.dens_hi):
+            return None
+        limit = min(self.denom, WINDOW)
+        idxs = []
+        prev = -1
+        for pos in sig:
+            x = pos * self.denom
+            if x.denominator != 1:
+                return None  # off this bar's lattice
+            i = int(x)
+            if not (0 <= i < limit) or i <= prev:
+                return None
+            idxs.append(i)
+            prev = i
+        if self.open_span and idxs and idxs[0] == 0:
+            return None  # the carried span has nowhere to close first
+        return idxs
+
+    def _repeat_redundant(self, delta):
+        """True when a smaller available δ' carries the identical signature:
+        the realized bar would relabel as REPEAT_δ', so REPEAT_δ is masked
+        (gold always emits the minimal δ, so this never touches gold)."""
+        sig = self.sigs[self.bar_i - delta]
+        return any(d < delta and 0 <= self.bar_i - d < len(self.sigs)
+                   and self.sigs[self.bar_i - d] == sig
+                   for d in self.spec.motif_deltas)
+
+    def _repeat_remaining(self):
+        return (self.repeat_target is not None
+                and self.repeat_i < len(self.repeat_target))
+
+    def _repeat_span_start_ok(self):
+        """A span opened on the current target must leave a slot to close in
+        before the NEXT target (the last target may carry across the bar)."""
+        tgt, k = self.repeat_target, self.repeat_i
+        return k + 1 >= len(tgt) or tgt[k + 1] - tgt[k] >= 2
+
+    def _repeat_end_hi(self):
+        """Highest legal SPAN_END index: strictly before the next forced
+        target (or the bar's last slot once the targets are exhausted)."""
+        if self._repeat_remaining():
+            return self.repeat_target[self.repeat_i] - 1
+        return self.denom - 1
+
+    # -- masks -------------------------------------------------------------
+    def _legal_positions(self):
+        """Legal position indices for a plain (countable) skeleton position:
+        strictly increasing, on-grid, leaving room for the DENS lower bound
+        to stay reachable.  Under a committed REPEAT_δ the position is fully
+        determined by the reference occupancy."""
+        if self.denom is None:
+            return range(0)
+        if self.repeat_target is not None:
+            if not self._repeat_remaining():
+                return range(0)
+            nxt = self.repeat_target[self.repeat_i]
+            return range(nxt, nxt + 1)
+        lo = self.last_pos + 1
+        hi = self.denom - 1
+        need_after = max(self.dens_lo - (self.n_count + 1), 0)
+        return range(lo, min(hi, self.denom - 1 - need_after) + 1)
+
+    def _skel_mask_repeat(self, mask):
+        """Skeleton mask under a committed REPEAT_δ: countable positions are
+        forced onto the reference occupancy, [REAL] stays masked until every
+        target is placed, and the open-span carry keeps priority."""
+        v = self.v
+        remain = self._repeat_remaining()
+        any_skel = False
+        if self.open_span:
+            if self.last_pos + 1 <= self._repeat_end_hi():
+                mask[v.span_end] = True
+                any_skel = True
+        elif remain:
+            mask[v.time0 + self.repeat_target[self.repeat_i]] = True
+            any_skel = True
+            if ((self.budget is None or self.budget > 0)
+                    and self._repeat_span_start_ok()):
+                mask[v.span_start] = True
+        if not remain or not any_skel:
+            # targets met -> [REAL] closes the skeleton; the `not any_skel`
+            # arm is the same fail-open escape hatch as the soft path.
+            mask[v.real] = True
+
+    def _skel_mask(self, mask):
+        v = self.v
+        if self.repeat_target is not None:
+            self._skel_mask_repeat(mask)
+            return
+        room = self.last_pos < self.denom - 1
+        can_count = self.dens_hi is None or self.n_count < self.dens_hi
+        any_skel = False
+        if self.open_span:
+            # carry: only SPAN_END may place content (no swallowed hits, no
+            # nested spans); the span may also pass through ([REAL] below).
+            if room:
+                mask[v.span_end] = True
+                any_skel = True
+        else:
+            if can_count:
+                legal = self._legal_positions()
+                if len(legal):
+                    for i in legal:
+                        mask[v.time0 + i] = True
+                    any_skel = True
+                    # SPAN_START is only offered when a countable position
+                    # exists for it to land on: skel_pos reuses the same
+                    # _legal_positions() range, so gating on non-empty legal
+                    # keeps the never-dead-end guarantee (a room/budget-only
+                    # gate could open SPAN_START into an empty position mask
+                    # when the DENS room reservation exhausts the range,
+                    # e.g. a carried span closing late in a high-L bar).
+                    if self.budget is None or self.budget > 0:
+                        mask[v.span_start] = True
+        if self.n_count >= self.dens_lo or not any_skel:
+            # fail-open escape hatch (docstring): [REAL] is always legal when
+            # nothing else is, so generation can never dead-end. Gold
+            # sequences satisfy [L, U] by construction and never need it.
+            mask[v.real] = True
+
+    def legal_mask(self):
+        v = self.v
+        mask = np.zeros(v.size, dtype=bool)
+        if self.done:
+            return mask
+        ph = self.phase
+        if ph == "start":
+            mask[v.bar] = True
+            if not self.open_span:
+                mask[v.eos] = True
+        elif ph == "denom":
+            for t in v.bar_denom.values():
+                mask[t] = True
+        elif ph == "plan0":
+            for t in v.bar_dens.values():
+                mask[t] = True
+        elif ph == "plan1":
+            for t in v.span_budget.values():
+                mask[t] = True
+        elif ph == "plan2":
+            for t in v.bar_sync.values():
+                mask[t] = True
+        elif ph == "plan3":
+            mask[v.motif["new"]] = True
+            for name in MOTIF_REF_NAMES[1:]:
+                delta = int(name.rsplit("_", 1)[1])
+                if self.bar_i - delta < 0:
+                    continue
+                if self.motif_hard and name.startswith("repeat_"):
+                    # plan-stage prevention: never commit to a reference the
+                    # skeleton stage could not reproduce, and never offer a
+                    # δ that would relabel as a smaller one.
+                    if (self._repeat_indices(delta) is None
+                            or self._repeat_redundant(delta)):
+                        continue
+                mask[v.motif[name]] = True
+        elif ph == "skel":
+            self._skel_mask(mask)
+        elif ph == "skel_pos":
+            lo = self.last_pos + 1
+            if self.pending == "span_end":
+                # closing position: strictly increasing is the legality rule
+                # (same-bar closes sit after the start; cross-bar closes are
+                # later by construction); no DENS room reservation (an end
+                # position is not countable).  Under a committed REPEAT_δ the
+                # close must also land before the next forced target.
+                hi = (self._repeat_end_hi() if self.repeat_target is not None
+                      else self.denom - 1)
+                for i in range(lo, hi + 1):
+                    mask[v.time0 + i] = True
+            else:
+                for i in self._legal_positions():
+                    mask[v.time0 + i] = True
+        elif ph == "real":
+            idx, role = self.skeleton[self.real_i]
+            if role == "span_start":
+                for cls in SPAN_CLASSES:
+                    mask[v.note[cls]] = True
+            elif role == "span_end":
+                mask[v.note["end"]] = True
+            else:
+                for cls in HIT_CLASSES:
+                    mask[v.note[cls]] = True
+        elif ph == "real_dur":
+            for t in v.span_dur.values():
+                mask[t] = True
+        return mask
+
+    def state_summary(self):
+        motif = self.motif_ref or "-"
+        if self.repeat_target is not None:
+            motif += f"{self.repeat_target}@{self.repeat_i}"
+        return (f"bar={self.bar_i} phase={self.phase} denom={self.denom} "
+                f"dens=[{self.dens_lo},{self.dens_hi}] count={self.n_count} "
+                f"budget={self.budget} open_span={self.open_span} "
+                f"last_pos={self.last_pos} real_i={self.real_i} "
+                f"motif={motif}")
+
+    # -- transition --------------------------------------------------------
+    def step(self, tok):
+        mask = self.legal_mask()
+        if not (0 <= tok < len(mask)) or not mask[tok]:
+            raise IllegalTokenError(
+                f"token {tok} illegal at [{self.state_summary()}]")
+        v = self.v
+        ph = self.phase
+        if ph == "start":
+            if tok == v.eos:
+                self.done = True
+            else:
+                self.bar_i += 1
+                self.phase = "denom"
+        elif ph == "denom":
+            self.denom = self._denom_rev[tok]
+            self.last_pos = -1
+            self.n_count = 0
+            self.skeleton = []
+            self.real_i = 0
+            self.pending = None
+            self.motif_ref = None
+            self.repeat_target = None
+            self.repeat_i = 0
+            self.phase = "plan0"
+        elif ph == "plan0":
+            bucket = next(d for d, t in v.bar_dens.items() if t == tok)
+            self.dens_lo, self.dens_hi = self.spec.dens_bounds(bucket)
+            self.phase = "plan1"
+        elif ph == "plan1":
+            b = next(s for s, t in v.span_budget.items() if t == tok)
+            self.budget = None if b == N_SPAN_BUDGET - 1 else b
+            self.phase = "plan2"
+        elif ph == "plan2":
+            self.phase = "plan3"
+        elif ph == "plan3":
+            name = self._motif_rev[tok]
+            self.motif_ref = name
+            if self.motif_hard and name.startswith("repeat_"):
+                self.repeat_target = self._repeat_indices(
+                    int(name.rsplit("_", 1)[1]))
+            self.repeat_i = 0
+            self.phase = "skel"
+        elif ph == "skel":
+            if tok == v.real:
+                self._record_bar_sig()
+                self.phase = "real" if self.skeleton else "start"
+            elif tok == v.span_start:
+                self.pending = "span_start"
+                self.phase = "skel_pos"
+            elif tok == v.span_end:
+                self.pending = "span_end"
+                self.phase = "skel_pos"
+            else:  # plain position
+                self._add_pos(tok - v.time0, "hit")
+        elif ph == "skel_pos":
+            role = self.pending
+            self.pending = None
+            self._add_pos(tok - v.time0, role)
+            self.phase = "skel"
+        elif ph == "real":
+            _idx, role = self.skeleton[self.real_i]
+            if role == "span_start":
+                self.phase = "real_dur"
+            else:
+                self.real_i += 1
+                if self.real_i == len(self.skeleton):
+                    self.phase = "start"
+        elif ph == "real_dur":
+            self.real_i += 1
+            self.phase = "real" if self.real_i < len(self.skeleton) else "start"
+
+    def _add_pos(self, idx, role):
+        self.last_pos = idx
+        self.skeleton.append((idx, role))
+        if role == "span_end":
+            self.open_span = False
+            return
+        if role == "span_start":
+            self.open_span = True
+            if self.budget is not None:
+                self.budget -= 1
+        self.n_count += 1
+        if self.repeat_target is not None:
+            self.repeat_i += 1
+
+
+def gold_masks(token_ids, spec, motif_hard=True):
+    """Precompute the (L, V) bool legal-mask array for a gold sequence (the
+    batch path the trainer consumes; the stepwise FSM is the correctness
+    oracle it is verified against). Raises IllegalTokenError if the sequence
+    leaves the grammar."""
+    fsm = BarscriptFSM(spec, motif_hard=motif_hard)
+    masks = np.zeros((len(token_ids), VOCAB.size), dtype=bool)
+    for i, tok in enumerate(token_ids):
+        masks[i] = fsm.legal_mask()
+        fsm.step(tok)
+    return masks

softchart/sc2/checkpoint.py

@@ -0,0 +1,337 @@
+"""Strict generator-checkpoint metadata and architecture reconstruction.
+
+V1.8 condition semantics cannot be recovered from tensor shapes.  A missing
+``dual`` or ``ctx`` flag changes the decoder prefix while all weights still
+load successfully, so release loading must reject incomplete or contradictory
+metadata instead of guessing from state-dict keys.
+"""
+
+from __future__ import annotations
+
+import math
+from collections.abc import Mapping
+from numbers import Integral, Real
+
+import torch
+
+from .vocab import N_MELS, VOCAB
+
+
+# SoftChart 2.0 checkpoint args (2026-07-25 interface ruling): part of the
+# REQUIRED set + presence checks. Pre-2.0 checkpoints lack the keys entirely,
+# so parse_generator_checkpoint BACKFILLS them to False before validation —
+# older checkpoints keep loading bit-identically. ``axis_knobs`` is a pure
+# capability flag (big_rate/span_rate/stream prefix conditioning; no state
+# keys of its own).
+V2_MODEL_ARGS = frozenset({
+    "stage_emb", "span_duration_head", "tempo_head", "rich_section_stats",
+    "axis_knobs",
+})
+
+REQUIRED_GENERATOR_ARGS = frozenset({
+    "d_model", "nhead", "dropout", "enc_layers", "dec_layers", "ffn",
+    "emb_factor", "func_time", "aux", "global_ctx", "hierarchical_ctx",
+    "section_layers", "section_ffn", "clean_phase", "enc_share",
+    "dec_share", "adapter_rank", "unique_layernorm", "adapter_rank_ffn",
+    "depth_emb", "unshare_last_dec", "align", "beat_head", "beat_hires",
+    "ctx", "sibling", "style", "sync_token", "plan", "mask_infill",
+    "complexity", "slot", "dual",
+}) | V2_MODEL_ARGS
+
+_BOOL_ARGS = frozenset({
+    "func_time", "aux", "global_ctx", "hierarchical_ctx", "clean_phase",
+    "unique_layernorm", "depth_emb", "unshare_last_dec", "align",
+    "beat_head", "beat_hires", "ctx", "sibling", "style", "sync_token",
+    "plan", "complexity", "slot", "dual",
+}) | V2_MODEL_ARGS
+
+GENERATOR_CAPABILITY_KEYS = (
+    "ctx", "sibling", "style", "sync_token", "plan", "beat_head", "align",
+    "aux", "func_time", "mask_infill", "complexity", "slot", "dual",
+    "global_ctx", "hierarchical_ctx", "axis_knobs",
+)
+
+VOCAB_EXPANSION_SEED = 20260725  # deterministic init for appended token rows
+
+
+def expand_vocab_rows(state, vocab_size, seed=VOCAB_EXPANSION_SEED):
+    """Grow ``tok_emb.weight`` (and a tied ``out.weight``) to ``vocab_size``
+    rows, initializing the appended rows N(0, 0.02) exactly like ChartModel's
+    embedding init (deterministic via ``seed``).
+
+    The vocabulary only ever grows append-only (vocab.py contract), so an
+    older checkpoint's rows are a prefix of the current table. Returns a new
+    mapping; tensors that need no change are passed through untouched.
+    Raises when the checkpoint has MORE rows than requested (a newer-vocab
+    checkpoint cannot be truncated to an older runtime).
+    """
+    if "tok_emb.weight" not in state:
+        raise ValueError("state has no tok_emb.weight to expand")
+    old_rows = int(state["tok_emb.weight"].shape[0])
+    if old_rows > vocab_size:
+        raise ValueError(
+            f"checkpoint vocab has {old_rows} rows; cannot shrink to "
+            f"{vocab_size} (newer-vocab checkpoint under an older runtime)")
+    if old_rows == vocab_size:
+        return state
+    out = dict(state)
+    gen = torch.Generator(device="cpu").manual_seed(seed)
+    grown = None
+    for key in ("tok_emb.weight", "out.weight"):
+        w = state.get(key)
+        if w is None:
+            continue
+        if int(w.shape[0]) != old_rows:
+            raise ValueError(
+                f"{key} has {int(w.shape[0])} rows but tok_emb.weight has "
+                f"{old_rows}; refusing to expand an inconsistent state")
+        if grown is None:  # one shared block keeps tied weights tied
+            grown = torch.empty(
+                vocab_size - old_rows, int(w.shape[1]), dtype=torch.float32,
+            ).normal_(mean=0.0, std=0.02, generator=gen)
+        out[key] = torch.cat(
+            [w, grown.to(device=w.device, dtype=w.dtype)], dim=0)
+    return out
+
+
+def _state_has(state: Mapping[str, torch.Tensor], prefix: str) -> bool:
+    return any(key.startswith(prefix) for key in state)
+
+
+def _require_presence(
+    state: Mapping[str, torch.Tensor], prefix: str, expected: bool, label: str,
+) -> None:
+    present = _state_has(state, prefix)
+    if present != expected:
+        raise ValueError(
+            f"checkpoint args/state disagree for {label}: "
+            f"metadata={expected}, state={present}"
+        )
+
+
+def _positive_int(args: Mapping[str, object], name: str) -> int:
+    value = args[name]
+    if not isinstance(value, Integral) or isinstance(value, bool) or int(value) <= 0:
+        raise ValueError(f"checkpoint arg {name} must be a positive integer")
+    return int(value)
+
+
+def _nonnegative_int(args: Mapping[str, object], name: str) -> int:
+    value = args[name]
+    if not isinstance(value, Integral) or isinstance(value, bool) or int(value) < 0:
+        raise ValueError(f"checkpoint arg {name} must be a non-negative integer")
+    return int(value)
+
+
+def parse_generator_checkpoint(checkpoint: Mapping[str, object]) -> tuple[
+    Mapping[str, torch.Tensor], dict, dict, dict
+]:
+    """Validate and return ``state, args, model_config, capabilities``.
+
+    The returned configuration is suitable for ``ChartModel``.  Tensor shapes
+    are still checked a second time by its strict ``load_state_dict`` call.
+    """
+    if not isinstance(checkpoint, Mapping):
+        raise ValueError("generator checkpoint must be a mapping")
+    state = checkpoint.get("model")
+    args = checkpoint.get("args")
+    if not isinstance(state, Mapping) or not state:
+        raise ValueError("generator checkpoint model must be a non-empty mapping")
+    if not isinstance(args, Mapping):
+        raise ValueError("generator checkpoint args must be a mapping")
+    if not all(isinstance(key, str) and isinstance(value, torch.Tensor)
+               for key, value in state.items()):
+        raise ValueError("generator state must map string keys to tensors")
+    args = dict(args)
+    for name in V2_MODEL_ARGS:  # pre-2.0 checkpoints: backfill, keep loading
+        args.setdefault(name, False)
+    missing = sorted(REQUIRED_GENERATOR_ARGS - set(args))
+    if missing:
+        raise ValueError(f"generator checkpoint lacks required args: {missing}")
+
+    for name in _BOOL_ARGS:
+        if not isinstance(args[name], bool):
+            raise ValueError(f"checkpoint arg {name} must be boolean")
+    for name in ("d_model", "nhead", "enc_layers", "dec_layers", "ffn",
+                 "section_layers"):
+        _positive_int(args, name)
+    for name in ("emb_factor", "section_ffn", "adapter_rank",
+                 "adapter_rank_ffn"):
+        _nonnegative_int(args, name)
+    dropout = args["dropout"]
+    if (not isinstance(dropout, Real) or isinstance(dropout, bool)
+            or not math.isfinite(float(dropout)) or not 0.0 <= float(dropout) < 1.0):
+        raise ValueError("checkpoint arg dropout must be finite in [0, 1)")
+    if args["enc_share"] is not None and not isinstance(args["enc_share"], (str, int)):
+        raise ValueError("checkpoint arg enc_share must be null, integer, or string")
+    if args["dec_share"] is not None and not isinstance(args["dec_share"], (str, int)):
+        raise ValueError("checkpoint arg dec_share must be null, integer, or string")
+    if args["global_ctx"] and args["hierarchical_ctx"]:
+        raise ValueError("global_ctx and hierarchical_ctx are mutually exclusive")
+    if args["hierarchical_ctx"] and not args["aux"]:
+        raise ValueError("hierarchical_ctx requires aux skeleton supervision")
+    if args["beat_hires"] and not args["beat_head"]:
+        raise ValueError("beat_hires requires beat_head")
+    if (args["slot"] or args["dual"]) and not args["clean_phase"]:
+        raise ValueError("V1.8 slot/dual checkpoints require clean_phase")
+    if args["complexity"]:
+        raise ValueError(
+            "complexity-conditioned checkpoints are not supported by the "
+            "release generation API"
+        )
+
+    required_tensors = ("tok_emb.weight", "frontend.0.weight")
+    absent = [name for name in required_tensors if name not in state]
+    if absent:
+        raise ValueError(f"generator state lacks required tensors: {absent}")
+    token_weight = state["tok_emb.weight"]
+    front_weight = state["frontend.0.weight"]
+    if token_weight.ndim != 2 or front_weight.ndim != 3:
+        raise ValueError("generator embedding/frontend tensors have invalid rank")
+
+    d_model = int(args["d_model"])
+    nhead = int(args["nhead"])
+    if d_model % nhead:
+        raise ValueError("checkpoint d_model must be divisible by nhead")
+    vocab_size, embedding_width = map(int, token_weight.shape)
+    if vocab_size > VOCAB.size:
+        raise ValueError(
+            f"checkpoint vocab has {vocab_size} rows; current vocabulary has "
+            f"only {VOCAB.size} (newer-vocab checkpoint under an older runtime)"
+        )
+    if vocab_size < VOCAB.size:
+        # Append-only vocab growth (SoftChart 2.0): older checkpoints load by
+        # deterministically initializing the appended token rows.
+        state = expand_vocab_rows(state, VOCAB.size)
+        token_weight = state["tok_emb.weight"]
+        vocab_size = VOCAB.size
+    emb_factor = int(args["emb_factor"])
+    expected_width = emb_factor or d_model
+    if embedding_width != expected_width:
+        raise ValueError(
+            "checkpoint args/state disagree for emb_factor: "
+            f"expected width {expected_width}, state has {embedding_width}"
+        )
+    if int(front_weight.shape[0]) != d_model:
+        raise ValueError("frontend output channels disagree with d_model")
+    expected_front_channels = (
+        N_MELS if args["clean_phase"]
+        else (N_MELS + 2 if (args["slot"] or args["dual"]) else N_MELS)
+    )
+    if int(front_weight.shape[1]) != expected_front_channels:
+        raise ValueError(
+            "frontend input channels disagree with clean slot/time metadata: "
+            f"expected {expected_front_channels}, state has {front_weight.shape[1]}"
+        )
+
+    _require_presence(state, "emb_proj.", emb_factor > 0, "emb_factor")
+    _require_presence(
+        state, "section_encoder.", args["hierarchical_ctx"], "hierarchical_ctx")
+    _require_presence(
+        state, "skeleton_proj.", args["hierarchical_ctx"], "hierarchical skeleton")
+    _require_presence(state, "gsum_proj.", args["global_ctx"], "global_ctx")
+    _require_presence(state, "aux.", args["aux"], "aux")
+    _require_presence(state, "phase_proj.", args["clean_phase"], "clean_phase")
+    _require_presence(state, "time_proj.", args["func_time"], "func_time")
+    _require_presence(state, "ptr.", args["align"], "align")
+    _require_presence(state, "beat.", args["beat_head"], "beat_head")
+    _require_presence(
+        state, "beat.up.", args["beat_head"] and args["beat_hires"], "beat_hires")
+
+    # SoftChart 2.0 presence checks (state-key prefixes per the 2026-07-25
+    # interface ruling; axis_knobs carries no state keys)
+    _require_presence(state, "stage_emb", args["stage_emb"], "stage_emb")
+    _require_presence(state, "span_dur.", args["span_duration_head"],
+                      "span_duration_head")
+    _require_presence(state, "tempo_logp.", args["tempo_head"], "tempo_head")
+    _require_presence(state, "tempo_k.", args["tempo_head"], "tempo_head")
+    _require_presence(
+        state, "section_encoder.inp_rich.",
+        args["hierarchical_ctx"] and args["rich_section_stats"],
+        "rich_section_stats")
+    max_tgt = args.get("max_tgt")
+    if max_tgt is not None and (not isinstance(max_tgt, Integral)
+                                or isinstance(max_tgt, bool)
+                                or int(max_tgt) <= 0):
+        raise ValueError("checkpoint arg max_tgt must be a positive integer")
+
+    beat: bool | str = (
+        "hires" if args["beat_head"] and args["beat_hires"]
+        else bool(args["beat_head"])
+    )
+    model_config = {
+        "d_model": d_model,
+        "nhead": nhead,
+        "dropout": float(dropout),
+        "enc_layers": int(args["enc_layers"]),
+        "dec_layers": int(args["dec_layers"]),
+        "ffn": int(args["ffn"]),
+        "vocab_size": vocab_size,
+        "aux": bool(args["aux"]),
+        "global_ctx": bool(args["global_ctx"]),
+        "hierarchical_ctx": bool(args["hierarchical_ctx"]),
+        "section_layers": int(args["section_layers"]),
+        "section_ffn": int(args["section_ffn"]) or None,
+        "func_time": bool(args["func_time"]),
+        "ptr": bool(args["align"]),
+        "beat": beat,
+        "in_ch": N_MELS + 2 if (args["slot"] or args["dual"]) else N_MELS,
+        "emb_factor": emb_factor or None,
+        "clean_phase": bool(args["clean_phase"]),
+        "enc_share": args["enc_share"],
+        "dec_share": args["dec_share"],
+        "adapter_rank": int(args["adapter_rank"]),
+        "unique_layernorm": bool(args["unique_layernorm"]),
+        "adapter_rank_ffn": int(args["adapter_rank_ffn"]),
+        "depth_emb": bool(args["depth_emb"]),
+        "unshare_last_dec": bool(args["unshare_last_dec"]),
+        "stage_emb": bool(args["stage_emb"]),
+        "span_duration_head": bool(args["span_duration_head"]),
+        "tempo_head": bool(args["tempo_head"]),
+        "rich_section_stats": bool(args["rich_section_stats"]),
+    }
+    if max_tgt is not None:
+        model_config["max_tgt"] = int(max_tgt)
+    capabilities = {
+        name: (bool(args[name]) if name != "hierarchical_ctx"
+               else bool(args["hierarchical_ctx"]))
+        for name in GENERATOR_CAPABILITY_KEYS
+    }
+    # BarScript prefixes ALWAYS carry the sync slot -- [BOS] course level dens
+    # SYNC big_rate span_rate stream -- but train.py records the v1.8
+    # `sync_token` flag, which sc2_train.sh never sets.  A loader that trusted
+    # the recorded False would build a 7-token prefix for an 8-token model and
+    # silently collapse every sync bucket onto one conditioning: no error
+    # raised, just a dead knob.  Derive it from the mode here, at the single
+    # point where capabilities are read off the checkpoint, so every consumer
+    # (attach_generator_capabilities, the packager, the HF path) agrees.
+    # v1.8 artifacts (mode absent, or slot/time) keep the recorded flag.
+    if str(args.get("mode", "")) in ("b0", "c1"):
+        capabilities["sync_token"] = True
+    return state, args, model_config, capabilities
+
+
+def attach_generator_capabilities(model, capabilities: Mapping[str, bool]) -> None:
+    """Attach the exact semantic flags consumed by generation."""
+    capabilities = dict(capabilities)
+    # pre-2.0 capability dicts (released HF repos) lack the new knob flag
+    capabilities.setdefault("axis_knobs", False)
+    missing = sorted(set(GENERATOR_CAPABILITY_KEYS) - set(capabilities))
+    if missing:
+        raise ValueError(f"generator capabilities are incomplete: {missing}")
+    invalid = sorted(
+        name for name in GENERATOR_CAPABILITY_KEYS
+        if not isinstance(capabilities[name], bool)
+    )
+    if invalid:
+        raise ValueError(f"generator capabilities must be boolean: {invalid}")
+    model._has_axis_knobs = bool(capabilities["axis_knobs"])
+    model._has_ctx = bool(capabilities["ctx"])
+    model._has_sib = bool(capabilities["sibling"])
+    model._has_style = bool(capabilities["style"])
+    model._has_sync = bool(capabilities["sync_token"])
+    model._has_plan = bool(capabilities["plan"])
+    model._dual = bool(capabilities["dual"])
+    model._slot = bool(capabilities["slot"]) or model._dual
+    model._hierarchical_ctx = bool(capabilities["hierarchical_ctx"])
+    model._has_gctx = bool(capabilities["global_ctx"] or model._hierarchical_ctx)

softchart/sc2/meter.py

@@ -0,0 +1,125 @@
+"""Exact rational-meter helpers for the 96-slot-per-4/4 TJA lattice."""
+
+from fractions import Fraction
+from math import gcd
+from numbers import Real
+
+import numpy as np
+
+from .vocab import SLOTS, WINDOW
+
+
+def _expand(value, n_measures, name, dtype):
+    array = np.asarray(value)
+    if array.ndim == 0:
+        array = np.full(n_measures, array.item())
+    else:
+        array = array.reshape(-1)
+        if len(array) != n_measures:
+            raise ValueError(
+                f"{name} must contain one value per measure "
+                f"({n_measures}), got {len(array)}"
+            )
+    if np.issubdtype(np.dtype(dtype), np.integer):
+        info = np.iinfo(dtype)
+        values = []
+        for raw in array.flat:
+            if isinstance(raw, (bool, np.bool_)):
+                raise ValueError(f"{name} must contain finite integers")
+            if isinstance(raw, (int, np.integer)):
+                integer = int(raw)
+            elif isinstance(raw, (float, np.floating)):
+                number = float(raw)
+                if not np.isfinite(number) or not number.is_integer():
+                    raise ValueError(f"{name} must contain finite integers")
+                integer = int(number)
+            else:
+                raise ValueError(f"{name} must contain finite integers")
+            if not info.min <= integer <= info.max:
+                raise ValueError(f"{name} is outside the {np.dtype(dtype)} range")
+            values.append(integer)
+        return np.asarray(values, dtype=dtype).reshape(array.shape)
+    values = []
+    for raw in array.flat:
+        if isinstance(raw, (bool, np.bool_)) or not isinstance(raw, Real):
+            raise ValueError(f"{name} must contain numeric values")
+        values.append(float(raw))
+    return np.asarray(values, dtype=dtype).reshape(array.shape)
+
+
+def rational_meter_arrays(measure_num, measure_den, n_measures=None):
+    """Validate TJA ``#MEASURE`` fractions and return lattice properties.
+
+    Returns ``(slots, numerator, denominator, quarter_beats)``. A meter is
+    representable exactly iff ``96 * numerator / denominator`` is an integer;
+    no rounding or coercion is permitted.
+    """
+    num = np.asarray(measure_num)
+    den = np.asarray(measure_den)
+    if n_measures is None:
+        if num.ndim == 0 and den.ndim == 0:
+            n_measures = 1
+        else:
+            n_measures = max(num.size, den.size)
+    num = _expand(num, n_measures, "measure_num", np.int64)
+    den = _expand(den, n_measures, "measure_den", np.int64)
+    if (num <= 0).any() or (den <= 0).any():
+        raise ValueError("measure numerator and denominator must be positive")
+
+    slot_values = []
+    for numerator, denominator in zip(num.tolist(), den.tolist()):
+        common = gcd(int(numerator), int(denominator))
+        reduced_num = int(numerator) // common
+        reduced_den = int(denominator) // common
+        if SLOTS % reduced_den:
+            raise ValueError(
+                "meter is not exactly representable on the 96-slot lattice")
+        slot_count = (SLOTS // reduced_den) * reduced_num
+        if not 1 <= slot_count <= WINDOW:
+            raise ValueError(
+                f"measure lattice length must be within 1..{WINDOW} slots")
+        slot_values.append(slot_count)
+    slots = np.asarray(slot_values, dtype=np.int64)
+    quarter_beats = 4.0 * num / den
+    return slots, num, den, quarter_beats.astype(np.float64)
+
+
+def resolve_meter_layout(grid, n_measures):
+    """Resolve exact per-measure layout from a generation grid.
+
+    Preferred input is ``measure_num``/``measure_den`` with optional
+    ``measure_slots`` for an integrity check. A uniform ``meter`` or a
+    ``beats_per_measure`` array remains a supported grid API: each value is a
+    quarter-note count and is converted to an exact reduced TJA fraction.
+    """
+    has_num = "measure_num" in grid
+    has_den = "measure_den" in grid
+    if has_num != has_den:
+        raise ValueError("measure_num and measure_den must be provided together")
+    if has_num:
+        num = _expand(grid["measure_num"], n_measures, "measure_num", np.int64)
+        den = _expand(grid["measure_den"], n_measures, "measure_den", np.int64)
+    else:
+        raw = grid.get("beats_per_measure", grid.get("meter"))
+        if raw is None:
+            raise ValueError(
+                "slot generation requires measure_num/measure_den or an "
+                "explicit uniform meter/beats_per_measure grid"
+            )
+        quarter = _expand(raw, n_measures, "beats_per_measure", np.float64)
+        if not np.isfinite(quarter).all() or (quarter <= 0).any():
+            raise ValueError("quarter beats per measure must be positive and finite")
+        fractions = [Fraction(float(q)).limit_denominator(768) for q in quarter]
+        if any(abs(float(f) - q) > 1e-9
+               for f, q in zip(fractions, quarter)):
+            raise ValueError("quarter-beat meter values must be exact rationals")
+        num = np.asarray([f.numerator for f in fractions], dtype=np.int64)
+        den = np.asarray([4 * f.denominator for f in fractions], dtype=np.int64)
+
+    slots, num, den, quarter = rational_meter_arrays(num, den, n_measures)
+    if "measure_slots" in grid:
+        supplied = _expand(
+            grid["measure_slots"], n_measures, "measure_slots", np.int64)
+        if not np.array_equal(supplied, slots):
+            raise ValueError("measure_slots disagrees with measure_num/measure_den")
+    return slots, num, den, quarter

softchart/sc2/model.py

@@ -0,0 +1,615 @@
+"""Encoder-decoder Transformer: log-mel windows -> chart token sequences."""
+
+import math
+
+import torch
+import torch.nn as nn
+
+from .vocab import (
+    N_MELS, N_SPAN_DUR_BUCKETS, N_STAGES, N_TEMPO_K, VOCAB, WINDOW,
+)
+
+
+def sinusoidal(length, dim):
+    pos = torch.arange(length, dtype=torch.float32)[:, None]
+    i = torch.arange(dim // 2, dtype=torch.float32)[None]
+    angle = pos / torch.pow(10000.0, 2 * i / dim)
+    emb = torch.zeros(length, dim)
+    emb[:, 0::2] = torch.sin(angle)
+    emb[:, 1::2] = torch.cos(angle)
+    return emb
+
+
+class BeatHiRes(nn.Module):
+    """Frame-resolution beat/downbeat head: encoder states (T/4) are
+    upsampled 4x so supervision and peaks live on the 11.6 ms frame grid
+    instead of 46 ms encoder bins (4x finer timing per peak)."""
+
+    def __init__(self, d):
+        super().__init__()
+        self.up = nn.ConvTranspose1d(d, d // 2, 4, stride=4)
+        self.out = nn.Conv1d(d // 2, 2, 3, padding=1)
+
+    def forward(self, mem):  # (B, L, d) -> (B, 4L, 2)
+        h = nn.functional.gelu(self.up(mem.transpose(1, 2)))
+        return self.out(h).transpose(1, 2)
+
+
+class HierarchicalSectionEncoder(nn.Module):
+    """Encode a fixed relative-time summary of the complete song.
+
+    The local encoder remains dedicated to the current 20-second window. This
+    separate branch models section-to-section structure, marks the section that
+    contains the current window, and exposes its states directly to decoder
+    cross-attention. It is trained from scratch with the chart objective; no
+    external audio representation is used.
+    """
+
+    def __init__(self, d, nhead, ffn, dropout, layers=2, sections=100,
+                 rich_stats=False):
+        super().__init__()
+        self.sections = sections
+        # rich_stats (SoftChart 2.0, default off): each section summary is the
+        # concat of four frame statistics [mean, std, max, last-first] instead
+        # of the mean alone. The rich projection lives under its OWN key
+        # (inp_rich) so v1.8 checkpoints (inp only) load with strict=False.
+        self.rich_stats = bool(rich_stats)
+        if self.rich_stats:
+            self.inp = None
+            self.inp_rich = nn.Linear(4 * N_MELS, d)
+        else:
+            self.inp = nn.Linear(N_MELS, d)
+            self.inp_rich = None
+        self.pos = nn.Parameter(torch.randn(sections, d) * 0.02)
+        self.current = nn.Parameter(torch.randn(d) * 0.02)
+        layer = nn.TransformerEncoderLayer(
+            d, nhead, ffn, dropout, activation="gelu",
+            batch_first=True, norm_first=True,
+        )
+        self.encoder = nn.TransformerEncoder(
+            layer, layers, nn.LayerNorm(d), enable_nested_tensor=False)
+
+    def forward(self, summary, position):
+        feat = 4 * N_MELS if self.rich_stats else N_MELS
+        if summary.ndim != 3 or summary.shape[1:] != (self.sections, feat):
+            raise ValueError(
+                f"expected song summary (B,{self.sections},{feat}), "
+                f"got {tuple(summary.shape)}"
+            )
+        position = position.to(summary.device).long().clamp(0, self.sections - 1)
+        proj = self.inp_rich if self.rich_stats else self.inp
+        h = proj(summary) + self.pos
+        marker = torch.zeros_like(h)
+        marker[torch.arange(h.shape[0], device=h.device), position] = self.current
+        return self.encoder(h + marker)
+
+
+def _parse_pattern(spec, n):
+    """Layer-sharing schedule. spec is either an int (number of UNIQUE
+    physical layers, cycled to length n) or a comma string like "E0,E1,E0,E1"
+    (labels; each distinct label is one physical layer). Returns a list of n
+    physical-layer indices, e.g. [0,1,0,1] for 2-unique/4-physical-depth."""
+    if spec is None:
+        return list(range(n))  # no sharing: every layer unique
+    if isinstance(spec, int):
+        return [k % spec for k in range(n)]
+    labels = [s.strip() for s in str(spec).split(",") if s.strip()]
+    assert len(labels) == n, f"pattern {spec} has {len(labels)} labels, need {n}"
+    order = {}
+    out = []
+    for lb in labels:
+        if lb not in order:
+            order[lb] = len(order)
+        out.append(order[lb])
+    return out
+
+
+class Adapter(nn.Module):
+    """Rank-r residual adapter applied after a (possibly reused) transformer
+    block: x + gate * up(gelu(down(x))). Both projections and the gate start on
+    an active gradient path. The up projection is initialized very small so the
+    initial adapter is a near-identity perturbation without the dead-gradient
+    state caused by zero-initializing both ``up`` and ``gate``."""
+
+    def __init__(self, d, rank):
+        super().__init__()
+        self.down = nn.Linear(d, rank, bias=False)
+        self.up = nn.Linear(rank, d, bias=False)
+        nn.init.normal_(self.down.weight, std=0.02)
+        nn.init.normal_(self.up.weight, std=1e-3)
+        self.gate = nn.Parameter(torch.ones(1))
+
+    def forward(self, x):
+        return x + self.gate * self.up(nn.functional.gelu(self.down(x)))
+
+
+class SharedEncoder(nn.Module):
+    """Encoder whose physical layers are reused according to a sharing pattern.
+    Each POSITION in the depth keeps its own (a) rank-r adapter and (b) final
+    LayerNorm, so every logical depth can specialize while reusing the block.
+    With pattern=None this is exactly n unique layers + a norm, matching the
+    stock nn.TransformerEncoder param count (adapters/pos-norms are opt-in).
+
+    v1.7 depth-diversity options (all default-off):
+      depth_emb: per-depth learned d-vector ADDED to the block input at each
+        reuse (params = depth x d, zero-init so init == hard sharing).
+      adapter_rank_ffn > 0: SPLIT adapters — instead of one post-block adapter,
+        each depth owns a rank-`adapter_rank` adapter on the attention sublayer
+        output and a rank-`adapter_rank_ffn` adapter on the FFN sublayer output
+        (patterning lives in the FFN).
+    """
+
+    def __init__(self, d, nhead, ffn, dropout, pattern, adapter_rank=0,
+                 unique_layernorm=False, adapter_rank_ffn=0, depth_emb=False):
+        super().__init__()
+        # `pattern` is the resolved list of physical-layer indices per depth
+        n_phys = max(pattern) + 1
+        self.layers = nn.ModuleList([
+            nn.TransformerEncoderLayer(d, nhead, ffn, dropout, activation="gelu",
+                                       batch_first=True, norm_first=True)
+            for _ in range(n_phys)])
+        self.plan = pattern
+        self.split = adapter_rank_ffn > 0
+        if self.split:  # v1.7: per-sublayer adapters (attention rank-a, FFN rank-f)
+            self.attn_adapters = nn.ModuleList([
+                Adapter(d, adapter_rank) if adapter_rank else nn.Identity()
+                for _ in pattern])
+            self.ffn_adapters = nn.ModuleList([
+                Adapter(d, adapter_rank_ffn) for _ in pattern])
+        else:  # v1.6: one residual adapter after the whole block
+            self.adapters = nn.ModuleList([
+                Adapter(d, adapter_rank) if adapter_rank else nn.Identity()
+                for _ in pattern])
+        self.depth_emb = (nn.Parameter(torch.zeros(len(pattern), d))
+                          if depth_emb else None)
+        self.pos_norms = nn.ModuleList([
+            nn.LayerNorm(d) if unique_layernorm else nn.Identity()
+            for _ in pattern])
+        self.norm = nn.LayerNorm(d)
+
+    def forward(self, x, src_key_padding_mask=None):
+        for k, phys in enumerate(self.plan):
+            if self.depth_emb is not None:
+                x = x + self.depth_emb[k]
+            layer = self.layers[phys]
+            if self.split:
+                # norm_first decomposition of nn.TransformerEncoderLayer with
+                # per-depth adapters applied to each sublayer OUTPUT (Houlsby
+                # placement)
+                x = x + self.attn_adapters[k](
+                    layer._sa_block(layer.norm1(x), None, src_key_padding_mask))
+                x = x + self.ffn_adapters[k](layer._ff_block(layer.norm2(x)))
+            else:
+                x = layer(x, src_key_padding_mask=src_key_padding_mask)
+                x = self.adapters[k](x)
+            x = self.pos_norms[k](x)
+        return self.norm(x)
+
+
+class SharedDecoder(nn.Module):
+    """Decoder counterpart of SharedEncoder (see that docstring). Split mode
+    puts rank-`adapter_rank` adapters on BOTH attention sublayers (self and
+    cross) and rank-`adapter_rank_ffn` on the FFN sublayer."""
+
+    def __init__(self, d, nhead, ffn, dropout, pattern, adapter_rank=0,
+                 unique_layernorm=False, adapter_rank_ffn=0, depth_emb=False):
+        super().__init__()
+        n_phys = max(pattern) + 1
+        self.layers = nn.ModuleList([
+            nn.TransformerDecoderLayer(d, nhead, ffn, dropout, activation="gelu",
+                                       batch_first=True, norm_first=True)
+            for _ in range(n_phys)])
+        self.plan = pattern
+        self.split = adapter_rank_ffn > 0
+        if self.split:
+            self.sa_adapters = nn.ModuleList([
+                Adapter(d, adapter_rank) if adapter_rank else nn.Identity()
+                for _ in pattern])
+            self.ca_adapters = nn.ModuleList([
+                Adapter(d, adapter_rank) if adapter_rank else nn.Identity()
+                for _ in pattern])
+            self.ffn_adapters = nn.ModuleList([
+                Adapter(d, adapter_rank_ffn) for _ in pattern])
+        else:
+            self.adapters = nn.ModuleList([
+                Adapter(d, adapter_rank) if adapter_rank else nn.Identity()
+                for _ in pattern])
+        self.depth_emb = (nn.Parameter(torch.zeros(len(pattern), d))
+                          if depth_emb else None)
+        self.pos_norms = nn.ModuleList([
+            nn.LayerNorm(d) if unique_layernorm else nn.Identity()
+            for _ in pattern])
+        self.norm = nn.LayerNorm(d)
+
+    def forward(self, tgt, memory, tgt_mask=None, tgt_key_padding_mask=None,
+                tgt_is_causal=False):
+        x = tgt
+        for k, phys in enumerate(self.plan):
+            if self.depth_emb is not None:
+                x = x + self.depth_emb[k]
+            layer = self.layers[phys]
+            if self.split:
+                x = x + self.sa_adapters[k](layer._sa_block(
+                    layer.norm1(x), tgt_mask, tgt_key_padding_mask, tgt_is_causal))
+                x = x + self.ca_adapters[k](layer._mha_block(
+                    layer.norm2(x), memory, None, None))
+                x = x + self.ffn_adapters[k](layer._ff_block(layer.norm3(x)))
+            else:
+                x = layer(x, memory, tgt_mask=tgt_mask,
+                          tgt_key_padding_mask=tgt_key_padding_mask,
+                          tgt_is_causal=tgt_is_causal)
+                x = self.adapters[k](x)
+            x = self.pos_norms[k](x)
+        return self.norm(x)
+
+
+class ChartModel(nn.Module):
+    def __init__(
+        self,
+        d_model=512,
+        nhead=8,
+        enc_layers=6,
+        dec_layers=6,
+        ffn=2048,
+        dropout=0.1,
+        vocab_size=None,
+        max_tgt=None,
+        aux=False,
+        global_ctx=False,
+        hierarchical_ctx=False,
+        section_layers=2,
+        section_ffn=None,
+        func_time=False,
+        ptr=False,
+        beat=False,
+        in_ch=None,
+        emb_factor=None,
+        clean_phase=False,
+        enc_share=None,
+        dec_share=None,
+        adapter_rank=0,
+        unique_layernorm=False,
+        adapter_rank_ffn=0,
+        depth_emb=False,
+        unshare_last_dec=False,
+        stage_emb=False,
+        span_duration_head=False,
+        tempo_head=False,
+        rich_section_stats=False,
+    ):
+        # SoftChart 2.0 flags (all default-off; defaults are bit-identical to
+        # v1.8 in parameters and forward outputs):
+        #   stage_emb: 4 learned d_model vectors (header/plan/skeleton/
+        #     realization), zero-init, added to decoder inputs per the
+        #     caller-supplied `stage_ids` (skipped entirely when None).
+        #   span_duration_head: Linear(d, N_SPAN_DUR_BUCKETS) over decoder
+        #     states; supervised at SPAN_START realization steps
+        #     (SOFTCHART2_DESIGN §1.3).
+        #   tempo_head: slot+ TEMPO factorization heads — log-p regression
+        #     Linear(d, 1) and octave-k classification Linear(d, N_TEMPO_K)
+        #     over decoder states; supervised at TEMPO steps
+        #     (SLOTPLUS_DESIGN §2.2/§3.2/§3.3).
+        #   rich_section_stats: HierarchicalSectionEncoder consumes
+        #     [mean, std, max, last-first] concat (4*N_MELS) per section.
+        super().__init__()
+        from .vocab import MAX_TGT
+
+        vocab_size = vocab_size or VOCAB.size
+        max_tgt = max_tgt or MAX_TGT
+        self.d_model = d_model
+        if global_ctx and hierarchical_ctx:
+            raise ValueError("global_ctx and hierarchical_ctx are mutually exclusive")
+        self.hierarchical_ctx = bool(hierarchical_ctx)
+        self._hierarchical_ctx = self.hierarchical_ctx
+        # clean_phase (v1.6): the encoder never sees the 2 grid-phase channels
+        # (in_ch stays N_MELS), and phase is injected into the DECODER memory
+        # only, via a small projection. This removes the condition leak that
+        # forced beat supervision to be masked on slot windows in v1.5.
+        self.clean_phase = clean_phase
+        self.in_ch = in_ch or N_MELS  # legacy slot mode appends 2 grid-phase channels
+        front_ch = N_MELS if clean_phase else self.in_ch
+
+        # conv frontend: (B, front_ch, T) -> (B, T/4, d)
+        self.frontend = nn.Sequential(
+            nn.Conv1d(front_ch, d_model, 3, stride=2, padding=1),
+            nn.GELU(),
+            nn.Conv1d(d_model, d_model, 3, stride=2, padding=1),
+            nn.GELU(),
+        )
+        enc_len = WINDOW // 4
+        self.register_buffer("enc_pos", sinusoidal(enc_len, d_model), persistent=False)
+        self.register_buffer("dec_pos", sinusoidal(max_tgt, d_model), persistent=False)
+
+        # phase injection (clean_phase only): 2 grid-phase channels at frame
+        # rate -> encoder rate (T/4) -> additive conditioning on the memory the
+        # DECODER reads. The beat head reads the pre-injection (clean) memory.
+        # phase_proj input = [measure_phase, beat_phase, valid_flag]. valid=0 on
+        # time-mode windows (their raw phase is -1), giving the decoder a clean
+        # zero-phase conditioning there; slot windows pass valid=1 + real phase.
+        if clean_phase:
+            self.phase_proj = nn.Sequential(
+                nn.Conv1d(3, d_model, 3, stride=2, padding=1),
+                nn.GELU(),
+                nn.Conv1d(d_model, d_model, 3, stride=2, padding=1),
+            )
+        else:
+            self.phase_proj = None
+
+        # sharing schedule (v1.6-small/tiny): reuse physical layers per pattern.
+        # enc_share/dec_share: int (#unique layers) or "E0,E1,E0,E1" label string.
+        self.enc_share = enc_share
+        self.dec_share = dec_share
+        if (enc_share is None and dec_share is None and not adapter_rank
+                and not adapter_rank_ffn and not depth_emb and not unshare_last_dec):
+            enc_layer = nn.TransformerEncoderLayer(
+                d_model, nhead, ffn, dropout, activation="gelu",
+                batch_first=True, norm_first=True)
+            self.encoder = nn.TransformerEncoder(
+                enc_layer, enc_layers, nn.LayerNorm(d_model),
+                enable_nested_tensor=False)
+            dec_layer = nn.TransformerDecoderLayer(
+                d_model, nhead, ffn, dropout, activation="gelu",
+                batch_first=True, norm_first=True)
+            self.decoder = nn.TransformerDecoder(dec_layer, dec_layers,
+                                                 nn.LayerNorm(d_model))
+        else:
+            enc_plan = _parse_pattern(enc_share, enc_layers)
+            dec_plan = _parse_pattern(dec_share, dec_layers)
+            # v1.7: un-share the LAST decoder layer (pattern-shaping layer in
+            # front of the generation head gets its own physical parameters);
+            # no-op if that depth is already unique.
+            if unshare_last_dec and dec_plan.count(dec_plan[-1]) > 1:
+                dec_plan = dec_plan[:-1] + [max(dec_plan) + 1]
+            self.encoder = SharedEncoder(d_model, nhead, ffn, dropout, enc_plan,
+                                         adapter_rank, unique_layernorm,
+                                         adapter_rank_ffn, depth_emb)
+            self.decoder = SharedDecoder(d_model, nhead, ffn, dropout, dec_plan,
+                                         adapter_rank, unique_layernorm,
+                                         adapter_rank_ffn, depth_emb)
+
+        # ALBERT-style factorized embeddings (v1.5): vocab->E->d. Position
+        # tokens dominate the table (1728/1810 rows), so E=64 cuts the
+        # embedding block ~3.5x; logits stay tied via the projected matrix.
+        self.emb_factor = emb_factor
+        if emb_factor:
+            self.tok_emb = nn.Embedding(vocab_size, emb_factor, padding_idx=VOCAB.pad)
+            nn.init.normal_(self.tok_emb.weight, std=0.02)
+            self.emb_proj = nn.Linear(emb_factor, d_model, bias=False)
+            nn.init.normal_(self.emb_proj.weight, std=1.0 / (emb_factor ** 0.5) * 0.16)
+            self.out = None
+        else:
+            self.tok_emb = nn.Embedding(vocab_size, d_model, padding_idx=VOCAB.pad)
+            nn.init.normal_(self.tok_emb.weight, std=0.02)  # keep tied logits well-scaled
+            self.emb_proj = None
+            self.out = nn.Linear(d_model, vocab_size, bias=False)
+            self.out.weight = self.tok_emb.weight  # weight tying
+        self.dropout = nn.Dropout(dropout)
+        # Auxiliary local rhythmic skeleton. In the hierarchical model its
+        # predicted occupancy is also projected back into decoder memory, so
+        # the event realizer is explicitly conditioned on a coarse audio-led
+        # skeleton rather than learning placement and pattern in one flat pass.
+        if hierarchical_ctx and not aux:
+            raise ValueError("hierarchical_ctx requires the auxiliary skeleton head")
+        self.aux = nn.Linear(d_model, 1) if aux else None
+        self.skeleton_proj = nn.Linear(1, d_model) if hierarchical_ctx else None
+        if self.skeleton_proj is not None:
+            nn.init.zeros_(self.skeleton_proj.weight)
+            nn.init.zeros_(self.skeleton_proj.bias)
+        # pointer alignment head: each generated note must point to its audio
+        # frame (differentiable provenance; explainability that trains alignment)
+        self.ptr = nn.Linear(d_model, d_model) if ptr else None
+        # beat/downbeat head: explicit metrical percept on the encoder.
+        # beat="hires" builds the frame-resolution head (11.6 ms bins)
+        self.beat = (BeatHiRes(d_model) if beat == "hires"
+                     else (nn.Linear(d_model, 2) if beat else None))
+        # song-level context: coarse whole-song summary + window position (v4).
+        # Lets the model see beyond the window, so intentional gaps (a breath
+        # before a strong section) are informed decisions, not failures.
+        # v2 fix (after the v4 negative result): summary chunks get DEDICATED
+        # positional embeddings instead of reusing window positions.
+        if global_ctx:
+            self.gsum_proj = nn.Linear(N_MELS, d_model)
+            self.seg_emb = nn.Parameter(torch.zeros(2, d_model))
+            self.pos_emb = nn.Embedding(100, d_model)  # relative song position
+            self.gpos = nn.Parameter(torch.randn(128, d_model) * 0.02)
+        else:
+            self.gsum_proj = None
+        if rich_section_stats and not hierarchical_ctx:
+            raise ValueError("rich_section_stats requires hierarchical_ctx")
+        if hierarchical_ctx:
+            self.section_encoder = HierarchicalSectionEncoder(
+                d_model, nhead, section_ffn or (2 * d_model), dropout,
+                layers=section_layers, rich_stats=rich_section_stats,
+            )
+            self.hier_seg_emb = nn.Parameter(torch.zeros(2, d_model))
+        else:
+            self.section_encoder = None
+        # functional time embeddings: the 1728 TIME tokens share a sinusoidal
+        # basis + small projection instead of free embeddings (fewer params,
+        # neighbouring times get similar representations)
+        if func_time:
+            self.register_buffer("time_basis", sinusoidal(WINDOW, 64), persistent=False)
+            self.time_proj = nn.Linear(64, d_model)
+            # match the 0.02-std scale of tok_emb rows (basis row norm ~ sqrt(32))
+            nn.init.normal_(self.time_proj.weight, std=0.004)
+            nn.init.zeros_(self.time_proj.bias)
+        else:
+            self.time_proj = None
+        # --- SoftChart 2.0 additions (all None when their flag is off, so
+        # v1.8 state dicts and RNG init order are untouched by default) ---
+        # BarScript stage embedding: zero-init so a flags-on model with any
+        # stage_ids starts bit-identical to the flags-off decoder input.
+        self.stage_emb = (nn.Parameter(torch.zeros(N_STAGES, d_model))
+                          if stage_emb else None)
+        # span duration head: log-grid bucket logits at SPAN_START steps
+        self.span_dur = (nn.Linear(d_model, N_SPAN_DUR_BUCKETS)
+                         if span_duration_head else None)
+        # slot+ TEMPO heads: canonical-period log-p regression + octave-k class
+        if tempo_head:
+            self.tempo_logp = nn.Linear(d_model, 1)
+            self.tempo_k = nn.Linear(d_model, N_TEMPO_K)
+        else:
+            self.tempo_logp = None
+            self.tempo_k = None
+
+    def _local_input(self, mel):
+        audio = mel[:, :N_MELS] if self.clean_phase else mel
+        h = self.frontend(audio).transpose(1, 2)  # (B, T/4, d)
+        return h + self.enc_pos[: h.shape[1]]
+
+    def encode_local(self, mel):
+        """Encode only the current window for beat/downbeat estimation."""
+        h = self._local_input(mel)
+        if self.section_encoder is not None:
+            h = h + self.hier_seg_emb[0]
+        return self.encoder(self.dropout(h))
+
+    def encode(self, mel, gsum=None, pos_bucket=None):
+        # mel: (B, C, T). clean_phase: C may be N_MELS (audio only) or N_MELS+2
+        # (audio + 2 grid-phase channels); only the audio channels reach the
+        # encoder, so the returned memory is CLEAN (phase-free) and safe for the
+        # beat head. gsum: (B, 100, n_mels) complete-song section summaries
+        # ((B, 100, 4*n_mels) with rich_section_stats: [mean,std,max,last-first]);
+        # pos_bucket: (B,) current relative song section in [0, 100).
+        h = self._local_input(mel)
+        if self.section_encoder is not None:
+            if gsum is None or pos_bucket is None:
+                raise ValueError("hierarchical_ctx requires whole-song summary and position")
+            local = self.encoder(self.dropout(h + self.hier_seg_emb[0]))
+            sections = self.section_encoder(gsum, pos_bucket) + self.hier_seg_emb[1]
+            return torch.cat([sections, local], dim=1)
+        if self.gsum_proj is not None and gsum is not None:
+            g = self.gsum_proj(gsum) + self.seg_emb[1] + self.gpos[: gsum.shape[1]]
+            h = h + self.seg_emb[0]
+            p = self.pos_emb(pos_bucket).unsqueeze(1)  # (B, 1, d)
+            h = torch.cat([p, g, h], dim=1)
+        return self.encoder(self.dropout(h))
+
+    def phase_mem(self, memory, mel):
+        """Phase-conditioned copy of `memory` for the DECODER to cross-attend.
+        Adds a projection of the grid-phase channels to the audio-frame slice
+        of memory; the beat head keeps reading the clean `memory`. No-op unless
+        clean_phase is on and phase channels are present. Time-mode windows
+        (raw phase = -1) get valid=0 and exactly zero phase conditioning."""
+        out = memory.clone()
+        L = WINDOW // 4
+        nch = N_MELS
+        if self.clean_phase and mel.shape[1] >= nch + 2:
+            ph = mel[:, nch:nch + 2]  # [0,1); -1 marks no-grid frames
+            valid = (ph[:, :1] >= 0).float()
+            ph_in = torch.cat([ph.clamp(min=0.0), valid], dim=1)
+            p = self.phase_proj(ph_in)
+            # Mask *after* both biased convolutions. Masking only their input
+            # leaves a learned bias residual in gridless time-mode windows.
+            valid_out = nn.functional.adaptive_max_pool1d(valid, p.shape[-1])
+            p = (p * valid_out).transpose(1, 2)
+            L = p.shape[1]
+            out[:, -L:] = out[:, -L:] + p
+        if self.skeleton_proj is not None:
+            clean_local = memory[:, -L:]
+            skeleton = torch.sigmoid(self.aux(clean_local))
+            out[:, -L:] = out[:, -L:] + self.skeleton_proj(skeleton)
+        return out
+
+    def emb_matrix(self):
+        """Full embedding matrix; factorized and/or functional rows resolved."""
+        E = self.tok_emb.weight
+        if self.emb_proj is not None:
+            E = self.emb_proj(E)
+        if self.time_proj is not None:
+            t = self.time_proj(self.time_basis)  # (WINDOW, d)
+            E = torch.cat([E[: VOCAB.time0], t, E[VOCAB.time0 + WINDOW :]], dim=0)
+        return E
+
+    def _add_stage(self, h, stage_ids):
+        """Add BarScript stage embeddings to embedded decoder inputs.
+
+        stage_ids: (B, L) ints in [0, N_STAGES) aligned position-for-position
+        with the embedded tokens; None skips the addition entirely (decoder
+        input is bit-identical to v1.8)."""
+        if stage_ids is None:
+            return h
+        if self.stage_emb is None:
+            raise ValueError("stage_ids supplied but stage_emb flag is off")
+        if stage_ids.shape != h.shape[:2]:
+            raise ValueError(
+                f"stage_ids shape {tuple(stage_ids.shape)} does not match "
+                f"decoder input {tuple(h.shape[:2])}"
+            )
+        return h + self.stage_emb[stage_ids.long()]
+
+    def decode(self, tgt_in, memory, stage_ids=None, return_hidden=False):
+        # tgt_in: (B, L) token ids; stage_ids (optional): (B, L) BarScript
+        # stage ids aligned with tgt_in. return_hidden=True also returns the
+        # decoder states so callers can apply span_dur/tempo heads during AR
+        # generation.
+        L = tgt_in.shape[1]
+        E = self.emb_matrix()
+        h = nn.functional.embedding(tgt_in, E) * math.sqrt(self.d_model) + self.dec_pos[:L]
+        h = self._add_stage(h, stage_ids)
+        mask = torch.ones((L, L), dtype=torch.bool, device=tgt_in.device).triu(1)
+        pad_mask = tgt_in == VOCAB.pad
+        h = self.decoder(
+            self.dropout(h),
+            memory,
+            tgt_mask=mask,
+            tgt_key_padding_mask=pad_mask,
+            tgt_is_causal=True,
+        )
+        logits = h @ E.T  # tied output projection (functional rows included)
+        return (logits, h) if return_hidden else logits
+
+    def enable_ptr(self):
+        self.ptr = nn.Linear(self.d_model, self.d_model)
+
+    def enable_beat(self, hires=False):
+        self.beat = BeatHiRes(self.d_model) if hires else nn.Linear(self.d_model, 2)
+
+    def forward(self, mel, tgt, return_aux=False, gsum=None, pos_bucket=None,
+                return_extras=False, in_tgt=None, stage_ids=None,
+                return_v2_heads=False):
+        # in_tgt: optional corrupted decoder input (e.g. MASKed types for the
+        # skeleton->color infill curriculum); gold targets stay = tgt
+        # stage_ids: optional (B, L) BarScript stage ids aligned with tgt
+        #   (sliced [:, :-1] alongside the decoder input); None skips.
+        # return_v2_heads: returns the fixed 7-tuple (logits, aux, ptr, beat,
+        #   span_dur_logits, tempo_logp, tempo_k_logits); entries whose head
+        #   is disabled are None. Default False keeps v1.8 return shapes.
+        return_extras = return_extras or return_v2_heads
+        memory = self.encode(mel, gsum=gsum, pos_bucket=pos_bucket)  # CLEAN
+        dec_mem = self.phase_mem(memory, mel)  # phase-conditioned for the decoder
+        dec_in = (in_tgt if in_tgt is not None else tgt)[:, :-1]
+        L = dec_in.shape[1]
+        E = self.emb_matrix()
+        import math as _m
+        h = nn.functional.embedding(dec_in, E) * _m.sqrt(self.d_model) + self.dec_pos[:L]
+        h = self._add_stage(h, None if stage_ids is None else stage_ids[:, :-1])
+        dm = torch.ones((L, L), dtype=torch.bool, device=tgt.device).triu(1)
+        h = self.decoder(self.dropout(h), dec_mem, tgt_mask=dm,
+                         tgt_key_padding_mask=dec_in == VOCAB.pad, tgt_is_causal=True)
+        logits = h @ E.T
+        if not (return_aux or return_extras):
+            return logits
+        outs = [logits]
+        # aux/ptr/beat heads read the CLEAN memory (no phase leak)
+        aux_mem = memory[:, -(WINDOW // 4):]
+        outs.append(self.aux(aux_mem).squeeze(-1) if self.aux is not None else None)
+        if return_extras:
+            ptr_logits = None
+            if self.ptr is not None:
+                q = self.ptr(h)  # (B, L, d)
+                ptr_logits = q @ aux_mem.transpose(1, 2) / _m.sqrt(self.d_model)
+            beat_logits = self.beat(aux_mem) if self.beat is not None else None
+            outs += [ptr_logits, beat_logits]
+        if return_v2_heads:
+            # per-step logits over ALL decoder steps; training masks them to
+            # SPAN_START steps (span_dur) / TEMPO steps (log-p, k)
+            outs.append(self.span_dur(h) if self.span_dur is not None else None)
+            outs.append(self.tempo_logp(h).squeeze(-1)
+                        if self.tempo_logp is not None else None)
+            outs.append(self.tempo_k(h) if self.tempo_k is not None else None)
+        return tuple(outs)
+
+
+def count_params(m):
+    return sum(p.numel() for p in m.parameters() if p.requires_grad)

softchart/sc2/tja.py

@@ -0,0 +1,148 @@
+"""Meter-aware TJA emission for slot-mode charts.
+
+No quantization happens here — generate_song_slot() already produced exact
+lattice positions (96 slots per 4/4 measure), so this writer just prints them.
+Preview rendering and the .tja are identical by construction. Authored measure
+edges determine exact per-measure tempo; meter changes emit #MEASURE directives.
+"""
+
+import numpy as np
+
+from .meter import resolve_meter_layout
+from .vocab import SLOTS
+
+CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
+        "roll": "5", "roll_big": "6", "balloon": "7"}
+
+
+def _format_number(value):
+    """Emit enough digits for a float to round-trip through the TJA text."""
+    value = float(value)
+    if not np.isfinite(value):
+        raise ValueError("TJA numeric fields must be finite")
+    if value == 0.0:
+        value = 0.0  # avoid a cosmetic negative zero in OFFSET
+    return format(value, ".17g")
+
+
+def write_tja_slots(gen, grid, title, course, level, wave, out_path=None,
+                    balloon_count=10):
+    """Serialize a slot generation on an explicit trusted meter timeline.
+
+    ``grid`` must contain exact rational meter metadata and ``measure_edges``
+    with one trusted terminal edge. Returns the TJA text and optionally writes
+    it to ``out_path``.
+    """
+    if int(gen.get("slots_per_4_4", SLOTS)) != SLOTS:
+        raise ValueError(f"expected {SLOTS} slots per 4/4 measure")
+    n_meas = max(int(gen.get("n_measures", 0)),
+                 (max((m for m, _, _ in gen.get("hits_slots", [])), default=-1) + 1),
+                 (max((m1 for _, _, m1, _, _ in gen.get("spans_slots", [])),
+                      default=-1) + 1), 1)
+    fields = ("measure_slots", "measure_num", "measure_den")
+    gen_has_layout = any(name in gen for name in fields)
+    grid_has_layout = any(name in grid for name in fields) \
+        or "meter" in grid or "beats_per_measure" in grid
+    gen_layout = resolve_meter_layout(gen, n_meas) if gen_has_layout else None
+    grid_layout = resolve_meter_layout(grid, n_meas) if grid_has_layout else None
+    if gen_layout is not None and grid_layout is not None:
+        gen_slots, gen_num, gen_den, _ = gen_layout
+        grid_slots, grid_num, grid_den, _ = grid_layout
+        same_fraction = all(
+            int(gn) * int(gd) == int(gdn) * int(gnd)
+            for gn, gnd, gdn, gd in zip(
+                gen_num, gen_den, grid_num, grid_den)
+        )
+        if not np.array_equal(gen_slots, grid_slots) or not same_fraction:
+            raise ValueError("generated meter layout disagrees with the timing grid")
+    layout = grid_layout if grid_layout is not None else gen_layout
+    if layout is None:
+        raise ValueError(
+            "TJA slot export requires exact meter metadata; refusing to assume 4/4")
+    slots_per_measure, measure_num, measure_den, quarter_beats = layout
+    offsets = np.concatenate([[0], np.cumsum(slots_per_measure)])
+
+    def flat(measure, slot):
+        measure, slot = int(measure), int(slot)
+        if not 0 <= measure < n_meas:
+            raise ValueError(f"measure index {measure} is outside 0..{n_meas - 1}")
+        if not 0 <= slot < slots_per_measure[measure]:
+            raise ValueError(
+                f"slot {slot} is outside measure {measure}'s "
+                f"0..{slots_per_measure[measure] - 1} range"
+            )
+        return int(offsets[measure] + slot)
+
+    slots = {}
+    for me, sl, cls in gen["hits_slots"]:
+        position = flat(me, sl)
+        if position in slots:
+            raise ValueError(
+                f"slot collision at measure {int(me)}, slot {int(sl)}")
+        slots[position] = CHAR[cls]
+    for m0, s0, m1, s1, typ in gen.get("spans_slots", []):
+        a = flat(m0, s0)
+        b = flat(m1, s1)
+        if b <= a:
+            raise ValueError("span end must be strictly after its start")
+        if a in slots:
+            raise ValueError(
+                f"slot collision at span start measure {int(m0)}, slot {int(s0)}")
+        if b in slots:
+            raise ValueError(
+                f"slot collision at span end measure {int(m1)}, slot {int(s1)}")
+        slots[a] = CHAR[typ]
+        slots[b] = "8"
+
+    # Slot output is timing-exact only when every start and the terminal edge
+    # are explicit.  Never fall back to BPM/downbeat extrapolation here: that
+    # would make the serialized TJA disagree with the lattice used to decode.
+    if "measure_edges" not in grid:
+        raise ValueError(
+            "slot TJA export requires explicit measure_edges including the "
+            "trusted terminal edge"
+        )
+    tempo_edges = np.asarray(grid["measure_edges"], dtype=float)
+    if (tempo_edges.ndim != 1 or len(tempo_edges) != n_meas + 1
+            or not np.all(np.isfinite(tempo_edges))
+            or np.any(np.diff(tempo_edges) <= 0)):
+        raise ValueError(
+            "measure_edges must contain one finite, increasing edge per "
+            "measure plus the final edge")
+    lines = []
+    base_bpm = float(grid["bpm"])
+    if not np.isfinite(base_bpm) or base_bpm <= 0:
+        raise ValueError("grid bpm must be positive and finite")
+    cur_bpm = base_bpm
+    cur_meter = (4, 4)
+    for m in range(n_meas):
+        duration = float(tempo_edges[m + 1] - tempo_edges[m])
+        if not np.isfinite(duration) or duration <= 0:
+            raise ValueError(f"measure {m} duration must be positive and finite")
+        bpm_m = 60.0 * float(quarter_beats[m]) / duration
+        # No integer snap and no perceptual threshold: even a small tempo
+        # delta accumulates into a real chart/audio timing error.
+        if bpm_m != cur_bpm:
+            lines.append(f"#BPMCHANGE {_format_number(bpm_m)}")
+            cur_bpm = bpm_m
+        meter = (int(measure_num[m]), int(measure_den[m]))
+        if meter != cur_meter:
+            lines.append(f"#MEASURE {meter[0]}/{meter[1]}")
+            cur_meter = meter
+        start = int(offsets[m])
+        lines.append("".join(slots.get(start + k, "0")
+                             for k in range(int(slots_per_measure[m]))) + ",")
+    balloons = [balloon_count] * sum(1 for s in gen.get("spans_slots", [])
+                                     if s[4] == "balloon")
+    offset = float(tempo_edges[0])
+    tja = "\n".join([
+        f"TITLE:{title} (SoftChart)", f"BPM:{_format_number(base_bpm)}",
+        f"WAVE:{wave}", f"OFFSET:{_format_number(-offset)}",
+        f"COURSE:{'Oni' if course == 'oni' else str(course).capitalize()}",
+        f"LEVEL:{level}",
+        f"BALLOON:{','.join(map(str, balloons))}" if balloons else "BALLOON:",
+        "", "#START", *lines, "#END"]) + "\n"
+    if out_path:
+        with open(out_path, "w") as f:
+            f.write(tja)
+    return tja

softchart/sc2/trace.py

@@ -0,0 +1,519 @@
+"""Command-trace parser: authored TJA -> exact rational command stream (slot+ S0).
+
+A trace is the timing fragment of one TJA course represented as an ordered
+command stream over the authored-bar rational lattice (SLOTPLUS_DESIGN.md
+section 2 and 3): events are NOTE(pos, type), TEMPO(value), DELAY(dur) and
+MEASURE(num/den); tempo/meter are persistent state that only events change,
+and every bar-internal position is an exact ``fractions.Fraction`` whose
+denominator is the authored row's note-character count.  Nothing here goes
+through floating point on the position axis; absolute times are derived
+quantities (``trace_to_times``) computed with exact rational arithmetic and
+converted to float only at the boundary.
+
+Schema
+------
+Parse layer (lossless per ``#START``/``#END`` play section):
+
+``ChartCommand``
+    ``offset``  number of note characters of the bar emitted before the
+                command line (0 .. len(digits)),
+    ``name``    command name, uppercased, without the leading ``#``,
+    ``args``    verbatim argument text.
+``ChartBar``
+    ``digits``    the bar's note characters with commas removed (may be
+                  empty: an authored empty measure),
+    ``commands``  list of ChartCommand in source order (all commands are
+                  kept here, including GOGO/SCROLL, for losslessness).
+``Chart``
+    ``headers``           dict of raw header text (keys uppercased),
+    ``course``            canonical course name (easy/normal/hard/oni/ura),
+    ``level``             int level (0 when absent),
+    ``bpm_text``          authored header BPM, verbatim ("120" when absent),
+    ``offset_text``       authored header OFFSET, verbatim ("0" when absent);
+                          bar 0 starts at absolute time ``-OFFSET`` seconds,
+    ``start_args``        arguments of ``#START`` ("" or "P1"/"P2"),
+    ``bars``              list of ChartBar,
+    ``trailing_commands`` commands after the last completed bar that precede
+                          ``#END`` without any note characters.
+
+Trace layer (the command stream itself):
+
+``NoteEvent``   ``bar``, ``pos`` (Fraction in [0, 1)), ``type`` (the raw TJA
+                note character, '1'..'9', kept verbatim; see NOTE_CHAR_CLASS).
+``TempoEvent``  ``bar``, ``pos``, ``bpm`` (float), ``authored`` (verbatim
+                directive argument), and the (k, p) tempo-octave
+                factorisation of SLOTPLUS_DESIGN section 2.2: the authored
+                beat period ``60/bpm`` equals ``p * 2**k`` with ``p`` inside
+                one octave window.
+``DelayEvent``  ``bar``, ``pos``, ``seconds`` (signed float), ``authored``.
+``MeterEvent``  ``bar``, ``num``, ``den`` (ints), ``authored_num``,
+                ``authored_den`` (verbatim).  Meter events sit at bar
+                boundaries only (grammar, SLOTPLUS_DESIGN section 3.2).
+``BarInfo``     ``index``, ``denom`` (authored row note-character count; 0
+                for an empty measure), ``num``/``den`` (meter state effective
+                for the bar) and ``k`` (tempo-octave notation state at the
+                bar start) -- the per-bar notation coset of section 2.3.
+``Trace``       ``course``, ``level``, ``bpm``/``bpm_authored`` (initial
+                tempo state), ``offset``/``offset_authored``, ``bars`` (list
+                of BarInfo) and ``events`` (the ordered command stream).
+
+Semantics rulings (documented because raw TJA leaves them implicit):
+
+- ``#BPMCHANGE``/``#DELAY`` are legal mid-bar; their position is the exact
+  fraction ``written_digits / row_length``.  ``#MEASURE`` is bar-boundary
+  only (SLOTPLUS_DESIGN section 3.2): a mid-row ``#MEASURE`` takes effect at
+  the next bar's start (the current bar's length is already determined by
+  its meter).  Commands written after every note character of a bar but
+  before its terminating comma are timing-equivalent to the next bar's
+  position 0 and are carried there; if no next bar exists, one empty bar is
+  synthesized so the trace stays a lossless verbatim carrier.
+- ``#GOGOSTART``/``#GOGOEND``/``#SCROLL`` and other non-timing commands are
+  excluded from the trace (SLOTPLUS_DESIGN section 3.1); they remain in the
+  Chart layer and never affect reconstructed times.
+- Branch commands (``#BRANCHSTART`` etc.) fail closed with
+  ``BranchUnsupportedError``: a branched section is three parallel note
+  streams and cannot be represented as one digit stream (upstream
+  taiko-1000-parsed is already flattened to a single stream).
+- ``#DELAY`` is a signed second value added to the timeline at its position.
+  A negative delay that makes reconstructed note times non-monotonic raises
+  ``NonMonotonicTimeError`` (fail-closed, SLOTPLUS_DESIGN section 3.1).
+- Tempo-octave labels: the song's initial k is pinned from the header BPM by
+  the canonical window ``[PERIOD_LO, PERIOD_HI)``; each subsequent TEMPO
+  event keeps ``p`` log-nearest to the previous canonical period, so k moves
+  only on ~x2/x0.5 notation switches (SLOTPLUS_DESIGN section 2.2).  (k, p)
+  are annotation: time reconstruction always uses the authored value.
+
+Round-trip contract (SLOTPLUS_DESIGN section 8-S0): the trace carries every
+authored TEMPO/MEASURE/DELAY value verbatim and reproduces every note's
+absolute time exactly (rational arithmetic; decimal header/directive text is
+parsed as exact decimal fractions).
+"""
+
+import math
+from dataclasses import dataclass, field
+from fractions import Fraction
+
+# Canonical tempo-octave window (draft values per SLOTPLUS_DESIGN section
+# 2.2: [1.0, 2.0) seconds ~ BPM (120, 240]; re-derived and frozen before S1).
+PERIOD_LO = 1.0
+PERIOD_HI = 2.0
+
+# Timing-bearing TJA commands (the trace's command alphabet besides NOTE).
+TIMING_COMMANDS = ("BPMCHANGE", "MEASURE", "DELAY")
+# Branch structure diverges into parallel streams: fail closed.
+BRANCH_COMMANDS = ("BRANCHSTART", "BRANCHEND", "N", "E", "M")
+
+# Raw TJA note characters -> canonical note classes (vocab.NOTE_CLASSES).
+# '9' (kusudama) is balloon-class, mirroring vocab.NOTE_TYPE_MAP.
+NOTE_CHAR_CLASS = {"1": "don", "2": "ka", "3": "don_big", "4": "ka_big",
+                   "5": "roll", "6": "roll_big", "7": "balloon", "8": "end",
+                   "9": "balloon"}
+
+_COURSE_ALIASES = {
+    "easy": "easy", "0": "easy",
+    "normal": "normal", "1": "normal",
+    "hard": "hard", "2": "hard",
+    "oni": "oni", "3": "oni",
+    "edit": "ura", "ura": "ura", "4": "ura",
+}
+
+
+class TraceError(ValueError):
+    """A TJA construct the trace layer refuses to guess about."""
+
+
+class BranchUnsupportedError(TraceError):
+    """#BRANCHSTART sections fail closed (parallel streams, upstream-flattened)."""
+
+
+class NonMonotonicTimeError(TraceError):
+    """A negative #DELAY produced non-monotonic note times (fail-closed)."""
+
+
+@dataclass(frozen=True)
+class ChartCommand:
+    offset: int
+    name: str
+    args: str
+
+
+@dataclass
+class ChartBar:
+    digits: str
+    commands: list
+
+
+@dataclass
+class Chart:
+    headers: dict
+    course: str
+    level: int
+    bpm_text: str
+    offset_text: str
+    start_args: str
+    bars: list
+    trailing_commands: list = field(default_factory=list)
+
+
+@dataclass(frozen=True)
+class NoteEvent:
+    bar: int
+    pos: Fraction
+    type: str
+
+
+@dataclass(frozen=True)
+class TempoEvent:
+    bar: int
+    pos: Fraction
+    bpm: float
+    authored: str
+    k: int
+    p: float
+
+
+@dataclass(frozen=True)
+class DelayEvent:
+    bar: int
+    pos: Fraction
+    seconds: float
+    authored: str
+
+
+@dataclass(frozen=True)
+class MeterEvent:
+    bar: int
+    num: int
+    den: int
+    authored_num: str
+    authored_den: str
+
+
+@dataclass(frozen=True)
+class BarInfo:
+    index: int
+    denom: int
+    num: int
+    den: int
+    k: int
+
+
+@dataclass
+class Trace:
+    course: str
+    level: int
+    bpm: float
+    bpm_authored: str
+    offset: float
+    offset_authored: str
+    bars: list
+    events: list
+
+
+def note_class(char):
+    """Canonical note class for a raw TJA note character (None if unknown)."""
+    return NOTE_CHAR_CLASS.get(char)
+
+
+def _fraction_text(text):
+    """Exact rational value of authored numeric text (decimal-exact when the
+    text is decimal; otherwise the exact rational of the parsed float)."""
+    try:
+        return Fraction(text)
+    except (ValueError, ZeroDivisionError):
+        return Fraction(float(text))
+
+
+def _build_chart(headers, start_args, bars, trailing):
+    course_text = headers.get("COURSE", "Oni").strip()
+    course = _COURSE_ALIASES.get(course_text.lower(), course_text.lower())
+    level_text = headers.get("LEVEL", "").strip()
+    try:
+        level = int(float(level_text)) if level_text else 0
+    except ValueError:
+        level = 0
+    bpm_text = headers.get("BPM", "").strip() or "120"
+    offset_text = headers.get("OFFSET", "").strip() or "0"
+    return Chart(dict(headers), course, level, bpm_text, offset_text,
+                 start_args, bars, trailing)
+
+
+def parse_tja(text):
+    """Parse TJA text into one Chart per #START/#END play section.
+
+    Headers accumulate across the file (TJA semantics: BPM/OFFSET persist
+    unless re-declared before the next course section).  Comments (``//``)
+    are stripped everywhere; a measure may span several lines and a line may
+    hold several comma-terminated measures.
+    """
+    charts = []
+    headers = {}
+    in_body = False
+    start_args = ""
+    bars = commands = None
+    digits = ""
+    for raw in text.lstrip("\ufeff").splitlines():
+        cut = raw.find("//")
+        line = (raw[:cut] if cut >= 0 else raw).strip()
+        if not line:
+            continue
+        if not in_body:
+            if line.startswith("#"):
+                head = line[1:].split(None, 1)
+                if head and head[0].upper() == "START":
+                    in_body = True
+                    start_args = head[1].strip() if len(head) > 1 else ""
+                    bars, digits, commands = [], "", []
+                # other pre-body commands carry no timing; ignored
+            elif ":" in line:
+                key, value = line.split(":", 1)
+                headers[key.strip().upper()] = value.strip()
+            continue
+        if line.startswith("#"):
+            head = line[1:].split(None, 1)
+            name = head[0].upper() if head else ""
+            args = head[1].strip() if len(head) > 1 else ""
+            if name == "END":
+                trailing = []
+                if digits:
+                    bars.append(ChartBar(digits, commands))
+                else:
+                    trailing = commands
+                charts.append(_build_chart(headers, start_args, bars, trailing))
+                in_body = False
+                bars, digits, commands = None, "", None
+                continue
+            if name in BRANCH_COMMANDS:
+                raise BranchUnsupportedError(
+                    f"branch command #{name} is not supported (fail-closed)")
+            commands.append(ChartCommand(len(digits), name, args))
+            continue
+        for ch in line:
+            if ch == ",":
+                bars.append(ChartBar(digits, commands))
+                digits, commands = "", []
+            elif ch.isspace():
+                continue
+            elif ch.isascii() and ch.isalnum():
+                digits += ch
+            else:
+                raise TraceError(
+                    f"unexpected character {ch!r} in note line {raw!r}")
+    if in_body:
+        raise TraceError("unterminated #START section (missing #END)")
+    return charts
+
+
+def parse_tja_course(text, course):
+    """Return the first Chart whose canonical course matches ``course``."""
+    wanted = _COURSE_ALIASES.get(str(course).strip().lower(),
+                                 str(course).strip().lower())
+    for chart in parse_tja(text):
+        if chart.course == wanted:
+            return chart
+    raise TraceError(f"no {wanted!r} course section in this TJA")
+
+
+def chart_to_trace(chart):
+    """Convert a parsed Chart into its command Trace.
+
+    Positions are exact ``Fraction(written_digits, row_length)``; authored
+    numeric text is kept verbatim on every event.  Non-timing commands are
+    excluded (they cannot move any note time); branch charts never reach
+    here (parse_tja fails closed).
+    """
+    try:
+        bpm0 = float(chart.bpm_text)
+    except ValueError:
+        raise TraceError(f"unparseable header BPM {chart.bpm_text!r}") from None
+    if not math.isfinite(bpm0) or bpm0 <= 0:
+        raise TraceError(f"header BPM must be finite and positive: {chart.bpm_text!r}")
+    try:
+        offset0 = float(chart.offset_text)
+    except ValueError:
+        raise TraceError(
+            f"unparseable header OFFSET {chart.offset_text!r}") from None
+    if not math.isfinite(offset0):
+        raise TraceError(f"header OFFSET must be finite: {chart.offset_text!r}")
+
+    state = {"num": 4, "den": 4}
+    period0 = 60.0 / bpm0
+    # Pin the song's initial octave label from the canonical window
+    # (SLOTPLUS_DESIGN section 2.2); with PERIOD_LO = 1.0 this is floor(log2).
+    state["k"] = math.floor(math.log2(period0 / PERIOD_LO))
+    state["p"] = period0 / 2.0 ** state["k"]
+
+    events = []
+    bars = []
+
+    def emit(bar_index, pos, cmd):
+        if cmd.name == "BPMCHANGE":
+            try:
+                value = float(cmd.args)
+            except ValueError:
+                raise TraceError(
+                    f"bar {bar_index}: unparseable #BPMCHANGE {cmd.args!r}"
+                ) from None
+            if not math.isfinite(value) or value <= 0:
+                raise TraceError(
+                    f"bar {bar_index}: #BPMCHANGE must be finite and positive: "
+                    f"{cmd.args!r}")
+            period = 60.0 / value
+            # k moves only when the new period leaves the +-half-octave band
+            # around the previous canonical period p (~x2/x0.5 switches).
+            k = round(math.log2(period / state["p"]))
+            state["k"], state["p"] = k, period / 2.0 ** k
+            events.append(TempoEvent(bar_index, pos, value, cmd.args, k,
+                                     state["p"]))
+        elif cmd.name == "MEASURE":
+            parts = cmd.args.split("/")
+            if len(parts) != 2:
+                raise TraceError(
+                    f"bar {bar_index}: unparseable #MEASURE {cmd.args!r}")
+            authored_num, authored_den = parts[0].strip(), parts[1].strip()
+            try:
+                num, den = int(authored_num), int(authored_den)
+            except ValueError:
+                raise TraceError(
+                    f"bar {bar_index}: #MEASURE terms must be integers: "
+                    f"{cmd.args!r}") from None
+            if num <= 0 or den <= 0:
+                raise TraceError(
+                    f"bar {bar_index}: #MEASURE terms must be positive: "
+                    f"{cmd.args!r}")
+            state["num"], state["den"] = num, den
+            events.append(MeterEvent(bar_index, num, den, authored_num,
+                                     authored_den))
+        else:  # DELAY
+            try:
+                seconds = float(cmd.args)
+            except ValueError:
+                raise TraceError(
+                    f"bar {bar_index}: unparseable #DELAY {cmd.args!r}"
+                ) from None
+            if not math.isfinite(seconds):
+                raise TraceError(
+                    f"bar {bar_index}: #DELAY must be finite: {cmd.args!r}")
+            events.append(DelayEvent(bar_index, pos, seconds, cmd.args))
+
+    carry = []  # timing commands owed to the next bar's position 0
+    source_bars = list(chart.bars)
+    trailing = [c for c in chart.trailing_commands if c.name in TIMING_COMMANDS]
+    index = 0
+    while index < len(source_bars) or carry or trailing:
+        if index < len(source_bars):
+            bar = source_bars[index]
+        else:
+            # Synthesized empty bar hosting end-of-chart carried commands so
+            # their authored values stay in the trace (module docstring).
+            bar = ChartBar("", [])
+        length = len(bar.digits)
+        local, carry_out = [], []
+        for cmd in bar.commands:
+            if cmd.name not in TIMING_COMMANDS:
+                continue
+            if (cmd.name == "MEASURE" and cmd.offset > 0) or \
+                    (length > 0 and cmd.offset >= length):
+                carry_out.append(cmd)
+            else:
+                local.append(cmd)
+        if index >= len(source_bars):
+            local = list(trailing)
+            trailing = []
+        head = carry + [c for c in local if c.offset == 0]
+        tail = [c for c in local if c.offset > 0]
+        carry = carry_out
+        for cmd in head:
+            emit(index, Fraction(0), cmd)
+        bars.append(BarInfo(index, length, state["num"], state["den"],
+                            state["k"]))
+        ti = 0
+        for i, ch in enumerate(bar.digits):
+            while ti < len(tail) and tail[ti].offset <= i:
+                emit(index, Fraction(tail[ti].offset, length), tail[ti])
+                ti += 1
+            if ch != "0":
+                events.append(NoteEvent(index, Fraction(i, length), ch))
+        # remaining tail commands all had offset == length and were carried
+        index += 1
+
+    return Trace(chart.course, chart.level, bpm0, chart.bpm_text,
+                 offset0, chart.offset_text, bars, events)
+
+
+def _iter_timed(trace):
+    """Walk the command stream with exact rational time.
+
+    Yields ("edge", bar_index, t) at every bar start plus the terminal edge
+    (bar_index == len(bars)), and ("note", NoteEvent, t) for every note, all
+    in stream order.  The terminal edge is evaluated from trace state, never
+    extrapolated (SLOTPLUS_DESIGN section 8-S0).
+    """
+    t = -_fraction_text(trace.offset_authored)
+    tau = Fraction(60) / _fraction_text(trace.bpm_authored)
+    num, den = 4, 4
+    bar_i = 0
+    cursor = Fraction(0)
+    n_bars = len(trace.bars)
+    yield "edge", 0, t
+    for ev in trace.events:
+        while bar_i < ev.bar:
+            t += (1 - cursor) * Fraction(4 * num, den) * tau
+            bar_i += 1
+            cursor = Fraction(0)
+            yield "edge", bar_i, t
+        if isinstance(ev, MeterEvent):
+            if cursor != 0:
+                raise TraceError(
+                    f"bar {ev.bar}: MEASURE away from the bar boundary is "
+                    "outside the grammar")
+            num, den = ev.num, ev.den
+            continue
+        if ev.pos != cursor:
+            t += (ev.pos - cursor) * Fraction(4 * num, den) * tau
+            cursor = ev.pos
+        if isinstance(ev, TempoEvent):
+            tau = Fraction(60) / _fraction_text(ev.authored)
+        elif isinstance(ev, DelayEvent):
+            t += _fraction_text(ev.authored)
+        else:
+            yield "note", ev, t
+    while bar_i < n_bars:
+        t += (1 - cursor) * Fraction(4 * num, den) * tau
+        bar_i += 1
+        cursor = Fraction(0)
+        yield "edge", bar_i, t
+
+
+def trace_to_times(trace, exact=False, check_monotonic=True):
+    """Reconstruct every note's absolute time (seconds) and type from a trace.
+
+    Returns a list of (time, type_char) in stream order; times are floats, or
+    exact Fractions with ``exact=True``.  Raises NonMonotonicTimeError when a
+    negative #DELAY makes note times non-increasing (fail-closed).
+    """
+    out = []
+    last = None
+    for kind, payload, t in _iter_timed(trace):
+        if kind != "note":
+            continue
+        if check_monotonic and last is not None and t <= last:
+            raise NonMonotonicTimeError(
+                f"non-monotonic note time at bar {payload.bar} "
+                f"pos {payload.pos}: {float(t)} after {float(last)}")
+        last = t
+        out.append((t if exact else float(t), payload.type))
+    return out
+
+
+def bar_edge_times(trace, exact=False):
+    """Absolute times of every bar start plus the terminal edge.
+
+    Returns ``len(trace.bars) + 1`` values; the terminal edge comes from
+    evaluating trace state across the final bar, not from extrapolation.
+    """
+    edges = []
+    for kind, _payload, t in _iter_timed(trace):
+        if kind == "edge":
+            edges.append(t if exact else float(t))
+    return edges

softchart/sc2/vocab.py

@@ -0,0 +1,301 @@
+"""Token vocabulary for chart sequences.
+
+Sequence layout (decoder):
+    [BOS] [COURSE_c] [LEVEL_l] [DENS_d] ([TIME_t] [NOTE_e])* [EOS]
+Conditions may be UNK (condition dropout / unspecified at inference).
+"""
+
+import math
+from dataclasses import dataclass
+from numbers import Real
+
+# --- audio / windowing constants (single source of truth) ---
+SR = 22050
+N_FFT = 2048
+HOP = 256
+N_MELS = 128
+FPS = SR / HOP  # 86.1328125 frames/sec
+WINDOW = 1728  # frames per training window (~20.06 s)
+MAX_TGT = 960  # max decoder length (prefix + events + eos); 960 covers densest ura
+# BarScript window (912 content tokens + prefix, train-split measured) with headroom.
+# Frozen 2026-07-25 (spec ruling); was 768 through v1.8.
+
+SLOTS = 96  # TJA lattice: slots per 4/4 measure (LCM of 16ths/triplets/32nds)
+MEAS_MAX = WINDOW // SLOTS  # max whole measures per window in slot mode (18)
+SLOTS_PER_BEAT = 24  # meter-general lattice (v3): 3/4 -> 72, 5/4 -> 120 slots
+BEATS_MAX = WINDOW // SLOTS_PER_BEAT  # 72 beats of token space per window
+
+COURSES = ["easy", "normal", "hard", "oni", "ura"]
+N_LEVELS = 12  # levels clamped to 1..12
+N_DENS = 16
+DENS_BUCKET_NPS = 0.75  # bucket width in notes/sec
+
+# canonical note classes; parsed TJA note_type strings are mapped onto these
+NOTE_CLASSES = ["don", "ka", "don_big", "ka_big", "roll", "roll_big", "balloon", "end"]
+
+# --- SoftChart 2.0 BarScript constants (SOFTCHART2_DESIGN §1.2/§8.1) ---
+# Per-bar plan-token families. These are DISTINCT from the window-prefix
+# condition families (pdens/sync): plan tokens live inside the bar stream at
+# bar granularity, the older families are window-level prefix conditions.
+N_DENS_D8 = 8       # per-bar onset-count density bucket (DENS_d8)
+N_SPAN_BUDGET = 4   # per-bar span-start budget
+N_BAR_SYNC = 6      # per-bar LHL syncopation band
+MOTIF_DELTAS = (1, 2, 4, 8)  # lookback (bars) for REPEAT/VAR references
+MOTIF_REF_NAMES = (
+    ("new",)
+    + tuple(f"repeat_{d}" for d in MOTIF_DELTAS)
+    + tuple(f"var_{d}" for d in MOTIF_DELTAS)
+)  # 9: NEW, REPEAT x {1,2,4,8}, VAR x {1,2,4,8}
+N_MOTIF_REF = len(MOTIF_REF_NAMES)
+# split-axis condition buckets (prefix conditions, per-axis CFG)
+N_BIG_RATE = 8
+N_SPAN_RATE = 8
+N_STREAM = 6
+# span duration buckets (log grid anchored on authored p50/p90/p99).
+# TENTATIVE: the design doc fixes the grid family but not the bucket count;
+# 16 is a placeholder pending the frozen label distribution report.
+N_SPAN_DUR_BUCKETS = 16
+# per-bar rational-lattice denominator support (BarScript skeleton positions
+# are bar-local indices on a D_m grid, D_m = LCM of the bar's reduced event
+# denominators; SLOTPLUS_DESIGN §2.1). 1..192 is a draft SUPERSET: it covers
+# dyadic 64ths (denom 192) and the observed 5/7/9/11 tuplet families; the
+# frozen support set comes from the S0 census. Bars needing D_m > 192 stay
+# fail-closed (counted), never snapped.
+MAX_BAR_DENOM = 192
+
+# BarScript stage ids (stage embedding routing; SOFTCHART2_DESIGN §1.1)
+STAGE_HEADER = 0
+STAGE_PLAN = 1
+STAGE_SKELETON = 2
+STAGE_REALIZATION = 3
+N_STAGES = 4
+
+# slot+ TEMPO factorization heads (SLOTPLUS_DESIGN §2.2/§3.2/§3.3):
+# beat period = p * 2^k with canonical p in [TEMPO_P_LO, TEMPO_P_HI) seconds
+# (draft window; frozen from bpm_map before S1). The regression head predicts
+# log p (no hard clamp); k is a classification label. TENTATIVE: the spec
+# leaves k in Z; classification needs finite support, so k is clamped to
+# [TEMPO_K_MIN, TEMPO_K_MIN + N_TEMPO_K).
+TEMPO_P_LO = 1.0
+TEMPO_P_HI = 2.0
+TEMPO_K_MIN = -3
+N_TEMPO_K = 7  # k in {-3..+3}
+
+# mapping from strings observed in the dataset -> canonical class
+# (observed in taiko-1000-parsed: Don, Ka, DonBig, KaBig, Roll, RollBig,
+#  Balloon, BalloonAlt, EndOf)
+NOTE_TYPE_MAP = {
+    "Don": "don",
+    "Ka": "ka",
+    "DonBig": "don_big",
+    "KaBig": "ka_big",
+    "Roll": "roll",
+    "RollBig": "roll_big",
+    "Balloon": "balloon",
+    "BalloonAlt": "balloon",  # kusudama, treated as balloon-class span
+    "EndOf": "end",
+}
+
+
+@dataclass
+class Vocab:
+    pad: int = 0
+    eos: int = 1
+    bos: int = 2
+    unk_cond: int = 3
+
+    def __post_init__(self):
+        base = 4
+        self.course = {c: base + i for i, c in enumerate(COURSES)}
+        base += len(COURSES)
+        self.level = {l: base + l - 1 for l in range(1, N_LEVELS + 1)}
+        base += N_LEVELS
+        self.dens = {d: base + d for d in range(N_DENS)}
+        base += N_DENS
+        self.note = {n: base + i for i, n in enumerate(NOTE_CLASSES)}
+        base += len(NOTE_CLASSES)
+        self.time0 = base
+        base += WINDOW
+        # v2+ tokens appended at the end so older checkpoints stay loadable
+        self.sep = base  # separates prev-window context from conditions
+        self.sib = base + 1  # marks the sibling-chart (easier course) segment
+        self.style = {s: base + 2 + s for s in range(8)}  # charting-intent codes
+        self.sync = {s: base + 10 + s for s in range(6)}  # LHL syncopation bands
+        # plan-realize tokens: [PLAN] marker + block density (8) + block flags (3)
+        self.plan = base + 16
+        self.pdens = {d: base + 17 + d for d in range(8)}
+        self.pflag = {f: base + 25 + f for f in range(3)}  # 0 none / 1 gap / 2 climax
+        self.mask = base + 28  # type-infill placeholder
+        self.cplx = {c: base + 29 + c for c in range(6)}  # rhythmic-complexity band
+        # dual-mode models: explicit output-semantics switch (v3). mode "slot"
+        # = positions are lattice indices on a supplied grid; "time" = frames.
+        self.mode = {"slot": base + 35, "time": base + 36}
+        # --- SoftChart 2.0 BarScript tokens (append-only; older checkpoints
+        # keep every pre-existing id). SOFTCHART2_DESIGN §1.2/§8.1. ---
+        b2 = base + 37
+        self.bar = b2  # [BAR] header-stage marker (TEMPO/MEASURE/k anchor)
+        self.real = b2 + 1  # [REAL] realization-stage marker
+        self.bar_dens = {d: b2 + 2 + d for d in range(N_DENS_D8)}
+        self.span_budget = {s: b2 + 10 + s for s in range(N_SPAN_BUDGET)}
+        self.bar_sync = {s: b2 + 14 + s for s in range(N_BAR_SYNC)}
+        self.motif = {m: b2 + 20 + i for i, m in enumerate(MOTIF_REF_NAMES)}
+        self.big_rate = {r: b2 + 29 + r for r in range(N_BIG_RATE)}
+        self.span_rate = {r: b2 + 37 + r for r in range(N_SPAN_RATE)}
+        self.stream = {s: b2 + 45 + s for s in range(N_STREAM)}
+        self.span_dur = {b: b2 + 51 + b for b in range(N_SPAN_DUR_BUCKETS)}
+        # --- BarScript skeleton markers + notation denominator (append-only,
+        # 2026-07-24; SOFTCHART2_DESIGN §1.1 skeleton stage, §1.3 span object).
+        # Skeleton POSITION tokens reuse the time0 family as BAR-LOCAL lattice
+        # indices (v3 slot-mode precedent: "positions are lattice indices on a
+        # supplied grid"), so hit token counts match the flat encoding exactly.
+        b3 = b2 + 51 + N_SPAN_DUR_BUCKETS
+        self.span_start = b3  # skeleton marker: next position opens a span
+        self.span_end = b3 + 1  # skeleton marker: next position closes the span
+        self.bar_denom = {d: b3 + 2 + (d - 1)
+                          for d in range(1, MAX_BAR_DENOM + 1)}
+        self.size = b3 + 2 + MAX_BAR_DENOM
+
+        self.id2note = {v: k for k, v in self.note.items()}
+        self.id2course = {v: k for k, v in self.course.items()}
+
+    def time(self, frame: int) -> int:
+        assert 0 <= frame < WINDOW
+        return self.time0 + frame
+
+    def is_time(self, tok: int) -> bool:
+        return self.time0 <= tok < self.time0 + WINDOW
+
+    def is_note(self, tok: int) -> bool:
+        return tok in self.id2note
+
+    def dens_bucket(self, nps: float) -> int:
+        return min(N_DENS - 1, max(0, int(nps / DENS_BUCKET_NPS)))
+
+
+VOCAB = Vocab()
+
+
+def v2_token_ids(vocab=VOCAB):
+    """Explicit name -> id map for every SoftChart 2.0 appended token."""
+    ids = {"BAR": vocab.bar, "REAL": vocab.real}
+    ids.update({f"DENS_D8_{d}": t for d, t in vocab.bar_dens.items()})
+    ids.update({f"SPAN_BUDGET_{s}": t for s, t in vocab.span_budget.items()})
+    ids.update({f"BAR_SYNC_{s}": t for s, t in vocab.bar_sync.items()})
+    ids.update({f"MOTIF_{m.upper()}": t for m, t in vocab.motif.items()})
+    ids.update({f"BIG_RATE_{r}": t for r, t in vocab.big_rate.items()})
+    ids.update({f"SPAN_RATE_{r}": t for r, t in vocab.span_rate.items()})
+    ids.update({f"STREAM_{s}": t for s, t in vocab.stream.items()})
+    ids.update({f"SPAN_DUR_{b}": t for b, t in vocab.span_dur.items()})
+    ids["SPAN_START"] = vocab.span_start
+    ids["SPAN_END"] = vocab.span_end
+    ids.update({f"BAR_DENOM_{d}": t for d, t in vocab.bar_denom.items()})
+    return ids
+
+
+V2_TOKEN_IDS = v2_token_ids()
+
+
+SIB_EVENTS = 12  # fixed number of sibling-chart events in the prefix (24 tokens)
+
+
+PLAN_SLOTS = 5  # fixed number of plan blocks in the prefix (2 tokens each)
+
+
+def complexity_band(frames):
+    """IOI-class entropy of a window's hit sequence -> band 0-5 (rhythmic
+    complexity independent of raw density)."""
+    import math
+    hits = sorted(f for f, c in frames if NOTE_CLASSES[c] in ("don","ka","don_big","ka_big"))
+    if len(hits) < 4:
+        return 0
+    iois = [b - a for a, b in zip(hits, hits[1:]) if b > a]
+    if not iois:
+        return 0
+    from collections import Counter
+    cls = Counter(int(round(math.log2(max(i,1)) * 2)) for i in iois)
+    tot = sum(cls.values())
+    ent = -sum((n/tot) * math.log2(n/tot) for n in cls.values())
+    return min(5, int(ent / 0.5))
+
+
+def encode_window(vocab, course, level, notes, cond_drop=0.0, rng=None, ctx_types=None,
+                  sib_pairs=None, style=None, sync_band=None, plan_slice=None,
+                  complexity=None, mode=None, duration_seconds=None):
+    """Build a token sequence for one window.
+
+    notes: list of (frame_idx, note_class_id) sorted by frame, frame in [0, WINDOW).
+    ctx_types: optional list of note-class ids from the previous window's tail
+        (pattern continuity context, v2). Encoded as [BOS] ctx.. [SEP] conds..
+    sib_pairs: optional list of (frame, class_id) events from an easier course of
+        the same song (skeleton hint, easy⊂hard). Fixed SIB_EVENTS slots,
+        missing slots filled with UNK. Encoded after [SIB].
+    Returns (tokens, prefix_len) where loss should be applied after the prefix.
+    """
+    if duration_seconds is None:
+        duration_seconds = WINDOW / FPS
+    if (not isinstance(duration_seconds, Real)
+            or isinstance(duration_seconds, bool)
+            or not math.isfinite(duration_seconds)
+            or duration_seconds <= 0):
+        raise ValueError("duration_seconds must be positive and finite")
+    n_hits = sum(1 for _, c in notes if NOTE_CLASSES[c] not in ("end",))
+    nps = n_hits / float(duration_seconds)
+    d = vocab.dens_bucket(nps)
+
+    def maybe(tok):
+        if cond_drop > 0 and rng is not None and rng.random() < cond_drop:
+            return vocab.unk_cond
+        return tok
+
+    lvl = max(1, min(N_LEVELS, level if level and level > 0 else 1))
+    seq = [vocab.bos]
+    if mode is not None:  # dual-mode: explicit output-semantics token (never dropped)
+        seq.append(vocab.mode[mode])
+    if ctx_types is not None:  # fixed-length, left-padded with UNK (id < 0 = pad)
+        seq += [vocab.note[NOTE_CLASSES[c]] if c >= 0 else vocab.unk_cond
+                for c in ctx_types]
+        seq.append(vocab.sep)
+    if sib_pairs is not None:
+        seq.append(vocab.sib)
+        pairs = list(sib_pairs)[:SIB_EVENTS]
+        for f, c in pairs:
+            seq += [vocab.time(int(f)), vocab.note[NOTE_CLASSES[c]]]
+        seq += [vocab.unk_cond] * (2 * (SIB_EVENTS - len(pairs)))
+    seq += [
+        maybe(vocab.course[course]),
+        maybe(vocab.level[lvl]),
+        maybe(vocab.dens[d]),
+    ]
+    if style is not None:  # charting-intent code (style >= 0; -1 = unknown)
+        seq.append(maybe(vocab.style[style]) if style >= 0 else vocab.unk_cond)
+    if sync_band is not None:  # LHL syncopation band (groove-intensity control)
+        seq.append(maybe(vocab.sync[sync_band]) if sync_band >= 0 else vocab.unk_cond)
+    if complexity is not None:  # rhythmic-complexity band (difficulty beyond density)
+        seq.append(maybe(vocab.cplx[complexity]) if complexity >= 0 else vocab.unk_cond)
+    if plan_slice is not None:  # song-level plan blocks overlapping this window
+        seq.append(vocab.plan)
+        blocks = list(plan_slice)[:PLAN_SLOTS]
+        for d8, fl in blocks:
+            seq += [maybe(vocab.pdens[min(7, max(0, d8))]),
+                    maybe(vocab.pflag[min(2, max(0, fl))])]
+        seq += [vocab.unk_cond] * (2 * (PLAN_SLOTS - len(blocks)))
+    prefix_len = len(seq)
+    for f, c in notes:
+        seq.append(vocab.time(int(f)))
+        seq.append(vocab.note[NOTE_CLASSES[c]])
+    seq.append(vocab.eos)
+    return seq, prefix_len
+
+
+def decode_tokens(vocab, tokens):
+    """Token ids -> list of (frame_idx, note_class_name). Ignores malformed pairs."""
+    out = []
+    cur_t = None
+    for tok in tokens:
+        if vocab.is_time(tok):
+            cur_t = tok - vocab.time0
+        elif tok in vocab.id2note and cur_t is not None:
+            out.append((cur_t, vocab.id2note[tok]))
+        elif tok == vocab.eos:
+            break
+    return out

softchart/sc2_loader.py

@@ -0,0 +1,944 @@
+"""Load and serve the BarScript release package.
+
+This is the only bridge between the Space and ``softchart.sc2``.  It reads a
+release package (``config.json``, ``spec.json``, ``model.safetensors``) and
+returns an object whose ``generate`` call produces TJA text.
+
+The package is resolved in one order, most explicit first -- ``$SC2_PACKAGE_DIR``,
+a local directory, then the Hub repo (see ``resolve_package_dir``).  Whichever
+source wins, the package directory is the whole source of truth from that point
+on: every integrity check and every serving parameter is read out of it, so a
+downloaded package and a local one are served identically or not at all.
+
+The Hub repo is **private**, so the download needs a token.  When none can be
+found the loader says exactly that, rather than reporting a missing model --
+a token problem and an absent package have different fixes.
+
+Every serving parameter comes from ``config.json``.  The package records not
+just the values but the reasoning behind each one in ``config.json:
+serving_basis``, so a constant re-typed in the Space would be a second,
+unaudited contract.  ``serving_kwargs`` therefore mirrors the package's own
+``scripts/sc2_package.py:serving_kwargs`` and passes the recorded values
+explicitly into the decoder rather than letting module defaults apply.
+
+Deployment grid: a uniform ``/16`` lattice synthesized from user BPM and
+offset, decoded with ``bar_denoms='supplied'`` -- the ``SHIP_DECISION``
+ruling, and the ``default`` serving profile.  The package also carries a
+``deploy_bpm_grid`` profile (model-emitted denominators masked to divisors of
+96); the model card does NOT recommend it for this checkpoint (§2.8).  A
+``/16`` grid makes triplets arithmetically impossible: 3 does not divide 16.
+
+The returned dict always reports the grid that was actually used.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+import re
+import threading
+
+import torch
+
+from .sc2 import generate as sc2_generate
+from .sc2.barscript import BarscriptSpec
+from .sc2.checkpoint import (GENERATOR_CAPABILITY_KEYS,
+                             attach_generator_capabilities)
+from .sc2.generate import (SlotExportError, barscript_grid_from_bpm,
+                           generate_song_barscript, require_slot_export)
+from .sc2.model import ChartModel
+from .sc2.tja import write_tja_slots
+from .sc2.vocab import (COURSES, DENS_BUCKET_NPS, FPS, N_DENS, SLOTS,
+                        V2_TOKEN_IDS, VOCAB, WINDOW)
+
+CONFIG_SCHEMA = "softchart2_release_config_v1"
+
+# The Hub repo the release is published to. PRIVATE: reading it needs a token
+# (see resolve_token). Overridable so a fork or a re-upload does not need a code
+# change, but the default is the published release.
+DEFAULT_HUB_REPO = "JacobLinCool/softchart-barscript"
+
+# The release's own directory name. This is the RUN identifier, not the public
+# name: it is what the package, its manifest and every provenance record call
+# this checkpoint, so it stays as built.
+RELEASE_DIRNAME = "sc2_c1_v13_s1234_best"
+
+# Exactly the files the release package consists of. snapshot_download is given
+# this as its allow-list, so a repo that later grows unrelated files cannot pull
+# them into the Space.
+PACKAGE_FILES = ("MODEL_CARD.md", "config.json", "model.safetensors",
+                 "preprocessor_config.json", "release_manifest.json",
+                 "spec.json", "vocab.json")
+
+# Files without which the package cannot be served at all.
+REQUIRED_FILES = ("config.json", "model.safetensors", "preprocessor_config.json",
+                  "spec.json", "vocab.json")
+
+# The uniform bar lattice the release ships on (SHIP_DECISION / MODEL_CARD §3,
+# "Recommended deployment configuration": --grid bpm --grid-bpm-denom 16).
+SHIP_GRID_DENOM = 16
+
+# md5 of every vendored BarScript module, as recorded by the release package in
+# config.json:code.module_md5.  Checked at load time -- see verify_code().
+VENDORED_MODULES = ("barscript.py", "checkpoint.py", "generate.py",
+                    "meter.py", "model.py", "tja.py", "trace.py", "vocab.py")
+
+# TJA `COURSE:` header literals, measured on the corpus rather than assumed.
+# Over the 1 155 songs of JacobLinCool/taiko-1000-parsed, 229 carry an `ura`
+# chart, and every one of them writes that chart's header as `Edit` (223) or
+# the numeric `4` (6).  The string "Ura" appears zero times: it is the
+# dataset's normalized course NAME, not the TJA header literal.  `tja.py`
+# derives the header as `str(course).capitalize()`, which would emit
+# `COURSE:Ura`, so the header is rewritten here -- in code this Space owns --
+# and the vendored module stays byte-identical to the shipping tree.
+TJA_COURSE_HEADER = {"easy": "Easy", "normal": "Normal", "hard": "Hard",
+                     "oni": "Oni", "ura": "Edit"}
+
+# Straight from MODEL_CARD.md §2.  These are the limitations a user must see
+# before reading anything into a generated chart; the section numbers point at
+# the measurement.  Do not soften them.
+KNOWN_LIMITATIONS = (
+    ("champion", "§0",
+     "The champion comparison is INCONCLUSIVE: the challenger was run at one "
+     "seed."),
+    ("nesting", "§2.1",
+     "The difficulty ladder is not nested. Independently decoded courses "
+     "share 0.658/0.667/0.660 of their notes with the next harder course; "
+     "authored charts share 0.95-0.99."),
+    ("density", "§2.9",
+     "Notes come out too dense: 102 sub-40 ms gaps across 29 charts (authored: "
+     "2 across 1), 19 of them below the series record of 29.41 ms, and up to "
+     "17 notes inside one second."),
+    ("micro_spans", "§2.7",
+     "One drumroll in seven is a micro-span (14.7 % vs 3.8 % authored)."),
+    ("tuplets", "§2.12",
+     "Fine-lattice and tuplet placement is much worse than average: 0.345 hit "
+     "rate at positions whose denominator is divisible by 3."),
+    ("grid", "§2.8",
+     "The deployment grid is a uniform /16 lattice, so triplets are "
+     "arithmetically impossible -- 3 does not divide 16."),
+    ("density_knob", "§2.4",
+     "At the top of the density knob, drumrolls are squeezed out: span yield "
+     "falls to 33 % of authored. The highest setting is not recommended."),
+    ("ura", "corpus",
+     "Ura was trained on 208 charts against roughly 1 100 per other course; "
+     "expect lower quality."),
+)
+
+
+class PackageError(RuntimeError):
+    """The package directory is missing, incomplete or inconsistent."""
+
+
+# ---------------------------------------------------------------------------
+# locating the package
+
+
+def deploy_package_dir():
+    """``spaces/models/<release>`` -- the copy that travels with the Space.
+
+    Named separately from :func:`local_candidates` so it stays available as the
+    path to quote in a message even when the candidate list is empty.
+    """
+    spaces_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+    return os.path.join(spaces_dir, "models", RELEASE_DIRNAME)
+
+
+def local_candidates():
+    """On-disk directories searched for the package, in order.
+
+    1. :func:`deploy_package_dir`.
+    2. the frozen build artifact in the development worktree, for local runs
+       from this checkout.
+    """
+    spaces_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+    repo_root = os.path.dirname(spaces_dir)
+    return (
+        deploy_package_dir(),
+        os.path.join(repo_root, ".codex", "worktrees", "v18-8h", "artifacts",
+                     "sc2", RELEASE_DIRNAME),
+    )
+
+
+def default_package_dir():
+    """The package directory reachable WITHOUT the network.
+
+    ``$SC2_PACKAGE_DIR`` if set, else the first populated local candidate, else
+    the deploy location as a fallback so callers always get a path to name in a
+    message.  This deliberately never downloads: it is what ``is_available``
+    and the test suites use to ask whether a package is already on disk.
+    Full resolution, Hub included, is :func:`resolve_package_dir`.
+    """
+    env = os.environ.get("SC2_PACKAGE_DIR")
+    if env:
+        return os.path.abspath(os.path.expanduser(env))
+    for candidate in local_candidates():
+        if os.path.isfile(os.path.join(candidate, "config.json")):
+            return candidate
+    return deploy_package_dir()
+
+
+# ---------------------------------------------------------------------------
+# the Hub fallback
+
+
+def hub_repo_id():
+    """The Hub repo the release is published to."""
+    return os.environ.get("SC2_REPO_ID", DEFAULT_HUB_REPO)
+
+
+def hub_enabled():
+    """Whether the Hub is allowed as a source at all.
+
+    ``SC2_NO_HUB=1`` pins the Space to on-disk packages, which is how a
+    deployment proves it is not silently depending on a download.
+    """
+    if os.environ.get("SC2_NO_HUB", "").strip().lower() in ("1", "true", "yes"):
+        return False
+    return bool(hub_repo_id())
+
+
+def resolve_token(token=None):
+    """A Hub token, or ``None``.
+
+    Explicit argument, then ``$HF_TOKEN`` / ``$HUGGING_FACE_HUB_TOKEN``, then
+    whatever ``hf auth login`` cached.  On a Space the first is what a secret
+    named ``HF_TOKEN`` becomes; locally the last is the usual case.
+    """
+    for candidate in (token, os.environ.get("HF_TOKEN"),
+                      os.environ.get("HUGGING_FACE_HUB_TOKEN")):
+        if candidate:
+            return candidate
+    try:
+        from huggingface_hub import get_token
+    except ImportError:
+        return None
+    try:
+        return get_token() or None
+    except Exception:
+        return None
+
+
+def _no_token_error(repo_id):
+    return PackageError(
+        f"BarScript is published as a PRIVATE Hub repo ({repo_id}) and no "
+        "HuggingFace token was found, so its weights cannot be downloaded. "
+        "This is a credentials problem, not a missing model. Fix it in one of "
+        "three ways: (1) set $HF_TOKEN to a token with read access to "
+        f"{repo_id} -- on a Space that is Settings -> Variables and secrets -> "
+        "New secret, named HF_TOKEN; (2) run `hf auth login` on this machine; "
+        "or (3) point $SC2_PACKAGE_DIR at a local copy of the release package "
+        "and skip the download entirely. Looked in: $HF_TOKEN, "
+        "$HUGGING_FACE_HUB_TOKEN, and the HuggingFace token cache.")
+
+
+def _hub_download_error(repo_id, exc):
+    kind = type(exc).__name__
+    hint = ""
+    if kind in ("RepositoryNotFoundError", "GatedRepoError") \
+            or "401" in str(exc) or "403" in str(exc):
+        hint = (" The repo is PRIVATE, so this is what a token that is absent, "
+                "expired, or lacking read access on it looks like -- the Hub "
+                "reports a private repo as missing to callers who may not see "
+                "it. Check the token before concluding the repo is gone.")
+    elif kind == "LocalEntryNotFoundError" or "offline" in str(exc).lower():
+        hint = (" Offline mode is on and the package is not in the local "
+                "HuggingFace cache. Unset $HF_HUB_OFFLINE, or point "
+                "$SC2_PACKAGE_DIR at a local copy.")
+    return PackageError(
+        f"could not download the BarScript package from {repo_id}: "
+        f"{kind}: {exc}.{hint}")
+
+
+def hub_snapshot_cached(repo_id=None, revision=None):
+    """The cached snapshot directory for the repo, or ``None``. No network."""
+    repo_id = repo_id or hub_repo_id()
+    try:
+        from huggingface_hub import try_to_load_from_cache
+    except ImportError:
+        return None
+    try:
+        hit = try_to_load_from_cache(repo_id, "config.json",
+                                     revision=revision or None)
+    except Exception:
+        return None
+    if isinstance(hit, str) and os.path.isfile(hit):
+        return os.path.dirname(hit)
+    return None
+
+
+def download_from_hub(repo_id=None, revision=None, token=None):
+    """Fetch the release package from the Hub and return its directory.
+
+    Only :data:`PACKAGE_FILES` are fetched.  The snapshot is then checked
+    against the ``release_manifest.json`` that travels with it, because these
+    bytes crossed a network: the manifest records the sha256 of every file as
+    built, so a truncated or substituted file is caught here rather than
+    surfacing later as an unexplained decode.  Local packages are not
+    re-hashed -- this check exists for the transport, and adding it to the
+    on-disk paths would change how they already behave.
+    """
+    repo_id = repo_id or hub_repo_id()
+    revision = revision or os.environ.get("SC2_REPO_REVISION") or None
+    resolved = resolve_token(token)
+    if resolved is None:
+        raise _no_token_error(repo_id)
+    try:
+        from huggingface_hub import snapshot_download
+    except ImportError as exc:
+        raise PackageError(
+            "huggingface_hub is required to download the BarScript package "
+            f"from {repo_id}; install it or set $SC2_PACKAGE_DIR") from exc
+    try:
+        path = snapshot_download(repo_id=repo_id, repo_type="model",
+                                 revision=revision, token=resolved,
+                                 allow_patterns=list(PACKAGE_FILES))
+    except Exception as exc:  # noqa: BLE001 -- re-raised as PackageError
+        raise _hub_download_error(repo_id, exc) from exc
+    missing = [n for n in REQUIRED_FILES
+               if not os.path.isfile(os.path.join(path, n))]
+    if missing:
+        raise PackageError(
+            f"the Hub repo {repo_id} is not a complete release package; "
+            f"missing {missing}")
+    verify_manifest(path, source=repo_id)
+    return path
+
+
+def verify_manifest(package_dir, *, source="package"):
+    """Check a package's files against its own ``release_manifest.json``.
+
+    A missing manifest is accepted -- not every package carries one -- but a
+    manifest that disagrees with the bytes beside it is refused.
+    """
+    manifest_path = os.path.join(package_dir, "release_manifest.json")
+    if not os.path.isfile(manifest_path):
+        return None
+    with open(manifest_path, encoding="utf-8") as f:
+        manifest = json.load(f)
+    bad = []
+    for name, meta in (manifest.get("files") or {}).items():
+        path = os.path.join(package_dir, name)
+        if not os.path.isfile(path):
+            if name in REQUIRED_FILES:
+                bad.append(f"{name}: missing")
+            continue
+        size = os.path.getsize(path)
+        if size != int(meta["size_bytes"]):
+            bad.append(f"{name}: {size} bytes, manifest says {meta['size_bytes']}")
+            continue
+        digest = hashlib.sha256()
+        with open(path, "rb") as f:
+            for chunk in iter(lambda: f.read(1 << 20), b""):
+                digest.update(chunk)
+        if digest.hexdigest() != meta["sha256"]:
+            bad.append(f"{name}: sha256 {digest.hexdigest()[:16]} != "
+                       f"{meta['sha256'][:16]}")
+    if bad:
+        raise PackageError(
+            f"the release package from {source} does not match its own "
+            "release_manifest.json: " + "; ".join(bad))
+    return manifest
+
+
+def resolve_package_dir(*, allow_hub=True, token=None):
+    """``(package_dir, source)`` -- the full resolution order.
+
+    1. ``$SC2_PACKAGE_DIR`` -- an explicit request. If it holds no package this
+       RAISES rather than falling through: someone who names a directory wants
+       that directory, and silently downloading a different copy instead would
+       hide the typo.
+    2. a populated local candidate (:func:`local_candidates`).
+    3. the Hub repo, downloaded (:func:`download_from_hub`).
+
+    ``source`` is one of ``"env"``, ``"local"``, ``"hub"`` and is reported by
+    ``SoftChart2.describe`` so a running Space can say where its weights came
+    from.
+    """
+    env = os.environ.get("SC2_PACKAGE_DIR")
+    if env:
+        path = os.path.abspath(os.path.expanduser(env))
+        if not os.path.isfile(os.path.join(path, "config.json")):
+            raise PackageError(
+                f"$SC2_PACKAGE_DIR points at {path!r}, which has no "
+                "config.json. Unset it to fall back to a local copy or the "
+                f"Hub repo {hub_repo_id()}, or point it at a release package.")
+        return path, "env"
+    for candidate in local_candidates():
+        if os.path.isfile(os.path.join(candidate, "config.json")):
+            return candidate, "local"
+    if allow_hub and hub_enabled():
+        return download_from_hub(token=token), "hub"
+    raise PackageError(
+        "no BarScript release package found. Searched $SC2_PACKAGE_DIR and "
+        f"{list(local_candidates())}"
+        + ("; the Hub fallback is disabled by $SC2_NO_HUB."
+           if not hub_enabled() else "; the Hub fallback was not attempted."))
+
+
+# ---------------------------------------------------------------------------
+# integrity
+
+
+def _canonical_sha256(value):
+    payload = json.dumps(value, ensure_ascii=False, sort_keys=True,
+                         separators=(",", ":"), allow_nan=False).encode("utf-8")
+    return hashlib.sha256(payload).hexdigest()
+
+
+def _file_md5(path):
+    h = hashlib.md5()
+    with open(path, "rb") as f:
+        for chunk in iter(lambda: f.read(1 << 20), b""):
+            h.update(chunk)
+    return h.hexdigest()
+
+
+def vocab_tokens():
+    """``tokens_by_id`` rebuilt from the VENDORED vocabulary.
+
+    Mirrors ``scripts/sc2_package.py:vocab_contract`` field for field, so its
+    sha256 can be compared with the one the package recorded.  A mismatch
+    means the vendored ``vocab.py`` is not the table this checkpoint's
+    embedding rows were trained against, and the ids in every prefix would be
+    silently wrong.
+    """
+    tokens = [None] * VOCAB.size
+
+    def assign(token_id, name):
+        token_id = int(token_id)
+        if not 0 <= token_id < VOCAB.size:
+            raise PackageError(f"token id {token_id} out of range")
+        if tokens[token_id] is not None:
+            raise PackageError(f"duplicate token id {token_id}")
+        tokens[token_id] = name
+
+    for name in ("pad", "eos", "bos", "unk_cond"):
+        assign(getattr(VOCAB, name), name.upper())
+    for prefix, mapping in (("COURSE", VOCAB.course), ("LEVEL", VOCAB.level),
+                            ("DENS", VOCAB.dens), ("NOTE", VOCAB.note)):
+        for value, token_id in mapping.items():
+            assign(token_id, f"{prefix}_{value}")
+    for frame in range(WINDOW):
+        assign(VOCAB.time0 + frame, f"TIME_{frame}")
+    assign(VOCAB.sep, "SEP")
+    assign(VOCAB.sib, "SIB")
+    for prefix, mapping in (("STYLE", VOCAB.style), ("SYNC", VOCAB.sync),
+                            ("PDENS", VOCAB.pdens), ("PFLAG", VOCAB.pflag),
+                            ("CPLX", VOCAB.cplx), ("MODE", VOCAB.mode)):
+        for value, token_id in mapping.items():
+            assign(token_id, f"{prefix}_{value}")
+    assign(VOCAB.plan, "PLAN")
+    assign(VOCAB.mask, "MASK")
+    for name, token_id in V2_TOKEN_IDS.items():
+        assign(token_id, name)
+    missing = [i for i, t in enumerate(tokens) if t is None]
+    if missing:
+        raise PackageError(f"vocabulary mapping has unmapped ids: {missing[:8]}")
+    return tokens
+
+
+def verify_code(config):
+    """Refuse to serve with vendored modules that drifted from the package.
+
+    The package's numbers describe ONE tree of serving code, identified by md5
+    in ``config.json:code.module_md5``.  Vendored copies that no longer match
+    are a different decoder, and nothing the model card says would still apply.
+    """
+    recorded = config.get("code", {}).get("module_md5", {})
+    here = os.path.join(os.path.dirname(os.path.abspath(__file__)), "sc2")
+    bad = []
+    for name in VENDORED_MODULES:
+        want = recorded.get(f"src/softchart/{name}")
+        if want is None:
+            bad.append(f"{name}: package records no md5")
+            continue
+        got = _file_md5(os.path.join(here, name))
+        if got != want:
+            bad.append(f"{name}: {got} != {want}")
+    if bad:
+        raise PackageError(
+            "vendored BarScript serving code does not match the release "
+            "package; re-copy from the frozen shipping tree: " + "; ".join(bad))
+    return {f"softchart/sc2/{n}": recorded[f"src/softchart/{n}"]
+            for n in VENDORED_MODULES}
+
+
+def verify_vocab(config, package_dir):
+    """Cross-check the vendored token table against the package's record."""
+    tokens = vocab_tokens()
+    got = _canonical_sha256(tokens)
+    want = config["vocab"]["mapping_sha256"]
+    if got != want:
+        raise PackageError(
+            f"vendored vocab mapping sha256 {got} != package {want}")
+    if int(config["vocab"]["size"]) != VOCAB.size:
+        raise PackageError(
+            f"vendored vocab size {VOCAB.size} != package "
+            f"{config['vocab']['size']}")
+    shipped = os.path.join(package_dir, "vocab.json")
+    if os.path.isfile(shipped):
+        with open(shipped, encoding="utf-8") as f:
+            if json.load(f)["tokens_by_id"] != tokens:
+                raise PackageError("vocab.json disagrees with the vendored "
+                                   "token table")
+    if int(config["vocab"]["window_frames"]) != WINDOW \
+            or int(config["vocab"]["slots_per_4_4"]) != SLOTS:
+        raise PackageError("vendored WINDOW/SLOTS disagree with the package")
+    return got
+
+
+def preprocessing_contract(package_dir):
+    """The audio front end the package was trained and evaluated under."""
+    path = os.path.join(package_dir, "preprocessor_config.json")
+    with open(path, encoding="utf-8") as f:
+        return json.load(f)
+
+
+def check_preprocessing(package_dir, *, sr, n_fft, hop_length, n_mels,
+                        fmin, fmax):
+    """Assert a caller's log-mel front end matches the package contract.
+
+    The Space computes its own mel (``app.load_logmel``); if any parameter
+    drifts, the encoder sees a distribution it never trained on and the chart
+    degrades silently.  Raises ``PackageError`` listing every mismatch.
+    """
+    c = preprocessing_contract(package_dir)
+    want = {
+        "sample_rate_hz": float(c["audio"]["sample_rate_hz"]),
+        "n_fft": float(c["stft"]["n_fft"]),
+        "hop_length": float(c["stft"]["hop_length"]),
+        "n_mels": float(c["mel"]["n_mels"]),
+        "fmin_hz": float(c["mel"]["fmin_hz"]),
+        "fmax_hz": float(c["mel"]["fmax_hz"]),
+    }
+    got = {"sample_rate_hz": float(sr), "n_fft": float(n_fft),
+           "hop_length": float(hop_length), "n_mels": float(n_mels),
+           "fmin_hz": float(fmin), "fmax_hz": float(fmax)}
+    bad = [f"{k}: {got[k]} != {want[k]}" for k in want if got[k] != want[k]]
+    if bad:
+        raise PackageError("log-mel front end disagrees with the package "
+                           "contract: " + "; ".join(bad))
+    return c
+
+
+# ---------------------------------------------------------------------------
+# the serving contract, read from the package
+
+
+def serving_kwargs(config, profile="default"):
+    """``generate_song_barscript`` keyword arguments this release ships.
+
+    Field-for-field the same function as ``scripts/sc2_package.py:
+    serving_kwargs``: values are passed EXPLICITLY so the contract the package
+    recorded is the one that reaches the decoder, and never a module default
+    that happens to agree today.
+    """
+    profiles = config.get("serving_profiles") or {"default": {"overrides": {}}}
+    if profile not in profiles:
+        raise PackageError(f"unknown serving profile {profile!r}; "
+                           f"have {sorted(profiles)}")
+    s = dict(config["serving"], **profiles[profile].get("overrides", {}))
+    return {
+        "span_min_sec": s["span_min_sec"],
+        "span_min_beats": s["span_min_beats"],
+        "degenerate_bar_sec": s["degenerate_bar_sec"],
+        "min_onset_gap_sec": s["min_onset_gap_sec"],
+        "bar_denoms": s["bar_denoms"],
+        "bar_denom_mask": s["bar_denom_mask"],
+        "greedy": s["greedy"],
+        "temperature": s["temperature"],
+        "plan_temperature": s["plan_temperature"],
+        "top_p": s["top_p"],
+    }
+
+
+class MotifConstraint:
+    """Serve the motif-hard value the package RESOLVED, not a module default.
+
+    ``generate._decode_barscript_window`` constructs ``BarscriptFSM(spec)``
+    and exposes no parameter, so the module default would otherwise be the
+    only value a decode could get.  The vendored ``generate.py`` is
+    byte-identical to the shipping tree and stays that way, so the serving
+    value is injected by wrapping ``BarscriptFSM.__init__`` for the duration
+    of the decode -- the same mechanism, for the same reason, as
+    ``scripts/sc2_package.py`` and ``scripts/sc2_generate_eval.py``.  The
+    class object is untouched, so ``isinstance`` checks and any caller that
+    passes ``motif_hard`` explicitly keep working.
+
+    ``observed_state`` reports what was SERVED, so a decode that never built
+    an FSM cannot be reported as compliant.
+    """
+
+    def __init__(self, motif_hard):
+        self.motif_hard = bool(motif_hard)
+        self.n_constructed = 0
+        self.observed = set()
+
+    def __enter__(self):
+        from .sc2 import barscript as bs_mod
+
+        self._mod = bs_mod
+        self._orig = bs_mod.BarscriptFSM.__init__
+        orig, owner = self._orig, self
+
+        def init(fsm, spec, vocab=bs_mod.VOCAB, motif_hard=None):
+            hard = owner.motif_hard if motif_hard is None else bool(motif_hard)
+            owner.n_constructed += 1
+            owner.observed.add(hard)
+            orig(fsm, spec, vocab, hard)
+
+        bs_mod.BarscriptFSM.__init__ = init
+        return self
+
+    def __exit__(self, *exc):
+        self._mod.BarscriptFSM.__init__ = self._orig
+        return False
+
+    def observed_state(self):
+        return {"n_fsm_constructed": self.n_constructed,
+                "observed_motif_hard": sorted(self.observed),
+                "verified": bool(self.n_constructed > 0
+                                 and self.observed == {self.motif_hard})}
+
+
+def motif_context(config):
+    """The serving FSM configuration this package resolved at build time."""
+    return MotifConstraint(
+        config["train_provenance"]["motif_constraint"]["resolved_motif_hard"])
+
+
+# ---------------------------------------------------------------------------
+# loading
+
+
+def load_release(package_dir, device="cpu"):
+    """Rebuild the model from a package, reading nothing outside it.
+
+    Returns ``(model, config, spec)``.  Ports ``scripts/sc2_package.py:
+    load_release`` onto the vendored namespace; the checkpoint file, the
+    architecture arguments, the capability flags and the label spec all come
+    from ``package_dir``.
+    """
+    package_dir = os.path.abspath(package_dir)
+    config_path = os.path.join(package_dir, "config.json")
+    if not os.path.isfile(config_path):
+        raise PackageError(
+            f"no BarScript package at {package_dir!r} (config.json "
+            "missing). Point $SC2_PACKAGE_DIR at a release directory, or "
+            f"leave it unset to download {hub_repo_id()} from the Hub.")
+    with open(config_path, encoding="utf-8") as f:
+        config = json.load(f)
+    if config.get("schema_version") != CONFIG_SCHEMA:
+        raise PackageError(
+            f"unsupported package schema {config.get('schema_version')!r}")
+
+    verify_code(config)
+    verify_vocab(config, package_dir)
+
+    from safetensors.torch import load_file
+
+    weights = os.path.join(package_dir, config["weights"]["file"])
+    state = load_file(weights)
+    # Weights ship bfloat16 (config.weights.cast_from == float32); the forward
+    # pass runs under autocast, so they are restored to float32 storage, as
+    # the package's own loader does.
+    state = {k: (v.to(torch.float32) if v.is_floating_point() else v)
+             for k, v in state.items()}
+    model = ChartModel(**config["model"]).to(device)
+    model.load_state_dict(state, strict=True)
+    model.eval()
+
+    caps = dict(config["capabilities"])
+    missing = sorted(set(GENERATOR_CAPABILITY_KEYS) - set(caps))
+    if missing:
+        raise PackageError(f"package capabilities are incomplete: {missing}")
+    attach_generator_capabilities(model, caps)
+    model._has_axis_knobs = bool(caps.get("axis_knobs", False))
+    # config.serving_prefix_fix: train.py records sync_token=False while the
+    # training prefix carries the SYNC slot. A loader that trusts the recorded
+    # flag drops the slot and every sync bucket reaches the decoder as an
+    # identical prefix.
+    model._has_sync = True
+
+    spec = BarscriptSpec.load(os.path.join(package_dir, "spec.json"))
+    return model, config, spec
+
+
+# ---------------------------------------------------------------------------
+# the interface the app calls
+
+
+def _course_header(text, course):
+    header = TJA_COURSE_HEADER.get(str(course).lower())
+    if header is None:
+        return text
+    out, n = re.subn(r"(?m)^COURSE:.*$", f"COURSE:{header}", text, count=1)
+    if n != 1:
+        raise PackageError("exported TJA has no COURSE header to normalize")
+    return out
+
+
+class SoftChart2:
+    """A loaded BarScript release, ready to generate.
+
+    Construct via :func:`load` (which caches by directory).  ``generate`` is
+    the whole serving surface.
+    """
+
+    def __init__(self, model, config, spec, package_dir, device):
+        self.model = model
+        self.config = config
+        self.spec = spec
+        self.package_dir = package_dir
+        self.device = device
+        # Where the package came from: "env", "local", "hub" or "explicit".
+        # Set by load(); a directly constructed instance has not been resolved.
+        self.source = "explicit"
+        self._lock = threading.Lock()
+
+    # -- description ------------------------------------------------------
+
+    @property
+    def release_name(self):
+        return self.config.get("release_name", "sc2")
+
+    @property
+    def parameters(self):
+        return int(self.config["weights"]["parameters"])
+
+    @property
+    def courses(self):
+        return list(COURSES)
+
+    def serving(self, profile="default"):
+        return serving_kwargs(self.config, profile)
+
+    def limitations(self):
+        """``[(key, section, text)]`` -- MODEL_CARD §2, for the UI."""
+        return list(KNOWN_LIMITATIONS)
+
+    def describe(self):
+        return {
+            "release_name": self.release_name,
+            "parameters": self.parameters,
+            "experimental": True,
+            "package_dir": self.package_dir,
+            "package_source": self.source,
+            "hub_repo": hub_repo_id() if self.source == "hub" else None,
+            "source_seed": self.config.get("source", {}).get("seed"),
+            "source_step": self.config.get("source", {}).get("step"),
+            "weights_dtype": self.config["weights"]["dtype"],
+            "grid": {"source": "bpm", "denom": SHIP_GRID_DENOM,
+                     "bar_denoms": self.serving()["bar_denoms"]},
+            "serving": self.serving(),
+            "motif_hard": self.config["train_provenance"][
+                "motif_constraint"]["resolved_motif_hard"],
+            "limitations": self.limitations(),
+        }
+
+    # -- conditions -------------------------------------------------------
+
+    def density_bucket_from_nps(self, nps):
+        """Notes-per-second -> DENS bucket, using the package's bucket width.
+
+        MODEL_CARD §2.5: absolute bucket calibration is off on the density
+        axis, so treat this as a knob, not a promise of a note count.
+        """
+        width = float(preprocessing_contract(self.package_dir).get(
+            "density_bucket_nps", DENS_BUCKET_NPS))
+        return int(min(N_DENS - 1, max(0, int(float(nps) / width))))
+
+    # -- generation -------------------------------------------------------
+
+    def generate(self, mel, course, level=None, *, bpm,
+                 offset_sec=0.0, duration_sec=None, density_bucket=None,
+                 density_nps=None, meter=(4, 4), grid_denom=SHIP_GRID_DENOM,
+                 seed=0, big_rate=None, span_rate=None, stream=None,
+                 sync_band=None, title="SoftChart", wave="song.ogg",
+                 balloon_count=10, profile="default", out_path=None,
+                 on_progress=None, **overrides):
+        """Generate one chart and return it as TJA text plus provenance.
+
+        ``mel``      (n_mels, T) natural-log mel, the front end described by
+                     ``preprocessor_config.json`` (see ``check_preprocessing``).
+        ``course``   one of ``COURSES`` (easy/normal/hard/oni/ura), or ``None``
+                     to decode unconditioned.
+        ``level``    TJA level, clamped to 1..12 by the prefix builder.
+        ``bpm``      tempo of the synthesized grid.
+        ``offset_sec``  audio time of the FIRST DOWNBEAT, in seconds.  This is
+                     the negative of the TJA ``OFFSET`` field.
+        ``duration_sec``  defaults to ``mel.shape[1] / FPS``.
+        ``density_bucket`` DENS bucket 0..15, or ``density_nps`` in notes per
+                     second, which is bucketed with the package's width.
+        ``grid_denom``  bar-local lattice denominator teacher-forced into every
+                     [BAR] header.  16 is the shipped deployment grid.
+        ``overrides``  serving parameters (``temperature``, ``top_p``,
+                     ``min_onset_gap_sec``, ...).  The package's recorded
+                     values apply unless one is named here; anything overridden
+                     is reported back under ``serving_overrides``, because the
+                     model card's numbers no longer describe that decode.
+
+        Returns a dict:  ``tja`` (text), ``grid`` (the lattice actually used),
+        ``serving`` (the kwargs that reached the decoder), ``serving_overrides``,
+        ``motif`` (what the FSM was actually served), ``counters`` (decoder
+        clamp/floor tallies), ``gen`` (the raw generation), plus ``course``,
+        ``level``, ``density_bucket``, ``seed``, ``n_notes``, ``n_spans``,
+        ``duration_sec`` and ``release_name``.
+
+        Raises ``SlotExportError`` when a decoded position does not sit on the
+        96-slot TJA export lattice; the generation is intact but no TJA can be
+        written for it.
+        """
+        if course is not None and str(course).lower() not in COURSES:
+            raise ValueError(f"unknown course {course!r}; "
+                             f"choose one of {COURSES}")
+        course = None if course is None else str(course).lower()
+        if mel.ndim != 2:
+            raise ValueError("mel must be (n_mels, T)")
+        if duration_sec is None:
+            duration_sec = mel.shape[1] / FPS
+        if density_bucket is None and density_nps is not None:
+            density_bucket = self.density_bucket_from_nps(density_nps)
+        if density_bucket is not None:
+            density_bucket = int(density_bucket)
+            if not 0 <= density_bucket < N_DENS:
+                raise ValueError(f"density_bucket must be in [0, {N_DENS})")
+
+        kwargs = serving_kwargs(self.config, profile)
+        unknown = sorted(set(overrides) - set(kwargs))
+        if unknown:
+            raise ValueError(f"not serving parameters: {unknown}")
+        applied = {k: v for k, v in overrides.items()
+                   if k in kwargs and v is not None and v != kwargs[k]}
+        kwargs.update({k: v for k, v in overrides.items() if v is not None})
+
+        grid = barscript_grid_from_bpm(bpm, float(offset_sec),
+                                       float(duration_sec), meter=meter,
+                                       denom=int(grid_denom))
+        grid_info = {
+            "source": "bpm",
+            "denom": int(grid_denom),
+            "bpm": float(bpm),
+            "offset_sec": float(offset_sec),
+            "duration_sec": float(duration_sec),
+            "meter": [int(meter[0]), int(meter[1])],
+            "n_bars": len(grid["bar_denoms"]),
+            "bar_denoms": kwargs["bar_denoms"],
+            "bar_denom_mask": kwargs["bar_denom_mask"],
+            "triplets_representable": (int(grid_denom) % 3 == 0),
+            "profile": profile,
+        }
+
+        # config.serving_basis: span_bucket0_profile is null in this release
+        # and is a KNOWN GAP (§2.7), not a tuned choice. A package that ever
+        # ships one needs a builder here, so refuse rather than drop it.
+        if self.config["serving"].get("span_bucket0_profile") is not None:
+            raise PackageError(
+                "package ships a span_bucket0_profile; this loader cannot "
+                "build one and will not serve without it")
+
+        motif = motif_context(self.config)
+        with self._lock:
+            with motif:
+                gen = generate_song_barscript(
+                    self.model, mel, grid, course, level=level,
+                    density_bucket=density_bucket, spec=self.spec,
+                    device=self.device, seed=int(seed), sync_band=sync_band,
+                    big_rate=big_rate, span_rate=span_rate, stream=stream,
+                    span_bucket0_profile=None,
+                    on_progress=on_progress, **kwargs)
+
+        require_slot_export(gen)
+        tja = write_tja_slots(gen, grid, title=title, course=course or "oni",
+                              level=level or 0, wave=wave,
+                              balloon_count=balloon_count)
+        tja = _course_header(tja, course or "oni")
+        if out_path:
+            with open(out_path, "w", encoding="utf-8") as f:
+                f.write(tja)
+
+        return {
+            "tja": tja,
+            "grid": grid_info,
+            "serving": kwargs,
+            "serving_overrides": applied,
+            "motif": motif.observed_state(),
+            "counters": {k: int(v) for k, v in gen["counters"].items()},
+            "gen": gen,
+            "course": course,
+            "level": level,
+            "density_bucket": density_bucket,
+            "seed": int(seed),
+            "n_notes": len(gen["hits"]),
+            "n_spans": len(gen["spans"]),
+            "duration_sec": float(duration_sec),
+            "release_name": self.release_name,
+            "experimental": True,
+        }
+
+
+_LOADED = {}
+_LOAD_LOCK = threading.Lock()
+
+
+def load(package_dir=None, device="cpu", *, token=None):
+    """Load (and cache) the release package.
+
+    With no ``package_dir`` the source is chosen by :func:`resolve_package_dir`,
+    which may download from the Hub.  Either way the package that wins is put
+    through the same gate: ``load_release`` verifies the eight vendored serving
+    modules by md5 against ``config.json:code.module_md5``, cross-checks the
+    vendored token table, and rejects an unknown package schema.
+    """
+    if package_dir is None:
+        package_dir, source = resolve_package_dir(token=token)
+    else:
+        package_dir, source = os.path.abspath(package_dir), "explicit"
+    key = (package_dir, device)
+    with _LOAD_LOCK:
+        if key not in _LOADED:
+            model, config, spec = load_release(package_dir, device=device)
+            release = SoftChart2(model, config, spec, package_dir, device)
+            release.source = source
+            _LOADED[key] = release
+        return _LOADED[key]
+
+
+def is_available(package_dir=None):
+    """Whether the model can be offered, without loading any weights.
+
+    True when a package is on disk, and also when the Hub fallback has a real
+    chance of producing one -- the repo is already cached, or a token exists and
+    the process is not offline.  Offering the entry on a resolvable-but-not-yet-
+    downloaded Hub is deliberate: the failure a user then sees names the missing
+    token, whereas hiding the entry would tell them the model does not exist.
+    """
+    if package_dir is not None:
+        return os.path.isfile(os.path.join(package_dir, "config.json"))
+    if os.path.isfile(os.path.join(default_package_dir(), "config.json")):
+        return True
+    if os.environ.get("SC2_PACKAGE_DIR"):
+        # An explicit request that is not satisfiable: do not quietly answer it
+        # from the Hub. resolve_package_dir raises with the path that was named.
+        return False
+    if not hub_enabled():
+        return False
+    if hub_snapshot_cached() is not None:
+        return True
+    offline = os.environ.get("HF_HUB_OFFLINE", "").strip().lower() \
+        in ("1", "true", "yes")
+    return (not offline) and resolve_token() is not None
+
+
+__all__ = ["SoftChart2", "SlotExportError", "PackageError", "load",
+           "load_release", "is_available", "default_package_dir",
+           "local_candidates", "resolve_package_dir", "download_from_hub",
+           "hub_repo_id", "hub_enabled", "hub_snapshot_cached",
+           "resolve_token", "verify_manifest",
+           "serving_kwargs", "motif_context", "check_preprocessing",
+           "preprocessing_contract", "verify_code", "verify_vocab",
+           "vocab_tokens", "KNOWN_LIMITATIONS", "TJA_COURSE_HEADER",
+           "SHIP_GRID_DENOM", "DEFAULT_HUB_REPO", "PACKAGE_FILES",
+           "REQUIRED_FILES", "sc2_generate"]

softchart/tja.py

@@ -1,143 +1,114 @@
-"""Meter-aware TJA emission for slot-mode charts.
+"""TJA emission for slot-mode charts: slot indices ARE the chart.
 
 No quantization happens here — generate_song_slot() already produced exact
-lattice positions (96 slots per 4/4 measure), so this writer just prints them.
-Preview rendering and the .tja are identical by construction. Authored measure
-edges determine exact per-measure tempo; meter changes emit #MEASURE directives.
+lattice positions (measure, slot in 0..95), so this writer just prints them.
+Preview rendering and the .tja are identical by construction. Piecewise
+grids additionally emit #BPMCHANGE lines between measures.
 """
 
 import numpy as np
 
-from .meter import resolve_meter_layout
 from .vocab import SLOTS
 
 CHAR = {"don": "1", "ka": "2", "don_big": "3", "ka_big": "4",
         "roll": "5", "roll_big": "6", "balloon": "7"}
 
 
-def _format_number(value):
-    """Emit enough digits for a float to round-trip through the TJA text."""
-    value = float(value)
-    if not np.isfinite(value):
-        raise ValueError("TJA numeric fields must be finite")
-    if value == 0.0:
-        value = 0.0  # avoid a cosmetic negative zero in OFFSET
-    return format(value, ".17g")
+def grid_measure_starts(grid, n_measures):
+    """Return measure boundary times for a slot grid, extending if needed."""
+    db = np.asarray(grid.get("downbeats", []), dtype=float)
+    if len(db) >= n_measures + 1:
+        return db[: n_measures + 1]
+    if len(db) >= 2:
+        step = float(np.median(np.diff(db)))
+        start = float(db[0])
+    else:
+        step = 240.0 / float(grid["bpm"])
+        start = float(db[0]) if len(db) else 0.0
+    return start + np.arange(n_measures + 1, dtype=float) * step
 
 
-def write_tja_slots(gen, grid, title, course, level, wave, out_path=None,
-                    balloon_count=10):
-    """Serialize a slot generation on an explicit trusted meter timeline.
+def gogo_measure_mask(plan, measure_starts, n_measures):
+    """Map plan climax blocks (flag == 2) to TJA measures."""
+    mask = [False] * n_measures
+    if not plan:
+        return mask
+    starts = np.asarray(measure_starts, dtype=float)
+    if len(starts) < n_measures + 1:
+        return mask
+    for block in plan:
+        if len(block) < 4 or int(block[3]) != 2:
+            continue
+        a, b = float(block[0]), float(block[1])
+        if b <= a:
+            continue
+        for m in range(n_measures):
+            if starts[m] < b and starts[m + 1] > a:
+                mask[m] = True
+    return mask
+
 
-    ``grid`` must contain exact rational meter metadata and ``measure_edges``
-    with one trusted terminal edge. Returns the TJA text and optionally writes
-    it to ``out_path``.
-    """
-    if int(gen.get("slots_per_4_4", SLOTS)) != SLOTS:
-        raise ValueError(f"expected {SLOTS} slots per 4/4 measure")
-    n_meas = max(int(gen.get("n_measures", 0)),
-                 (max((m for m, _, _ in gen.get("hits_slots", [])), default=-1) + 1),
-                 (max((m1 for _, _, m1, _, _ in gen.get("spans_slots", [])),
-                      default=-1) + 1), 1)
-    fields = ("measure_slots", "measure_num", "measure_den")
-    gen_has_layout = any(name in gen for name in fields)
-    grid_has_layout = any(name in grid for name in fields) \
-        or "meter" in grid or "beats_per_measure" in grid
-    gen_layout = resolve_meter_layout(gen, n_meas) if gen_has_layout else None
-    grid_layout = resolve_meter_layout(grid, n_meas) if grid_has_layout else None
-    if gen_layout is not None and grid_layout is not None:
-        gen_slots, gen_num, gen_den, _ = gen_layout
-        grid_slots, grid_num, grid_den, _ = grid_layout
-        same_fraction = all(
-            int(gn) * int(gd) == int(gdn) * int(gnd)
-            for gn, gnd, gdn, gd in zip(
-                gen_num, gen_den, grid_num, grid_den)
-        )
-        if not np.array_equal(gen_slots, grid_slots) or not same_fraction:
-            raise ValueError("generated meter layout disagrees with the timing grid")
-    layout = grid_layout if grid_layout is not None else gen_layout
-    if layout is None:
-        raise ValueError(
-            "TJA slot export requires exact meter metadata; refusing to assume 4/4")
-    slots_per_measure, measure_num, measure_den, quarter_beats = layout
-    offsets = np.concatenate([[0], np.cumsum(slots_per_measure)])
+def append_measure_with_gogo(lines, measure_line, measure_idx, gogo_mask, in_gogo):
+    """Append a measure line, opening/closing #GOGO commands at boundaries."""
+    want_gogo = bool(gogo_mask[measure_idx]) if measure_idx < len(gogo_mask) else False
+    if in_gogo and not want_gogo:
+        lines.append("#GOGOEND")
+        in_gogo = False
+    if want_gogo and not in_gogo:
+        lines.append("#GOGOSTART")
+        in_gogo = True
+    lines.append(measure_line)
+    return in_gogo
 
-    def flat(measure, slot):
-        measure, slot = int(measure), int(slot)
-        if not 0 <= measure < n_meas:
-            raise ValueError(f"measure index {measure} is outside 0..{n_meas - 1}")
-        if not 0 <= slot < slots_per_measure[measure]:
-            raise ValueError(
-                f"slot {slot} is outside measure {measure}'s "
-                f"0..{slots_per_measure[measure] - 1} range"
-            )
-        return int(offsets[measure] + slot)
 
+def write_tja_slots(gen, grid, title, course, level, wave, out_path=None,
+                    balloon_count=10, plan=None):
+    """gen: generate_song_slot() result; grid: fit_grid() result (or a dict
+    with downbeats+bpm). Returns the TJA text (and writes it if out_path)."""
     slots = {}
     for me, sl, cls in gen["hits_slots"]:
-        position = flat(me, sl)
-        if position in slots:
-            raise ValueError(
-                f"slot collision at measure {int(me)}, slot {int(sl)}")
-        slots[position] = CHAR[cls]
+        slots.setdefault((me, sl), CHAR[cls])
     for m0, s0, m1, s1, typ in gen.get("spans_slots", []):
-        a = flat(m0, s0)
-        b = flat(m1, s1)
-        if b <= a:
-            raise ValueError("span end must be strictly after its start")
-        if a in slots:
-            raise ValueError(
-                f"slot collision at span start measure {int(m0)}, slot {int(s0)}")
-        if b in slots:
-            raise ValueError(
-                f"slot collision at span end measure {int(m1)}, slot {int(s1)}")
+        a = (m0, s0)
+        while a in slots:  # span start yields to hits: next free slot
+            a = (a[0] + (a[1] + 1) // SLOTS, (a[1] + 1) % SLOTS)
+        b = (m1, s1)
+        while b in slots or b <= a:
+            b = (b[0] + (b[1] + 1) // SLOTS, (b[1] + 1) % SLOTS)
         slots[a] = CHAR[typ]
         slots[b] = "8"
-
-    # Slot output is timing-exact only when every start and the terminal edge
-    # are explicit.  Never fall back to BPM/downbeat extrapolation here: that
-    # would make the serialized TJA disagree with the lattice used to decode.
-    if "measure_edges" not in grid:
-        raise ValueError(
-            "slot TJA export requires explicit measure_edges including the "
-            "trusted terminal edge"
-        )
-    tempo_edges = np.asarray(grid["measure_edges"], dtype=float)
-    if (tempo_edges.ndim != 1 or len(tempo_edges) != n_meas + 1
-            or not np.all(np.isfinite(tempo_edges))
-            or np.any(np.diff(tempo_edges) <= 0)):
-        raise ValueError(
-            "measure_edges must contain one finite, increasing edge per "
-            "measure plus the final edge")
+    n_meas = max(gen.get("n_measures", 0),
+                 (max(m for m, _ in slots) + 1) if slots else 1)
+    # piecewise-tempo grids: emit #BPMCHANGE whenever the per-measure BPM
+    # (from consecutive fitted barlines) moves; TJA measure lines are unchanged
+    db = np.asarray(grid.get("downbeats", []), float) if grid.get("piecewise") else None
+    measure_starts = grid_measure_starts(grid, n_meas)
+    gogo_mask = gogo_measure_mask(plan, measure_starts, n_meas)
     lines = []
-    base_bpm = float(grid["bpm"])
-    if not np.isfinite(base_bpm) or base_bpm <= 0:
-        raise ValueError("grid bpm must be positive and finite")
-    cur_bpm = base_bpm
-    cur_meter = (4, 4)
+    cur_bpm = float(grid["bpm"])
+    in_gogo = False
     for m in range(n_meas):
-        duration = float(tempo_edges[m + 1] - tempo_edges[m])
-        if not np.isfinite(duration) or duration <= 0:
-            raise ValueError(f"measure {m} duration must be positive and finite")
-        bpm_m = 60.0 * float(quarter_beats[m]) / duration
-        # No integer snap and no perceptual threshold: even a small tempo
-        # delta accumulates into a real chart/audio timing error.
-        if bpm_m != cur_bpm:
-            lines.append(f"#BPMCHANGE {_format_number(bpm_m)}")
-            cur_bpm = bpm_m
-        meter = (int(measure_num[m]), int(measure_den[m]))
-        if meter != cur_meter:
-            lines.append(f"#MEASURE {meter[0]}/{meter[1]}")
-            cur_meter = meter
-        start = int(offsets[m])
-        lines.append("".join(slots.get(start + k, "0")
-                             for k in range(int(slots_per_measure[m]))) + ",")
+        if in_gogo and not gogo_mask[m]:
+            lines.append("#GOGOEND")
+            in_gogo = False
+        if db is not None and m + 1 < len(db):
+            bpm_m = 240.0 / (db[m + 1] - db[m])
+            if abs(bpm_m - round(bpm_m)) < 0.05:
+                bpm_m = float(round(bpm_m))
+            if abs(bpm_m - cur_bpm) > 0.05:
+                lines.append(f"#BPMCHANGE {bpm_m:g}")
+                cur_bpm = bpm_m
+        in_gogo = append_measure_with_gogo(
+            lines, "".join(slots.get((m, k), "0") for k in range(SLOTS)) + ",",
+            m, gogo_mask, in_gogo)
+    if in_gogo:
+        lines.append("#GOGOEND")
     balloons = [balloon_count] * sum(1 for s in gen.get("spans_slots", [])
                                      if s[4] == "balloon")
-    offset = float(tempo_edges[0])
+    offset = float(grid["downbeats"][0]) if len(grid.get("downbeats", [])) else 0.0
     tja = "\n".join([
-        f"TITLE:{title} (SoftChart)", f"BPM:{_format_number(base_bpm)}",
-        f"WAVE:{wave}", f"OFFSET:{_format_number(-offset)}",
+        f"TITLE:{title} (SoftChart)", f"BPM:{grid['bpm']:g}", f"WAVE:{wave}",
+        f"OFFSET:{-offset:.3f}",
         f"COURSE:{'Oni' if course == 'oni' else str(course).capitalize()}",
         f"LEVEL:{level}",
         f"BALLOON:{','.join(map(str, balloons))}" if balloons else "BALLOON:",

softchart/train.py

@@ -0,0 +1,433 @@
+"""Training loop.
+
+    python -m softchart.train --cache /workspace/softchart/cache --out runs/base
+"""
+
+import argparse
+import json
+import math
+import os
+import time
+
+import torch
+import torch.nn.functional as F
+from torch.utils.data import DataLoader
+
+from .data import ChartWindowDataset, collate, load_split_ids
+from .model import ChartModel, count_params
+from .vocab import VOCAB
+
+
+# class-balanced weights for rare note types (~1/sqrt(train frequency), capped)
+TYPE_WEIGHTS = {"don": 1.0, "ka": 1.23, "don_big": 3.19, "ka_big": 5.29,
+                "roll": 7.18, "roll_big": 8.0, "balloon": 8.0, "end": 5.0}
+
+
+def build_class_weights(device, cap=8.0):
+    w = torch.ones(VOCAB.size, device=device)
+    for name, weight in TYPE_WEIGHTS.items():
+        w[VOCAB.note[name]] = min(weight, cap)
+    return w
+
+
+def loss_fn(logits, tgt, loss_mask, label_smoothing=0.1, class_weight=None):
+    # logits: (B, L-1, V) predicting tgt[:, 1:]
+    gold = tgt[:, 1:]
+    mask = loss_mask[:, 1:]
+    ls = F.cross_entropy(
+        logits.reshape(-1, logits.shape[-1]),
+        gold.reshape(-1),
+        reduction="none",
+        label_smoothing=label_smoothing,
+        weight=class_weight,
+    ).reshape(gold.shape)
+    return (ls * mask).sum() / mask.sum().clamp(min=1)
+
+
+def corrupt_events(tgt, loss_mask, rng_gen, hard=False):
+    """Synthetic negative for DPO. hard=True keeps corruptions NEAR the snap
+    tolerance (small jitter, few swaps) so negatives resemble the policy's own
+    mistakes rather than being trivially separable (fixes dpo_loss->0 collapse)."""
+    from .vocab import WINDOW
+
+    rej = tgt.clone()
+    ev = loss_mask.clone()
+    t0 = VOCAB.time0
+    is_time = (rej >= t0) & (rej < t0 + WINDOW) & ev
+    if hard:  # +-1..3 frames (~12-35ms, straddles the snap grid) on 60% of notes
+        jit = torch.randint(-3, 4, rej.shape, device=rej.device, generator=rng_gen)
+        jit = torch.where(jit == 0, torch.ones_like(jit), jit)
+        do_t = torch.rand(rej.shape, device=rej.device, generator=rng_gen) < 0.6
+        jit = torch.where(do_t, jit, torch.zeros_like(jit))
+        swap_p = 0.12
+    else:
+        jit = torch.randint(-5, 6, rej.shape, device=rej.device, generator=rng_gen)
+        jit = torch.where(jit.abs() < 2, jit.sign() * 2, jit)
+        swap_p = 0.25
+    rej = torch.where(is_time, (rej + jit).clamp(t0, t0 + WINDOW - 1), rej)
+    hit0 = VOCAB.note["don"]
+    is_hit = (rej >= hit0) & (rej < hit0 + 4) & ev
+    swap = torch.randint(0, 4, rej.shape, device=rej.device, generator=rng_gen) + hit0
+    do = torch.rand(rej.shape, device=rej.device, generator=rng_gen) < swap_p
+    rej = torch.where(is_hit & do, swap, rej)
+    return rej
+
+
+def seq_logprob(model, mel, tgt, mask):
+    import torch.nn.functional as F
+
+    logits = model(mel, tgt)
+    lp = F.log_softmax(logits.float(), dim=-1)
+    gold = tgt[:, 1:]
+    g = lp.gather(-1, gold.unsqueeze(-1)).squeeze(-1)
+    return (g * mask[:, 1:]).sum(1)
+
+
+@torch.no_grad()
+def validate(model, loader, device):
+    model.eval()
+    tot, n = 0.0, 0
+    for mel, tgt, mask, _aux, gsum, posb, _beat, _in in loader:
+        mel, tgt, mask = mel.to(device), tgt.to(device), mask.to(device)
+        gsum = gsum.to(device) if gsum is not None else None
+        with torch.autocast("cuda", dtype=torch.bfloat16):
+            logits = model(mel, tgt, gsum=gsum, pos_bucket=posb.to(device))
+            l = loss_fn(logits, tgt, mask, label_smoothing=0.0)
+        tot += l.item() * mel.shape[0]
+        n += mel.shape[0]
+    model.train()
+    return tot / max(n, 1)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--cache", required=True)
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--steps", type=int, default=60000)
+    ap.add_argument("--batch", type=int, default=32)
+    ap.add_argument("--lr", type=float, default=3e-4)
+    ap.add_argument("--warmup", type=int, default=1000)
+    ap.add_argument("--cond-drop", type=float, default=0.15)
+    ap.add_argument("--no-augment", action="store_true")
+    ap.add_argument("--balanced", action="store_true", help="class-balanced type loss")
+    ap.add_argument("--balance-cap", type=float, default=8.0,
+                    help="cap for class weights (v3: 2.5 after v2 over-corrected)")
+    ap.add_argument("--ctx", action="store_true", help="prev-window context tokens")
+    ap.add_argument("--aux", action="store_true", help="onset-heatmap auxiliary head")
+    ap.add_argument("--global-ctx", action="store_true",
+                    help="whole-song summary + window-position context (v4)")
+    ap.add_argument("--importance-sampling", action="store_true",
+                    help="bias window sampling toward informative regions (v4)")
+    ap.add_argument("--d-model", type=int, default=512)
+    ap.add_argument("--enc-layers", type=int, default=6)
+    ap.add_argument("--dec-layers", type=int, default=6)
+    ap.add_argument("--ffn", type=int, default=2048)
+    ap.add_argument("--func-time", action="store_true",
+                    help="functional (sinusoidal-basis) TIME token embeddings")
+    ap.add_argument("--emb-factor", type=int, default=0,
+                    help="ALBERT-style factorized embeddings (e.g. 64); 0 = off")
+    ap.add_argument("--distill", default=None,
+                    help="teacher checkpoint for knowledge distillation")
+    ap.add_argument("--use-aug", action="store_true",
+                    help="include waveform speed-augmented variants in training")
+    ap.add_argument("--init-from", default=None,
+                    help="initialize matching weights (decoder/tok_emb) from a checkpoint")
+    ap.add_argument("--kd-alpha", type=float, default=0.5, help="CE weight (rest = KL)")
+    ap.add_argument("--kd-tau", type=float, default=2.0, help="distillation temperature")
+    ap.add_argument("--tempo-aug", action="store_true",
+                    help="rhythm-preserving tempo/pitch augmentation in mel domain")
+    ap.add_argument("--sibling", action="store_true",
+                    help="easier-course skeleton hint in the prefix (easy subset-of hard)")
+    ap.add_argument("--style", action="store_true",
+                    help="charting-intent style token (cache/styles.json)")
+    ap.add_argument("--align", action="store_true",
+                    help="pointer alignment loss: notes must attend to their onset frame")
+    ap.add_argument("--beat-head", action="store_true",
+                    help="beat/downbeat auxiliary head (labels from TJA measures)")
+    ap.add_argument("--beat-weight", type=float, default=0.1,
+                    help="beat-head loss weight (raise for a dedicated beat model)")
+    ap.add_argument("--beat-hires", action="store_true",
+                    help="frame-resolution beat head (11.6 ms bins instead of 46 ms)")
+    ap.add_argument("--sync-token", action="store_true",
+                    help="LHL syncopation-band condition token")
+    ap.add_argument("--plan", action="store_true",
+                    help="song-level plan-block conditioning (cache/plans.json)")
+    ap.add_argument("--mask-infill", type=float, default=0.0,
+                    help="probability of type-mask (skeleton->color) curriculum per window")
+    ap.add_argument("--complexity", action="store_true",
+                    help="rhythmic-complexity band token (density-independent difficulty)")
+    ap.add_argument("--slot", action="store_true",
+                    help="slot-token mode: notes as exact TJA lattice indices "
+                         "(measure*96+slot), measure-aligned windows, grid-phase "
+                         "input channels — zero quantization error by construction")
+    ap.add_argument("--dual-slot-p", type=float, default=0.65,
+                    help="dual mode: probability of a slot window (rest = time)")
+    ap.add_argument("--dual", action="store_true",
+                    help="dual-mode: mixed slot/time windows with an explicit "
+                         "MODE token; one model serves both the grid-exact path "
+                         "and the gridless time path")
+    ap.add_argument("--dpo", default=None,
+                    help="reference checkpoint: switch to DPO finetuning with synthetic negatives")
+    ap.add_argument("--dpo-beta", type=float, default=0.1)
+    ap.add_argument("--dpo-hard", action="store_true",
+                    help="near-tolerance negatives (resemble policy errors)")
+    ap.add_argument("--val-every", type=int, default=2000)
+    ap.add_argument("--workers", type=int, default=12)
+    ap.add_argument("--resume", default=None)
+    ap.add_argument("--seed", type=int, default=1234)
+    ap.add_argument("--data-frac", type=float, default=1.0,
+                    help="fraction of train songs to use (data-scaling ablation); "
+                         "val/test splits are unaffected")
+    args = ap.parse_args()
+    torch.manual_seed(args.seed)
+
+    os.makedirs(args.out, exist_ok=True)
+    device = "cuda"
+    torch.backends.cuda.matmul.allow_tf32 = True
+    torch.backends.cudnn.allow_tf32 = True
+
+    train_ids, val_ids, _ = load_split_ids(args.cache, use_aug=args.use_aug,
+                                           frac=args.data_frac)
+    train_ds = ChartWindowDataset(
+        args.cache, train_ids, train=True, cond_drop=args.cond_drop,
+        spec_augment=not args.no_augment, use_ctx=args.ctx, aux=args.aux,
+        global_ctx=args.global_ctx, importance_sampling=args.importance_sampling,
+        tempo_aug=args.tempo_aug, sibling=args.sibling, style=args.style,
+        beat_head=args.beat_head, sync_token=args.sync_token,
+        plan=args.plan, mask_infill=args.mask_infill, complexity=args.complexity,
+        slot=args.slot, dual=args.dual, dual_slot_p=args.dual_slot_p,
+        beat_hires=args.beat_hires,
+    )
+    val_ds = ChartWindowDataset(args.cache, val_ids, train=False, windows_per_chart=1,
+                                use_ctx=args.ctx, aux=args.aux, global_ctx=args.global_ctx,
+                                sibling=args.sibling, style=args.style,
+                                beat_head=args.beat_head, sync_token=args.sync_token,
+                                plan=args.plan, complexity=args.complexity,
+                                slot=args.slot, dual=args.dual,
+                                beat_hires=args.beat_hires)
+    print(f"train charts {len(train_ds.items)}, val charts {len(val_ds.items)}")
+
+    train_loader = DataLoader(
+        train_ds, batch_size=args.batch, shuffle=True, collate_fn=collate,
+        num_workers=args.workers, pin_memory=True, drop_last=True, persistent_workers=True,
+    )
+    val_loader = DataLoader(
+        val_ds, batch_size=args.batch, shuffle=False, collate_fn=collate, num_workers=4,
+    )
+
+    from .vocab import N_MELS as _NM
+    model = ChartModel(
+        d_model=args.d_model, enc_layers=args.enc_layers, dec_layers=args.dec_layers,
+        ffn=args.ffn, aux=args.aux, global_ctx=args.global_ctx,
+        func_time=args.func_time, in_ch=(_NM + 2) if (args.slot or args.dual) else None,
+        emb_factor=args.emb_factor or None,
+    ).to(device)
+    if args.align:
+        model.enable_ptr()
+    if args.beat_head:
+        model.enable_beat(hires=args.beat_hires)
+    model.to(device)
+    print(f"params: {count_params(model)/1e6:.1f}M, vocab {VOCAB.size}")
+    if args.init_from and os.path.exists(args.init_from):
+        src = torch.load(args.init_from, map_location=device)["model"]
+        own = model.state_dict()
+        hit = {k: v for k, v in src.items()
+               if k in own and own[k].shape == v.shape
+               and k.startswith(("decoder", "tok_emb", "out", "dec_pos"))}
+        own.update(hit)
+        model.load_state_dict(own)
+        print(f"initialized {len(hit)} tensors from {args.init_from}")
+    class_weight = build_class_weights(device, args.balance_cap) if args.balanced else None
+
+    teacher = None
+    if args.distill:
+        from .generate import load_model
+
+        teacher = load_model(args.distill, device=device)
+        for p in teacher.parameters():
+            p.requires_grad_(False)
+        t_vocab = teacher.tok_emb.weight.shape[0]
+        print(f"distilling from {args.distill} (teacher vocab {t_vocab})")
+    dpo_ref = None
+    if args.dpo:
+        from .generate import load_model as _lm
+
+        dpo_ref = _lm(args.dpo, device=device)
+        for p_ in dpo_ref.parameters():
+            p_.requires_grad_(False)
+        # initialize the policy from the reference (vocab may have grown since:
+        # copy overlapping embedding rows, everything else matches the 12M arch)
+        ref_sd = dpo_ref.state_dict()
+        own = model.state_dict()
+        for k, v in ref_sd.items():
+            if k in own:
+                if own[k].shape == v.shape:
+                    own[k] = v
+                elif k == "tok_emb.weight":
+                    own[k][: v.shape[0]] = v
+        model.load_state_dict(own)
+        # disable dropout for DPO: logprob-margin noise from dropout on long
+        # sequences swamps the beta-scaled objective (verified: init loss 3.3
+        # instead of log 2)
+        import torch.nn as _nn
+
+        for m_ in model.modules():
+            if isinstance(m_, _nn.Dropout):
+                m_.p = 0.0
+        print(f"DPO: policy initialized from {args.dpo}, beta={args.dpo_beta}, dropout off")
+
+    opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=0.01, betas=(0.9, 0.98))
+
+    def lr_at(step):
+        if step < args.warmup:
+            return args.lr * step / args.warmup
+        p = (step - args.warmup) / max(1, args.steps - args.warmup)
+        return args.lr * (0.1 + 0.9 * 0.5 * (1 + math.cos(math.pi * p)))
+
+    step = 0
+    best_val = float("inf")
+    if args.resume and os.path.exists(args.resume):
+        ck = torch.load(args.resume, map_location=device)
+        model.load_state_dict(ck["model"])
+        opt.load_state_dict(ck["opt"])
+        step = ck["step"]
+        best_val = ck.get("best_val", best_val)
+        print(f"resumed from {args.resume} at step {step}")
+
+    log_path = os.path.join(args.out, "train_log.jsonl")
+    model.train()
+    t0 = time.time()
+    run_loss, run_n = 0.0, 0
+    done = False
+    while not done:
+        for mel, tgt, mask, aux_t, gsum, posb, beat_t, in_tgt in train_loader:
+            step += 1
+            if step > args.steps:
+                done = True
+                break
+            for g in opt.param_groups:
+                g["lr"] = lr_at(step)
+            mel, tgt, mask = mel.to(device, non_blocking=True), tgt.to(device), mask.to(device)
+            gsum = gsum.to(device) if gsum is not None else None
+            posb = posb.to(device)
+            in_tgt = in_tgt.to(device)
+            if dpo_ref is not None:
+                gen_rng = torch.Generator(device=device)
+                gen_rng.manual_seed(args.seed * 100003 + step)
+                rej = corrupt_events(tgt, mask, gen_rng, hard=args.dpo_hard)
+                with torch.autocast("cuda", dtype=torch.bfloat16):
+                    pc = seq_logprob(model, mel, tgt, mask)
+                    pr = seq_logprob(model, mel, rej, mask)
+                    with torch.no_grad():
+                        rc = seq_logprob(dpo_ref, mel, tgt, mask)
+                        rr = seq_logprob(dpo_ref, mel, rej, mask)
+                    loss = -F.logsigmoid(
+                        args.dpo_beta * ((pc - pr) - (rc - rr))).mean()
+                opt.zero_grad(set_to_none=True)
+                loss.backward()
+                torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
+                opt.step()
+                run_loss += loss.item()
+                run_n += 1
+                if step % 100 == 0:
+                    msg = {"step": step, "dpo_loss": round(run_loss / run_n, 4)}
+                    print(json.dumps(msg), flush=True)
+                    with open(log_path, "a") as f:
+                        f.write(json.dumps(msg) + "\n")
+                    run_loss, run_n = 0.0, 0
+                if step % args.val_every == 0 or step == args.steps:
+                    ck = {"model": model.state_dict(), "opt": opt.state_dict(),
+                          "step": step, "best_val": 0.0, "args": vars(args)}
+                    torch.save(ck, os.path.join(args.out, "last.pt"))
+                    torch.save(ck, os.path.join(args.out, "best.pt"))
+                continue
+            with torch.autocast("cuda", dtype=torch.bfloat16):
+                if args.align or args.beat_head:
+                    logits, aux_logits, ptr_logits, beat_logits = model(
+                        mel, tgt, return_aux=True, return_extras=True,
+                        gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
+                elif args.aux and aux_t is not None:
+                    logits, aux_logits = model(mel, tgt, return_aux=True,
+                                               gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
+                    ptr_logits = beat_logits = None
+                else:
+                    logits = model(mel, tgt, gsum=gsum, pos_bucket=posb, in_tgt=in_tgt)
+                    aux_logits = ptr_logits = beat_logits = None
+                aux_loss = 0.0
+                if args.aux and aux_t is not None and aux_logits is not None:
+                    aux_loss = F.binary_cross_entropy_with_logits(
+                        aux_logits.float(), aux_t.to(device))
+                extra = 0.0
+                if ptr_logits is not None:
+                    # pointer alignment: positions whose GOLD token is TIME must
+                    # point at that frame's encoder position
+                    gold = tgt[:, 1:]
+                    is_t = (gold >= VOCAB.time0) & (gold < VOCAB.time0 + 1728) & mask[:, 1:]
+                    if is_t.any():
+                        target_pos = ((gold - VOCAB.time0) // 4).clamp(0, ptr_logits.shape[-1] - 1)
+                        pl = F.cross_entropy(
+                            ptr_logits.reshape(-1, ptr_logits.shape[-1]).float(),
+                            target_pos.reshape(-1), reduction="none",
+                        ).reshape(gold.shape)
+                        extra = extra + 0.2 * (pl * is_t).sum() / is_t.sum()
+                if beat_logits is not None and beat_t is not None:
+                    bt = beat_t.to(device)
+                    bm = bt >= 0  # -1 marks masked (grid-visible) windows
+                    if bm.any():
+                        extra = extra + args.beat_weight * F.binary_cross_entropy_with_logits(
+                            beat_logits.float()[bm], bt[bm])
+                loss = loss_fn(logits, tgt, mask, class_weight=class_weight) + 0.1 * aux_loss + extra
+                if teacher is not None:
+                    with torch.no_grad():
+                        # teacher may have a smaller vocab (no SIB/STYLE tokens):
+                        # clamp unseen ids to UNK and distill over the shared slice
+                        t_tgt = torch.where(tgt >= t_vocab,
+                                            torch.full_like(tgt, VOCAB.unk_cond), tgt)
+                        t_logits = teacher(mel, t_tgt)
+                    V = min(t_logits.shape[-1], logits.shape[-1])
+                    tau = args.kd_tau
+                    kl = F.kl_div(
+                        F.log_softmax(logits[..., :V].float() / tau, dim=-1),
+                        F.softmax(t_logits[..., :V].float() / tau, dim=-1),
+                        reduction="none",
+                    ).sum(-1)
+                    kl = (kl * mask[:, 1:]).sum() / mask[:, 1:].sum().clamp(min=1)
+                    loss = args.kd_alpha * loss + (1 - args.kd_alpha) * (tau * tau) * kl
+            opt.zero_grad(set_to_none=True)
+            loss.backward()
+            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
+            opt.step()
+            run_loss += loss.item()
+            run_n += 1
+
+            if step % 100 == 0:
+                msg = {
+                    "step": step,
+                    "loss": round(run_loss / run_n, 4),
+                    "lr": round(lr_at(step), 6),
+                    "sec": round(time.time() - t0, 1),
+                }
+                print(json.dumps(msg), flush=True)
+                with open(log_path, "a") as f:
+                    f.write(json.dumps(msg) + "\n")
+                run_loss, run_n = 0.0, 0
+
+            if step % args.val_every == 0 or step == args.steps:
+                vl = validate(model, val_loader, device)
+                msg = {"step": step, "val_loss": round(vl, 4)}
+                print(json.dumps(msg), flush=True)
+                with open(log_path, "a") as f:
+                    f.write(json.dumps(msg) + "\n")
+                ck = {"model": model.state_dict(), "opt": opt.state_dict(),
+                      "step": step, "best_val": best_val, "args": vars(args)}
+                torch.save(ck, os.path.join(args.out, "last.pt"))
+                if vl < best_val:
+                    best_val = vl
+                    ck["best_val"] = best_val
+                    torch.save(ck, os.path.join(args.out, "best.pt"))
+
+    print(f"done. best_val={best_val:.4f}")
+
+
+if __name__ == "__main__":
+    main()

softchart/vocab.py

@@ -5,9 +5,7 @@ Sequence layout (decoder):
 Conditions may be UNK (condition dropout / unspecified at inference).
 """
 
-import math
 from dataclasses import dataclass
-from numbers import Real
 
 # --- audio / windowing constants (single source of truth) ---
 SR = 22050
@@ -16,7 +14,7 @@ HOP = 256
 N_MELS = 128
 FPS = SR / HOP  # 86.1328125 frames/sec
 WINDOW = 1728  # frames per training window (~20.06 s)
-MAX_TGT = 768  # max decoder length (prefix + events + eos)
+MAX_TGT = 1536  # max decoder length (prefix + events + eos)
 
 SLOTS = 96  # TJA lattice: slots per 4/4 measure (LCM of 16ths/triplets/32nds)
 MEAS_MAX = WINDOW // SLOTS  # max whole measures per window in slot mode (18)
@@ -66,7 +64,7 @@ class Vocab:
         base += len(NOTE_CLASSES)
         self.time0 = base
         base += WINDOW
-        # v2+ tokens appended at the end so older checkpoints stay loadable
+        # Extended condition tokens append after the original token ids.
         self.sep = base  # separates prev-window context from conditions
         self.sib = base + 1  # marks the sibling-chart (easier course) segment
         self.style = {s: base + 2 + s for s in range(8)}  # charting-intent codes
@@ -127,7 +125,7 @@ def complexity_band(frames):
 
 def encode_window(vocab, course, level, notes, cond_drop=0.0, rng=None, ctx_types=None,
                   sib_pairs=None, style=None, sync_band=None, plan_slice=None,
-                  complexity=None, mode=None, duration_seconds=None):
+                  complexity=None, mode=None):
     """Build a token sequence for one window.
 
     notes: list of (frame_idx, note_class_id) sorted by frame, frame in [0, WINDOW).
@@ -138,15 +136,8 @@ def encode_window(vocab, course, level, notes, cond_drop=0.0, rng=None, ctx_type
         missing slots filled with UNK. Encoded after [SIB].
     Returns (tokens, prefix_len) where loss should be applied after the prefix.
     """
-    if duration_seconds is None:
-        duration_seconds = WINDOW / FPS
-    if (not isinstance(duration_seconds, Real)
-            or isinstance(duration_seconds, bool)
-            or not math.isfinite(duration_seconds)
-            or duration_seconds <= 0):
-        raise ValueError("duration_seconds must be positive and finite")
     n_hits = sum(1 for _, c in notes if NOTE_CLASSES[c] not in ("end",))
-    nps = n_hits / float(duration_seconds)
+    nps = n_hits / (WINDOW / FPS)
     d = vocab.dens_bucket(nps)
 
     def maybe(tok):

static/app.css

@@ -531,7 +531,7 @@ audio {
 
 .course-selector {
   display: grid;
-  grid-template-columns: repeat(4, 1fr);
+  grid-template-columns: repeat(5, minmax(0, 1fr));
   padding: 4px;
   background: var(--paper-deep);
   border-radius: 13px;
@@ -617,6 +617,42 @@ audio {
   outline-offset: 1px;
 }
 
+/* Known limitations of the selected model / difficulty. Rendered verbatim from
+   the server's /api/capabilities, which reads the model card. */
+.model-notices {
+  padding: 12px 14px;
+  margin-top: 10px;
+  color: var(--red-deep);
+  background: var(--red-soft);
+  border-radius: 11px;
+  font-size: 11px;
+  line-height: 1.5;
+}
+
+.model-notices strong {
+  display: block;
+  margin-bottom: 6px;
+  font-size: 11px;
+  letter-spacing: .04em;
+  text-transform: uppercase;
+}
+
+.model-notices ul {
+  padding-left: 16px;
+  margin: 0;
+}
+
+.model-notices li + li {
+  margin-top: 5px;
+}
+
+.model-notices .notice-section {
+  margin-right: 4px;
+  font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
+  font-weight: 750;
+  opacity: .75;
+}
+
 .range {
   --range-progress: 88.89%;
   width: 100%;
@@ -1081,7 +1117,7 @@ audio {
 .loading-core,
 .metronome,
 .beat-pulses circle,
-.structure-cursor,
+.plan-cursor,
 .note,
 .export-sheet,
 .mix-wave,
@@ -1119,19 +1155,19 @@ audio {
 .is-active .beat-pulses circle:nth-child(5) { animation-delay: -1.6s; }
 
 /* Plan */
-.structure-bars rect { fill: currentColor; opacity: .13; }
-.structure-bars .structure-peak { fill: var(--red); opacity: .42; }
-.is-active .structure-bars rect { animation: structure-rise 1.9s var(--ease) infinite alternate both; transform-box: fill-box; transform-origin: center bottom; }
-.is-active .structure-bars rect:nth-child(2) { animation-delay: .12s; }
-.is-active .structure-bars rect:nth-child(3) { animation-delay: .24s; }
-.is-active .structure-bars rect:nth-child(4) { animation-delay: .36s; }
-.is-active .structure-bars rect:nth-child(5) { animation-delay: .48s; }
-.is-active .structure-bars rect:nth-child(6) { animation-delay: .6s; }
-.is-active .structure-bars rect:nth-child(7) { animation-delay: .72s; }
-.is-active .structure-bars rect:nth-child(8) { animation-delay: .84s; }
-.structure-path { stroke-dasharray: 520; stroke-dashoffset: 520; }
-.is-active .structure-path { animation: structure-draw 2.2s var(--ease) infinite; }
-.is-active .structure-cursor { animation: structure-cursor 2.2s var(--ease) infinite; }
+.plan-bars rect { fill: currentColor; opacity: .13; }
+.plan-bars .plan-climax { fill: var(--red); opacity: .42; }
+.is-active .plan-bars rect { animation: plan-rise 1.9s var(--ease) infinite alternate both; transform-box: fill-box; transform-origin: center bottom; }
+.is-active .plan-bars rect:nth-child(2) { animation-delay: .12s; }
+.is-active .plan-bars rect:nth-child(3) { animation-delay: .24s; }
+.is-active .plan-bars rect:nth-child(4) { animation-delay: .36s; }
+.is-active .plan-bars rect:nth-child(5) { animation-delay: .48s; }
+.is-active .plan-bars rect:nth-child(6) { animation-delay: .6s; }
+.is-active .plan-bars rect:nth-child(7) { animation-delay: .72s; }
+.is-active .plan-bars rect:nth-child(8) { animation-delay: .84s; }
+.plan-path { stroke-dasharray: 520; stroke-dashoffset: 520; }
+.is-active .plan-path { animation: plan-draw 2.2s var(--ease) infinite; }
+.is-active .plan-cursor { animation: plan-cursor 2.2s var(--ease) infinite; }
 
 /* Generate */
 .note { fill: var(--red); transform: translate(560px, 140px); }
@@ -1483,17 +1519,17 @@ footer a:hover {
   48%, 100% { opacity: .16; transform: scale(.75); }
 }
 
-@keyframes structure-rise {
+@keyframes plan-rise {
   0% { opacity: .12; transform: scaleY(.45); }
   100% { opacity: .35; transform: scaleY(1); }
 }
 
-@keyframes structure-draw {
+@keyframes plan-draw {
   0% { stroke-dashoffset: 520; }
   75%, 100% { stroke-dashoffset: 0; }
 }
 
-@keyframes structure-cursor {
+@keyframes plan-cursor {
   0% { transform: translate(0, 0); opacity: 0; }
   8% { opacity: 1; }
   12% { transform: translate(0, 0); }

static/app.js

@@ -16,10 +16,16 @@ const ui = {
   removeFile: $("#remove-file"),
   fileError: $("#file-error"),
   model: $("#model"),
+  experimentalModels: $("#experimental-models"),
+  modelNotices: $("#model-notices"),
+  modelNoticesTitle: $("#model-notices-title"),
+  modelNoticesList: $("#model-notices-list"),
   level: $("#level"),
   levelOutput: $("#level-output"),
   bpm: $("#bpm"),
+  autoPlan: $("#auto-plan"),
   useBeat: $("#use-beat"),
+  usePlanner: $("#use-planner"),
   sampling: $("#sampling"),
   samplingParameters: $("#sampling-parameters"),
   temperature: $("#temperature"),
@@ -51,13 +57,13 @@ const ui = {
   metricsGrid: $("#metrics-grid"),
   previewBlock: $(".preview-block"),
   chartTab: $("#chart-tab"),
-  structureTab: $("#structure-tab"),
+  planTab: $("#plan-tab"),
   chartPreview: $("#chart-preview"),
-  structurePreview: $("#structure-preview"),
+  planPreview: $("#plan-preview"),
   chartImage: $("#chart-image"),
-  structureImage: $("#structure-image"),
+  planImage: $("#plan-image"),
   chartEmpty: $("#chart-empty"),
-  structureEmpty: $("#structure-empty"),
+  planEmpty: $("#plan-empty"),
   downloadTja: $("#download-tja"),
   downloadAudio: $("#download-audio"),
   newChartButton: $("#new-chart-button"),
@@ -80,10 +86,10 @@ const stageDefaults = {
     title: "Finding the beat",
     detail: "Finding beats and tempo."
   },
-  structure: {
-    eyebrow: "STRUCTURE",
-    title: "Reading the structure",
-    detail: "Building whole-song hierarchical context."
+  plan: {
+    eyebrow: "PLANNING",
+    title: "Shaping the arc",
+    detail: "Planning density and climaxes."
   },
   generate: {
     eyebrow: "COMPOSING",
@@ -108,15 +114,26 @@ const stageDefaults = {
 };
 
 const metricLabels = {
-  model: "Model",
   bpm: "BPM",
   notes: "Notes",
   spans: "Drumroll sections",
   timing: "Timing mode",
   grid_rms_ms: "Grid error",
-  preview_hits: "Audition hits"
+  preview_hits: "Audition hits",
+  model: "Model",
+  difficulty: "Difficulty",
+  plan_source: "Song plan",
+  chart_grid: "Chart lattice",
+  grid_anchor: "Grid anchor",
+  density_bucket: "Density bucket",
+  sampling: "Sampling",
+  tempo_map: "Tempo map"
 };
 
+// Filled from /api/capabilities: the server owns the model list and the model
+// card's limitation text, so the interface never restates either from memory.
+let capabilities = null;
+
 let client = null;
 let connectPromise = null;
 let currentFile = null;
@@ -409,6 +426,10 @@ function setDownload(anchor, fileValue, defaultName) {
 
 function metricValue(key, value) {
   if (value === null || value === undefined || value === "") return "—";
+  if (key === "timing") {
+    if (value === "slot-exact") return "Grid-exact";
+    if (value === "time-quantized") return "Time-quantized";
+  }
   if (key === "grid_rms_ms" && Number.isFinite(Number(value))) return `${Number(value).toFixed(1)} ms`;
   if (key === "bpm" && Number.isFinite(Number(value))) return Number(value).toFixed(1);
   if (typeof value === "boolean") return value ? "Yes" : "No";
@@ -424,11 +445,11 @@ function renderMetrics(metrics) {
     return;
   }
 
-  const entries = Object.entries(metricLabels).map(([key, label]) => [
-    key,
-    label,
-    metrics[key]
-  ]);
+  // Only metrics the server actually reported. A key it omitted (the 2.0
+  // lattice fields on a 1.x run, say) is not a missing value to show as "—".
+  const entries = Object.entries(metricLabels)
+    .filter(([key]) => key in metrics)
+    .map(([key, label]) => [key, label, metrics[key]]);
   ui.metricsGrid.hidden = entries.length === 0;
   entries.forEach(([key, label, value]) => {
     const wrapper = document.createElement("div");
@@ -446,14 +467,14 @@ function renderMetrics(metrics) {
 function activatePreview(name, focus = false) {
   const chartActive = name === "chart";
   ui.chartTab.classList.toggle("is-active", chartActive);
-  ui.structureTab.classList.toggle("is-active", !chartActive);
+  ui.planTab.classList.toggle("is-active", !chartActive);
   ui.chartTab.setAttribute("aria-selected", String(chartActive));
-  ui.structureTab.setAttribute("aria-selected", String(!chartActive));
+  ui.planTab.setAttribute("aria-selected", String(!chartActive));
   ui.chartTab.tabIndex = chartActive ? 0 : -1;
-  ui.structureTab.tabIndex = chartActive ? -1 : 0;
+  ui.planTab.tabIndex = chartActive ? -1 : 0;
   ui.chartPreview.hidden = !chartActive;
-  ui.structurePreview.hidden = chartActive;
-  if (focus) (chartActive ? ui.chartTab : ui.structureTab).focus();
+  ui.planPreview.hidden = chartActive;
+  if (focus) (chartActive ? ui.chartTab : ui.planTab).focus();
 }
 
 function renderResult(payload) {
@@ -470,15 +491,11 @@ function renderResult(payload) {
 
   renderMetrics(payload.metrics);
   const hasChart = setImage(ui.chartImage, ui.chartEmpty, files.chart_image);
-  const hasStructure = setImage(
-    ui.structureImage,
-    ui.structureEmpty,
-    files.structure_image
-  );
-  ui.previewBlock.hidden = !hasChart && !hasStructure;
+  const hasPlan = setImage(ui.planImage, ui.planEmpty, files.plan_image);
+  ui.previewBlock.hidden = !hasChart && !hasPlan;
   ui.chartTab.hidden = !hasChart;
-  ui.structureTab.hidden = !hasStructure;
-  activatePreview(hasChart ? "chart" : "structure");
+  ui.planTab.hidden = !hasPlan;
+  activatePreview(hasChart ? "chart" : "plan");
 
   setDownload(ui.downloadTja, files.tja, "softchart.tja");
   setDownload(ui.downloadAudio, files.audio, "softchart-taiko-preview.wav");
@@ -514,6 +531,74 @@ function handlePayload(payload) {
   }
 }
 
+function selectedCourseValue() {
+  const selected = $('input[name="course"]:checked', ui.form);
+  return selected ? selected.value : "oni";
+}
+
+function selectedModelValue() {
+  return ui.model ? ui.model.value : "v1.7";
+}
+
+// The model card's wording, verbatim. Anything shown here is a defect the
+// tester is meant to recognise as already known, so it is never paraphrased.
+function renderNotices() {
+  if (!capabilities || !ui.modelNotices) return;
+  const model = (capabilities.models || []).find((m) => m.value === selectedModelValue());
+  const course = selectedCourseValue();
+  const courseNotices = (capabilities.course_notices || {})[course] || [];
+  const plannerOn = ui.usePlanner ? ui.usePlanner.checked : false;
+  const items = [];
+
+  if (model && model.experimental) {
+    (model.limitations || []).forEach((entry) => items.push(entry));
+    (model.notes || []).forEach((text) => items.push({ text }));
+  }
+  courseNotices.forEach((entry) => {
+    if (entry.when === "planner" && !plannerOn) return;
+    if (entry.when === "planner" && model && model.experimental) return;
+    items.push({ text: entry.text });
+  });
+
+  ui.modelNoticesList.replaceChildren();
+  items.forEach((entry) => {
+    const li = document.createElement("li");
+    if (entry.section) {
+      const tag = document.createElement("span");
+      tag.className = "notice-section";
+      tag.textContent = entry.section;
+      li.append(tag);
+    }
+    li.append(document.createTextNode(entry.text));
+    ui.modelNoticesList.append(li);
+  });
+  ui.modelNoticesTitle.textContent = model && model.experimental
+    ? "Experimental — known limitations"
+    : "Known limitations";
+  ui.modelNotices.hidden = items.length === 0;
+}
+
+async function loadCapabilities() {
+  const response = await fetch("/api/capabilities", { headers: { Accept: "application/json" } });
+  if (!response.ok) throw new Error(`capabilities ${response.status}`);
+  capabilities = await response.json();
+
+  const experimental = (capabilities.models || []).filter((m) => m.experimental);
+  if (ui.experimentalModels && experimental.length) {
+    ui.experimentalModels.replaceChildren();
+    experimental.forEach((model) => {
+      const option = document.createElement("option");
+      option.value = model.value;
+      option.textContent = model.label;
+      ui.experimentalModels.append(option);
+    });
+    if (experimental[0].group) ui.experimentalModels.label = experimental[0].group;
+    ui.experimentalModels.disabled = false;
+    ui.experimentalModels.hidden = false;
+  }
+  renderNotices();
+}
+
 function requestValues() {
   const selectedCourse = $('input[name="course"]:checked', ui.form);
   const bpm = ui.bpm.value.trim() === "" ? 0 : Number(ui.bpm.value);
@@ -523,12 +608,14 @@ function requestValues() {
     selectedCourse ? selectedCourse.value : "oni",
     Number.parseInt(ui.level.value, 10),
     Number.isFinite(bpm) ? bpm : 0,
+    ui.autoPlan.checked,
     ui.useBeat.checked,
+    ui.usePlanner.checked,
     ui.sampling.checked,
     Number(ui.temperature.value),
     Number(ui.topP.value),
     Number(ui.drumVolume.value),
-    ui.model.value
+    ui.model ? ui.model.value : "v1.7"
   ];
 }
 
@@ -644,6 +731,11 @@ function bindEvents() {
     range.addEventListener("input", updateRanges);
   });
   ui.sampling.addEventListener("change", () => setSamplingEnabled(ui.sampling.checked));
+  if (ui.model) ui.model.addEventListener("change", renderNotices);
+  if (ui.usePlanner) ui.usePlanner.addEventListener("change", renderNotices);
+  $$('input[name="course"]', ui.form).forEach((radio) => {
+    radio.addEventListener("change", renderNotices);
+  });
   ui.cancelButton.addEventListener("click", cancelGeneration);
   ui.retryButton.addEventListener("click", () => ui.form.requestSubmit());
   ui.newChartButton.addEventListener("click", () => {
@@ -654,13 +746,13 @@ function bindEvents() {
   });
 
   ui.chartTab.addEventListener("click", () => activatePreview("chart"));
-  ui.structureTab.addEventListener("click", () => activatePreview("structure"));
+  ui.planTab.addEventListener("click", () => activatePreview("plan"));
   $$(".preview-tab").forEach((tab) => {
     tab.addEventListener("keydown", (event) => {
       if (!["ArrowLeft", "ArrowRight"].includes(event.key)) return;
       event.preventDefault();
-      const next = tab === ui.chartTab ? "structure" : "chart";
-      const target = next === "chart" ? ui.chartTab : ui.structureTab;
+      const next = tab === ui.chartTab ? "plan" : "chart";
+      const target = next === "chart" ? ui.chartTab : ui.planTab;
       if (!target.hidden) activatePreview(next, true);
     });
   });
@@ -674,6 +766,10 @@ bindEvents();
 updateRanges();
 setSamplingEnabled(ui.sampling.checked);
 resetProcess();
+loadCapabilities().catch(() => {
+  // The seven published models are in the markup already; only the
+  // experimental entry and the limitation text depend on this call.
+});
 connect().catch(() => {
   ui.queueLabel.textContent = "We will reconnect when you start generation";
 });

static/index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
-  <meta name="description" content="SoftChart V1.8 turns music into playable Taiko charts with a scratch-trained hierarchical model.">
+  <meta name="description" content="SoftChart turns music into playable, musical Taiko charts with an instant drum-mix audition.">
   <meta name="theme-color" content="#f6f3ec">
   <title>SoftChart — AI Taiko Chart Studio</title>
   <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='25' fill='%23e14b3f'/%3E%3Ccircle cx='32' cy='32' r='18' fill='%23fff8ed'/%3E%3Ccircle cx='32' cy='32' r='9' fill='%23e14b3f'/%3E%3C/svg%3E">
@@ -34,11 +34,11 @@
     <section class="hero" aria-labelledby="hero-title">
       <div class="hero-copy">
         <h1 id="hero-title">Music in.<br><em>Taiko out.</em></h1>
-        <p class="hero-lede">Hierarchical V1.8 listens across the whole song, writes a playable TJA, and lets you hear it instantly.</p>
+        <p class="hero-lede">Upload a track. Get a playable TJA. Hear it instantly.</p>
       </div>
       <ul class="hero-features" aria-label="Key features">
-        <li><strong>V1.8 scratch model</strong></li>
-        <li><strong>Whole-song hierarchy</strong></li>
+        <li><strong>Beat-locked</strong></li>
+        <li><strong>Structure-aware</strong></li>
         <li><strong>Instant audition</strong></li>
       </ul>
     </section>
@@ -86,23 +86,30 @@
               <label for="model">Model</label>
             </div>
             <select id="model" class="model-select">
-              <optgroup label="V1.8 — hierarchical scratch (recommended)">
-                <option value="v1.8" selected>V1.8 · 9.0M — best research model</option>
+              <optgroup label="v1.7 — richest patterns (recommended)">
+                <option value="v1.7" selected>v1.7 · 7.9M — best quality</option>
+                <option value="v1.7-small">v1.7-small · 3.6M</option>
+                <option value="v1.7-tiny">v1.7-tiny · 1.4M — fastest</option>
               </optgroup>
-              <optgroup label="V1.7 — original planner path">
-                <option value="v1.7">V1.7 · 7.9M — best legacy quality</option>
-                <option value="v1.7-small">V1.7 small · 3.6M</option>
-                <option value="v1.7-tiny">V1.7 tiny · 1.4M — fastest</option>
+              <optgroup label="v1.6 — scaling ladder">
+                <option value="v1.6">v1.6 · 7.9M</option>
+                <option value="v1.6-small">v1.6-small · 3.6M</option>
+                <option value="v1.6-tiny">v1.6-tiny · 1.4M</option>
               </optgroup>
-              <optgroup label="V1.6 — original planner path">
-                <option value="v1.6">V1.6 · 7.9M</option>
-                <option value="v1.6-small">V1.6 small · 3.6M</option>
-                <option value="v1.6-tiny">V1.6 tiny · 1.4M</option>
-              </optgroup>
-              <optgroup label="V1.5 — original planner path">
-                <option value="v1.5">V1.5 · 7.9M</option>
+              <optgroup label="v1.5 — single model">
+                <option value="v1.5">v1.5 · 7.9M</option>
               </optgroup>
+              <!-- BarScript is the experimental preview line, served from its
+                   own release package. The group stays disabled and hidden
+                   until /api/capabilities reports the package resolved; the
+                   label below is a placeholder that app.js overwrites with the
+                   group name the API returns. -->
+              <optgroup label="BarScript — EXPERIMENTAL PREVIEW, under evaluation" id="experimental-models" hidden disabled></optgroup>
             </select>
+            <div class="model-notices" id="model-notices" hidden>
+              <strong id="model-notices-title">Known limitations</strong>
+              <ul id="model-notices-list"></ul>
+            </div>
           </div>
 
           <fieldset class="field-group">
@@ -112,6 +119,7 @@
               <label><input type="radio" name="course" value="normal"><span><small>N</small>Normal</span></label>
               <label><input type="radio" name="course" value="hard"><span><small>H</small>Hard</span></label>
               <label><input type="radio" name="course" value="oni" checked><span><small>O</small>Oni</span></label>
+              <label><input type="radio" name="course" value="ura"><span><small>U</small>Ura</span></label>
             </div>
           </fieldset>
 
@@ -141,11 +149,21 @@
           </summary>
           <div class="advanced-body">
             <div class="toggle-list">
+              <label class="toggle-card compact">
+                <span><strong>Structure</strong></span>
+                <input id="auto-plan" type="checkbox" checked>
+                <span class="switch" aria-hidden="true"></span>
+              </label>
               <label class="toggle-card compact">
                 <span><strong>Beat grid</strong></span>
                 <input id="use-beat" type="checkbox" checked>
                 <span class="switch" aria-hidden="true"></span>
               </label>
+              <label class="toggle-card compact">
+                <span><strong>AI planner</strong></span>
+                <input id="use-planner" type="checkbox" checked>
+                <span class="switch" aria-hidden="true"></span>
+              </label>
             </div>
             <label class="toggle-card compact">
               <span><strong>Creative sampling</strong></span>
@@ -230,20 +248,20 @@
               </svg>
             </div>
 
-            <div class="stage-visual" data-stage="structure" aria-hidden="true">
+            <div class="stage-visual" data-stage="plan" aria-hidden="true">
               <svg viewBox="0 0 520 280">
-                <g class="structure-bars">
+                <g class="plan-bars">
                   <rect x="42" y="166" width="45" height="60" rx="5"/>
                   <rect x="96" y="143" width="45" height="83" rx="5"/>
                   <rect x="150" y="153" width="45" height="73" rx="5"/>
                   <rect x="204" y="105" width="45" height="121" rx="5"/>
                   <rect x="258" y="127" width="45" height="99" rx="5"/>
                   <rect x="312" y="83" width="45" height="143" rx="5"/>
-                  <rect x="366" y="53" width="45" height="173" rx="5" class="structure-peak"/>
+                  <rect x="366" y="53" width="45" height="173" rx="5" class="plan-climax"/>
                   <rect x="420" y="118" width="45" height="108" rx="5"/>
                 </g>
-                <path class="structure-path" d="M64 158 118 135l54 10 54-49 54 22 54-43 54-31 54 65" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
-                <circle class="structure-cursor" cx="64" cy="158" r="7" fill="currentColor"/>
+                <path class="plan-path" d="M64 158 118 135l54 10 54-49 54 22 54-43 54-31 54 65" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
+                <circle class="plan-cursor" cx="64" cy="158" r="7" fill="currentColor"/>
               </svg>
             </div>
 
@@ -344,16 +362,16 @@
           <div class="preview-block">
             <div class="preview-tabs" role="tablist" aria-label="Preview type">
               <button id="chart-tab" class="preview-tab is-active" type="button" role="tab" aria-selected="true" aria-controls="chart-preview">Chart preview</button>
-              <button id="structure-tab" class="preview-tab" type="button" role="tab" aria-selected="false" aria-controls="structure-preview" tabindex="-1">Song structure</button>
+              <button id="plan-tab" class="preview-tab" type="button" role="tab" aria-selected="false" aria-controls="plan-preview" tabindex="-1">Song structure</button>
             </div>
             <div class="preview-frame">
               <div id="chart-preview" role="tabpanel" aria-labelledby="chart-tab">
                 <img id="chart-image" alt="Preview of the generated Taiko chart">
                 <p class="preview-empty" id="chart-empty" hidden>No chart preview is available.</p>
               </div>
-              <div id="structure-preview" role="tabpanel" aria-labelledby="structure-tab" hidden>
-                <img id="structure-image" alt="Full-song mel spectrogram used for hierarchical context">
-                <p class="preview-empty" id="structure-empty" hidden>No song structure preview is available.</p>
+              <div id="plan-preview" role="tabpanel" aria-labelledby="plan-tab" hidden>
+                <img id="plan-image" alt="Song energy and chart section plan">
+                <p class="preview-empty" id="plan-empty" hidden>No song structure preview is available.</p>
               </div>
             </div>
           </div>