SteerTTS v1 (9.6M acoustic model + Vocos; flow refiner, prompted-flow cloning, prompt adapter); CC BY-NC 4.0 weights, MIT code

LICENSE.md

@@ -0,0 +1,34 @@
+# Licence
+
+This repository holds two kinds of material, each under its own licence.
+
+## 1. Code: MIT License
+
+This covers `steertts/`, `tests/`, `inference.py` and `leaderboard/`.
+
+Copyright (c) 2026 The SteerTTS authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
+documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
+persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
+Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## 2. Model weights and voice data: CC BY-NC 4.0
+
+This covers `*.safetensors`, `voices.json`, `control_stats.json` and `config.json`.
+
+- **Licence.** Creative Commons Attribution-NonCommercial 4.0 International,
+  https://creativecommons.org/licenses/by-nc/4.0/. Non-commercial use only.
+- **Why.** The weights were trained partly on CC BY-NC 4.0 corpora (EARS, Expresso), on NonverbalTTS (its source
+  terms apply), and on synthetic speech from a commercial TTS API, whose terms of service include a
+  competing-model clause. Use of the weights must also respect the most restrictive of these terms. The model card
+  (`README.md`, "Training data") lists every source.
+- **Voice cloning.** Clone only voices whose owners have consented. See "Intended use and misuse" in the model card.

README.md

@@ -0,0 +1,366 @@
+---
+language:
+- en
+license: cc-by-nc-4.0
+library_name: steertts
+pipeline_tag: text-to-speech
+tags:
+- text-to-speech
+- tts
+- speech-synthesis
+- english
+- controllable-tts
+- steerable-tts
+- prosody-control
+- expressive-tts
+- style-control
+- inline-markup
+- word-emphasis
+- zero-shot-voice-cloning
+- voice-cloning
+- interpretable
+- non-autoregressive
+- fastpitch
+- mamba
+- mamba-3
+- state-space-model
+- hybrid-attention
+- flow-matching
+- vocos
+- small-model
+- on-device
+- streaming
+- non-commercial
+---
+
+# SteerTTS v1
+
+A small English text-to-speech model that you steer with numbers. Each request can set any subset of **12 acoustic
+controls**, for example speaking rate, loudness, median pitch, pitch range, final rise or fall, pauses, spectral tilt
+and breathiness. Each control is a z-score relative to the chosen voice's own neutral speech. You can also pick a
+**style tag**, give a **free-text direction** (which a small adapter turns into a request you can inspect), or use
+**inline markup**. The model can **clone** a voice from a short reference clip.
+
+| | |
+|---|---|
+| Acoustic model | **9.59M** parameters (encoder/decoder: attention + bidirectional Mamba-3, AdaLN conditioning) |
+| + Vocos vocoder | 13.53M (`charactr/vocos-mel-24khz`, frozen), **23.1M** total |
+| Flow-matching refiner (optional, on by default) | 17.64M → 40.8M total |
+| Prompted flow (cloning only) | 24.67M, used in place of the refiner when `ref_wav` is given → 47.8M total |
+| Frozen external encoders | WavLM-base-plus-sv 101M (cloning only), all-MiniLM-L6-v2 22.7M (free-text prompts only) |
+| Output | 24 kHz mono, fp32, CPU or CUDA |
+| Voices | 2,825 built-in voices under anonymised names (`voice_0001` …), default `voice_1361` |
+
+> **Licence: weights CC BY-NC 4.0 (non-commercial use only); package code MIT.** The weights were trained partly on
+> non-commercial data (see [Training data](#training-data)).
+
+## Quick start
+
+```bash
+git clone https://huggingface.co/WhissleAI/steer-tts && cd steer-tts      # or huggingface_hub.snapshot_download("WhissleAI/steer-tts")
+pip install ".[all]"          # core: torch, vocos, g2p_en …; [all] adds transformers (prompts, cloning) + parselmouth (measuring)
+steertts say "Hello there, nice to meet you." -o hello.wav
+steertts say "I can't believe it worked." --tag happiness --ctrl rate=1,f0_iqr=1.5 -o happy.wav
+steertts say "Please keep this quiet." --prompt "whispering, slow and secretive" -o secret.wav   # prints the resolved request
+steertts stream "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" -o markup.wav
+steertts say "Now in your voice." --ref-wav consenting_speaker.wav -o clone.wav
+steertts voices --gender female | head
+```
+
+The CLI looks for the model in the current directory. From anywhere else, pass it before the command
+(`steertts --model WhissleAI/steer-tts say "Hi." -o hi.wav`) or set `STEERTTS_MODEL`.
+
+```python
+from steertts import SteerTTS
+
+tts = SteerTTS.from_pretrained("WhissleAI/steer-tts")   # HF repo id or a local dir; device: cuda if available, else cpu
+wav = tts("The meeting moved to Thursday.")       # float32 numpy, 24 kHz, default voice voice_1361
+wav = tts("Are you serious?", voice="voice_0042", ctrl={"f0_end": 2.0, "f0_iqr": 1.5})   # rising, lively
+wav = tts("It is over.", tag="sadness", ctrl={"loud": -1}, guidance=1.5)
+
+req = tts.resolve({"text": "See you soon.", "prompt": "excited, a bit faster"})
+print(req)    # {'text': ..., 'voice': 'voice_1361', 'tag': 'excited', 'ctrl': {'rate': 0.29}, 'prompt_mapping': {...}, ...}
+wav = tts.synthesize(req)                         # a resolved request renders exactly as the prompt would
+
+for chunk in tts.stream("First sentence. Second sentence."):   # sentence-level streaming
+    play(chunk.audio)
+tts.save(wav, "out.wav")
+```
+
+`inference.py` renders one example of each feature. `python -m steertts.cli --help` lists the CLI.
+
+## The request
+
+Every synthesis is one JSON object. It is documented field by field in [`request.schema.json`](request.schema.json)
+(JSON Schema 2020-12; `steertts schema` prints it). Only `text` is required.
+
+| Field | Meaning |
+|---|---|
+| `text` | English text. Numbers, units, currency, ordinals, ranges and unknown acronyms are spelled out. |
+| `voice` / `ref_wav` | A built-in voice, **or** a reference clip to clone (first 8 s, at least 0.5 s). |
+| `tag` | One of 51 style tags (`none`, `whisper`, `sadness`, `anger`, `fast`, `high_pitch`, …). |
+| `ctrl` | `{control: z}` over the 12 controls below, each in [-5, 5]. |
+| `prompt` | Free-text direction. `resolve()` maps it to `tag` + `ctrl`; an explicit `tag` or `ctrl` dim wins. |
+| `guidance` | Steering scale s (default 1). See [Guidance](#how-well-it-steers). |
+| `pace` | Global tempo divisor (>1 is faster). It does not change the style. |
+| `word_shift` | `{word_index: {pitch: st, energy: dB, dur: ×}}`, per-word prosody edits. |
+| `markup` | Parse inline markup in `text` (see below). `stream()` always parses it. |
+| `flow`, `seed` | Use the flow refiner (default true) and set the sampler seed. Renders are deterministic per device. |
+
+**Controls and the mask.** The 12 controls are utterance-level acoustic measurements:
+
+- `rate`: articulation rate
+- `loud`: speech level
+- `f0_med`: median pitch
+- `f0_iqr`: pitch range
+- `f0_end`: final pitch slope
+- `pause`: pause share
+- `tilt`: spectral tilt
+- `voiced`: voiced share
+- `flat`: spectral flatness
+- `centroid`: spectral centroid
+- `cpps`: cepstral peak prominence
+- `h1h2`: H1–H2
+
+Each value is a z-score against the chosen voice's own neutral speech, in units of the pooled within-speaker SD
+(`control_stats.json`). The model also receives an availability mask:
+
+- A dim named in `ctrl` is requested.
+- A dim left out is unspecified, and the model chooses it from voice, tag and text.
+- `{"rate": 0}` asks for this voice's neutral rate. Omitting `rate` lets a `fast` tag or the text decide.
+
+`steertts measure out.wav --text "…" --voice voice_1361` reads the controls back with the same measurement code
+that labelled the training data.
+
+**Markup** (`markup: true` or `stream()`):
+
+| Markup | Effect |
+|---|---|
+| `*word*` or `<emph>…</emph>` | Emphasis: +2.5 st, +4 dB, ×1.2 duration |
+| `<slow> <fast> <high> <low> <loud> <soft>` | Per-word changes; the sentence is still rendered in one pass |
+| `[pause]`, `[pause 300ms]`, `[pause 1.5s]` | Exact silence |
+| `<whisper>…</whisper>` (any tag), `<style prompt="…">…</style>` | The span is rendered as its own segment |
+
+Text is rendered sentence by sentence, so the first sentence can play while the rest renders.
+
+## Architecture
+
+FastPitch-style explicit-variance TTS:
+
+1. **Front end.** g2p_en ARPAbet phonemes with word indices.
+2. **Encoder.** 4 blocks.
+3. **Variance predictors.** Per-phone duration, pitch and energy.
+4. **Length regulator.**
+5. **Decoder.** 4 blocks.
+6. **Output.** A 100-band log-mel, rendered by Vocos.
+
+Each encoder/decoder block is a **hybrid**: multi-head self-attention, then a **bidirectional Mamba-3** branch (two
+Mamba-3 layers run forward and backward and summed; complex-valued state, trapezoidal discretisation; 3.70M
+parameters in total), then a conv FFN.
+
+Every LayerNorm in every block and predictor is an **adaptive LayerNorm**. Its scale and shift are predicted from
+g = MLP([speaker vector, tag embedding, ctrl ⊙ mask, mask]). The speaker slot is a linear projection of a frozen
+512-d WavLM-base-plus-sv x-vector. Built-in voices use their stored mean vector; cloning uses the vector of the
+reference clip.
+
+**Guidance** renders the request and the uninstructed request, then extrapolates durations, pitch, energy and mel:
+out = out₀ + s·(out − out₀).
+
+Optional decoders:
+
+- **Flow refiner.** A conditional flow-matching ConvNeXt network (OT-CFM, 10 midpoint steps, temperature 0.4). It
+  samples mel detail given the acoustic model's mel and g, so every control still comes from the acoustic model.
+- **Prompted flow.** The same network plus attention, given the reference clip's own mel as context
+  (Voicebox-style infilling). Used for cloning.
+
+The `steertts` package here is self-contained. It is the inference subset of the training code, operation for
+operation, and its outputs are bit-identical to the training repository's synthesis on the same device (see
+[Release verification](#release-verification)).
+
+## How well it steers
+
+The numbers below are **held-out**: 16 LibriTTS-R test-clean speakers who share no speaker, chapter or recording
+with the training data (8 female, 8 male by a median-F0 rule; the corpus has no gender labels on disk), each
+conditioned on the speaker vector of one recording, × 8 texts. They are for the acoustic model at guidance 1 **without the flow refiner**.
+Steering through the refiner was not measured in the paper; the release check below confirms only the direction.
+
+Columns:
+
+- **Hit**: share of renders that move more than 0.5 SD the requested way.
+- **Δz**: achieved change for a request of ±2.
+- **ρ**: Spearman correlation between request and readback (by our extractor / by Praat).
+
+| Tier | Control | Hit at −2 / +2 | Δz at −2 / +2 | ρ (ours / Praat) |
+|---|---|---|---|---|
+| **Reliable** | loud | 1.00 / 1.00 | −1.57 / +1.81 | .97 / .97 |
+| | f0_med | .98 / 1.00 | −1.15 / +1.18 | .95 / .95 |
+| | h1h2 | 1.00 / 1.00 | −1.59 / +1.49 | .96 / .96 |
+| | tilt | 1.00 / .99 | −1.39 / +0.97 | .96 / .87 |
+| | centroid | .78 / .94 | −0.79 / +1.20 | .92 / .92 |
+| | f0_iqr | .66 / .91 | −0.63 / +1.09 | .88 / .88 |
+| **Usable** | rate | 1.00 / 1.00 | −1.61 / +1.30 | .96 / .66 |
+| | f0_end | .96 / .99 | −0.85 / +1.31 | .94 / .83 |
+| | flat | .62 / .41 | −0.64 / +0.51 | .85 / .68 |
+| **Weak** | pause | .38 / .64 | −0.37 / +0.71 | .76 / .70 |
+| | voiced | .39 / .12 | −0.44 / +0.26 | .79 / .60 |
+| | cpps | .56 / .35 | −0.45 / +0.30 | .58 / .46 |
+
+- **Tiers.** Reliable: ρ ≥ 0.85 under both extractors. Usable: under one. Weak: under neither.
+- **Gain.** The mean gain (achieved / requested) is about 0.5. `rate` and `f0_med` at +1 hit only .48 / .42.
+- **Cross-talk.** Mean leakage into the other 11 dims is 0.15–0.30 SD. The off-diagonal / diagonal ratio is 0.10.
+- **CPPS** is not a working control: a request moves pitch and level about as much as CPPS itself.
+- **Intelligibility.** WER (Whisper-large-v3) is 0.94 % uninstructed and 0.84–1.01 % across all requests.
+
+**Guidance** (pooled over loud, rate, f0_med, f0_iqr and tilt; ±2 requests; n = 1,280):
+
+| s | Steer ratio (Δz / requested) | Hit | Leak | WER |
+|---|---|---|---|---|
+| 1 | 0.65 | .95 | .20 | 0.9 % |
+| 2 | 1.35 | .99 | .34 | 1.0 % |
+| 3 | 1.97 | .98 | .48 | 1.0 % |
+
+- s ≈ 1.5 roughly delivers the requested size, at the cost of more leakage.
+- Guidance on tags is riskier: `whisper` at s = 3 raises WER to 16 %.
+
+**Markup** (8 held-out voices × 6 sentences × 2 words):
+
+- Emphasis achieved +2.58 st and +3.92 dB, against +2.5 st and +4 dB requested.
+- `<high>` / `<low>` gave +3.18 / −2.96 st for ±3.
+- `<loud>` / `<soft>` gave +4.84 / −5.54 dB for ±6. The decoder attenuates level by 10–20 %.
+- `<slow>` / `<fast>` and pauses are exact by construction.
+- Neighbouring words move by at most 0.19 st and 0.16 dB.
+
+**Free-text prompts** (161 hand-written prompts):
+
+- Which-dims precision / recall: .85 / .66. Sign accuracy: .88.
+- Tag top-1 / top-3: .56 / .69.
+- Composition (.23) and negation (.22) mostly fail. "Not fast" can come out fast.
+- Absolute-register prompts ("a deep male voice") are not supported. Pick a voice instead (`voices.json`).
+- **Always look at the resolved request.**
+
+## Quality
+
+Results under the Open TTS Leaderboard protocol (run locally on an RTX 5070, not an official submission;
+Qwen3-ASR-1.7B WER, WavLM-large SIM):
+
+| Setting | Seed-TTS en | CV3-Eval en | Avg |
+|---|---|---|---|
+| Default voice `voice_1361`, acoustic model + Vocos, WER | 1.00 % | 2.63 % | **1.81 %** |
+| Cloning (prompted flow), WER / SIM ×100 | 1.01 % / 37.4 | 2.73 % / 28.7 | 1.87 % / 33.1 |
+
+- The flow refiner was scored with an earlier acoustic checkpoint: 1.04 % / 2.70 % WER (no change).
+- **Cloning similarity is low.** The board's lowest cloning SIM is about 37, and its median is about 65. Do not
+  expect a convincing likeness.
+
+## Speed
+
+Measured on an RTX 5070 and on a shared, heavily loaded 8-thread CPU, so treat CPU figures as lower bounds:
+
+- **GPU.** Batch-1 RTFx 61–81 for the acoustic model + Vocos (leaderboard runs).
+- **CPU, 8 threads.** RTFx 3.6–7.9 for the acoustic model + Vocos. The flow refiner adds 12× the compute and
+  drops RTFx from 6.7 to 0.8 (0.4 with 1 thread). Use `flow=False` / `--no-flow` for real-time CPU use.
+- **Cloning on CPU.** RTFx 0.2 with the prompted flow, including the WavLM speaker encoder.
+- **Streaming on CPU.** First audio after about 1 s.
+- See [Release verification](#release-verification) for the numbers measured on this exact package.
+
+## Limitations
+
+- **English only.** The front end is ARPAbet. Single speaker per request, and the controls are utterance-level.
+- **Emotion tags are style switches, not emotions.** An independent emotion classifier (emotion2vec+ large)
+  recognises only `sadness` (0.16 top-1 vs 0.01 uninstructed). Every other emotion tag scores at most 0.02. The same
+  classifier recognises 7 of 10 of these emotions on real EARS recordings. 27 of 49 tags are acoustically within 0.5
+  of `neutral`. Voice-quality tags such as `whisper` act strongly (voiced share −9.9 SD).
+- **Event tokens only reserve time.** `[laugh]`, `[cough]`, `[dog]`, … add about 0.12 s of near-silence. An audio
+  tagger detects 0 of 1,728 renders. The project's demo mixed recorded sounds in, but that mixer is not part of this
+  release.
+- **The acoustic-environment input is untrained.** The weights contain an environment projection, but the final
+  training stage never saw a non-zero environment vector, so this package does not expose it.
+- **Cloning similarity is low** (see above), and cloning speed on CPU is slow with the prompted flow.
+- **The controls are acoustic measurements, not percepts.** `voiced`, `flat` and `cpps` barely move. `rate` and
+  `tilt` depend on how they are measured. z = 0 is the voice's typical render, not its real recordings.
+- **Test-set-informed choices.** The prompted-flow checkpoint was chosen on subsets of the Seed-TTS / CV3 test
+  sets, and the text-normalisation rules came from CV3 error analysis. The default voice `voice_1361` is the
+  pre-registered one: a sweep picked a lower-WER voice on the test sets, but it is not the default here.
+- **No human evaluation.** Every number above is from automatic metrics, with one training seed.
+
+## Intended use and misuse
+
+**Intended uses:**
+
+- Research on controllable / interpretable speech synthesis.
+- Accessibility and prototyping, where explicit, inspectable control matters more than peak naturalness.
+- On-device experiments with a ~23M-parameter TTS.
+
+**Voice cloning requires consent.** Clone only a voice whose owner has agreed, and do not use this model to:
+
+- impersonate real people;
+- produce deceptive or fraudulent audio;
+- bypass voice authentication;
+- create non-consensual content.
+
+Disclose synthetic speech to listeners. There is no watermark yet.
+
+**Built-in voices are anonymised.** Built-in voices are training speakers under opaque names. Speaker IDs from the
+source corpora are not included. Speakers from a celebrity-interview corpus (VoxCeleb, via NonverbalTTS) were
+excluded from the voice list.
+
+## Training data
+
+410,566 utterances (659 h) from 4,919 speakers:
+
+| Source | Hours | Licence |
+|---|---|---|
+| LibriTTS-R (clean-100, clean-360, other-500) | 516 | CC BY 4.0 |
+| EARS (read + freeform) | 78 | CC BY-NC 4.0 |
+| Expresso | 11 | CC BY-NC 4.0 |
+| NonverbalTTS (VoxCeleb and Expresso sources) | 12.5 | Source terms apply |
+| Synthetic speech from a commercial TTS API | 43 | The provider's terms of service, incl. a competing-model clause |
+
+Control labels are measured from the audio. Style tags come from the corpora's own style/emotion labels.
+
+**Licence: CC BY-NC 4.0. Non-commercial use only.**
+
+## Files
+
+| File | Contents |
+|---|---|
+| `model.safetensors` | Acoustic model (fp32) |
+| `flow.safetensors`, `pflow.safetensors` | Flow refiner and prompted flow |
+| `prompt_adapter.safetensors` | Free-text → request adapter (0.5M) |
+| `speaker_vectors.safetensors` | 512-d vector per built-in voice, in `config.json:voices_order` |
+| `voices.json` | Measured descriptions (see below) |
+| `control_stats.json` | Pooled SDs, tag list, and each voice's neutral control means (for `measure_z`) |
+| `config.json` | Architecture and file map |
+| `request.schema.json` | The request schema |
+| `steertts/` | The package (incl. `steertts/mamba3` with its own invariant tests) |
+| `tests/` | pytest suite |
+| `leaderboard/` | Open TTS Leaderboard backend |
+| `TEST_LOG.md` | Release verification log |
+| `LICENSE.md` | Code (MIT) and weights (CC BY-NC 4.0) |
+
+`voices.json` descriptions are measured, not copied from corpus metadata:
+
+- **Gender.** A probe on the speaker vector; 99.3 % 5-fold accuracy on 534 self-labelled speakers.
+- **Pitch register.** Median F0 within that gender.
+- **Pace.** Articulation rate.
+
+## Release verification
+
+The package was verified in a fresh CPU environment; `TEST_LOG.md` has the full log.
+
+- **Tests.** The pytest suite passes (16 tests).
+- **Parity.** Seven renders are bit-identical to the training code's synthesis on the same device: acoustic
+  only, controls, tag + guidance, per-word shifts, the flow refiner, and cloning through the prompted flow.
+- **Speed.** On this package, CPU with 4 threads on a loaded machine: RTFx 4.9 without the refiner, 0.7 with it.
+  First streamed audio arrives after 0.5 s.
+
+## Citation
+
+```bibtex
+@misc{steertts2026,
+  title  = {SteerTTS: A Tiny, Measurably Steerable Speech Synthesizer},
+  author = {{Whissle AI}},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/WhissleAI/steer-tts}}
+}
+```

TEST_LOG.md

@@ -0,0 +1,78 @@
+# Release verification log: steertts-v1
+
+Date: 2026-10-11. Everything ran on the CPU (4 threads, `nice -n 19`) of a shared 48-core workstation, with a 1-min
+load average of 20 to 40 from other jobs. The timings are lower bounds.
+
+## Weights
+
+| File | Source checkpoint | sha256 (first 16) |
+|---|---|---|
+| `model.safetensors` | acoustic model, step 312,500 | `adc11912871113e3` |
+| `flow.safetensors` | flow refiner (EMA), step 100,000 | `585ff2c931811c30` |
+| `pflow.safetensors` | prompted flow with prompt attention (EMA), step 40,000 | `877f81523dfaba50` |
+| `prompt_adapter.safetensors` | free-text → request adapter | `aca34bf753456e46` |
+| `speaker_vectors.safetensors` | 2,825 built-in voices (public names only) | `edcabfd02dd4a520` |
+
+These are the checkpoints behind the model card's numbers. The prompted flow is the one in the leaderboard cloning row
+(1.01 % / 37.4 and 2.73 % / 28.7).
+
+## 1. Fresh-environment install
+
+- A new Python 3.11 venv (uv) with the CPU torch wheel. Then `pip install ".[test]"` from a copy of the package
+  code, holding no weights.
+- Resolved versions: torch 2.11.0+cpu, torchaudio 2.11.0+cpu, numpy 2.4.6, safetensors 0.8.0, vocos 0.1.0,
+  g2p_en 2.1.0, nltk 3.10.3, num2words 0.5.14, soundfile 0.14.0, soxr 1.1.0, transformers 5.19.0,
+  praat-parselmouth 0.4.7, pytest 9.1.1, jsonschema 4.26.0.
+- Not fresh: the Hugging Face hub cache and `~/nltk_data` came from the host. A first run on a new machine
+  downloads `charactr/vocos-mel-24khz`, plus `microsoft/wavlm-base-plus-sv` for cloning and
+  `sentence-transformers/all-MiniLM-L6-v2` for free-text prompts, and the NLTK tagger and CMUdict.
+
+## 2. Test suite
+
+`STEERTTS_MODEL=<this dir> pytest` from the fresh env gave **16 passed** in 6 min 11 s. The suite covers:
+
+- the Mamba-3 invariant suite;
+- text normalisation and phonemes;
+- the request schema and its examples;
+- request validation errors;
+- deterministic renders, both acoustic-only and through the flow refiner;
+- pace and rate;
+- Praat readback direction for `loud`, `f0_med` and `h1h2`;
+- markup and streaming;
+- cloning;
+- prompt → request mapping.
+
+## 3. Parity with the training code
+
+The same inputs were rendered on the same device (CPU) by this package and by the training repository's
+`steer.synth.Synth`, given the source checkpoints above. Every case was **bit-identical** (max |Δ| = 0):
+
+| Case | Samples | Identical |
+|---|---|---|
+| default voice, acoustic only | 73,472 | yes |
+| `ctrl` rate/f0_med/f0_end, acoustic only | 68,096 | yes |
+| tag `sadness` + `loud` −1, guidance 1.5 | 85,760 | yes |
+| `word_shift` (pitch/energy/dur on one word) | 37,376 | yes |
+| another built-in voice, flow refiner | 73,216 | yes |
+| `ctrl` loud/f0_iqr, flow refiner | 75,008 | yes |
+| cloning from a reference clip, prompted flow | 51,456 | yes |
+
+Cross-environment check: two renders (acoustic only, and flow + ctrl) were bit-identical between the fresh venv
+(torch 2.11.0+cpu) and the training environment (torch 2.11.0+cu128, run on the CPU). GPU renders are deterministic
+per device but are not bit-identical to CPU renders.
+
+## 4. Speed on this package (CPU, 4 threads, load average about 22)
+
+| Path | RTFx (3 texts, 1.7–5 s) |
+|---|---|
+| Acoustic model + Vocos (`flow=False`) | 4.35–5.59 (median 4.89) |
+| + flow refiner (default) | 0.62–0.79 (median 0.71) |
+| Streaming, first chunk (`flow=False`) | 0.49 s |
+
+## 5. Hygiene
+
+- No source-corpus speaker or clip identifiers appear in any file. The build fails on the pattern check in
+  `steer.release`, and the check was re-run here with grep.
+- VoxCeleb-derived speakers are excluded from the voice list.
+- The schema `$id` is a URN (`urn:steertts:request-schema:v1`), not a URL.
+- `leaderboard/` names a placeholder Hugging Face repo id. It is not referenced by the package code.

build/lib/steertts/__init__.py

@@ -0,0 +1,10 @@
+"""SteerTTS: a small, numerically steerable English TTS (hybrid attention + bidirectional Mamba-3, AdaLN on a
+masked 12-dim control request), with optional flow-matching refiners, a free-text prompt adapter and zero-shot
+cloning from a reference clip."""
+from .config import CONTROL_DIMS, DEFAULT_VOICE, SR
+from .pipeline import SteerTTS, SteerTTSPipeline, resolve_model_dir
+from .request import normalize_request, schema
+
+__all__ = ["SteerTTS", "SteerTTSPipeline", "CONTROL_DIMS", "DEFAULT_VOICE", "SR", "normalize_request", "schema",
+           "resolve_model_dir"]
+__version__ = "1.0.0"

build/lib/steertts/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+main()

build/lib/steertts/audio.py

@@ -0,0 +1,36 @@
+"""Waveform I/O and the Vocos mel front end (24 kHz, 100 bands, hop 256)."""
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from .config import HOP, N_FFT, N_MELS, SR
+
+_mel_fe = None
+
+
+def mel_extractor():
+    global _mel_fe
+    if _mel_fe is None:
+        from vocos.feature_extractors import MelSpectrogramFeatures
+        _mel_fe = MelSpectrogramFeatures(sample_rate=SR, n_fft=N_FFT, hop_length=HOP, n_mels=N_MELS, padding="center")
+    return _mel_fe
+
+
+def load_wav(x, sr=None):
+    """path or float array (at `sr`, default 24 kHz) -> (mono float32 at 24 kHz, original sample rate)."""
+    if isinstance(x, (str, Path)):
+        import soundfile as sf
+        y, sr = sf.read(str(x), dtype="float32")
+    else:
+        y, sr = np.asarray(x, np.float32), (sr or SR)
+    y = y.mean(1) if y.ndim > 1 else y
+    if sr != SR:
+        import soxr
+        y = soxr.resample(y, sr, SR)
+    return np.asarray(y, np.float32), sr
+
+
+def save_wav(wav, path):
+    import soundfile as sf
+    sf.write(str(path), np.asarray(wav, np.float32), SR)

build/lib/steertts/cli.py

@@ -0,0 +1,157 @@
+"""steertts command line.
+
+  steertts say "Hello there." -o hello.wav                                    # default voice, default style
+  steertts say "I can't believe it." --voice voice_1361 --tag anger --ctrl loud=1.5,f0_med=1 -o angry.wav
+  steertts say "Keep it down." --prompt "whispering and slow" --show-request -o soft.wav
+  steertts say "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" --markup -o m.wav
+  steertts say "Now in your voice." --ref-wav me.wav -o clone.wav
+  steertts say --request req.json -o out.wav                                  # a JSON request (request.schema.json)
+  steertts stream "First sentence. Second sentence." -o s.wav                 # prints per-chunk timing
+  steertts voices [--gender female] | steertts tags | steertts schema | steertts prompt "soft and sad"
+  steertts measure out.wav --text "..." --voice voice_1361                    # read the 12 controls back (z)
+
+The model directory comes from --model, else $STEERTTS_MODEL, else the current directory if it holds config.json.
+"""
+import argparse
+import json
+import os
+import sys
+import time
+
+
+def _model_dir(a):
+    if a.model:
+        return a.model
+    if os.environ.get("STEERTTS_MODEL"):
+        return os.environ["STEERTTS_MODEL"]
+    if os.path.exists("config.json") and os.path.exists("voices.json"):
+        return "."
+    sys.exit("steertts: no model directory; pass --model DIR (or an HF repo id) or set STEERTTS_MODEL")
+
+
+def _ctrl(s):
+    out = {}
+    for kv in filter(None, (s or "").split(",")):
+        k, _, v = kv.partition("=")
+        out[k.strip()] = float(v)
+    return out
+
+
+def _load(a, **kw):
+    from . import SteerTTS
+    t0 = time.time()
+    tts = SteerTTS.from_pretrained(_model_dir(a), device=a.device, flow=not a.no_flow, **kw)
+    print(f"loaded {tts} in {time.time() - t0:.1f}s", file=sys.stderr)
+    return tts
+
+
+def _request(a):
+    req = json.load(open(a.request)) if a.request else {}
+    for k, v in dict(text=a.text, voice=a.voice, ref_wav=a.ref_wav, prompt=a.prompt, tag=a.tag,
+                     pace=a.pace, guidance=a.guidance, seed=a.seed).items():
+        if v is not None:
+            req[k] = v
+    if a.ctrl:
+        req["ctrl"] = {**req.get("ctrl", {}), **_ctrl(a.ctrl)}
+    if getattr(a, "markup", False):
+        req["markup"] = True
+    if a.no_flow:
+        req["flow"] = False
+    return req
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog="steertts", description="SteerTTS: steerable English TTS")
+    ap.add_argument("--model", help="model directory or HF repo id (default: $STEERTTS_MODEL or .)")
+    ap.add_argument("--device", help="cpu | cuda (default: cuda if available)")
+    ap.add_argument("--no-flow", action="store_true", help="skip the flow-matching refiner")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    for name in ("say", "stream"):
+        p = sub.add_parser(name)
+        p.add_argument("text", nargs="?")
+        p.add_argument("--request", help="JSON request file (fields as in request.schema.json)")
+        p.add_argument("--voice")
+        p.add_argument("--ref-wav", dest="ref_wav", help="clone this reference clip (consent required)")
+        p.add_argument("--prompt", help="free-text direction, e.g. 'soft and sad'")
+        p.add_argument("--tag")
+        p.add_argument("--ctrl", help="comma list dim=z, e.g. rate=1.5,f0_med=-1")
+        p.add_argument("--pace", type=float)
+        p.add_argument("--guidance", type=float)
+        p.add_argument("--seed", type=int)
+        p.add_argument("--markup", action="store_true")
+        p.add_argument("--show-request", action="store_true", help="print the resolved request JSON")
+        p.add_argument("-o", "--out", default="out.wav")
+    p = sub.add_parser("voices")
+    p.add_argument("--gender", choices=["female", "male"])
+    p.add_argument("--json", action="store_true")
+    sub.add_parser("tags")
+    sub.add_parser("schema")
+    p = sub.add_parser("prompt")
+    p.add_argument("prompt")
+    p = sub.add_parser("measure")
+    p.add_argument("wav")
+    p.add_argument("--text")
+    p.add_argument("--voice")
+    a = ap.parse_args(argv)
+
+    if a.cmd == "schema":
+        from .request import schema
+        print(json.dumps(schema(), indent=1))
+        return
+    if a.cmd in ("voices", "tags"):
+        from .pipeline import resolve_model_dir
+        d = resolve_model_dir(_model_dir(a))
+        if a.cmd == "tags":
+            print(" ".join(json.load(open(d / "control_stats.json"))["tags"]))
+            return
+        vs = json.load(open(d / "voices.json"))["voices"]
+        vs = [v for v in vs if not a.gender or v.get("gender") == a.gender]
+        if a.json:
+            print(json.dumps(vs, indent=1))
+        else:
+            for v in vs:
+                print(f"{v['name']}  {v['description']}")
+        return
+    if a.cmd == "measure":
+        from .audio import load_wav
+        from .measure import measure, measure_z, n_phones
+        from .pipeline import resolve_model_dir
+        y, _ = load_wav(a.wav)
+        if a.voice:
+            st = json.load(open(resolve_model_dir(_model_dir(a)) / "control_stats.json"))
+            z = measure_z(y, a.text, st["spk_mean"][a.voice], st["pooled_sd"])
+            print(json.dumps({k: round(v, 2) for k, v in z.items()}))
+        else:
+            from .config import CONTROL_DIMS
+            raw = measure(y, n_phones=n_phones(a.text) if a.text else None, want_mel=False)["controls"]
+            print(json.dumps({k: round(float(v), 3) for k, v in zip(CONTROL_DIMS, raw)}))
+        return
+    tts = _load(a)
+    if a.cmd == "prompt":
+        print(json.dumps(tts.map_prompt(a.prompt), indent=1))
+        return
+    req = _request(a)
+    if not req.get("text"):
+        sys.exit("steertts: no text (positional argument or --request)")
+    req = tts.resolve(req)
+    if getattr(a, "show_request", False) or "prompt_mapping" in req:
+        print(json.dumps(req, indent=1), file=sys.stderr)
+    import numpy as np
+    if a.cmd == "say":
+        t0 = time.time()
+        wav = tts.synthesize(req)
+        dt = time.time() - t0
+    else:
+        t0, chunks = time.time(), []
+        for i, c in enumerate(tts.stream(request=req)):
+            chunks.append(c.audio)
+            print(f"chunk {i}: {len(c.audio) / tts.sample_rate:.2f}s audio at +{(time.time() - t0) * 1000:.0f} ms "
+                  f"({c.segment.text or ('pause %d ms' % c.segment.pause_ms)!r})", file=sys.stderr)
+        wav = np.concatenate(chunks) if chunks else np.zeros(1, np.float32)
+        dt = time.time() - t0
+    tts.save(wav, a.out)
+    print(f"{a.out}: {len(wav) / tts.sample_rate:.2f}s audio in {dt:.2f}s", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

build/lib/steertts/config.py

@@ -0,0 +1,27 @@
+"""Audio / control constants. They must match training: the charactr/vocos-mel-24khz mel front end."""
+SR = 24000
+N_FFT = 1024
+HOP = 256
+N_MELS = 100
+FPS = SR / HOP  # 93.75 mel frames per second
+PITCH_SCALE = 3.0  # predicted token pitch is semitones re the speaker median / 3; word_shift pitch is given in st
+
+# Utterance-level controls. Each is a SPEAKER-RELATIVE z-score: 0 = this voice's own neutral speech, +-1 = one
+# pooled within-speaker standard deviation (pooled_sd in control_stats.json gives the raw unit per dim).
+CONTROL_DIMS = [
+    "rate",      # articulation rate, phones per second of speech (pauses excluded)
+    "loud",      # mean RMS level of speech frames, dB
+    "f0_med",    # median F0, semitones
+    "f0_iqr",    # F0 interquartile range, semitones (pitch range / liveliness)
+    "f0_end",    # F0 slope over the final 300 ms of voicing, st/s (+ rising, question-like)
+    "pause",     # share of the speech span that is silent pauses >= 100 ms
+    "tilt",      # alpha ratio: energy 1-5 kHz vs 50 Hz-1 kHz, dB (+ brighter / pressed)
+    "voiced",    # voiced share of speech frames (whisper -> low)
+    "flat",      # spectral flatness of speech frames, log (breathy / whisper -> high)
+    "centroid",  # spectral centroid of speech frames, log Hz (brightness)
+    "cpps",      # smoothed cepstral peak prominence, dB (periodicity; breathy / tense -> low)
+    "h1h2",      # H1-H2, dB (breathy -> high)
+]
+CONTROL_RANGE = (-5.0, 5.0)  # accepted request range; training labels rarely exceed +-3
+
+DEFAULT_VOICE = "voice_1361"  # pre-registered default (chosen before any test-set voice sweep)

build/lib/steertts/flow.py

@@ -0,0 +1,278 @@
+"""Flow-matching mel decoders (optional), inference subset of recipes/steer_tts/steer/flow.py.
+
+refiner (flow.safetensors): a conditional flow (OT-CFM, as the Matcha-TTS decoder) that samples a detailed mel
+    given the acoustic model's mel mu and its conditioning g. Durations, pitch, energy, the coarse spectrum and
+    every control still come from the acoustic model; the flow adds texture (sharper harmonics / noise).
+prompted flow (pflow.safetensors): the same with an audio prompt - the reference clip's own mel is given as
+    context before the target (Voicebox / F5-TTS-style infilling). Used only for zero-shot cloning.
+"""
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .config import N_MELS
+
+SIGMA = 1e-4
+
+class TimeEmb(nn.Module):
+    def __init__(self, d):
+        super().__init__()
+        self.d = d
+        self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.SiLU(), nn.Linear(4 * d, d))
+
+    def forward(self, t):  # [B] in [0, 1]
+        half = self.d // 2
+        f = torch.exp(-math.log(10000) * torch.arange(half, device=t.device) / half)
+        a = 1000 * t[:, None] * f[None]
+        return self.mlp(torch.cat([a.sin(), a.cos()], -1))
+
+
+class Block(nn.Module):
+    """ConvNeXt block (depthwise dilated conv -> LN -> MLP) with AdaLN-zero conditioning on (t, g)."""
+
+    def __init__(self, d, c, k=7, dil=1, mult=3):
+        super().__init__()
+        self.dw = nn.Conv1d(d, d, k, padding=dil * (k // 2), dilation=dil, groups=d)
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.pw1, self.pw2 = nn.Linear(d, mult * d), nn.Linear(mult * d, d)
+
+    def forward(self, x, m, c):  # x [B,d,T], m [B,1,T], c [B,c]
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.dw(x * m).transpose(1, 2)
+        h = self.ln(h) * (1 + sc) + sh
+        h = self.pw2(F.gelu(self.pw1(h))) * gate
+        return x + h.transpose(1, 2)
+
+
+class AttnBlock(nn.Module):
+    """Full self-attention over the sequence with AdaLN-zero conditioning: lets the audio prompt (prompted flow)
+    reach every target frame, beyond the conv receptive field (~170 frames each side)."""
+
+    def __init__(self, d, c, heads=6):
+        super().__init__()
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.att = nn.MultiheadAttention(d, heads, batch_first=True)
+
+    def forward(self, x, m, c):
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.ln(x.transpose(1, 2)) * (1 + sc) + sh
+        h, _ = self.att(h, h, h, key_padding_mask=~m[:, 0].bool(), need_weights=False)
+        return x + (h * gate).transpose(1, 2) * m
+
+
+def rope(x):  # x [B,H,T,dh]: rotary position embedding (Su et al. 2021) over the frame axis
+    T, dh = x.shape[-2], x.shape[-1]
+    f = torch.exp(-math.log(10000) * torch.arange(0, dh, 2, device=x.device, dtype=torch.float32) / dh)
+    a = torch.arange(T, device=x.device, dtype=torch.float32)[:, None] * f[None]
+    cos, sin = a.cos().to(x.dtype), a.sin().to(x.dtype)
+    x1, x2 = x[..., 0::2], x[..., 1::2]
+    return torch.stack([x1 * cos - x2 * sin, x1 * sin + x2 * cos], -1).flatten(-2)
+
+
+class PromptPool(nn.Module):
+    """Perceiver-resampler pooling of the prompt (XTTS; SupertonicTTS's learnable timbre queries): n learnable
+    queries cross-attend to the encoded prompt frames -> n global voice tokens. A learned null key is always
+    present, so an item without a prompt still gets defined (prompt-free) tokens."""
+
+    def __init__(self, d, n, heads=6, layers=2):
+        super().__init__()
+        self.q = nn.Parameter(torch.randn(n, d) * 0.02)
+        self.null = nn.Parameter(torch.zeros(1, 1, d))
+        self.heads = heads
+        self.layers = nn.ModuleList([nn.ModuleDict(dict(
+            lq=nn.LayerNorm(d), lk=nn.LayerNorm(d), q=nn.Linear(d, d), kv=nn.Linear(d, 2 * d), o=nn.Linear(d, d),
+            lf=nn.LayerNorm(d), f=nn.Sequential(nn.Linear(d, 2 * d), nn.GELU(), nn.Linear(2 * d, d))))
+            for _ in range(layers)])
+
+    def forward(self, e, cm):  # e [B,d,T] encoded prompt (zero off-prompt), cm [B,1,T]
+        B, d, T = e.shape
+        kv_in = torch.cat([self.null.expand(B, 1, d), e.transpose(1, 2)], 1)
+        keep = torch.cat([torch.ones(B, 1, device=e.device, dtype=torch.bool), cm[:, 0].bool()], 1)[:, None, None]
+        q = self.q[None].expand(B, -1, -1)
+        H = self.heads
+        for l in self.layers:
+            k, v = l["kv"](l["lk"](kv_in)).chunk(2, -1)
+            qq = l["q"](l["lq"](q))
+            sp = lambda z: z.view(B, -1, H, d // H).transpose(1, 2)
+            h = F.scaled_dot_product_attention(sp(qq), sp(k), sp(v), attn_mask=keep)
+            q = q + l["o"](h.transpose(1, 2).reshape(B, -1, d))
+            q = q + l["f"](l["lf"](q))
+        return q  # [B,n,d]
+
+
+class RAttnBlock(nn.Module):
+    """Self-attention with RoPE positions (frames), plus optional cross-attention to the pooled prompt tokens; both
+    zero-gated (AdaLN-zero gate / zero output projection), so adding these blocks to a trained net changes nothing
+    at init."""
+
+    def __init__(self, d, c, heads=6, xattn=False):
+        super().__init__()
+        self.heads = heads
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.qkv, self.o = nn.Linear(d, 3 * d), nn.Linear(d, d)
+        self.xattn = xattn
+        if xattn:
+            self.lx = nn.LayerNorm(d)
+            self.xq, self.xkv, self.xo = nn.Linear(d, d), nn.Linear(d, 2 * d), nn.Linear(d, d)
+            nn.init.zeros_(self.xo.weight); nn.init.zeros_(self.xo.bias)
+
+    def forward(self, x, m, c, tok=None):  # x [B,d,T], tok [B,n,d]
+        B, d, T = x.shape
+        H = self.heads
+        sp = lambda z: z.view(B, -1, H, d // H).transpose(1, 2)
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.ln(x.transpose(1, 2)) * (1 + sc) + sh
+        q, k, v = (sp(z) for z in self.qkv(h).chunk(3, -1))
+        h = F.scaled_dot_product_attention(rope(q), rope(k), v, attn_mask=m[:, 0].bool()[:, None, None])
+        h = self.o(h.transpose(1, 2).reshape(B, T, d)) * gate
+        if self.xattn and tok is not None:
+            hx = self.lx(x.transpose(1, 2) + h)
+            k2, v2 = self.xkv(tok).chunk(2, -1)
+            h = h + self.xo(F.scaled_dot_product_attention(sp(self.xq(hx)), sp(k2), sp(v2)).transpose(1, 2).reshape(B, T, d))
+        return x + h.transpose(1, 2) * m
+
+
+class FlowNet(nn.Module):
+    """prompt=True (prompted flow): also takes a context mel ctx with mask cm (1 = real audio given: the reference
+    clip's frames), as Voicebox / F5-TTS infilling. The context enters the input stack, a masked mean of it is added
+    to the conditioning (global voice), and attention layers at attn_at carry it across the whole sequence.
+
+    v5 additions (all zero-initialised, so a v4 checkpoint loads into a v5 net unchanged):
+      rope_at   layers that get an extra RoPE self-attention block (RAttnBlock)
+      n_ptok    pooled prompt tokens (PromptPool) that the RoPE blocks cross-attend to
+      skips     E2-TTS-style long skips, block i -> block n-1-i
+      align_at  layers whose target-frame mean is mapped to a speaker-verification vector (TLA-SA-style feature
+                alignment loss; training only)"""
+
+    def __init__(self, n_mel=N_MELS, d=384, n_layers=12, g_dim=128, dils=(1, 2, 4, 1, 2, 4, 1, 2, 4, 1, 2, 4),
+                 prompt=False, attn_at=(3, 7, 11), rope_at=(), n_ptok=0, skips=False, align_at=(), align_dim=512):
+        super().__init__()
+        self.cfg = dict(n_mel=n_mel, d=d, n_layers=n_layers, g_dim=g_dim, dils=tuple(dils), prompt=prompt,
+                        attn_at=tuple(attn_at), rope_at=tuple(rope_at), n_ptok=n_ptok, skips=skips,
+                        align_at=tuple(align_at), align_dim=align_dim)
+        self.prompt = prompt
+        self.inp = nn.Conv1d((3 * n_mel + 1) if prompt else 2 * n_mel, d, 3, padding=1)
+        self.temb = TimeEmb(d)
+        self.gemb = nn.Sequential(nn.Linear(g_dim, d), nn.SiLU(), nn.Linear(d, d))
+        self.blocks = nn.ModuleList([Block(d, d, dil=dils[i % len(dils)]) for i in range(n_layers)])
+        if prompt:
+            self.attn = nn.ModuleDict({str(i): AttnBlock(d, d) for i in attn_at})
+            self.ctx_enc = nn.Sequential(nn.Conv1d(n_mel, d, 3, padding=1), nn.SiLU(), nn.Conv1d(d, d, 3, padding=1))
+            self.ctx_out = nn.Linear(d, d)
+            nn.init.zeros_(self.ctx_out.weight); nn.init.zeros_(self.ctx_out.bias)
+            self.rattn = nn.ModuleDict({str(i): RAttnBlock(d, d, xattn=n_ptok > 0) for i in rope_at})
+            self.pool = PromptPool(d, n_ptok) if n_ptok else None
+        self.skips = skips
+        if skips:
+            self.skip = nn.ModuleDict({str(i): nn.Linear(d, d) for i in range(n_layers - n_layers // 2, n_layers)})
+            for l in self.skip.values():
+                nn.init.zeros_(l.weight); nn.init.zeros_(l.bias)
+        self.align_at = tuple(align_at)
+        if align_at:
+            self.align = nn.ModuleDict({str(i): nn.Sequential(nn.Linear(d, d), nn.SiLU(), nn.Linear(d, align_dim))
+                                        for i in align_at})
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.out = nn.Conv1d(d, n_mel, 1)
+        nn.init.zeros_(self.out.weight); nn.init.zeros_(self.out.bias)
+
+    def forward(self, x, t, mu, g, m, ctx=None, cm=None, feats=False):  # normalised mels [B,100,T]; m, cm [B,1,T]
+        """feats=True also returns {layer: speaker vector predicted from the target-frame mean} for align_at."""
+        c = self.temb(t) + self.gemb(g)
+        tok = None
+        if self.prompt:
+            if cm is None:
+                ctx, cm = torch.zeros_like(x), torch.zeros_like(m)
+            ctx = ctx * cm
+            e = self.ctx_enc(ctx) * cm
+            c = c + self.ctx_out(e.sum(-1) / cm.sum(-1).clamp_min(1))
+            if getattr(self, "pool", None) is not None:
+                tok = self.pool(e, cm)
+            h = self.inp(torch.cat([x, mu, ctx, cm], 1) * m)
+        else:
+            h = self.inp(torch.cat([x, mu], 1) * m)
+        rattn = getattr(self, "rattn", {})
+        stack, out = [], {}
+        n = len(self.blocks)
+        for i, b in enumerate(self.blocks):
+            if self.skips and str(i) in self.skip:
+                h = h + self.skip[str(i)](stack.pop().transpose(1, 2)).transpose(1, 2) * m
+            h = b(h, m, c)
+            if self.prompt and str(i) in self.attn:
+                h = self.attn[str(i)](h, m, c)
+            if str(i) in rattn:
+                h = rattn[str(i)](h, m, c, tok)
+            if self.skips and i < n // 2:
+                stack.append(h)
+            if feats and i in self.align_at:
+                w = m * (1 - cm) if cm is not None else m
+                out[i] = self.align[str(i)](((h * w).sum(-1) / w.sum(-1).clamp_min(1)).float())
+        v = self.out(self.ln(h.transpose(1, 2)).transpose(1, 2)) * m
+        return (v, out) if feats else v
+
+
+class Flow(nn.Module):
+    """FlowNet + mel normalisation; sample() maps a base mel to a refined mel (log-mel units, same frames)."""
+
+    def __init__(self, mean=-5.0, std=2.5, mu_cmn=False, **kw):
+        """mu_cmn (prompted flow v2): remove the per-band time-mean of the base mel (its static spectral envelope:
+        the long-term voice colouring the base model guessed from a 512-d speaker vector) so the flow must take the
+        voice from the audio prompt (or g); timing / pitch / energy / phonetic detail of mu are kept. Without it, mu
+        made from the TRUE speaker vector already carries the training speaker's colouring and the prompt adds
+        little (v3p @20k: Seed-TTS SIM 31.2 -> 33.2 only)."""
+        super().__init__()
+        self.net = FlowNet(**kw)
+        self.mu_cmn = mu_cmn
+        self.register_buffer("mean", torch.tensor(float(mean)))
+        self.register_buffer("std", torch.tensor(float(std)))
+
+    def norm(self, m):
+        return (m - self.mean) / self.std
+
+    def _cmn(self, mu_n, w):
+        """mu_n minus its per-band mean over the frames w marks (the target frames)."""
+        if not self.mu_cmn:
+            return mu_n
+        return mu_n - (mu_n * w).sum(-1, keepdim=True) / w.sum(-1, keepdim=True).clamp_min(1)
+
+    @torch.no_grad()
+    def sample(self, mu, g, m=None, steps=10, temp=0.8, method="midpoint", seed=None, ctx=None, cm=None, cfg=0.0):
+        """ctx / cm (prompted flow): real mel on the context frames (log-mel units), mask 1 there; mu is replaced by
+        the context on those frames, as in training. Returns every frame (callers slice off the context).
+        cfg (prompted flow): classifier-free guidance on the prompt, v + cfg * (v - v_no_prompt); the no-prompt
+        velocity is the same net with the context removed (trained on p_none of the items)."""
+        m = torch.ones_like(mu[:, :1]) if m is None else m
+        gen = None if seed is None else torch.Generator(device=mu.device).manual_seed(seed)
+        mu_n = self._cmn(self.norm(mu), m if cm is None else m * (1 - cm))
+        if cm is not None:
+            ctx = self.norm(ctx) * cm
+            mu_n = torch.where(cm.bool(), ctx, mu_n)
+        x = temp * torch.randn(mu_n.shape, device=mu.device, generator=gen) * m
+        ts = torch.linspace(0, 1, steps + 1, device=mu.device)
+
+        def vel(x, t):
+            v = self.net(x, t.expand(x.shape[0]), mu_n, g, m, ctx, cm)
+            if cfg and cm is not None:
+                v = v + cfg * (v - self.net(x, t.expand(x.shape[0]), mu_n, g, m, None, None))
+            return v
+
+        for i in range(steps):
+            t0, dt = ts[i], ts[i + 1] - ts[i]
+            v = vel(x, t0)
+            if method == "midpoint":
+                v = vel(x + 0.5 * dt * v, t0 + 0.5 * dt)
+            x = x + dt * v
+        return x * self.std + self.mean
+
+    @staticmethod
+    def from_state(state, cfg, device="cpu"):
+        f = Flow(**cfg)
+        f.load_state_dict(state)
+        return f.to(device).eval()

build/lib/steertts/mamba3/__init__.py

@@ -0,0 +1,25 @@
+"""Mamba-3: a paper-faithful, verified implementation of Lahoti et al. (arXiv 2603.15569).
+
+Ported from the MMSE research repo (drowsiness-uta-rldd/mamba3, commit c5280e6+), without the MMSE
+multi-stream extension or the LM tooling. Checked fwd+bwd at float64 against the recurrence:
+  - exponential-trapezoidal discretization with DATA-DEPENDENT lambda_t (Prop 1)
+  - complex state as real block-diagonal rotations / data-dependent RoPE (Prop 2-4, eq 11)
+  - MIMO rank-R (Sec 3.3)
+  - BCNorm/QKNorm on B,C + learnable data-independent B,C biases
+  - output gating
+Recorded deviations: data-independent A (Remark 1 says equivalent); MIMO input uses a full DPR
+projection rather than the paper's DP+PR (R=1 identical).
+
+Paths: `Mamba3Layer.forward` (O(T) recurrence, reference), `scan_forward` (chunked scan),
+`scan_forward_ssd` (SSD reduction, the fast path for training). Run `python -m
+steertts.mamba3.tests` after any change.
+"""
+from .layer import Mamba3Layer, rotate_pairs, ssm_scan
+from .scan import scan_forward, chunked_scan, rotate_by, cumulative_angles
+from .ssd import scan_forward_ssd, ssd_inputs, ssd_pytorch
+
+__all__ = [
+    "Mamba3Layer", "rotate_pairs", "ssm_scan",
+    "scan_forward", "chunked_scan", "rotate_by", "cumulative_angles",
+    "scan_forward_ssd", "ssd_inputs", "ssd_pytorch",
+]

build/lib/steertts/mamba3/layer.py

@@ -0,0 +1,338 @@
+"""Mamba-3 (Lahoti et al., arXiv 2603.15569), implemented to the paper.
+
+THREE CHANGES FROM MAMBA-2, implemented here in the order the paper derives
+them. Nothing is improvised: where this file departs from the paper it says
+so, and the equivalence tests at the bottom are how that claim is checked.
+
+1. EXPONENTIAL-TRAPEZOIDAL DISCRETIZATION (Sec 3.1). Mamba-1/2 approximate
+
+        integral_{t_{k-1}}^{t_k} e^{A(t_k - tau)} B(tau) x(tau) d tau
+
+   by holding the endpoint (Euler). Mamba-3 averages both endpoints
+   (trapezoidal), which makes the recurrence depend on x_{t-1} as well as
+   x_t:
+
+        h_t = alpha_t h_{t-1} + beta_t B_{t-1} x_{t-1} + gamma_t B_t x_t
+
+   with alpha_t = exp(Delta_t A), beta_t = (1 - lambda_t) Delta_t alpha_t,
+   gamma_t = lambda_t Delta_t and a data-dependent lambda_t = sigmoid(.)
+   (lambda = 1/2 is the classical trapezoid, initialised here). That two-tap dependence IS the implicit convolution
+   the paper describes - it is why Mamba-3 can drop the short causal conv
+   that Mamba-1/2 carry as a separate component (Sec 3.1, and eq 7's
+   factorisation of L into a 1-semiseparable matrix times a 2-band matrix).
+
+2. COMPLEX STATE via the RoPE trick (Sec 3.2, Prop 2 and 3). A complex SSM
+   of state N/2 is equivalent to a REAL SSM of state N whose transition is a
+   scalar decay times a block-diagonal of 2x2 rotations:
+
+        R_t = blockdiag({ R(Delta_t theta_t[i]) })
+        h_t = alpha_t R_t h_{t-1} + ...
+
+   Real non-negative eigenvalues cannot express rotation, which is why
+   Mamba-2 provably fails state tracking (parity). This file implements the
+   real-equivalent form, not complex arithmetic, exactly as Prop 2 licenses.
+
+3. MIMO (Sec 3.3). x_t: (P,) -> (P,R), B_t: (N,) -> (N,R), C_t likewise;
+   the state stays (N,P) and the update becomes a matmul B_t x_t^T rather
+   than an outer product. Memory traffic is dominated by h_t and barely
+   moves; FLOPs rise by R. Arithmetic intensity goes Theta(1) -> Theta(R),
+   which is the whole point during memory-bound decode.
+
+WHAT THIS FILE DOES NOT DO. No fused CUDA scan - the paper's speed comes
+from hardware-aware kernels and this runs on MPS/CPU with an explicit loop
+for the recurrent path and a materialised mask for the parallel path. The
+MODELLING claims are reproducible here; the LATENCY claims are not.
+"""
+from __future__ import annotations
+
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+def rotate_pairs(h: torch.Tensor, ang: torch.Tensor) -> torch.Tensor:
+    """Apply blockdiag 2x2 rotations to the state's (even, odd) channel pairs.
+
+    h:   (..., N)   state, N even
+    ang: (..., N/2) rotation angle per pair
+    """
+    a, b = h[..., 0::2], h[..., 1::2]
+    c, s = torch.cos(ang), torch.sin(ang)
+    out = torch.empty_like(h)
+    out[..., 0::2] = c * a - s * b
+    out[..., 1::2] = s * a + c * b
+    return out
+
+
+def ssm_scan(alpha, beta, gamma, theta, S, Cm, P, state=None, prev_S=None):
+    """The Mamba-3 recurrence (paper eq 11), shared by every Mamba-3 path.
+
+    This is the SINGLE SOURCE OF TRUTH for the recurrence, so a paper-faithfulness
+    fix here (e.g. the beta-term rotation) applies everywhere and cannot drift.
+
+        h_t = R_t(alpha_t h_{t-1} + beta_t S_{t-1}) + gamma_t S_t
+        y_t = C_t^T h_t
+
+    alpha,beta,gamma: (B,T,H)      per-step scalars
+    theta:            (B,T,H,N/2)  per-step rotation angles (already * dt)
+    S:                (B,T,H,N,P)  per-step write B_t x_t^T (or summed streams)
+    Cm:               (B,T,H,N,R)  readout
+    P:                head dim; state/prev_S carry across chunks for streaming.
+    Returns (y_flat (B,T,H*R*P), final state h, last write S[:,-1]).
+    """
+    B, T, H, N, _ = S.shape
+    R = Cm.shape[-1]
+    h = torch.zeros(B, H, N, P, device=S.device, dtype=S.dtype) \
+        if state is None else state
+    ys = []
+    for t in range(T):
+        prev = S[:, t - 1] if t > 0 else prev_S
+        pre = alpha[:, t, :, None, None] * h
+        if prev is not None:
+            pre = pre + beta[:, t, :, None, None] * prev
+        h = rotate_pairs(pre.transpose(-1, -2), theta[:, t].unsqueeze(-2)
+                         .expand(-1, -1, P, -1)).transpose(-1, -2)
+        h = h + gamma[:, t, :, None, None] * S[:, t]
+        ys.append((Cm[:, t].transpose(-1, -2) @ h).reshape(B, H * R * P))
+    return torch.stack(ys, 1), h, S[:, -1]
+
+
+class Mamba3Layer(nn.Module):
+    """One Mamba-3 layer. Recurrent and parallel paths, provably equal.
+
+    d_model  input/output width
+    d_state  N, the SSM state size (even; pairs are rotated)
+    rank     R of the MIMO lift. rank=1 is SISO.
+    head_dim P, channels sharing one (alpha, Delta, B) recurrence.
+    """
+
+    def __init__(self, d_model: int, d_state: int = 64, rank: int = 1,
+                 head_dim: int = 32, dt_min: float = 1e-3, dt_max: float = 0.5):
+        super().__init__()
+        assert d_state % 2 == 0, "d_state must be even: the state rotates in pairs"
+        assert d_model % head_dim == 0, "d_model must divide into heads of head_dim"
+        self.d_model, self.N, self.R, self.P = d_model, d_state, rank, head_dim
+        self.H = d_model // head_dim
+
+        # Gate branch (Mamba-2/3 block, Fig 2): the SSM output is multiplied by
+        # SiLU(in_proj(input)) before out_proj. Dimension d_model*R to match the
+        # pre-out_proj SSM output (H*R*P). Was defined-but-unused before.
+        self.in_proj = nn.Linear(d_model, d_model * rank)
+        self.B_proj = nn.Linear(d_model, self.H * d_state * rank)
+        self.C_proj = nn.Linear(d_model, self.H * d_state * rank)
+        self.x_proj = nn.Linear(d_model, d_model * rank)
+        self.dt_proj = nn.Linear(d_model, self.H)
+        self.theta_proj = nn.Linear(d_model, self.H * (d_state // 2))
+        # lambda_t: the DATA-DEPENDENT trapezoidal weight (paper Prop 1, Remark
+        # 3). gamma_t = lambda_t * dt, beta_t = (1-lambda_t) * dt * alpha_t.
+        # lambda=1/2 is the classical trapezoid; the paper found NOT enforcing
+        # that (letting the model choose per step) is empirically better. Bias 0
+        # -> sigmoid 0.5 at init, so training starts from the classical rule.
+        self.lam_proj = nn.Linear(d_model, self.H)
+        nn.init.zeros_(self.lam_proj.weight)
+        nn.init.zeros_(self.lam_proj.bias)
+        self.out_proj = nn.Linear(d_model * rank, d_model)
+        self.norm = nn.RMSNorm(d_model) if hasattr(nn, "RMSNorm") else nn.LayerNorm(d_model)
+
+        # BCNorm / QKNorm (paper section 3.2). RMS normalisation AFTER the B and C
+        # projections, mirroring QKNorm in Transformers. The paper reports this
+        # is what stabilises large-scale runs, to the point that they remove
+        # Mamba-2's post-gate RMSNorm on the strength of it. It was missing from
+        # the first version of this file and E17's ssm arm then diverged
+        # (0.330 -> 19.313) into a constant predictor, which was patched with
+        # gradient clipping - a generic fix for a problem the paper had already
+        # solved specifically. Normalising over the N state dimensions, which is
+        # the axis B and C index into.
+        rms = nn.RMSNorm if hasattr(nn, "RMSNorm") else nn.LayerNorm
+        self.B_norm = rms(d_state)
+        self.C_norm = rms(d_state)
+        # B, C biases (paper appendix): learnable, head-specific, channel-wise,
+        # data-INDEPENDENT, added after the norm, initialised to all ones. The
+        # paper finds models insensitive to the init as long as it is positive,
+        # and that B and C biases together beat either alone.
+        self.B_bias = nn.Parameter(torch.ones(self.H, d_state, rank))
+        self.C_bias = nn.Parameter(torch.ones(self.H, d_state, rank))
+
+        # A is data-INDEPENDENT here. Remark 1: the paper found data-dependent
+        # A_t performs similarly and chose it for consistency; the simpler
+        # form is used here and the difference is recorded rather than hidden.
+        self.A_log = nn.Parameter(torch.log(torch.rand(self.H) * 0.9 + 0.05))
+        # dt bias initialised so Delta spans [dt_min, dt_max] - the standard
+        # Mamba init. Delta sets the memory horizon, so this is the prior on
+        # how long the layer remembers before it learns anything.
+        dt = torch.exp(torch.rand(self.H) * (math.log(dt_max) - math.log(dt_min))
+                       + math.log(dt_min))
+        self.dt_proj.bias.data = dt + torch.log(-torch.expm1(-dt))
+
+    def _coeffs(self, u: torch.Tensor):
+        """alpha, beta, gamma, B, C, x, theta for every timestep."""
+        Bsz, T, _ = u.shape
+        H, N, R, P = self.H, self.N, self.R, self.P
+        dt = F.softplus(self.dt_proj(u))                      # (B,T,H) > 0
+        A = -torch.exp(self.A_log)                            # (H,) < 0
+        alpha = torch.exp(dt * A)                             # (B,T,H) in (0,1)
+        # Exponential-trapezoidal with a DATA-DEPENDENT weight lambda_t (paper
+        # Prop 1 / eq 5-6): gamma_t = lambda_t*dt (current input), beta_t =
+        # (1-lambda_t)*dt*alpha_t (previous input). lambda=1/2 recovers the
+        # classical trapezoid; lambda=1 recovers Mamba-2's exponential-Euler.
+        lam = torch.sigmoid(self.lam_proj(u))                 # (B,T,H) in (0,1)
+        gamma = lam * dt
+        beta = (1.0 - lam) * dt * alpha
+        # BCNorm then bias, in that order - the paper adds the bias AFTER the
+        # norm, so the bias is not rescaled by it.
+        Bm = self.B_proj(u).view(Bsz, T, H, R, N)
+        Bm = self.B_norm(Bm).permute(0, 1, 2, 4, 3) + self.B_bias
+        Cm = self.C_proj(u).view(Bsz, T, H, R, N)
+        Cm = self.C_norm(Cm).permute(0, 1, 2, 4, 3) + self.C_bias
+        x = self.x_proj(u).view(Bsz, T, H, P, R)
+        theta = self.theta_proj(u).view(Bsz, T, H, N // 2) * dt.unsqueeze(-1)
+        return alpha, beta, gamma, Bm, Cm, x, theta
+
+    def gate_and_proj(self, y_flat: torch.Tensor, nu: torch.Tensor):
+        """Apply the Mamba gate then the output projection.
+
+        y_flat: (..., d_model*R) SSM output before out_proj.
+        nu:     (..., d_model)   the NORMALISED layer input (in_proj sees the
+                                 same normalised input the SSM path does).
+        out = out_proj( y_flat * SiLU(in_proj(nu)) ).
+        """
+        return self.out_proj(y_flat * F.silu(self.in_proj(nu)))
+
+    def forward(self, u: torch.Tensor, state=None, return_state=False):
+        """Recurrent path. u: (B,T,d_model). This is the streaming form.
+
+        The recurrence itself lives in the shared `ssm_scan` (paper eq 11), so
+        callers cannot drift. Here the per-step write is the
+        single-stream outer product S_t = B_t x_t^T.
+
+        state: None or the (h, S_last) pair returned by a previous call with
+        return_state=True. Segment-wise streaming must carry S_last as well as
+        h: the first step of the next segment needs beta_t B_{t-1} x_{t-1}^T,
+        the trapezoidal term, and carrying h alone silently drops it.
+        """
+        nu = self.norm(u)
+        alpha, beta, gamma, Bm, Cm, x, theta = self._coeffs(nu)
+        S = Bm @ x.transpose(-1, -2)                    # (B,T,H,N,P)
+        h0, prev_S = (None, None) if state is None else state
+        y_flat, h, S_last = ssm_scan(alpha, beta, gamma, theta, S, Cm, self.P,
+                                     state=h0, prev_S=prev_S)
+        out = self.gate_and_proj(y_flat, nu)
+        return (out, (h, S_last)) if return_state else out
+
+    def step(self, u_t: torch.Tensor, state, prev_bx=None):
+        """One timestep, constant memory. The deployment path.
+
+        state: (B,H,N,P) or None. prev_bx: B_{t-1}x_{t-1}^T for the beta term,
+        which the trapezoidal rule requires and Mamba-2 does not have.
+        """
+        Bsz = u_t.shape[0]
+        nu = self.norm(u_t.unsqueeze(1))
+        alpha, beta, gamma, Bm, Cm, x, theta = self._coeffs(nu)
+        H, N, P = self.H, self.N, self.P
+        h = torch.zeros(Bsz, H, N, P, device=u_t.device, dtype=u_t.dtype) \
+            if state is None else state
+        # Match forward/eq(11): rotate (alpha*h + beta*prev_bx) by R_t, then add
+        # the current (unrotated) write. prev_bx = B_{t-1}x_{t-1}^T from the last
+        # step; bx = B_t x_t^T is returned for the next step's beta term.
+        pre = alpha[:, 0, :, None, None] * h
+        if prev_bx is not None:
+            pre = pre + beta[:, 0, :, None, None] * prev_bx
+        h = rotate_pairs(pre.transpose(-1, -2), theta[:, 0].unsqueeze(-2)
+                         .expand(-1, -1, P, -1)).transpose(-1, -2)
+        bx = Bm[:, 0] @ x[:, 0].transpose(-1, -2)
+        h = h + gamma[:, 0, :, None, None] * bx
+        y = (Cm[:, 0].transpose(-1, -2) @ h).reshape(Bsz, -1)
+        return self.gate_and_proj(y, nu[:, 0]), h, bx
+
+
+# ---------------------------------------------------------------------------
+def selftest(verbose: bool = True) -> dict:
+    """The four invariants, at float64.
+
+    These were an ad-hoc scratch script until BCNorm was added and there was
+    nothing to re-run. A verified file whose verification lives somewhere else
+    is an unverified file the moment anyone edits it.
+
+        python -m steertts.mamba3.layer
+    """
+    torch.manual_seed(0)
+    torch.set_default_dtype(torch.float64)
+    net = Mamba3Layer(d_model=32, d_state=16, rank=2, head_dim=8).eval()
+    B, T = 2, 20
+    u = torch.randn(B, T, 32)
+    out = {}
+
+    with torch.no_grad():
+        y = net(u)
+
+        # 1. CAUSALITY, as a SCALE test rather than an equality one. Appending
+        #    future timesteps moves the past by ~1e-15 here: the permute in
+        #    _coeffs leaves B and C non-contiguous, so matmul takes a different
+        #    kernel path at a different T and the last bits differ. An equality
+        #    assertion would call that a leak, and would also fail spuriously in
+        #    float32. A real leak scales with the size of the future; rounding
+        #    does not. Drive the future through four orders of magnitude and
+        #    require the drift to stay put.
+        drifts = []
+        for mag in (1.0, 1e2, 1e4):
+            u2 = torch.cat([u, torch.randn(B, 6, 32) * mag], 1)
+            drifts.append(float((net(u2)[:, :T] - y).abs().max()))
+        out["causality"] = max(drifts)
+        out["causality_scaling"] = max(drifts) / max(min(drifts), 1e-300)
+
+        # 2. STREAMING == BATCHED. The deployment path must equal training.
+        st = bx = None
+        steps = []
+        for t in range(T):
+            o, st, bx = net.step(u[:, t], st, bx)
+            steps.append(o)
+        out["streaming"] = float((torch.stack(steps, 1) - y).abs().max())
+
+        # 3. ROTATION IS ORTHOGONAL. State norm preserved, so it cannot blow up.
+        h = torch.randn(B, 4, 16, 8)
+        ang = torch.randn(B, 4, 1, 8)
+        hr = rotate_pairs(h.transpose(-1, -2), ang).transpose(-1, -2)
+        out["orthogonal"] = float(
+            (hr.norm(dim=-2) - h.norm(dim=-2)).abs().max())
+
+        # 4. TRAPEZOIDAL TWO-TAP. Perturbing x_{t-1} moves y_t, not y_{t-2}.
+        up = u.clone(); up[:, 3] += 1.0
+        yp = net(up)
+        out["twotap_y4"] = float((yp[:, 4] - y[:, 4]).abs().max())
+        out["twotap_y2"] = float((yp[:, 2] - y[:, 2]).abs().max())
+
+        # 5. BCNORM IS ACTIVE. B's pre-bias rows should have unit RMS. Without
+        #    the norm this is whatever the projection happens to produce, and
+        #    the first version of this file had no norm at all.
+        a, b, g, Bm, Cm, x, th = net._coeffs(net.norm(u))
+        #    Bm is (B,T,H,N,R) after the permute, so the normalised axis is -2.
+        rms = (Bm - net.B_bias).pow(2).mean(-2).sqrt()
+        out["bcnorm_rms"] = float((rms - 1.0).abs().max())
+
+    out["params"] = sum(p.numel() for p in net.parameters())
+    torch.set_default_dtype(torch.float32)
+
+    if verbose:
+        ok = lambda c: "OK" if c else "FAIL"
+        print("Mamba-3 selftest  (float64)")
+        print(f"  1 causality (future->past)   {out['causality']:.3e}"
+              f"   {ok(out['causality'] < 1e-12)}")
+        print(f"    drift x1 vs x10000 ratio   {out['causality_scaling']:.3f}"
+              f"   {ok(out['causality_scaling'] < 1.5)}  (a leak would scale)")
+        print(f"  2 step() == forward()        {out['streaming']:.3e}"
+              f"   {ok(out['streaming'] < 1e-10)}")
+        print(f"  3 rotation orthogonal        {out['orthogonal']:.3e}"
+              f"   {ok(out['orthogonal'] < 1e-10)}")
+        print(f"  4 trapezoidal two-tap        y4 {out['twotap_y4']:.3e}"
+              f"  y2 {out['twotap_y2']:.3e}"
+              f"   {ok(out['twotap_y4'] > 1e-6 and out['twotap_y2'] == 0.0)}")
+        print(f"  5 BCNorm rows unit RMS       {out['bcnorm_rms']:.3e}"
+              f"   {ok(out['bcnorm_rms'] < 1e-10)}")
+        print(f"  {out['params']:,} params")
+    return out
+
+
+if __name__ == "__main__":
+    selftest()

build/lib/steertts/mamba3/reference.py

@@ -0,0 +1,98 @@
+"""Independent Mamba-3 references, written from the paper rather than from this package.
+
+Nothing here calls Mamba3Layer._coeffs, ssm_scan or rotate_by: every coefficient is recomputed from
+the raw parameters with the paper's definitions, so a coefficient bug in layer.py cannot hide by being
+shared with the thing that checks it (tests.paper_eq11_check reuses _coeffs and so cannot catch one).
+
+    ref_complex   the complex-valued SSM itself (Prop 2; App B.3 eq 24), in complex128, MIMO as an
+                  explicit sum over ranks (eq 12-14). A sequential loop, the most literal form.
+    ref_parallel  the SSD parallel form, eq (7): L = (1-semiseparable decay) @ (2-band beta/gamma),
+                  applied to cumulative-RoPE'd B and C (Prop 3/4). Quadratic in T.
+
+The two are derived independently (complex arithmetic vs real rotations + masks) and agree to ~1e-15,
+so each layer path is checked against two references that do not share code with it or each other.
+Both are float64 and slow; they are for tests only.
+"""
+from __future__ import annotations
+
+import torch
+import torch.nn.functional as F
+
+
+def _rms(x, w, eps):
+    eps = torch.finfo(x.dtype).eps if eps is None else eps
+    return x * torch.rsqrt(x.pow(2).mean(-1, keepdim=True) + eps) * w
+
+
+def raw_coeffs(L, u):
+    """Every per-step quantity of a Mamba3Layer, from raw parameters (float64)."""
+    u = u.double()
+    Bsz, T, _ = u.shape
+    H, N, R, P = L.H, L.N, L.R, L.P
+    nu = _rms(u, L.norm.weight, L.norm.eps)
+    dt = F.softplus(F.linear(nu, L.dt_proj.weight, L.dt_proj.bias))              # (B,T,H)
+    A = -torch.exp(L.A_log)                                                      # (H,)
+    lam = torch.sigmoid(F.linear(nu, L.lam_proj.weight, L.lam_proj.bias))       # trapezoid weight
+    Bn = _rms(F.linear(nu, L.B_proj.weight, L.B_proj.bias).view(Bsz, T, H, R, N),
+              L.B_norm.weight, L.B_norm.eps) + L.B_bias.permute(0, 2, 1)         # (B,T,H,R,N)
+    Cn = _rms(F.linear(nu, L.C_proj.weight, L.C_proj.bias).view(Bsz, T, H, R, N),
+              L.C_norm.weight, L.C_norm.eps) + L.C_bias.permute(0, 2, 1)
+    x = F.linear(nu, L.x_proj.weight, L.x_proj.bias).view(Bsz, T, H, P, R)
+    th = F.linear(nu, L.theta_proj.weight, L.theta_proj.bias).view(Bsz, T, H, N // 2)
+    z = F.linear(nu, L.in_proj.weight, L.in_proj.bias)                          # gate pre-activation
+    return dt, A, lam, Bn, Cn, x, th, z
+
+
+def _finish(L, y, z):  # y (B,T,H,R,P) -> gate, out_proj
+    Bsz, T = y.shape[:2]
+    return F.linear(y.reshape(Bsz, T, -1) * F.silu(z), L.out_proj.weight, L.out_proj.bias)
+
+
+def ref_complex(L, u):
+    """h_t = e^{dt A} e^{i dt theta} h_{t-1} + beta_t e^{i dt theta} W_{t-1} + gamma_t W_t,
+    W_t = sum_r B_t^(r) x_t^(r)T,  y_t^(r) = Re(conj(C_t^(r)) . h_t),  state of N/2 complex numbers."""
+    dt, A, lam, Bn, Cn, x, th, z = raw_coeffs(L, u)
+    Bsz, T, H, R, N = Bn.shape
+    Bc = torch.complex(Bn[..., 0::2], Bn[..., 1::2])                             # (B,T,H,R,N/2)
+    Cc = torch.complex(Cn[..., 0::2], Cn[..., 1::2])
+    h = torch.zeros(Bsz, H, N // 2, L.P, dtype=torch.complex128, device=u.device)
+    Wprev = torch.zeros_like(h)
+    ys = []
+    for t in range(T):
+        d = dt[:, t]
+        alpha = torch.exp(d * A)
+        rot = torch.exp(1j * (d[..., None] * th[:, t]))[..., None]               # (B,H,N/2,1)
+        W = torch.einsum('bhrk,bhpr->bhkp', Bc[:, t], x[:, t].to(Bc.dtype))
+        beta = ((1 - lam[:, t]) * d * alpha)[..., None, None]
+        gamma = (lam[:, t] * d)[..., None, None]
+        h = alpha[..., None, None] * rot * h + beta * rot * Wprev + gamma * W
+        Wprev = W
+        ys.append(torch.einsum('bhrk,bhkp->bhrp', Cc[:, t].conj(), h).real)
+    return _finish(L, torch.stack(ys, 1), z)
+
+
+def ref_parallel(L, u):
+    """eq (7): y = (decay @ band) * (C~ B~^T) x with C~, B~ rotated by the cumulative angle."""
+    dt, A, lam, Bn, Cn, x, th, z = raw_coeffs(L, u)
+    Bsz, T, H, R, N = Bn.shape
+    ang = torch.cumsum(dt[..., None] * th, 1)[:, :, :, None]                    # Phi_t (B,T,H,1,N/2)
+    c, s = torch.cos(ang), torch.sin(ang)
+
+    def rope_T(v):  # (prod_{i<=t} R_i)^T v, i.e. rotate each pair by -Phi_t
+        a, b = v[..., 0::2], v[..., 1::2]
+        return torch.stack((c * a + s * b, -s * a + c * b), -1).flatten(-2)
+    Bb, Cb = rope_T(Bn), rope_T(Cn)
+    la = dt * A
+    alpha = torch.exp(la)
+    gamma, beta = lam * dt, (1 - lam) * dt * alpha
+    cs = torch.cumsum(la, 1)
+    tri = torch.tril(torch.ones(T, T, dtype=torch.bool, device=u.device))[None, :, :, None]
+    dec = torch.exp((cs[:, :, None] - cs[:, None, :]).masked_fill(~tri, float("-inf")))  # (B,t,s,H)
+    band = torch.zeros(Bsz, T, T, H, dtype=torch.float64, device=u.device)
+    i = torch.arange(T, device=u.device)
+    band[:, i, i] = gamma
+    band[:, i[1:], i[:-1]] = beta[:, 1:]
+    Lm = torch.einsum('btkh,bksh->btsh', dec, band)
+    G = torch.einsum('bthin,bshjn->bhtsij', Cb, Bb)
+    y = torch.einsum('btsh,bhtsij,bshpj->bthip', Lm, G, x)
+    return _finish(L, y, z)

build/lib/steertts/mamba3/scan.py

@@ -0,0 +1,193 @@
+"""Chunked parallel scan for the Mamba-3 recurrence.
+
+WHY. The recurrent path is a Python loop of T sequential steps. Measured on
+this laptop: d_model 256 / 4 layers / N 32 runs at 3,670 tok/s, while
+384 / 6 / 64 runs at 179 tok/s - 20x slower for ~4x the FLOPs. The gap is not
+arithmetic, it is kernel-launch overhead: 6 layers x 128 timesteps is 768
+sequential iterations each firing a handful of tiny kernels, so DEPTH
+MULTIPLIES LAUNCHES. That is what blocks scaling, not parameters.
+
+THE TRICK, and it is the paper's own (Proposition 3). The recurrence
+
+    h_t = alpha_t R(theta_t) h_{t-1} + beta_t S_{t-1} + gamma_t S_t
+
+has a data-dependent rotation inside it, which a scan cannot factor. But
+rotations compose: define the CUMULATIVE angle Phi_t = sum_{s<=t} theta_s and
+change basis by R(-Phi_t). In that frame the rotation disappears and what is
+left is a plain first-order linear recurrence
+
+    g_t = alpha_t g_{t-1} + w_t          w_t = R(-Phi_t) (beta_t S_{t-1} + gamma_t S_t)
+
+with h_t = R(Phi_t) g_t. The paper states this as applying data-dependent
+RoPE to B and C; the same identity in state space is what makes the scan
+possible at all.
+
+THE SCAN. g_t = alpha_t g_{t-1} + w_t has the closed form
+
+    g_t = A_t * sum_{s<=t} w_s / A_s     A_t = prod_{r<=t} alpha_r
+
+which is numerically hopeless directly - A_s underflows for long sequences
+and the division explodes. So this chunks: within a chunk of length L the sum
+is materialised as an L x L lower-triangular decay matrix (no division, decay
+is always relative to the chunk start), and chunk states are carried
+recurrently. T/L sequential steps instead of T. The intra-chunk work becomes
+one batched matmul, which is what GPUs are for.
+
+CORRECTNESS IS CHECKED AGAINST THE RECURRENCE, NOT AGAINST A LOSS CURVE.
+`verify()` compares this to Mamba3Layer.forward at float64. A fast scan that
+computes something slightly different would show up as a slightly worse loss
+curve and be indistinguishable from a hyperparameter difference - which is
+exactly how this project has lost time before.
+
+    python -m steertts.mamba3.scan
+"""
+from __future__ import annotations
+
+import math
+import torch
+
+
+def cumulative_angles(theta: torch.Tensor) -> torch.Tensor:
+    """Phi_t = sum_{s<=t} theta_s.  theta: (B,T,H,N/2)."""
+    return torch.cumsum(theta, dim=1)
+
+
+def rotate_by(v: torch.Tensor, ang: torch.Tensor) -> torch.Tensor:
+    """Rotate pairs of the N axis. v: (...,N,P), ang: (...,N/2) broadcastable.
+
+    Built with stack+reshape rather than writing into an empty tensor. The
+    first version used `out = torch.empty_like(v)` with strided assignment,
+    which is correct in the forward direction and BREAKS AUTOGRAD: in-place
+    writes bump the version counter of a tensor the graph still needs. The
+    forward equivalence check passed anyway, because it ran under no_grad.
+    """
+    a, b = v[..., 0::2, :], v[..., 1::2, :]
+    c, s = torch.cos(ang).unsqueeze(-1), torch.sin(ang).unsqueeze(-1)
+    even, odd = c * a - s * b, s * a + c * b
+    return torch.stack((even, odd), dim=-2).flatten(-3, -2)
+
+
+def chunked_scan(alpha: torch.Tensor, w: torch.Tensor, chunk: int = 64,
+                 state=None):
+    """g_t = alpha_t g_{t-1} + w_t, computed in chunks.
+
+    alpha: (B,T,H)          per-step decay, in (0,1)
+    w:     (B,T,H,N,P)      per-step input, already in the rotated frame
+    returns g: (B,T,H,N,P) and the final state.
+
+    Within a chunk, g_t = d_t * g_0 + sum_{s<=t} (d_t/d_s) w_s where
+    d_t = prod_{r<=t} alpha_r RELATIVE TO THE CHUNK START. Both d_t and d_t/d_s
+    stay bounded because the product never runs longer than `chunk` steps -
+    that is the whole reason for chunking rather than a global cumprod.
+    """
+    B, T, H = alpha.shape
+    N, P = w.shape[-2], w.shape[-1]
+    # Accumulate chunks in a list and cat once. Writing into a preallocated
+    # g_out and then reading g_out[:, e-1] back as the next chunk's state is
+    # an in-place write into a tensor autograd still needs.
+    pieces = []
+    g = torch.zeros(B, H, N, P, device=w.device, dtype=w.dtype) \
+        if state is None else state
+    la = torch.log(alpha.clamp_min(1e-30))                 # (B,T,H)
+    for s in range(0, T, chunk):
+        e = min(s + chunk, T)
+        L = e - s
+        cl = torch.cumsum(la[:, s:e], dim=1)               # (B,L,H) log d_t
+        d = torch.exp(cl)                                  # decay from chunk start
+        # (d_t / d_s) for s<=t, lower triangular, exp of a difference of logs
+        rel = cl.unsqueeze(2) - cl.unsqueeze(1)            # (B,L,L,H)  log d_t - log d_s
+        # mask before exp (see ssd.ssd_pytorch): exp of the positive upper triangle overflows and where()
+        # of an inf back-propagates NaN
+        mask = torch.tril(torch.ones(L, L, device=w.device, dtype=torch.bool))
+        M = torch.exp(rel.masked_fill(~mask.view(1, L, L, 1), float("-inf")))
+        # intra-chunk: sum_s M[t,s] * w_s  -> one batched matmul over (L,L) x (L, N*P)
+        wf = w[:, s:e].reshape(B, L, H, N * P).permute(0, 2, 1, 3)   # (B,H,L,NP)
+        Mp = M.permute(0, 3, 1, 2)                                   # (B,H,L,L)
+        acc = (Mp @ wf).permute(0, 2, 1, 3).reshape(B, L, H, N, P)
+        # carry the incoming chunk state, decayed
+        piece = acc + d.unsqueeze(-1).unsqueeze(-1) * g.unsqueeze(1)
+        pieces.append(piece)
+        g = piece[:, -1]
+    return torch.cat(pieces, dim=1), g
+
+
+def scan_forward(layer, u: torch.Tensor, chunk: int = 64):
+    """Mamba3Layer.forward computed with the chunked scan instead of the loop."""
+    Bsz, T, _ = u.shape
+    H, N, R, P = layer.H, layer.N, layer.R, layer.P
+    nu = layer.norm(u)
+    alpha, beta, gamma, Bm, Cm, x, theta = layer._coeffs(nu)
+    S = Bm @ x.transpose(-1, -2)                       # (B,T,H,N,P)
+    # trapezoidal: beta carries the PREVIOUS step's write. Built by cat, not
+    # by assigning into a zeros tensor, for the same autograd reason.
+    Sprev = torch.cat([torch.zeros_like(S[:, :1]), S[:, :-1]], dim=1)
+    Phi = cumulative_angles(theta)                     # (B,T,H,N/2) = Phi_s
+    Phi_prev = torch.cat([torch.zeros_like(Phi[:, :1]), Phi[:, :-1]], dim=1)
+    # Paper eq (11): the gamma (current) write rotates by R(-Phi_s), the beta
+    # (previous) write by R(-Phi_{s-1}) - one step less. Rotate the two terms
+    # separately into the global frame, then scan.
+    w = (gamma[..., None, None] * rotate_by(S, -Phi)
+         + beta[..., None, None] * rotate_by(Sprev, -Phi_prev))
+    g, _ = chunked_scan(alpha, w, chunk)
+    h = rotate_by(g, Phi)                              # back out
+    y = (Cm.transpose(-1, -2) @ h).reshape(Bsz, T, H * R * P)
+    return layer.gate_and_proj(y, nu)
+
+
+def verify(chunks=(8, 16, 64), verbose: bool = True) -> dict:
+    """Against the recurrence, at float64 - FORWARD AND BACKWARD.
+
+    The backward half is not optional. The first version of this file passed
+    every forward check and could not backpropagate at all: it built outputs
+    with in-place strided writes, which autograd rejects. The forward checks
+    ran under no_grad and saw nothing. A scan that cannot train is useless,
+    and one that trains with subtly wrong gradients is worse than useless -
+    it would surface as a slightly worse loss curve, indistinguishable from a
+    hyperparameter difference."""
+    from .layer import Mamba3Layer
+
+    torch.manual_seed(0)
+    torch.set_default_dtype(torch.float64)
+    out = {}
+    lay = Mamba3Layer(d_model=32, d_state=16, rank=2, head_dim=8).eval()
+    u = torch.randn(3, 40, 32)
+    with torch.no_grad():
+        ref = lay(u)
+        for c in chunks:
+            got = scan_forward(lay, u, chunk=c)
+            out[f"chunk{c}"] = float((got - ref).abs().max())
+        # a length that is not a multiple of the chunk
+        u2 = torch.randn(2, 37, 32)
+        r2 = lay(u2)
+        out["ragged"] = float((scan_forward(lay, u2, chunk=16) - r2).abs().max())
+        # causality of the scan path itself
+        u3 = torch.cat([u, torch.randn(3, 9, 32) * 50], 1)
+        out["causality"] = float(
+            (scan_forward(lay, u3, chunk=16)[:, :40]
+             - scan_forward(lay, u, chunk=16)).abs().max())
+
+    # ---- BACKWARD ----
+    ug = torch.randn(3, 24, 32, requires_grad=True)
+    lay.zero_grad()
+    lay(ug).pow(2).sum().backward()
+    gu_rec = ug.grad.clone()
+    gp_rec = {n: p.grad.clone() for n, p in lay.named_parameters()
+              if p.grad is not None}
+    ug.grad = None
+    lay.zero_grad()
+    scan_forward(lay, ug, chunk=8).pow(2).sum().backward()
+    out["grad_input"] = float((gu_rec - ug.grad).abs().max())
+    out["grad_params"] = max(
+        float((gp_rec[n] - p.grad).abs().max())
+        for n, p in lay.named_parameters() if p.grad is not None)
+    torch.set_default_dtype(torch.float32)
+    if verbose:
+        ok = lambda c: "OK  " if c else "FAIL"
+        print("chunked scan vs recurrence  (float64)")
+        for k, v in out.items():
+            print(f"  {k:<12}{v:.3e}   {ok(v < 1e-9)}")
+    return out
+
+
+if __name__ == "__main__":
+    verify()

build/lib/steertts/mamba3/ssd.py

@@ -0,0 +1,201 @@
+"""Fused Mamba-3 scan: the SSD reduction, verified, then a Triton kernel.
+
+WHY THIS FILE. scan.py's chunked scan materialises (B,T,H,N,P) intermediates
+(S, inp, w, g, h) in HBM and fires many kernels; on the RTX 5070 it is
+memory-bandwidth bound and runs at ~3.4k tok/s for a 12M-param model at ctx
+1024. The Mamba-3 paper's speed comes from a fused chunked kernel that keeps the
+chunk state in SRAM and never writes the expanded state to HBM. This file builds
+that, test-first, against the verified recurrence in layer.py.
+
+THE REDUCTION (paper Sec 3.2-3.3, "data-dependent RoPE on B and C"). The
+recurrence
+
+    h_t = alpha_t R(theta_t) h_{t-1} + gamma_t B_t x_t^T + beta_t B_{t-1} x_{t-1}^T
+    y_t = C_t^T h_t
+
+becomes, with Phi_t = cumsum(theta) and the rotations absorbed into B and C,
+
+    y_t = Ctil_t^T sum_{s<=t} decay(s,t) (Beff_s xeff_s^T),   decay(s,t)=prod_{s<r<=t} alpha_r
+    Ctil_t = R(-Phi_t) C_t
+    Beff_s = [ gamma_s R(-Phi_s)B_s , beta_s R(-Phi_s)B_{s-1} ]   (N, 2R)
+    xeff_s = [ x_s , x_{s-1} ]                                    (P, 2R)
+
+i.e. a standard Mamba-2 SSD scan on RoPE'd B/C with a rank-2R write that folds
+in the trapezoidal two-tap. SSD is the fusible, backward-known form.
+
+STAGE 1 (this file, first): a PyTorch chunked-SSD reference that implements
+exactly the arithmetic the Triton kernel will, and is checked fwd+bwd against
+Mamba3Layer at float64. Only once this matches does the Triton port begin.
+"""
+from __future__ import annotations
+
+import torch
+
+from .scan import rotate_by  # rotate (...,N,P) pairs of the N axis by ang (...,N/2)
+
+
+def ssd_inputs(layer, u: torch.Tensor):
+    """Reduce Mamba3Layer's coefficients to the SSD form.
+
+    Returns alpha (B,T,H), Beff (B,T,H,N,2R), xeff (B,T,H,P,2R), Ctil (B,T,H,N,R).
+    All the rotation and the trapezoidal fold happen here, in cheap elementwise
+    ops on tensors no larger than (.,.,.,N,2R) - never the (N,P) expanded state.
+    """
+    alpha, beta, gamma, Bm, Cm, x, theta = layer._coeffs(layer.norm(u))
+    Phi = torch.cumsum(theta, dim=1)                       # (B,T,H,N/2) = Phi_s
+    Phi_prev = torch.cat([torch.zeros_like(Phi[:, :1]), Phi[:, :-1]], dim=1)  # Phi_{s-1}
+    Btil = rotate_by(Bm, -Phi)                             # R(-Phi_s) B_s      (paper gamma term)
+    Ctil = rotate_by(Cm, -Phi)                             # R(-Phi_t) C_t
+    # Trapezoidal previous-step write, rotated by R(-Phi_{s-1}) per paper eq (11)
+    # (NOT Phi_s): B_{s-1} lands rotated by prod_{i<=s-1} R_i in the global frame.
+    Bm_prev = torch.cat([torch.zeros_like(Bm[:, :1]), Bm[:, :-1]], dim=1)
+    x_prev = torch.cat([torch.zeros_like(x[:, :1]), x[:, :-1]], dim=1)
+    Btil_prev = rotate_by(Bm_prev, -Phi_prev)             # R(-Phi_{s-1}) B_{s-1}
+    # rank-2R effective write: coefficients ride on B, so w_s = Beff_s xeff_s^T.
+    Beff = torch.cat([gamma[..., None, None] * Btil,
+                      beta[..., None, None] * Btil_prev], dim=-1)   # (B,T,H,N,2R)
+    xeff = torch.cat([x, x_prev], dim=-1)                          # (B,T,H,P,2R)
+    return alpha, Beff, xeff, Ctil
+
+
+def ssd_pytorch(alpha, Beff, xeff, Ctil, chunk: int = 64):
+    """Chunked SSD forward in PyTorch - the reference the Triton kernel matches.
+
+    alpha (B,T,H) ; Beff (B,T,H,N,K) ; xeff (B,T,H,P,K) ; Ctil (B,T,H,N,R).
+    Returns y (B,T,H,R,P). State g (N,P) is carried per chunk (kept small).
+    """
+    B, T, H, N, K = Beff.shape
+    P = xeff.shape[-2]
+    R = Ctil.shape[-1]
+    la = torch.log(alpha.clamp_min(1e-30))                 # (B,T,H)
+    g = torch.zeros(B, H, N, P, dtype=Beff.dtype, device=Beff.device)
+    ys = []
+    for s0 in range(0, T, chunk):
+        e = min(s0 + chunk, T)
+        L = e - s0
+        Bc = Beff[:, s0:e]          # (B,L,H,N,K)
+        xc = xeff[:, s0:e]          # (B,L,H,P,K)
+        Cc = Ctil[:, s0:e]          # (B,L,H,N,R)
+        cl = torch.cumsum(la[:, s0:e], dim=1)              # (B,L,H) = log prod_{s0..t}
+        d = torch.exp(cl)                                  # (B,L,H)  prod_{s0..t} alpha
+        # intra-chunk decay L[t,s] = exp(cl_t - cl_s) for s<=t else 0
+        rel = cl.unsqueeze(2) - cl.unsqueeze(1)            # (B,L(t),L(s),H)
+        # mask the exponent BEFORE exp: above the diagonal rel > 0 can overflow, and where() of an inf
+        # still back-propagates NaN into alpha (non-finite grads once a chunk's decay exceeds ~88 nats)
+        tril = torch.tril(torch.ones(L, L, dtype=torch.bool, device=Beff.device))
+        Ldec = torch.exp(rel.masked_fill(~tril.view(1, L, L, 1), float("-inf")))
+        # scores G[t,s] = Ctil_t^T Beff_s over N -> (B,H,L,L,R,K)
+        G = torch.einsum('blhnr,bmhnk->bhlmrk', Cc, Bc)
+        # intra y_t = sum_{s<=t} Ldec[t,s] G[t,s] xeff_s^T
+        y_intra = torch.einsum('blmh,bhlmrk,bmhpk->blhrp', Ldec, G, xc)
+        # inter y_t = d_t * (Ctil_t^T g)
+        y_inter = torch.einsum('blh,blhnr,bhnp->blhrp', d, Cc, g)
+        ys.append(y_intra + y_inter)
+        # state update: g <- d_{L-1} g + sum_s (d_{L-1}/d_s) Beff_s xeff_s^T
+        coef = torch.exp(cl[:, -1:, :] - cl)               # (B,L,H)
+        g = d[:, -1].unsqueeze(-1).unsqueeze(-1) * g \
+            + torch.einsum('blh,blhnk,blhpk->bhnp', coef, Bc, xc)
+    return torch.cat(ys, dim=1)                            # (B,T,H,R,P)
+
+
+def ssd_vectorized(alpha, Beff, xeff, Ctil, chunk: int = 64):
+    """ssd_pytorch with NO Python loop over chunks - same arithmetic, batched.
+
+    ssd_pytorch walks the chunks sequentially, firing ~10 kernels per chunk; at T ~ 2000 that loop,
+    not the FLOPs, is the cost. Here every chunk's intra-chunk term is one batched einsum, and the
+    chunk-to-chunk state pass - g_c = D_{c-1} g_{c-1} + U_{c-1} - is unrolled into a (C x C)
+    lower-triangular decay matrix over chunks, the formulation of Mamba-2's `ssd_minimal`. C = T/chunk
+    is small (tens), so that matrix is cheap.
+
+    Exponents are masked to -inf BEFORE exp, never exp-then-where: above the diagonal cl_t - cl_s is
+    positive and can overflow, and where() of an inf still back-propagates NaN.
+    Same signature and output as ssd_pytorch; verify() checks the two agree fwd+bwd at float64.
+    """
+    B, T, H, N, K = Beff.shape
+    P, R = xeff.shape[-2], Ctil.shape[-1]
+    pad = (-T) % chunk
+    if pad:  # alpha=1 (log 0) and zero writes/readouts: padded steps change nothing
+        alpha = torch.cat([alpha, alpha.new_ones(B, pad, H)], 1)
+        Beff, xeff, Ctil = (torch.cat([t, t.new_zeros(B, pad, *t.shape[2:])], 1) for t in (Beff, xeff, Ctil))
+    C, L = (T + pad) // chunk, chunk
+    la = torch.log(alpha.clamp_min(1e-30)).view(B, C, L, H)
+    Bc, xc, Cc = Beff.view(B, C, L, H, N, K), xeff.view(B, C, L, H, P, K), Ctil.view(B, C, L, H, N, R)
+    cl = torch.cumsum(la, 2)                                         # (B,C,L,H) log decay from chunk start
+    tril = torch.tril(torch.ones(L, L, dtype=torch.bool, device=Beff.device)).view(1, 1, L, L, 1)
+    Ldec = torch.exp((cl.unsqueeze(3) - cl.unsqueeze(2)).masked_fill(~tril, float("-inf")))  # (B,C,L,L,H)
+    G = torch.einsum('bclhnr,bcmhnk->bchlmrk', Cc, Bc)
+    y = torch.einsum('bclmh,bchlmrk,bcmhpk->bclhrp', Ldec, G, xc)   # intra-chunk
+    # each chunk's own contribution to the state at its end, then the state at each chunk's START
+    U = torch.einsum('bclh,bclhnk,bclhpk->bchnp', torch.exp(cl[:, :, -1:] - cl), Bc, xc)  # (B,C,H,N,P)
+    tot = cl[:, :, -1]                                               # (B,C,H) log decay across chunk c
+    ct = torch.cumsum(tot, 1)
+    # start state of chunk c = sum_{j<c} exp(ct_{c-1} - ct_j) U_j    (strictly lower triangular in c, j)
+    stril = torch.tril(torch.ones(C, C, dtype=torch.bool, device=Beff.device), -1).view(1, C, C, 1)
+    ctm1 = ct - tot                                                  # ct_{c-1} (0 for c=0)
+    Dc = torch.exp((ctm1.unsqueeze(2) - ct.unsqueeze(1)).masked_fill(~stril, float("-inf")))  # (B,C,C,H)
+    g0 = torch.einsum('bcjh,bjhnp->bchnp', Dc, U)
+    y = y + torch.einsum('bclh,bclhnr,bchnp->bclhrp', torch.exp(cl), Cc, g0)  # inter-chunk
+    return y.reshape(B, C * L, H, R, P)[:, :T]
+
+
+def scan_forward_ssd(layer, u: torch.Tensor, chunk: int = 64, vectorized: bool = True):
+    """Drop-in for scan.scan_forward, via the SSD reduction (PyTorch stage).
+
+    vectorized=True (default) uses ssd_vectorized (no per-chunk Python loop; much faster at long T);
+    False runs the chunk loop of ssd_pytorch, kept as the readable reference."""
+    Bsz, T, _ = u.shape
+    H, R, P = layer.H, layer.R, layer.P
+    nu = layer.norm(u)
+    alpha, Beff, xeff, Ctil = ssd_inputs(layer, u)
+    y = (ssd_vectorized if vectorized else ssd_pytorch)(alpha, Beff, xeff, Ctil, chunk)  # (B,T,H,R,P)
+    return layer.gate_and_proj(y.reshape(Bsz, T, H * R * P), nu)
+
+
+def verify(chunks=(8, 16, 64), verbose: bool = True) -> dict:
+    """SSD reduction vs the recurrence, fwd + bwd, at float64 - loop and vectorized paths."""
+    from .layer import Mamba3Layer
+    torch.manual_seed(0)
+    torch.set_default_dtype(torch.float64)
+    out = {}
+    lay = Mamba3Layer(d_model=32, d_state=16, rank=2, head_dim=8).eval()
+    u = torch.randn(3, 40, 32)
+    u2 = torch.randn(2, 37, 32)
+    u3 = torch.cat([u, torch.randn(3, 9, 32) * 50], 1)
+    ug = torch.randn(3, 24, 32, requires_grad=True)
+    with torch.no_grad():
+        ref, ref2 = lay(u), lay(u2)
+    lay.zero_grad(); lay(ug).pow(2).sum().backward()
+    gu = ug.grad.clone(); gp = {n: p.grad.clone() for n, p in lay.named_parameters()
+                                if p.grad is not None}
+    for vec in (False, True):
+        pre = "vec_" if vec else ""
+        f = lambda x, c: scan_forward_ssd(lay, x, c, vectorized=vec)
+        with torch.no_grad():
+            for c in chunks:
+                out[f"{pre}chunk{c}"] = float((f(u, c) - ref).abs().max())
+            out[f"{pre}ragged"] = float((f(u2, 16) - ref2).abs().max())
+            out[f"{pre}causality"] = float((f(u3, 16)[:, :40] - f(u, 16)).abs().max())
+        ug.grad = None; lay.zero_grad()
+        f(ug, 8).pow(2).sum().backward()
+        out[f"{pre}grad_input"] = float((gu - ug.grad).abs().max())
+        out[f"{pre}grad_params"] = max(float((gp[n] - p.grad).abs().max())
+                                       for n, p in lay.named_parameters() if p.grad is not None)
+    # long sequence with strong decay: the vectorized path's masked exponents must stay finite fwd+bwd
+    a = torch.full((1, 3000, 2), 1e-3)                     # log alpha ~ -6.9 per step
+    Bf, xf, Cf = (torch.randn(1, 3000, 2, 4, k, requires_grad=True) for k in (2, 2, 1))
+    yv = ssd_vectorized(a, Bf, xf, Cf, 64)
+    yv.sum().backward()
+    out["vec_long_finite"] = 0.0 if all(torch.isfinite(t).all() for t in (yv, Bf.grad, xf.grad, Cf.grad)) else 1.0
+    with torch.no_grad():
+        out["vec_long_vs_loop"] = float((yv - ssd_pytorch(a, Bf, xf, Cf, 64)).abs().max())
+    torch.set_default_dtype(torch.float32)
+    if verbose:
+        ok = lambda c: "OK  " if c else "FAIL"
+        print("SSD reduction vs recurrence  (float64, fwd+bwd)")
+        for k, v in out.items():
+            print(f"  {k:<18}{v:.3e}   {ok(v < 1e-9)}")
+    return out
+
+
+if __name__ == "__main__":
+    verify()

build/lib/steertts/mamba3/tests.py

@@ -0,0 +1,230 @@
+"""One runner for every Mamba-3 invariant. Exits non-zero on failure.
+
+    python -m steertts.mamba3.tests          # ~1 min on CPU
+    python -m steertts.mamba3.tests --full   # adds fp32/bf16 accuracy at T=3000
+
+Checks, at float64 unless stated:
+  1. layer selftest: causality, step == forward, rotation orthogonal, trapezoidal two-tap, BCNorm
+  2. chunked scan and SSD reduction vs the recurrence, fwd + bwd (their own verify())
+  3. PAPER FAITHFULNESS against reference.py, which recomputes every coefficient from raw parameters:
+     two independent references (complex arithmetic, eq (7) parallel form) agree with each other and
+     with every layer path (forward, step, scan, SSD loop, SSD vectorized) over ranks, lengths
+     (including T=1 and T not a multiple of the chunk) and chunk sizes
+  4. gradients: torch.autograd.gradcheck wrt the input AND every parameter on each path, and gradient
+     agreement of each fast path with the reference
+  5. causality: hard (perturb the future by 1e3, past outputs unchanged) and Jacobian (dy_t/du_s == 0
+     for s > t, exactly)
+  6. streaming: forward() in segments carrying (h, S_last) == one forward() over the whole sequence
+  7. strong decay: finite outputs and gradients (incl. wrt alpha) on every scan path in float32
+  8. (--full) float32 and bf16-autocast error vs the float64 reference at T = 100/1000/3000
+"""
+from __future__ import annotations
+
+import argparse
+import contextlib
+import math
+import sys
+
+import torch
+
+from .layer import Mamba3Layer, selftest as layer_selftest
+from .reference import ref_complex, ref_parallel
+from .scan import chunked_scan, scan_forward, verify as scan_verify
+from .ssd import scan_forward_ssd, ssd_pytorch, ssd_vectorized, verify as ssd_verify
+
+
+@contextlib.contextmanager
+def default_dtype(dt):
+    """set_default_dtype that is restored even when a check raises (the old runner leaked float64)."""
+    old = torch.get_default_dtype()
+    torch.set_default_dtype(dt)
+    try:
+        yield
+    finally:
+        torch.set_default_dtype(old)
+
+
+def make(d=16, N=8, R=1, P=8, seed=0, dtype=torch.float64, randomize=True):
+    """A layer moved off its symmetric init, so lambda, the biases, the norms and the rotation all matter."""
+    torch.manual_seed(seed)
+    L = Mamba3Layer(d_model=d, d_state=N, rank=R, head_dim=P).to(dtype).eval()
+    if randomize:
+        with torch.no_grad():
+            L.lam_proj.weight.normal_(0, 0.5); L.lam_proj.bias.normal_(0, 1)
+            L.B_bias.normal_(1, 0.5); L.C_bias.normal_(1, 0.5)
+            for n in (L.B_norm, L.C_norm, L.norm):
+                n.weight.normal_(1, 0.2)
+            L.theta_proj.weight.mul_(4)  # angles large enough that the rotation is far from identity
+    return L
+
+
+def paths(L, u, chunk):
+    """Every way the package can compute a layer's output."""
+    with torch.no_grad():
+        st = bx = None
+        steps = []
+        for t in range(u.shape[1]):
+            o, st, bx = L.step(u[:, t], st, bx)
+            steps.append(o)
+        return {"forward": L(u), "step": torch.stack(steps, 1), "scan": scan_forward(L, u, chunk),
+                "ssd_loop": scan_forward_ssd(L, u, chunk, vectorized=False),
+                "ssd_vec": scan_forward_ssd(L, u, chunk, vectorized=True)}
+
+
+class _Path(torch.nn.Module):
+    def __init__(self, L, path):
+        super().__init__(); self.L, self.path = L, path
+
+    def forward(self, u):
+        if self.path == "forward":
+            return self.L(u)
+        if self.path == "scan":
+            return scan_forward(self.L, u, 3)
+        return scan_forward_ssd(self.L, u, 3, vectorized=self.path == "ssd_vec")
+
+
+def run(full: bool) -> bool:
+    ok = True
+
+    def check(name, val, cond):
+        nonlocal ok
+        ok = ok and bool(cond)
+        print(f"  [{'PASS' if cond else 'FAIL'}] {name:<62} {val:.2e}")
+
+    print("== 1. layer selftest ==")
+    with default_dtype(torch.float64):
+        L = layer_selftest(verbose=False)
+    check("causality (future -> past)", L["causality"], L["causality"] < 1e-12)
+    check("step() == forward()", L["streaming"], L["streaming"] < 1e-10)
+    check("rotation orthogonal", L["orthogonal"], L["orthogonal"] < 1e-10)
+    check("trapezoidal two-tap (y4 moves, y2 does not)", L["twotap_y4"],
+          L["twotap_y4"] > 1e-6 and L["twotap_y2"] == 0.0)
+    check("BCNorm rows unit RMS", L["bcnorm_rms"], L["bcnorm_rms"] < 1e-10)
+
+    print("== 2. scan / SSD vs recurrence (fwd + bwd) ==")
+    with default_dtype(torch.float64):
+        for name, res in (("scan", scan_verify(verbose=False)), ("ssd", ssd_verify(verbose=False))):
+            for k, v in res.items():
+                check(f"{name} {k}", v, v < 1e-9)
+
+    with default_dtype(torch.float64):
+        print("== 3. paper faithfulness: independent references ==")
+        worst = {}
+        for d, N, R, P in [(16, 8, 1, 8), (16, 8, 2, 8), (24, 4, 3, 8), (8, 2, 1, 4)]:
+            lay = make(d, N, R, P, seed=R)
+            for T in (1, 2, 5, 16, 37):
+                u = torch.randn(2, T, d)
+                with torch.no_grad():
+                    rc, rp = ref_complex(lay, u), ref_parallel(lay, u)
+                worst["ref_complex vs ref_parallel"] = max(worst.get("ref_complex vs ref_parallel", 0),
+                                                           float((rc - rp).abs().max()))
+                for chunk in (1, 4, 16, 64):
+                    for k, v in paths(lay, u, chunk).items():
+                        worst[k] = max(worst.get(k, 0.0), float((v - rc).abs().max()))
+        for k, v in worst.items():
+            check(f"{k} (4 configs x 5 lengths x 4 chunks)", v, v < 1e-11)
+
+        print("== 4. gradients ==")
+        lay = make(8, 4, 2, 4, seed=5)
+        names = [n for n, _ in lay.named_parameters()]
+        params = tuple(p.detach().clone().requires_grad_(True) for p in lay.parameters())
+        u = torch.randn(1, 7, 8, requires_grad=True)
+        for path in ("forward", "scan", "ssd_loop", "ssd_vec"):
+            mod = _Path(lay, path)
+            fn = lambda u, *ps: torch.func.functional_call(mod, {"L." + n: p for n, p in zip(names, ps)}, (u,))
+            try:
+                good = torch.autograd.gradcheck(fn, (u,) + params, eps=1e-6, atol=1e-6, rtol=1e-5)
+            except Exception as ex:  # noqa: BLE001 - report, do not crash the runner
+                print("     ", str(ex).splitlines()[0][:120]); good = False
+            check(f"gradcheck {path} wrt input + all {len(params)} params", 0.0, good)
+        lay = make(16, 8, 2, 8, seed=7)
+        u, w = torch.randn(2, 37, 16), torch.randn(2, 37, 16)
+
+        def grads(f):
+            lay.zero_grad(); ui = u.clone().requires_grad_(True)
+            (f(ui) * w).sum().backward()
+            return [ui.grad.clone()] + [p.grad.clone() for p in lay.parameters()]
+        gr = grads(lambda v: ref_complex(lay, v))
+        for name, f in (("forward", lay), ("scan", lambda v: scan_forward(lay, v, 8)),
+                        ("ssd_loop", lambda v: scan_forward_ssd(lay, v, 8, vectorized=False)),
+                        ("ssd_vec", lambda v: scan_forward_ssd(lay, v, 8, vectorized=True))):
+            e = max(float((a - b).abs().max()) for a, b in zip(grads(f), gr))
+            check(f"grad(input, params) {name} vs ref_complex", e, e < 1e-9)
+
+        print("== 5. causality ==")
+        lay = make(16, 8, 2, 8, seed=3)
+        u = torch.randn(2, 40, 16)
+        base = paths(lay, u, 16)
+        for t0 in (15, 16, 39):  # inside a chunk, on a chunk boundary, last step
+            u2 = u.clone(); u2[:, t0:] += 1e3 * torch.randn_like(u2[:, t0:])
+            for k, v in paths(lay, u2, 16).items():
+                e = float((v[:, :t0] - base[k][:, :t0]).abs().max())
+                check(f"hard causality {k} t0={t0}", e, e < 1e-9)
+        ug = torch.randn(1, 20, 16, requires_grad=True)
+        for name, f in (("forward", lay), ("scan", lambda v: scan_forward(lay, v, 8)),
+                        ("ssd_loop", lambda v: scan_forward_ssd(lay, v, 8, vectorized=False)),
+                        ("ssd_vec", lambda v: scan_forward_ssd(lay, v, 8, vectorized=True))):
+            g, = torch.autograd.grad(f(ug)[0, 9].sum(), ug)
+            e = float(g[0, 10:].abs().max())
+            check(f"Jacobian causality {name}: |dy_9/du_s>9|", e, e == 0.0)
+
+        print("== 6. streaming in segments ==")
+        lay = make(16, 8, 2, 8, seed=9)
+        u = torch.randn(2, 30, 16)
+        with torch.no_grad():
+            whole = lay(u)
+            for cuts in ((17,), (1, 2, 16, 29), tuple(range(1, 30))):
+                outs, st, s0 = [], None, 0
+                for c in cuts + (30,):
+                    o, st = lay(u[:, s0:c], state=st, return_state=True)
+                    outs.append(o); s0 = c
+                e = float((torch.cat(outs, 1) - whole).abs().max())
+                check(f"forward(state) over segments cut at {cuts if len(cuts) < 5 else 'every step'}", e, e < 1e-10)
+
+    print("== 7. strong decay: finite fwd + bwd in float32 ==")
+    with default_dtype(torch.float32):
+        torch.manual_seed(0)
+        for la_step in (-1.5, -3.0, -20.0):  # per-step log decay; a 64-chunk sums to -96 .. -1280
+            al = torch.full((1, 64, 1), math.exp(la_step), requires_grad=True)
+            Bf = torch.randn(1, 64, 1, 4, 2, requires_grad=True)
+            xf, Cf = torch.randn(1, 64, 1, 2, 2), torch.randn(1, 64, 1, 4, 1)
+            for name, fn in (("ssd_loop", ssd_pytorch), ("ssd_vec", ssd_vectorized)):
+                al.grad = Bf.grad = None
+                y = fn(al, Bf, xf, Cf, 64); y.sum().backward()
+                bad = sum(int((~torch.isfinite(t)).sum()) for t in (y, al.grad, Bf.grad))
+                check(f"{name} log-decay/step {la_step}: non-finite entries", float(bad), bad == 0)
+            al.grad = None
+            g, _ = chunked_scan(al, torch.randn(1, 64, 1, 4, 2), 64); g.sum().backward()
+            bad = int((~torch.isfinite(al.grad)).sum()) + int((~torch.isfinite(g)).sum())
+            check(f"chunked_scan log-decay/step {la_step}: non-finite entries", float(bad), bad == 0)
+
+    if full:
+        print("== 8. float32 / bf16 accuracy vs float64 (d=256, N=32, P=32) ==")
+        for T in (100, 1000, 3000):
+            with default_dtype(torch.float64):
+                L64 = make(256, 32, 1, 32, seed=11, randomize=False)
+                u = torch.randn(2, T, 256)
+                with torch.no_grad():
+                    ref = scan_forward_ssd(L64, u)
+            L32 = make(256, 32, 1, 32, seed=11, dtype=torch.float32, randomize=False)
+            L32.load_state_dict({k: v.float() for k, v in L64.state_dict().items()})
+            with torch.no_grad():
+                sc = float(ref.abs().max())
+                e32 = float((scan_forward_ssd(L32, u.float()).double() - ref).abs().max()) / sc
+                with torch.autocast("cpu", dtype=torch.bfloat16):
+                    ebf = float((scan_forward_ssd(L32, u.float()).double() - ref).abs().max()) / sc
+            check(f"float32 relative error T={T}", e32, e32 < 1e-4)
+            check(f"bf16-autocast relative error T={T} (informational; BiMamba3 runs fp32)", ebf, ebf < 1e-1)
+    return ok
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--full", action="store_true", help="add the slow long-sequence precision checks")
+    ok = run(ap.parse_args().full)
+    print("\n" + ("ALL PASS" if ok else "FAILURES ABOVE"))
+    return 0 if ok else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

build/lib/steertts/markup.py

@@ -0,0 +1,187 @@
+"""Inline markup + sentence-level streaming (recipes/steer_tts/steer/markup.py without the recorded-sound mixer).
+
+Markup (nestable; unknown tags are dropped and their text spoken normally):
+  *word*  or  <emph>words</emph>          emphasis: pitch +2.5 st, energy +4 dB, 20% longer       (per word)
+  <slow> <fast> <high> <low> <loud> <soft>   per-word duration / pitch / energy changes inside the
+                                              sentence - prosody stays continuous                     (per word)
+  [pause] [pause 300ms] [pause 1.5s]          silence (default 400 ms, max 5 s)
+  <whisper>..</whisper>, <sadness>..</sadness>, any style tag the model knows
+                                              the span is rendered as its own segment with that tag   (span)
+  <style prompt="angry and fast">..</style>   the span gets the prompt adapter's tag + controls       (span)
+  [laugh] [cough] [dog] [knock] ...           event tokens: the model reserves time for them, but renders no
+                                              sound (the training-time demo mixed recorded sounds in; that mixer
+                                              is not part of this release)
+
+Rendering: the text is cut into segments at sentence ends (. ! ?), pauses and style-span boundaries; each segment
+is one forward pass with its tag/controls and per-word shifts, and segments are yielded in order, so a caller can
+play the first sentence while the rest renders (stream()). A style switch mid-sentence joins two separately
+rendered segments (a short audible boundary): there is no token-level style conditioning.
+"""
+import re
+from dataclasses import dataclass, field
+
+import numpy as np
+
+from .config import SR
+from .text import to_phonemes
+
+PROSODY = {  # Synth.word_shift user units: pitch semitones, energy dB, dur factor
+    "emph": dict(pitch=2.5, energy=4.0, dur=1.2), "slow": dict(dur=1.4), "fast": dict(dur=0.75),
+    "high": dict(pitch=3.0), "low": dict(pitch=-3.0), "loud": dict(energy=6.0), "soft": dict(energy=-6.0),
+}
+_TOKEN = re.compile(r'(<\s*(/?)\s*([A-Za-z_]+)(?:\s+prompt\s*=\s*"([^"]*)")?\s*>)'
+                    r'|(\[\s*([A-Za-z_]+)(?:\s+([\d.]+)\s*(ms|s)?)?\s*\])', re.I)
+_SENT_END = re.compile(r"(?<=[.!?])\s+")
+
+
+@dataclass
+class Segment:
+    text: str
+    tag: str
+    ctrl: dict
+    word_shift: dict = field(default_factory=dict)
+    pause_ms: int = 0  # silence instead of speech when > 0 (text empty)
+    event: str = ""  # environmental sound event rendered by the EnvMixer (text empty)
+
+
+@dataclass
+class Chunk:
+    audio: np.ndarray
+    segment: Segment
+    synth_ms: float
+
+
+def _n_words(text):
+    if not text.strip():
+        return 0
+    _, widx, _ = to_phonemes(text)
+    return max(widx) + 1 if max(widx) >= 0 else 0
+
+
+class MarkupSynth:
+    def __init__(self, tts):
+        """tts: a steertts.SteerTTS (its prompt adapter is loaded on first use of a prompt)."""
+        self.s = tts
+
+    # ------------------------------------------------------------------ parsing
+    def _style_of_prompt(self, prompt):
+        if not prompt:
+            return "none", {}
+        m = self.s.prompt_mapper(prompt)
+        # the adapter's best tag this checkpoint knows (an adapter trained on a newer tag set can rank a tag first
+        # that an older checkpoint lacks, e.g. 'irritation')
+        tag = next((t for t, _ in m["top_tags"] if t in self.s.tag2i and t != "none"), "none")
+        return tag, dict(m["ctrl"])
+
+    def parse(self, text, prompt=None, tag=None, ctrl=None):
+        """-> list[Segment]. prompt/tag/ctrl give the default style for unmarked text."""
+        d_tag, d_ctrl = self._style_of_prompt(prompt)
+        if tag:
+            d_tag = tag if tag in self.s.tag2i else d_tag
+        d_ctrl = {**d_ctrl, **(ctrl or {})}
+        text = re.sub(r"\*([^*\n]+)\*", r"<emph>\1</emph>", text)
+        runs = []  # (text, (tag, ctrl_items), frozenset(prosody)) or ("PAUSE", ms)
+        style_stack, pros_stack = [(d_tag, tuple(sorted(d_ctrl.items())))], []
+        pos = 0
+        for m in _TOKEN.finditer(text):
+            if m.start() > pos:
+                runs.append((text[pos:m.start()], style_stack[-1], frozenset(pros_stack)))
+            pos = m.end()
+            if m.group(5):  # [pause ...] / [event]
+                name = m.group(6).lower()
+                if name == "pause":
+                    v, unit = m.group(7), (m.group(8) or "ms").lower()
+                    ms = int(float(v) * (1000 if unit == "s" else 1)) if v else 400
+                    runs.append(("PAUSE", min(ms, 5000)))
+                else:  # event token (or unknown: dropped by to_phonemes) stays in the TTS text
+                    runs.append((f" [{name}] ", style_stack[-1], frozenset(pros_stack)))
+                continue
+            closing, name, prm = bool(m.group(2)), m.group(3).lower(), m.group(4)
+            if name in PROSODY:
+                if closing:
+                    if name in pros_stack:
+                        pros_stack.remove(name)
+                else:
+                    pros_stack.append(name)
+            elif name == "style" or name in self.s.tag2i:
+                if closing:
+                    if len(style_stack) > 1:
+                        style_stack.pop()
+                elif name == "style":
+                    t, c = self._style_of_prompt(prm)
+                    style_stack.append((t, tuple(sorted(c.items()))))
+                else:
+                    style_stack.append((name, ()))  # a named style tag resets the default controls
+            # unknown tag: the marker is dropped, its inner text is spoken normally
+        if pos < len(text):
+            runs.append((text[pos:], style_stack[-1], frozenset(pros_stack)))
+
+        # split runs at sentence ends so each sentence is its own (streamable) segment
+        pieces = []
+        for r in runs:
+            if r[0] in ("PAUSE", "EVENT"):
+                pieces.append(r); continue
+            parts = _SENT_END.split(r[0])
+            for i, p in enumerate(parts):
+                pieces.append((p, r[1], r[2]))
+                if i < len(parts) - 1:
+                    pieces.append(("BREAK",))
+        # group consecutive pieces with the same style into segments
+        segs, cur, cur_style = [], [], None
+
+        def flush():
+            if cur and "".join(p[0] for p in cur).strip():
+                segs.append(self._segment(cur, cur_style))
+
+        for p in pieces:
+            if p[0] in ("PAUSE", "BREAK", "EVENT"):
+                flush(); cur, cur_style = [], None
+                if p[0] == "PAUSE":
+                    segs.append(Segment("", "none", {}, pause_ms=p[1]))
+                elif p[0] == "EVENT":
+                    segs.append(Segment("", "none", {}, event=p[1]))
+                continue
+            if cur_style is not None and p[1] != cur_style:
+                flush(); cur = []
+            cur_style = p[1]
+            cur.append(p)
+        flush()
+        return segs
+
+    def _segment(self, pieces, style):
+        text, shift, offset = "", {}, 0
+        for p_text, _, pros in pieces:
+            before = _n_words(text)
+            text += p_text
+            if pros:
+                after = _n_words(text)
+                sh = {}
+                for name in pros:
+                    for k, v in PROSODY[name].items():
+                        sh[k] = sh.get(k, 1.0 if k == "dur" else 0.0) * v if k == "dur" else sh.get(k, 0.0) + v
+                for w in range(before, after):
+                    shift[w] = sh
+        tag, ctrl_items = style
+        return Segment(re.sub(r"\s+", " ", text).strip(), tag, dict(ctrl_items), shift)
+
+    # ------------------------------------------------------------------ rendering
+    def stream(self, text, voice=None, prompt=None, tag=None, ctrl=None, pace=1.0, ref_wav=None, guidance=None,
+               flow=True, seed=0):
+        """Yields Chunk per segment, in order (first chunk = first sentence / span)."""
+        import time
+        fade = int(0.01 * SR)
+        for seg in self.parse(text, prompt, tag, ctrl):
+            t0 = time.time()
+            if seg.pause_ms:
+                y = np.zeros(int(seg.pause_ms * SR / 1000), np.float32)
+            else:
+                y = np.asarray(self.s._render(seg.text, voice, tag=seg.tag, ctrl=seg.ctrl or None, pace=pace,
+                                              word_shift=seg.word_shift or None, ref_wav=ref_wav, guidance=guidance,
+                                              flow=flow, seed=seed), np.float32)
+                if len(y) > 2 * fade:  # 10 ms fades so concatenated segments never click
+                    ramp = np.linspace(0, 1, fade, dtype=np.float32)
+                    y[:fade] *= ramp; y[-fade:] *= ramp[::-1]
+            yield Chunk(y, seg, (time.time() - t0) * 1000)
+
+    def render(self, text, **kw):
+        return np.concatenate([c.audio for c in self.stream(text, **kw)] or [np.zeros(1, np.float32)])

build/lib/steertts/measure.py

@@ -0,0 +1,166 @@
+"""Read the 12 controls back from audio, in the units the model was conditioned on (the training-data measurement,
+recipes/steer_tts/steer/features.py). Needs `praat-parselmouth` (the `measure` extra).
+
+    from steertts.measure import measure_z
+    z = measure_z(wav, text, tts.voice_stats("voice_1361"), tts.pooled_sd)   # {dim: speaker-relative z}
+
+Raw measurements are converted to speaker-relative z with the voice's neutral mean (control_stats.json:
+spk_mean) and the pooled within-speaker SD. `rate` needs the text (phone count); a few dims can be undefined
+(NaN) on very short or unvoiced clips (f0_* need >= 10 voiced frames, f0_end >= 8 in the last 300 ms).
+"""
+import numpy as np
+import parselmouth
+import torch
+
+from .audio import mel_extractor
+from .config import CONTROL_DIMS, HOP, N_FFT, SR
+from .text import SP, SYMBOLS, to_phonemes
+
+_FREQS = np.fft.rfftfreq(N_FFT, 1 / SR)
+_PUNCT = {i for i, s in enumerate(SYMBOLS) if len(s) == 1}
+
+
+def stft_mag(y):
+    x = torch.from_numpy(y)
+    X = torch.stft(x, N_FFT, HOP, N_FFT, torch.hann_window(N_FFT), center=True, return_complex=True)
+    return X.abs().numpy()  # [513, T]
+
+
+def pitch_track(y, n_frames, lo=60.0, hi=700.0):
+    """Two-pass utterance-adaptive autocorrelation pitch (De Looze & Hirst) sampled at mel frame centres."""
+    snd = parselmouth.Sound(y.astype(np.float64), SR)
+
+    def run(lo, hi):
+        p = snd.to_pitch_ac(time_step=HOP / SR, pitch_floor=lo, pitch_ceiling=hi, very_accurate=False)
+        f = p.selected_array["frequency"]
+        return p.xs(), f
+
+    xs, f = run(lo, hi)
+    v = f[f > 0]
+    if len(v) > 20:
+        q25, q75 = np.percentile(v, [25, 75])
+        lo2, hi2 = max(50.0, 0.75 * q25), min(900.0, 1.75 * q75)
+        if hi2 > lo2 + 40:
+            xs, f = run(lo2, hi2)
+    t = np.arange(n_frames) * HOP / SR
+    idx = np.clip(np.searchsorted(xs, t), 1, len(xs) - 1) if len(xs) > 1 else np.zeros(n_frames, int)
+    if len(xs) > 1:
+        left = idx - 1
+        idx = np.where(np.abs(xs[left] - t) < np.abs(xs[idx] - t), left, idx)
+        near = np.abs(xs[idx] - t) <= HOP / SR
+        f0 = np.where(near, f[idx], 0.0)
+    else:
+        f0 = np.zeros(n_frames)
+    return f0.astype(np.float32), snd
+
+
+def cpps(P, f_lo=60.0, f_hi=330.0):
+    """Smoothed cepstral peak prominence (dB) over the given power-spectrum frames [513, n]:
+    real cepstrum of the dB spectrum, peak in the 60-330 Hz quefrency band above a linear
+    regression trend, cepstra smoothed over 3 frames x 3 quefrency bins first (Hillenbrand-style)."""
+    if P.shape[1] < 3:
+        return np.nan
+    L = 10 * np.log10(P + 1e-10)
+    C = np.fft.irfft(L, axis=0)[: N_FFT // 2]  # quefrency in samples
+    C = 20 * np.log10(np.abs(C) + 1e-10)
+    k = np.ones(3) / 3
+    C = np.apply_along_axis(lambda v: np.convolve(v, k, "same"), 1, C)  # time smoothing
+    C = np.apply_along_axis(lambda v: np.convolve(v, k, "same"), 0, C)  # quefrency smoothing
+    q = np.arange(C.shape[0]) / SR
+    band = (q >= 1 / f_hi) & (q <= 1 / f_lo)
+    fit = (q >= 0.001) & (q <= 0.02)
+    A = np.vstack([q[fit], np.ones(fit.sum())]).T
+    coef, *_ = np.linalg.lstsq(A, C[fit], rcond=None)  # [2, n]
+    i = np.argmax(C[band], 0)
+    qb = q[band][i]
+    peak = C[band][i, np.arange(C.shape[1])]
+    trend = coef[0] * qb + coef[1]
+    return float(np.mean(peak - trend))
+
+
+def measure(y, n_phones=None, want_mel=True):
+    """Returns dict(mel, f0, energy, controls(np.array len 12), speech_mask)."""
+    y = y.astype(np.float32)
+    out = {}
+    if want_mel:
+        mel = mel_extractor()(torch.from_numpy(y)[None])[0].numpy()  # [100, T]
+        out["mel"] = mel
+        T = mel.shape[1]
+    S = stft_mag(y)
+    T = S.shape[1] if not want_mel else min(T, S.shape[1])
+    S = S[:, :T]
+    P = S ** 2
+    energy = 10 * np.log10(P.mean(0) + 1e-10)
+    f0, snd = pitch_track(y, T)
+    voiced = f0 > 0
+    # adaptive VAD: 40% of the way from the noise floor (5th pct) to the speech peak (95th pct)
+    lo_e, hi_e = np.percentile(energy, [5, 95])
+    speech = energy > lo_e + 0.4 * (hi_e - lo_e)
+    out.update(f0=f0, energy=energy.astype(np.float32), speech=speech)
+
+    c = dict.fromkeys(CONTROL_DIMS, np.nan)
+    sp = np.where(speech)[0]
+    if len(sp) >= 5:
+        a, b = sp[0], sp[-1] + 1
+        span = (b - a) * HOP / SR
+        inside = np.zeros(T, bool); inside[a:b] = True
+        low = inside & ~speech
+        # silent runs >= 100 ms inside the speech span count as pauses
+        pause_frames, run = 0, 0
+        for x in np.r_[low, False]:
+            if x:
+                run += 1
+            else:
+                if run * HOP / SR >= 0.1:
+                    pause_frames += run
+                run = 0
+        pause_s = pause_frames * HOP / SR
+        c["pause"] = pause_s / span
+        if n_phones:
+            c["rate"] = n_phones / max(span - pause_s, 0.2)
+        sf_ = speech & inside
+        c["loud"] = float(energy[sf_].mean())
+        c["voiced"] = float(voiced[sf_].mean())
+        Ps = P[:, sf_]
+        lo_band = Ps[(_FREQS >= 50) & (_FREQS < 1000)].sum(0)
+        hi_band = Ps[(_FREQS >= 1000) & (_FREQS < 5000)].sum(0)
+        c["tilt"] = float(np.mean(10 * np.log10((hi_band + 1e-10) / (lo_band + 1e-10))))
+        Pb = Ps[(_FREQS >= 50) & (_FREQS < 8000)] + 1e-10
+        c["flat"] = float(np.mean(np.exp(np.log(Pb).mean(0)) / Pb.mean(0)))
+        c["flat"] = float(np.log(c["flat"]))
+        c["centroid"] = float(np.log((Ps * _FREQS[:, None]).sum(0).sum() / (Ps.sum() + 1e-10) + 1))
+        fv = f0[voiced & inside]
+        if len(fv) >= 10:
+            st = 12 * np.log2(fv / 100.0)
+            c["f0_med"] = float(np.median(st))
+            q25, q75 = np.percentile(st, [25, 75])
+            c["f0_iqr"] = float(q75 - q25)
+            tv = np.where(voiced & inside)[0] * HOP / SR
+            last = tv > tv[-1] - 0.3
+            if last.sum() >= 8:
+                c["f0_end"] = float(np.polyfit(tv[last], st[last], 1)[0])
+            # H1-H2 from harmonic peaks (uncorrected), voiced speech frames
+            Sdb = 10 * np.log10(P + 1e-10)
+            vals = []
+            for i in np.where(voiced & inside)[0][::2]:
+                f = f0[i]
+                m1 = (_FREQS > 0.85 * f) & (_FREQS < 1.15 * f)
+                m2 = (_FREQS > 1.85 * f) & (_FREQS < 2.15 * f)
+                if m1.any() and m2.any():
+                    vals.append(Sdb[m1, i].max() - Sdb[m2, i].max())
+            if vals:
+                c["h1h2"] = float(np.median(vals))
+        c["cpps"] = cpps(P[:, speech & inside])
+    out["controls"] = np.array([c[k] for k in CONTROL_DIMS], np.float32)
+    return out
+
+
+def n_phones(text):
+    ids, _, _ = to_phonemes(text)
+    return sum(1 for i in ids if i != SP and i not in _PUNCT and not SYMBOLS[i].startswith("["))
+
+
+def measure_z(wav, text, spk_mean, pooled_sd):
+    """-> {dim: z} relative to the voice's neutral speech (spk_mean: {dim: raw}, pooled_sd: {dim: raw SD})."""
+    raw = measure(np.asarray(wav, np.float32), n_phones=n_phones(text) if text else None, want_mel=False)["controls"]
+    return {k: float((r - spk_mean[k]) / pooled_sd[k]) for k, r in zip(CONTROL_DIMS, raw)}

build/lib/steertts/model.py

@@ -0,0 +1,231 @@
+"""SteerTTS acoustic model (inference path).
+
+text (ARPAbet) -> encoder -> per-phone duration / pitch / energy predictors -> length regulator -> decoder ->
+100-band log-mel (Vocos-compatible). Every block is conditioned through adaptive LayerNorm on
+g = MLP([speaker vector, style tag, control z-vector * mask, mask]). Encoder and decoder blocks are "hybrid":
+multi-head self-attention followed by a bidirectional Mamba-3 branch.
+
+This is the inference subset of the training code (recipes/steer_tts/steer/model.py): the same modules and
+operations in the same order, so a released checkpoint gives bit-identical mels. Training-only parts (the
+alignment module, losses, torch.compile paths) are left out; the released weights do not contain the aligner.
+"""
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .config import CONTROL_DIMS, N_MELS
+from .mamba3 import Mamba3Layer, scan_forward_ssd
+from .text import N_BASE, PAD
+
+N_CTRL = len(CONTROL_DIMS)
+
+
+def seq_mask(lens, max_len=None):
+    max_len = max_len or int(lens.max())
+    return torch.arange(max_len, device=lens.device)[None] < lens[:, None]
+
+
+class SinusoidalPos(nn.Module):
+    def __init__(self, d):
+        super().__init__()
+        self.d = d
+
+    def forward(self, T, device):
+        pos = torch.arange(T, device=device, dtype=torch.float32)[:, None]
+        div = torch.exp(torch.arange(0, self.d, 2, device=device).float() * (-math.log(10000.0) / self.d))
+        pe = torch.zeros(T, self.d, device=device)
+        pe[:, 0::2] = torch.sin(pos * div)
+        pe[:, 1::2] = torch.cos(pos * div)
+        return pe
+
+
+class AdaLN(nn.Module):
+    """LayerNorm whose scale/shift are predicted from the conditioning vector g."""
+
+    def __init__(self, d, g_dim):
+        super().__init__()
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.proj = nn.Linear(g_dim, 2 * d)
+
+    def forward(self, x, g):  # x [B,T,d], g [B,g]
+        s, b = self.proj(g)[:, None].chunk(2, -1)
+        return self.ln(x) * (1 + s) + b
+
+
+class BiMamba3(nn.Module):
+    """Bidirectional Mamba-3 token mixer: a forward and a backward Mamba3Layer, summed. The backward pass reverses
+    each sequence within its own length, so padding never enters the state. Runs the SSD path in fp32."""
+
+    def __init__(self, d, d_state=32, head_dim=32, rank=1, chunk=64):
+        super().__init__()
+        self.fwd = Mamba3Layer(d, d_state=d_state, rank=rank, head_dim=head_dim)
+        self.bwd = Mamba3Layer(d, d_state=d_state, rank=rank, head_dim=head_dim)
+        self.chunk = chunk
+
+    def forward(self, x, mask):  # x [B,T,d], mask [B,T] True=valid
+        T = x.shape[1]
+        t = torch.arange(T, device=x.device)[None]
+        lens = mask.sum(1, keepdim=True)
+        rev = torch.where(t < lens, lens - 1 - t, t)[..., None].expand(-1, -1, x.shape[2])  # an involution
+        with torch.autocast(x.device.type, enabled=False):
+            x = (x * mask[..., None]).float()
+            y = scan_forward_ssd(self.fwd, x, self.chunk, vectorized=True)
+            y = y + torch.gather(scan_forward_ssd(self.bwd, torch.gather(x, 1, rev), self.chunk, vectorized=True), 1, rev)
+        return y * mask[..., None]
+
+
+class FFTBlock(nn.Module):
+    """Pre-norm block: token mixer + conv FFN, all norms adaptive on g.
+    mixer: "attn" (self-attention), "mamba3" (bidirectional Mamba-3), "hybrid" (attention, then a Mamba-3 branch)."""
+
+    def __init__(self, d, heads, ffn, k, g_dim, p=0.1, mixer="attn", ssm_state=32):
+        super().__init__()
+        assert mixer in ("attn", "mamba3", "hybrid"), mixer
+        self.mixer = mixer
+        self.n1 = AdaLN(d, g_dim)
+        if mixer == "mamba3":
+            self.mix = BiMamba3(d, d_state=ssm_state)
+        else:
+            self.att = nn.MultiheadAttention(d, heads, dropout=p, batch_first=True)
+        if mixer == "hybrid":
+            self.n_ssm = AdaLN(d, g_dim)
+            self.ssm = BiMamba3(d, d_state=ssm_state)
+        self.n2 = AdaLN(d, g_dim)
+        self.c1 = nn.Conv1d(d, ffn, k, padding=k // 2)
+        self.c2 = nn.Conv1d(ffn, d, 1)
+        self.drop = nn.Dropout(p)
+
+    def forward(self, x, mask, g):  # mask [B,T] True=valid
+        h = self.n1(x, g)
+        if self.mixer == "mamba3":
+            h = self.mix(h, mask)
+        else:
+            h, _ = self.att(h, h, h, key_padding_mask=~mask, need_weights=False)
+        x = x + self.drop(h)
+        if self.mixer == "hybrid":
+            x = x + self.drop(self.ssm(self.n_ssm(x, g), mask))
+        h = self.n2(x, g).transpose(1, 2)
+        h = self.c2(F.gelu(self.c1(h))).transpose(1, 2)
+        x = x + self.drop(h)
+        return x * mask[..., None]
+
+
+class Stack(nn.Module):
+    def __init__(self, n, d, heads, ffn, k, g_dim, p, mixer="attn", ssm_state=32):
+        super().__init__()
+        self.pos = SinusoidalPos(d)
+        self.layers = nn.ModuleList([FFTBlock(d, heads, ffn, k, g_dim, p, mixer, ssm_state) for _ in range(n)])
+        self.out_norm = AdaLN(d, g_dim)
+
+    def forward(self, x, mask, g):
+        x = x + self.pos(x.shape[1], x.device)[None]
+        for l in self.layers:
+            x = l(x, mask, g)
+        return self.out_norm(x, g) * mask[..., None]
+
+
+class VariancePredictor(nn.Module):
+    def __init__(self, d, g_dim, k=3, p=0.15):
+        super().__init__()
+        self.c1 = nn.Conv1d(d, d, k, padding=k // 2)
+        self.n1 = AdaLN(d, g_dim)
+        self.c2 = nn.Conv1d(d, d, k, padding=k // 2)
+        self.n2 = AdaLN(d, g_dim)
+        self.out = nn.Linear(d, 1)
+        self.drop = nn.Dropout(p)
+
+    def forward(self, x, mask, g):
+        h = F.relu(self.c1(x.transpose(1, 2))).transpose(1, 2)
+        h = self.drop(self.n1(h, g))
+        h = F.relu(self.c2(h.transpose(1, 2))).transpose(1, 2)
+        h = self.drop(self.n2(h, g))
+        return self.out(h).squeeze(-1) * mask
+
+
+def length_regulate(x, dur, max_len=None):
+    """x [B,N,d], dur [B,N] int -> [B,T,d], mel_lens."""
+    lens = dur.sum(1)
+    T = int(max_len or lens.max())
+    idx = torch.cumsum(dur, 1)
+    frames = torch.arange(T, device=x.device)[None]
+    tok = (frames[:, :, None] >= idx[:, None, :]).sum(-1).clamp(max=x.shape[1] - 1)
+    out = torch.gather(x, 1, tok[..., None].expand(-1, -1, x.shape[2]))
+    return out * seq_mask(lens, T)[..., None], lens
+
+
+class SteerTTSModel(nn.Module):
+    """Arguments mirror the training class; `arch` in config.json holds the non-default ones."""
+
+    def __init__(self, n_speakers, n_tags, d=192, heads=2, ffn=384, k=3, enc_layers=4, dec_layers=4, g_dim=128,
+                 spk_dim=64, tag_dim=32, p=0.1, enc_mixer="attn", dec_mixer="attn", ssm_state=32, spk_enc_dim=0,
+                 n_symbols=None, n_env=0, ctrl_dims=None, no_mask=False, ctrl_norm="spk", level_norm=True):
+        super().__init__()
+        assert ctrl_dims is None and not no_mask and ctrl_norm == "spk", "ablation checkpoints are not supported"
+        n_symbols = n_symbols or N_BASE
+        self.emb = nn.Embedding(n_symbols, d, padding_idx=PAD)
+        self.spk = nn.Embedding(n_speakers + 1, spk_dim)  # training-speaker table (unused when a speaker vector is given)
+        self.spk_proj = nn.Linear(spk_enc_dim, spk_dim) if spk_enc_dim else None  # speaker-verification vector -> slot
+        self.tag = nn.Embedding(n_tags, tag_dim)  # index 0 = 'none'
+        self.cond = nn.Sequential(nn.Linear(spk_dim + tag_dim + 2 * N_CTRL, 256), nn.SiLU(), nn.Linear(256, g_dim))
+        # acoustic-environment input of the training code; present in the weights but never trained to do anything
+        # useful in this release, so the pipeline does not expose it
+        self.env_proj = nn.Sequential(nn.Linear(n_env, 128), nn.SiLU(), nn.Linear(128, g_dim)) if n_env else None
+        self.encoder = Stack(enc_layers, d, heads, ffn, k, g_dim, p, enc_mixer, ssm_state)
+        self.decoder = Stack(dec_layers, d, heads, ffn, k, g_dim, p, dec_mixer, ssm_state)
+        self.dur_pred = VariancePredictor(d, g_dim)
+        self.pitch_pred = VariancePredictor(d, g_dim)
+        self.energy_pred = VariancePredictor(d, g_dim)
+        self.pitch_emb = nn.Conv1d(1, d, 3, padding=1)
+        self.energy_emb = nn.Conv1d(1, d, 3, padding=1)
+        self.to_mel = nn.Linear(d, N_MELS)
+
+    def condition(self, spk, tag, ctrl, cmask, spk_vec=None, env=None):
+        """ctrl [B,12] z-values, cmask [B,12] 1 = this dim is requested, 0 = unspecified (the model picks it)."""
+        c = torch.nan_to_num(ctrl) * cmask
+        s = self.spk_proj(spk_vec) if (self.spk_proj is not None and spk_vec is not None) else self.spk(spk)
+        g = self.cond(torch.cat([s, self.tag(tag), c, cmask], -1))
+        if self.env_proj is not None and env is not None:
+            g = g + self.env_proj(env)
+        return g
+
+    def encode(self, ids, text_lens, g):
+        tmask = seq_mask(text_lens, ids.shape[1])
+        e = self.emb(ids)
+        return e, self.encoder(e, tmask, g), tmask
+
+    @torch.no_grad()
+    def infer(self, ids, spk, tag, ctrl, cmask, pace=1.0, word_idx=None, word_shift=None, spk_vec=None, env=None,
+              guidance=1.0, base=None):
+        """ids [1,N]. word_shift: {word_index: dict(pitch=st/PITCH_SCALE, energy=dB/10, dur=factor)}.
+        guidance s != 1 with base=(tag0, ctrl0, cmask0): out = out(base) + s * (out(request) - out(base)) on every
+        stage (durations first, so both passes share one timeline)."""
+        g = self.condition(spk, tag, ctrl, cmask, spk_vec, env)
+        g0 = self.condition(spk, *base, spk_vec, env) if (base is not None and guidance != 1.0) else None
+        text_lens = torch.tensor([ids.shape[1]], device=ids.device)
+        _, h, tmask = self.encode(ids, text_lens, g)
+        logdur, pitch, energy = self.dur_pred(h, tmask, g), self.pitch_pred(h, tmask, g), self.energy_pred(h, tmask, g)
+        if g0 is not None:
+            _, h0, _ = self.encode(ids, text_lens, g0)
+            mix = lambda a, a0: a0 + guidance * (a - a0)
+            logdur = mix(logdur, self.dur_pred(h0, tmask, g0))
+            pitch = mix(pitch, self.pitch_pred(h0, tmask, g0))
+            energy = mix(energy, self.energy_pred(h0, tmask, g0))
+        dur = (torch.exp(logdur) - 1).clamp_min(0) / pace
+        if word_shift and word_idx is not None:
+            for w, sh in word_shift.items():
+                sel = word_idx[0] == w
+                pitch[0, sel] += sh.get("pitch", 0.0)
+                energy[0, sel] += sh.get("energy", 0.0)
+                dur[0, sel] *= sh.get("dur", 1.0)
+        dur = torch.round(dur).long().clamp_min(0)
+        dur[:, 0] = dur[:, 0].clamp_min(1)
+        pe = self.pitch_emb(pitch[:, None]).transpose(1, 2) + self.energy_emb(energy[:, None]).transpose(1, 2)
+        x, lens = length_regulate(h + pe, dur)
+        mel = self.to_mel(self.decoder(x, seq_mask(lens, x.shape[1]), g)).transpose(1, 2)
+        if g0 is not None:
+            x0, _ = length_regulate(h0 + pe, dur)
+            mel0 = self.to_mel(self.decoder(x0, seq_mask(lens, x0.shape[1]), g0)).transpose(1, 2)
+            mel = mel0 + guidance * (mel - mel0)
+        return mel, dict(dur=dur, pitch=pitch, energy=energy, g=g)

build/lib/steertts/pipeline.py

@@ -0,0 +1,270 @@
+"""SteerTTS inference: a request (text + voice + tag / controls / free-text direction) -> 24 kHz waveform.
+
+    from steertts import SteerTTS
+    tts = SteerTTS.from_pretrained("path/to/steertts-v1")            # or an HF repo id; cuda if available
+    wav = tts("I can't believe it worked.", voice="voice_1361", tag="happiness", ctrl={"rate": 1.0})
+    req = tts.resolve({"text": "Hello there.", "prompt": "soft and sad"})  # the request the prompt maps to
+    wav = tts.synthesize(req)
+    for chunk in tts.stream("First sentence. <whisper>Second one.</whisper>"):   # sentence streaming + markup
+        play(chunk.audio)
+    wav = tts("Now in your voice.", ref_wav="me.wav")                 # zero-shot cloning
+
+The rendering path is the training repository's steer.synth.Synth, operation for operation (same text front end,
+model, flow sampler and seed, Vocos), so outputs match it bit for bit on the same device.
+"""
+import json
+import os
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from .audio import load_wav, mel_extractor, save_wav
+from .config import CONTROL_DIMS, DEFAULT_VOICE, PITCH_SCALE, SR
+from .model import SteerTTSModel
+from .request import normalize_request
+from .text import to_phonemes
+
+
+def resolve_model_dir(repo_or_path=None, revision=None, token=None):
+    """Local directory, else a Hugging Face repo id (downloaded once into the HF cache). None -> $STEERTTS_MODEL."""
+    repo_or_path = repo_or_path or os.environ.get("STEERTTS_MODEL")
+    if not repo_or_path:
+        raise ValueError("no model given: pass a local model directory or an HF repo id, or set STEERTTS_MODEL")
+    if os.path.isdir(repo_or_path):
+        return Path(repo_or_path)
+    from huggingface_hub import snapshot_download
+    return Path(snapshot_download(repo_or_path, revision=revision, token=token))
+
+
+class SteerTTS:
+    sample_rate = SR
+    controls = list(CONTROL_DIMS)
+
+    def __init__(self, model_dir, device=None, flow=True, flow_steps=10, flow_temp=0.4, guidance=1.0):
+        """flow: use the flow-matching refiner (and, for cloning, the prompted flow) when the weights are present.
+        guidance: default steering scale (1 = plain conditional render; see the model card for the trade-off)."""
+        from safetensors.torch import load_file
+        d = Path(model_dir)
+        self.model_dir = d
+        self.dev = device or ("cuda" if torch.cuda.is_available() else "cpu")
+        self.cfg = json.load(open(d / "config.json"))
+        stats = json.load(open(d / "control_stats.json"))
+        self.pooled_sd, self.spk_mean, self.tags = stats["pooled_sd"], stats["spk_mean"], list(stats["tags"])
+        self.tag2i = {t: i for i, t in enumerate(self.tags)}
+        self.voice_info = {v["name"]: v for v in json.load(open(d / "voices.json"))["voices"]}
+        ref = load_file(str(d / self.cfg["files"]["speaker_vectors"]))
+        self.voices = list(self.cfg["voices_order"]) if "voices_order" in self.cfg else list(self.voice_info)
+        assert ref["vec"].shape[0] == len(self.voices), "speaker_vectors / voices.json mismatch"
+        self.spk_ref = {v: ref["vec"][i].numpy() for i, v in enumerate(self.voices)}
+        b = self.cfg["acoustic"]
+        self.n_speakers = b["n_speakers"]
+        self.model = SteerTTSModel(b["n_speakers"], b["n_tags"], **b["arch"]).to(self.dev).eval()
+        self.model.load_state_dict(load_file(str(d / self.cfg["files"]["acoustic"]), device=self.dev))
+        from vocos import Vocos
+        self.vocos = Vocos.from_pretrained(self.cfg["vocoder"]).to(self.dev).eval()
+        self.use_flow, self.flow_steps, self.flow_temp, self.guidance = flow, flow_steps, flow_temp, guidance
+        self.flow = self._load_flow("refiner") if flow else None
+        self._pflow, self._xv, self._pm, self._vec_cache = None, None, None, {}
+        self.default_voice = self.cfg.get("default_voice", DEFAULT_VOICE)
+
+    @classmethod
+    def from_pretrained(cls, repo_or_path=None, revision=None, token=None, **kw):
+        return cls(resolve_model_dir(repo_or_path, revision, token), **kw)
+
+    # ------------------------------------------------------------------ lazy parts
+    def _load_flow(self, which):
+        from safetensors.torch import load_file
+        from .flow import Flow
+        spec = self.cfg.get(which)
+        if not spec or not (self.model_dir / spec["file"]).exists():
+            return None
+        return Flow.from_state(load_file(str(self.model_dir / spec["file"])), spec["cfg"], self.dev)
+
+    @property
+    def pflow(self):
+        if self._pflow is None and self.use_flow:
+            self._pflow = self._load_flow("prompted_flow") or False
+        return self._pflow or None
+
+    @property
+    def prompt_mapper(self):
+        if self._pm is None:
+            from safetensors.torch import load_file
+            from .prompt import PromptMapper
+            p = self.cfg["prompt_adapter"]
+            self._pm = PromptMapper(load_file(str(self.model_dir / p["file"])), p["tags"], self.dev)
+        return self._pm
+
+    # ------------------------------------------------------------------ requests
+    def map_prompt(self, prompt):
+        """Free-text direction -> dict(tag, ctrl, top_tags): the tag used is the adapter's highest-ranked tag that
+        this model knows."""
+        r = self.prompt_mapper(prompt)
+        tag = next((t for t, _ in r["top_tags"] if t in self.tag2i), "none")
+        return dict(tag=tag, ctrl=dict(r["ctrl"]), top_tags=[list(t) for t in r["top_tags"]])
+
+    def resolve(self, request=None, **kw):
+        """Validate a request (dict or keyword arguments; see request.schema.json) and return it fully resolved:
+        defaults filled in, the free-text `prompt` mapped to `tag` + `ctrl` (explicit `tag` wins, explicit `ctrl`
+        dims override the prompt's), and the mapping recorded under `prompt_mapping`. The result is itself a valid
+        request and renders identically."""
+        req = normalize_request({**(request or {}), **kw}, self)
+        if req.get("prompt"):
+            m = self.map_prompt(req["prompt"])
+            req["prompt_mapping"] = m
+            if req.get("tag") is None:
+                req["tag"] = m["tag"]
+            req["ctrl"] = {**m["ctrl"], **req.get("ctrl", {})}
+            del req["prompt"]
+        req.setdefault("tag", "none")
+        if req["tag"] is None:
+            req["tag"] = "none"
+        return req
+
+    def synthesize(self, request=None, **kw):
+        """Render a request -> float32 waveform at 24 kHz. Markup requests are rendered segment by segment."""
+        req = self.resolve(request, **kw)
+        if req["markup"]:
+            return self.markup().render(**self._stream_args(req))
+        return self._render(req["text"], self._voice(req), req["tag"], req["ctrl"], req["pace"], req.get("word_shift"),
+                            req.get("ref_wav"), req["guidance"], req["flow"], req["seed"])
+
+    def __call__(self, text, voice=None, **kw):
+        return self.synthesize(dict(kw, text=text, voice=voice) if voice else dict(kw, text=text))
+
+    say = __call__
+
+    def stream(self, text=None, request=None, **kw):
+        """Sentence-level streaming with inline markup: yields Chunk(audio, segment, synth_ms) in order, the first
+        as soon as the first sentence / span is rendered."""
+        if text is not None:
+            kw["text"] = text
+        req = self.resolve(request, **kw)
+        return self.markup().stream(**self._stream_args(req))
+
+    def markup(self):
+        from .markup import MarkupSynth
+        return MarkupSynth(self)
+
+    def _voice(self, req):
+        return None if req.get("ref_wav") else req["voice"]
+
+    def _stream_args(self, req):
+        return dict(text=req["text"], voice=self._voice(req), tag=req["tag"], ctrl=req["ctrl"], pace=req["pace"],
+                    ref_wav=req.get("ref_wav"), guidance=req["guidance"], flow=req["flow"], seed=req["seed"])
+
+    # ------------------------------------------------------------------ rendering (== steer.synth.Synth.__call__)
+    @torch.no_grad()
+    def speaker_vector(self, voice=None, ref_wav=None):
+        """[1,512]: a reference clip's x-vector (path or 24 kHz float array), else the stored vector of a voice."""
+        if ref_wav is not None:
+            key = str(ref_wav) if isinstance(ref_wav, (str, Path)) else None
+            if key is None or key not in self._vec_cache:
+                if self._xv is None:
+                    from .speaker import XVec
+                    self._xv = XVec(self.dev)
+                y, _ = load_wav(ref_wav)
+                v = self._xv([y])[0]
+                if key is not None:
+                    self._vec_cache[key] = v
+            v = self._vec_cache[key] if key is not None else v
+        else:
+            if voice not in self.spk_ref:
+                raise ValueError(f"unknown voice {voice!r}; see .voices")
+            v = self.spk_ref[voice]
+        return torch.from_numpy(np.asarray(v, np.float32))[None].to(self.dev)
+
+    @torch.no_grad()
+    def prompt_mel(self, ref_wav, mel, max_s=8.0):
+        """Log-mel of the reference clip (first max_s s), level-matched to this render; bands above a band-limited
+        clip's bandwidth are filled from the render's own spectral shape."""
+        y, sr = load_wav(ref_wav)
+        if not isinstance(ref_wav, (str, Path)):
+            sr = SR
+        y = y[: int(max_s * SR)]
+        if len(y) < SR // 2:
+            return None
+        pm = mel_extractor()(torch.from_numpy(y)[None]).to(self.dev).float()
+        mel = mel.float()
+        nb = pm.shape[1]
+        if sr < SR:
+            hz = 700 * (10 ** (np.linspace(0, 2595 * np.log10(1 + (SR / 2) / 700), nb + 2) / 2595) - 1)
+            kb = int(np.searchsorted(hz[2:], 0.95 * sr / 2))
+            nb = max(8, min(kb, pm.shape[1]))
+        loud = lambda m: m[:, :nb].mean(1)[0].sort().values[m.shape[-1] // 2:].mean()
+        pm = pm + (loud(mel) - loud(pm))
+        if nb < pm.shape[1]:
+            shape = mel.mean(-1, keepdim=True)
+            pm[:, nb:] = pm[:, nb - 1:nb] + (shape[:, nb:] - shape[:, nb - 1:nb])
+        return pm
+
+    def control_tensors(self, ctrl):
+        """Mask semantics: a dim named in ctrl is requested (value + mask 1); every other dim is unspecified (mask 0)
+        and the model chooses it from voice, tag and text. ctrl={'rate': 0} is NOT the same as omitting rate: it
+        asks for the voice's neutral rate."""
+        c = torch.zeros(1, len(CONTROL_DIMS), device=self.dev)
+        m = torch.zeros(1, len(CONTROL_DIMS), device=self.dev)
+        for k, v in (ctrl or {}).items():
+            i = CONTROL_DIMS.index(k)
+            c[0, i], m[0, i] = float(v), 1.0
+        return c, m
+
+    @torch.no_grad()
+    def _render(self, text, voice=None, tag="none", ctrl=None, pace=1.0, word_shift=None, ref_wav=None,
+                guidance=None, flow=True, seed=0, return_mel=False):
+        if ref_wav is None and voice is None:
+            voice = self.default_voice
+        ids, widx, _ = to_phonemes(text)
+        if not ids:
+            return np.zeros(0, np.float32)
+        n_sym = self.model.emb.num_embeddings
+        if max(ids) >= n_sym:
+            keep = [k for k, i in enumerate(ids) if i < n_sym]
+            ids, widx = [ids[k] for k in keep], [widx[k] for k in keep]
+        ids_t = torch.tensor([ids], device=self.dev)
+        widx_t = torch.tensor([widx], device=self.dev)
+        c, m = self.control_tensors(ctrl)
+        spk = torch.tensor([self.n_speakers], device=self.dev)  # table row: unused, the speaker vector conditions
+        spk_vec = self.speaker_vector(voice, ref_wav)
+        tg = torch.tensor([self.tag2i.get(tag, 0)], device=self.dev)
+        if word_shift:  # user units: pitch in semitones, energy in dB, dur factor
+            word_shift = {int(w): dict(pitch=s.get("pitch", 0) / PITCH_SCALE, energy=s.get("energy", 0) / 10.0,
+                                       dur=s.get("dur", 1.0)) for w, s in word_shift.items()}
+        kw = {}
+        s = self.guidance if guidance is None else guidance
+        if s != 1.0 and (ctrl or tag not in (None, "none")):
+            kw.update(guidance=s, base=(torch.zeros_like(tg), torch.zeros_like(c), torch.zeros_like(m)))
+        mel, aux = self.model.infer(ids_t, spk, tg, c, m, pace=pace, word_idx=widx_t, word_shift=word_shift,
+                                    spk_vec=spk_vec, **kw)
+        fl = None
+        if flow and self.flow is not None:
+            fl = self.pflow if (ref_wav is not None and self.pflow is not None) else self.flow
+        elif flow and ref_wav is not None:
+            fl = self.pflow
+        if fl is not None:
+            pm = self.prompt_mel(ref_wav, mel) if (ref_wav is not None and getattr(fl.net, "prompt", False)) else None
+            if pm is None:
+                mel = fl.sample(mel.float(), aux["g"].float(), steps=self.flow_steps, temp=self.flow_temp, seed=seed)
+            else:
+                P = pm.shape[-1]
+                full = torch.cat([pm, mel.float()], -1)
+                cm = torch.zeros_like(full[:, :1]); cm[..., :P] = 1
+                mel = fl.sample(full, aux["g"].float(), steps=self.flow_steps, temp=self.flow_temp, seed=seed,
+                                ctx=full, cm=cm, cfg=0.0)[..., P:]
+        wav = self.vocos.decode(mel.float()).squeeze().cpu().numpy().astype(np.float32)
+        return (wav, mel) if return_mel else wav
+
+    # ------------------------------------------------------------------ helpers
+    def voice_stats(self, voice):
+        """The voice's neutral raw control means (for steertts.measure.measure_z)."""
+        return self.spk_mean[voice]
+
+    def save(self, wav, path):
+        save_wav(wav, path)
+
+    def __repr__(self):
+        return f"SteerTTS({str(self.model_dir)!r}, device={self.dev!r}, flow={self.flow is not None})"
+
+
+SteerTTSPipeline = SteerTTS  # name used by whissle-sdk <= 0.7

build/lib/steertts/prompt.py

@@ -0,0 +1,69 @@
+"""Free-text voice direction -> request fields (style tag + which controls + their z values).
+
+A frozen sentence encoder (sentence-transformers/all-MiniLM-L6-v2, 22.7M, mean-pooled and L2-normalised exactly
+as the sentence-transformers wrapper does) feeds a ~0.5M-parameter adapter trained on acoustically anchored
+captions: for each control a "specified" probability (> 0.5 -> the dim is set) and a z value, plus a distribution
+over the style tags. Needs `transformers` (the `prompt` extra).
+"""
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .config import CONTROL_DIMS
+
+ENC = "sentence-transformers/all-MiniLM-L6-v2"
+N = len(CONTROL_DIMS)
+
+
+class Adapter(nn.Module):
+    def __init__(self, n_tags, d_in=384, h=512):
+        super().__init__()
+        self.body = nn.Sequential(nn.Linear(d_in, h), nn.GELU(), nn.Dropout(0.1), nn.Linear(h, h), nn.GELU())
+        self.tag = nn.Linear(h, n_tags)
+        self.spec = nn.Linear(h, N)
+        self.val = nn.Linear(h, N)
+
+    def forward(self, e):
+        h = self.body(e)
+        return self.tag(h), self.spec(h), self.val(h)
+
+
+class MiniLM:
+    """all-MiniLM-L6-v2 sentence embedding: token embeddings mean-pooled over the attention mask, L2-normalised,
+    inputs truncated at 256 word pieces (the model's sentence-transformers configuration)."""
+
+    def __init__(self, device="cpu"):
+        from transformers import AutoModel, AutoTokenizer
+        self.tok = AutoTokenizer.from_pretrained(ENC)
+        self.m = AutoModel.from_pretrained(ENC).to(device).eval()
+        self.dev = device
+
+    @torch.no_grad()
+    def __call__(self, texts):
+        b = self.tok(texts, padding=True, truncation=True, max_length=256, return_tensors="pt").to(self.dev)
+        h = self.m(**b).last_hidden_state
+        m = b["attention_mask"][..., None].to(h.dtype)
+        e = (h * m).sum(1) / m.sum(1).clamp(min=1e-9)
+        return F.normalize(e, p=2, dim=1)
+
+
+class PromptMapper:
+    def __init__(self, state_dict, tags, device="cpu"):
+        self.tags, self.dev = list(tags), device
+        self.m = Adapter(len(self.tags)).to(device).eval()
+        self.m.load_state_dict(state_dict)
+        self.enc = MiniLM(device)
+        self.cache = {}
+
+    @torch.no_grad()
+    def __call__(self, prompt, spec_threshold=0.5, top_tags=3):
+        """-> dict(top_tags=[(tag, p), ...], ctrl={dim: z}) for the dims whose specified-probability > threshold."""
+        key = " ".join(prompt.lower().split())
+        if key not in self.cache:
+            lt, ls, lv = self.m(self.enc([prompt]))
+            self.cache[key] = (torch.softmax(lt, -1)[0], torch.sigmoid(ls)[0], lv[0])
+        tp, sp, v = self.cache[key]
+        ctrl = {CONTROL_DIMS[i]: round(float(v[i]), 2) for i in range(N) if sp[i] > spec_threshold}
+        top = torch.topk(tp, top_tags)
+        return dict(top_tags=[(self.tags[i], round(float(p), 3)) for p, i in zip(top.values.tolist(), top.indices.tolist())],
+                    ctrl=ctrl)

build/lib/steertts/request.py

@@ -0,0 +1,95 @@
+"""The synthesis request: one JSON object, documented by request.schema.json (shipped inside this package and at the
+model repo root). normalize_request() validates a request against a loaded model and fills in defaults; it raises
+ValueError with a precise message instead of silently ignoring a misspelt field, tag or control.
+"""
+import json
+from importlib import resources
+
+from .config import CONTROL_DIMS, CONTROL_RANGE
+
+DEFAULTS = dict(pace=1.0, guidance=1.0, flow=True, seed=0, markup=False)
+FIELDS = {"text", "voice", "ref_wav", "prompt", "tag", "ctrl", "pace", "guidance", "word_shift", "markup", "flow",
+          "seed", "prompt_mapping", "$schema"}
+WORD_SHIFT_KEYS = {"pitch": (-12.0, 12.0), "energy": (-20.0, 20.0), "dur": (0.25, 4.0)}
+
+
+def schema():
+    """The JSON Schema (draft 2020-12) of a request, as a dict."""
+    return json.loads(resources.files(__package__).joinpath("request.schema.json").read_text())
+
+
+def _num(name, v, lo, hi):
+    if isinstance(v, bool) or not isinstance(v, (int, float)):
+        raise ValueError(f"{name} must be a number, got {v!r}")
+    if not lo <= float(v) <= hi:
+        raise ValueError(f"{name}={v} is outside [{lo}, {hi}]")
+    return float(v)
+
+
+def normalize_request(req, tts=None):
+    """-> a new dict with defaults filled in. tts (a SteerTTS) enables voice / tag checks."""
+    if not isinstance(req, dict):
+        raise ValueError("a request is a JSON object")
+    unknown = set(req) - FIELDS
+    if unknown:
+        raise ValueError(f"unknown request field(s) {sorted(unknown)}; see request.schema.json")
+    out = {k: v for k, v in req.items() if k != "$schema"}
+    text = out.get("text")
+    if not isinstance(text, str) or not text.strip():
+        raise ValueError("text must be a non-empty string")
+    if len(text) > 5000:
+        raise ValueError("text is longer than 5000 characters; split it (or stream it sentence by sentence)")
+    if out.get("ref_wav") is not None:
+        if out.get("voice"):
+            raise ValueError("give either voice (a built-in voice) or ref_wav (clone a reference clip), not both")
+    else:
+        out.pop("ref_wav", None)
+        out["voice"] = out.get("voice") or (tts.default_voice if tts is not None else None)
+        if tts is not None and out["voice"] not in tts.spk_ref:
+            raise ValueError(f"unknown voice {out['voice']!r}; list them with tts.voices / `steertts voices`")
+    if out.get("prompt") is not None and not isinstance(out["prompt"], str):
+        raise ValueError("prompt must be a string")
+    if out.get("tag") is not None:
+        if not isinstance(out["tag"], str):
+            raise ValueError("tag must be a string")
+        if tts is not None and out["tag"] not in tts.tag2i:
+            raise ValueError(f"unknown tag {out['tag']!r}; known: {', '.join(tts.tags)}")
+    ctrl = out.get("ctrl") or {}
+    if not isinstance(ctrl, dict):
+        raise ValueError("ctrl must be an object {control: z}")
+    for k in ctrl:
+        if k not in CONTROL_DIMS:
+            raise ValueError(f"unknown control {k!r}; known: {', '.join(CONTROL_DIMS)}")
+    out["ctrl"] = {k: _num(f"ctrl.{k}", v, *CONTROL_RANGE) for k, v in ctrl.items()}
+    defaults = dict(DEFAULTS)
+    if tts is not None:  # the loaded model's own defaults (SteerTTS(guidance=..., flow=...))
+        defaults.update(guidance=tts.guidance, flow=bool(tts.use_flow))
+    for k, v in defaults.items():
+        if out.get(k) is None:
+            out[k] = v
+    out["pace"] = _num("pace", out["pace"], 0.25, 4.0)
+    out["guidance"] = _num("guidance", out["guidance"], 0.0, 4.0)
+    for k in ("flow", "markup"):
+        if not isinstance(out[k], bool):
+            raise ValueError(f"{k} must be true or false")
+    if isinstance(out["seed"], bool) or not isinstance(out["seed"], int) or out["seed"] < 0:
+        raise ValueError("seed must be a non-negative integer")
+    ws = out.get("word_shift")
+    if ws:
+        if not isinstance(ws, dict):
+            raise ValueError("word_shift must be an object {word_index: {pitch, energy, dur}}")
+        if out["markup"]:
+            raise ValueError("word_shift and markup cannot be combined (markup re-indexes words per segment)")
+        norm = {}
+        for w, sh in ws.items():
+            try:
+                wi = int(w)
+            except (TypeError, ValueError):
+                raise ValueError(f"word_shift key {w!r} is not a word index") from None
+            if wi < 0 or not isinstance(sh, dict) or set(sh) - set(WORD_SHIFT_KEYS):
+                raise ValueError(f"word_shift[{w!r}] must be a non-negative index -> {{pitch, energy, dur}}")
+            norm[str(wi)] = {k: _num(f"word_shift.{w}.{k}", v, *WORD_SHIFT_KEYS[k]) for k, v in sh.items()}
+        out["word_shift"] = norm
+    else:
+        out.pop("word_shift", None)
+    return out

build/lib/steertts/request.schema.json

@@ -0,0 +1,280 @@
+{
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
+ "$id": "urn:steertts:request-schema:v1",
+ "title": "SteerTTS synthesis request",
+ "description": "One request = one rendered waveform (24 kHz mono float32). Every field except `text` is optional. Controls are SPEAKER-RELATIVE z-scores: 0 = the chosen voice's own neutral speech, +-1 = one pooled within-speaker standard deviation (raw units per dim: control_stats.json pooled_sd). Mask semantics: a control named in `ctrl` is requested (value + availability mask 1); every control NOT named is unspecified (mask 0) and the model chooses it from voice, tag and text. `{\"rate\": 0}` therefore asks for the voice's neutral rate, while omitting `rate` lets the tag/text decide. SteerTTS.resolve(request) returns the fully resolved request (defaults filled, `prompt` mapped to `tag` + `ctrl`, recorded in `prompt_mapping`), which is itself a valid request.",
+ "type": "object",
+ "required": [
+  "text"
+ ],
+ "additionalProperties": false,
+ "properties": {
+  "$schema": {
+   "type": "string"
+  },
+  "text": {
+   "type": "string",
+   "minLength": 1,
+   "maxLength": 5000,
+   "description": "English text. Numbers, units, currency, ordinals, ranges and unknown acronyms are spelled out by the front end. With `markup: true` it may contain inline markup (see `markup`). Event tokens such as [laugh] or [dog] are accepted: the model reserves a short stretch of time for them but renders no sound."
+  },
+  "voice": {
+   "type": "string",
+   "pattern": "^voice_[0-9]{4}$",
+   "description": "Built-in voice (anonymised public name, see voices.json). Default: the model's default_voice (voice_1361). Mutually exclusive with ref_wav."
+  },
+  "ref_wav": {
+   "type": "string",
+   "description": "Path to a reference recording (any sample rate, first 8 s used, >= 0.5 s) for zero-shot cloning: the voice is taken from its WavLM speaker vector and, when flow is on, the prompted flow uses the clip's own mel as context. Mutually exclusive with voice. Only clone voices whose owners consented."
+  },
+  "prompt": {
+   "type": "string",
+   "description": "Free-text voice direction ('soft and sad', 'excited, a bit faster'). Mapped by the prompt adapter to a style tag and a subset of controls. An explicit `tag` overrides the prompt's tag; explicit `ctrl` dims override the prompt's values for those dims."
+  },
+  "tag": {
+   "type": "string",
+   "enum": [
+    "none",
+    "adoration",
+    "amazement",
+    "amusement",
+    "anger",
+    "anxiety",
+    "apologetic",
+    "bored",
+    "confidence",
+    "confusion",
+    "contempt",
+    "contentment",
+    "cuteness",
+    "desire",
+    "determined",
+    "disappointment",
+    "disgust",
+    "distress",
+    "ecstasy",
+    "embarrassment",
+    "emphasis",
+    "enunciated",
+    "excited",
+    "fast",
+    "fear",
+    "frustration",
+    "guilt",
+    "happiness",
+    "hesitant",
+    "high_pitch",
+    "interest",
+    "irritation",
+    "laughing",
+    "loud",
+    "low_pitch",
+    "mysterious",
+    "neutral",
+    "nostalgia",
+    "pain",
+    "panic",
+    "pride",
+    "realization",
+    "relief",
+    "sadness",
+    "sarcasm",
+    "serenity",
+    "skeptical",
+    "slow",
+    "sympathy",
+    "tired",
+    "whisper"
+   ],
+   "default": "none",
+   "description": "Style tag; 'none' = unspecified. Tags act as style switches: voice-quality tags such as whisper change the voice strongly, but an independent emotion classifier recognises only sadness among the emotion tags, and about half of the tags are acoustically close to neutral (see the model card). For a specific acoustic change prefer ctrl."
+  },
+  "ctrl": {
+   "type": "object",
+   "additionalProperties": false,
+   "properties": {
+    "rate": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "articulation rate (phones per second of speech, pauses excluded). + faster"
+    },
+    "loud": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "mean speech level (dB). + louder"
+    },
+    "f0_med": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "median pitch (semitones). + higher"
+    },
+    "f0_iqr": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "pitch range, interquartile (semitones). + livelier, - monotone"
+    },
+    "f0_end": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "pitch slope over the last 300 ms of voicing (st/s). + rising (question-like), - falling"
+    },
+    "pause": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "share of the speech span spent in silent pauses >= 100 ms. + more pauses"
+    },
+    "tilt": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral tilt, energy 1-5 kHz vs 50 Hz-1 kHz (dB). + brighter / pressed"
+    },
+    "voiced": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "voiced share of speech frames. - whispery"
+    },
+    "flat": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral flatness (log). + breathier / noisier"
+    },
+    "centroid": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral centroid (log Hz). + brighter"
+    },
+    "cpps": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "cepstral peak prominence (dB), periodicity. - breathier / rougher"
+    },
+    "h1h2": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "H1-H2 (dB). + breathier"
+    }
+   },
+   "default": {},
+   "description": "Requested controls {dim: z}. Values beyond +-3 are outside most training data."
+  },
+  "pace": {
+   "type": "number",
+   "minimum": 0.25,
+   "maximum": 4.0,
+   "default": 1.0,
+   "description": "Global duration divisor applied after prediction (>1 faster). Unlike ctrl.rate it does not change the style, only the timing."
+  },
+  "guidance": {
+   "type": "number",
+   "minimum": 0.0,
+   "maximum": 4.0,
+   "default": 1.0,
+   "description": "Steering scale s: out = out(no request) + s * (out(request) - out(no request)), on durations, pitch, energy and mel. s > 1 strengthens the requested change (control labels are noisy, so the learned effect is attenuated) at the cost of more leakage into other dims and some quality; only applied when a tag or ctrl is given."
+  },
+  "word_shift": {
+   "type": "object",
+   "description": "Per-word prosody edits {word_index: {pitch: semitones, energy: dB, dur: factor}} (word_index counts words of the normalised text from 0). Not combinable with markup (use *emph*, <slow> etc. there).",
+   "patternProperties": {
+    "^[0-9]+$": {
+     "type": "object",
+     "additionalProperties": false,
+     "properties": {
+      "pitch": {
+       "type": "number",
+       "minimum": -12,
+       "maximum": 12
+      },
+      "energy": {
+       "type": "number",
+       "minimum": -20,
+       "maximum": 20
+      },
+      "dur": {
+       "type": "number",
+       "minimum": 0.25,
+       "maximum": 4
+      }
+     }
+    }
+   },
+   "additionalProperties": false
+  },
+  "markup": {
+   "type": "boolean",
+   "default": false,
+   "description": "Parse inline markup in text: *emphasis* / <emph>..</emph>; per-word <slow> <fast> <high> <low> <loud> <soft>; [pause], [pause 300ms], [pause 1.5s]; span styles <whisper>..</whisper> (any tag) and <style prompt=\"...\">..</style>. The text is rendered sentence by sentence (and per span), joined with 10 ms fades. stream() always parses markup."
+  },
+  "flow": {
+   "type": "boolean",
+   "default": true,
+   "description": "Refine the mel with the flow-matching decoder (refiner for built-in voices, prompted flow for ref_wav). false = the acoustic model's mel straight into Vocos (faster, slightly hazier)."
+  },
+  "seed": {
+   "type": "integer",
+   "minimum": 0,
+   "default": 0,
+   "description": "Noise seed of the flow sampler (renders are deterministic per device for a given seed)."
+  },
+  "prompt_mapping": {
+   "type": "object",
+   "description": "Written by resolve(): what `prompt` mapped to {tag, ctrl, top_tags}. Informational; ignored on input.",
+   "properties": {
+    "tag": {
+     "type": "string"
+    },
+    "ctrl": {
+     "type": "object"
+    },
+    "top_tags": {
+     "type": "array"
+    }
+   }
+  }
+ },
+ "examples": [
+  {
+   "text": "The meeting has been moved to Thursday afternoon at three.",
+   "voice": "voice_1361"
+  },
+  {
+   "text": "I can't believe you did that!",
+   "voice": "voice_1361",
+   "tag": "anger",
+   "ctrl": {
+    "loud": 1.5,
+    "f0_med": 1.0
+   }
+  },
+  {
+   "text": "Are you serious?",
+   "ctrl": {
+    "f0_end": 2.0,
+    "f0_iqr": 1.5
+   }
+  },
+  {
+   "text": "Please keep this between us.",
+   "prompt": "whispering, slow and secretive"
+  },
+  {
+   "text": "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>",
+   "markup": true
+  },
+  {
+   "text": "Now in a cloned voice.",
+   "ref_wav": "reference.wav"
+  }
+ ]
+}

build/lib/steertts/speaker.py

@@ -0,0 +1,27 @@
+"""Speaker vectors for zero-shot cloning: frozen WavLM-base-plus-sv x-vectors (L2-normalised, 512-d), the same
+encoder the acoustic model was trained with. Needs `transformers` (the `clone` extra)."""
+import numpy as np
+import torch
+
+from .config import SR
+
+MODEL = "microsoft/wavlm-base-plus-sv"
+DIM = 512
+
+
+class XVec:
+    def __init__(self, dev="cpu"):
+        from transformers import AutoFeatureExtractor, WavLMForXVector
+        self.fe = AutoFeatureExtractor.from_pretrained(MODEL)
+        self.m = WavLMForXVector.from_pretrained(MODEL).to(dev).eval()
+        self.dev = dev
+
+    @torch.no_grad()
+    def __call__(self, wavs24, max_s=8.0):
+        """list of float32 arrays at 24 kHz -> [n, 512] L2-normalised (first max_s seconds of each)."""
+        import soxr
+        ys = [soxr.resample(w[: int(max_s * SR)].astype(np.float32), SR, 16000) for w in wavs24]
+        ys = [np.tile(y, int(np.ceil(8000 / max(len(y), 1)))) if len(y) < 8000 else y for y in ys]
+        x = self.fe(ys, sampling_rate=16000, return_tensors="pt", padding=True).to(self.dev)
+        e = self.m(**x).embeddings
+        return torch.nn.functional.normalize(e, dim=-1).float().cpu().numpy()

build/lib/steertts/text.py

@@ -0,0 +1,133 @@
+"""Text -> ARPAbet phoneme ids with word indices (g2p_en). Identical to the training front end (steer/text.py)."""
+import re
+from functools import lru_cache
+
+_PUNCT = list(",.?!;:-")
+_ARPA = ["AA", "AE", "AH", "AO", "AW", "AY", "B", "CH", "D", "DH", "EH", "ER", "EY", "F", "G", "HH", "IH", "IY",
+         "JH", "K", "L", "M", "N", "NG", "OW", "OY", "P", "R", "S", "SH", "T", "TH", "UH", "UW", "V", "W", "Y", "Z", "ZH"]
+_VOWELS = {"AA", "AE", "AH", "AO", "AW", "AY", "EH", "ER", "EY", "IH", "IY", "OW", "OY", "UH", "UW"}
+SYMBOLS = ["<pad>", "<sp>"] + _PUNCT + [p + s for p in _ARPA for s in (["0", "1", "2"] if p in _VOWELS else [""])]
+N_BASE = len(SYMBOLS)  # vocabulary of every checkpoint trained before sound events (2026-10-09)
+# sound-event tokens, written inline as [laugh], [dog] ... : in-voice nonverbals first, then environmental events,
+# then reserved slots so the vocabulary can grow without resizing the embedding again. Appended after the phonemes,
+# so ids < N_BASE are unchanged and older checkpoints keep working (Synth drops tokens a checkpoint lacks).
+NONVERBAL = ["laugh", "cry", "scream", "yell", "cheer", "cough", "sneeze", "yawn", "throat", "breath", "sigh", "sniff"]
+EVENTS = ["dog", "cat", "bird", "knock", "door", "phone", "bell", "horn", "applause", "glass", "siren", "footsteps",
+          "typing", "thunder", "clock"]
+EVENT_TOKENS = NONVERBAL + EVENTS + [f"reserved{i}" for i in range(13)]
+SYMBOLS = SYMBOLS + [f"[{e}]" for e in EVENT_TOKENS]
+SYM2ID = {s: i for i, s in enumerate(SYMBOLS)}
+PAD, SP = 0, 1
+_EVENT_RE = re.compile(r"\[\s*([A-Za-z_]+)\s*\]")
+
+
+def _ensure_nltk():
+    """g2p_en needs NLTK's POS tagger + CMUdict. nltk>=3.9 looks the tagger up under its '_eng' name, which
+    g2p_en's own first-run download does not fetch; get whichever is missing once (into ~/nltk_data)."""
+    import nltk
+    for res, name in (("taggers/averaged_perceptron_tagger_eng", "averaged_perceptron_tagger_eng"),
+                      ("taggers/averaged_perceptron_tagger", "averaged_perceptron_tagger"),
+                      ("corpora/cmudict", "cmudict")):
+        try:
+            nltk.data.find(res)
+        except LookupError:
+            nltk.download(name, quiet=True)
+
+
+@lru_cache(maxsize=1)
+def _g2p():
+    _ensure_nltk()
+    from g2p_en import G2p
+    return G2p()
+
+
+_UNITS = {"km/h": "kilometers per hour", "km/hr": "kilometers per hour", "m/s": "meters per second", "mph": "miles per hour",
+          "km": "kilometers", "kg": "kilograms", "cm": "centimeters", "mm": "millimeters", "ft": "feet", "lbs": "pounds",
+          "lb": "pounds", "kHz": "kilohertz", "Hz": "hertz", "GB": "gigabytes", "MB": "megabytes"}
+_CURRENCY = {"$": "dollars", "£": "pounds", "€": "euros"}
+
+
+def _num(s, to="cardinal"):
+    from num2words import num2words
+    return num2words(s, to=to).replace("-", " ").replace(",", "")
+
+
+def _expand(text: str) -> str:
+    """Spell out what g2p_en gets wrong (CV3-Eval en, 2026-10-09: 0.6 + 0.6 WER points of 3.3 came from these).
+    g2p_en looks words up in CMUdict and sends anything else to a neural guesser - including every hyphenated
+    compound ("pay-per-view" -> P EY V ER P IY, "low-tech" -> L OW T CH EH) and "4x4" / "Il-76" / "km/h". It only
+    expands bare integers, reading "2,243" as "two" + "two hundred forty three"."""
+    text = re.sub(r"(\d)\s*[-–]\s*(\d)", r"\1 to \2", text)                     # ranges: 1990-1995, 10-20
+    text = re.sub(r"(\d+)\s*[x×]\s*(\d+)", r"\1 by \2", text)                   # 4x4, 1920x1080
+    text = re.sub(r"(?<=[A-Za-z0-9])[-‐/](?=[A-Za-z0-9])", " ", text)           # compounds: pay-per-view, Il-76, and/or
+    # the amount ends on a digit (a trailing comma is punctuation), and the space after it is only consumed together
+    # with a following "million/billion/thousand" (2026-10-11: "$3,000, 20%" read "three thousand, dollars20")
+    text = re.sub(r"([$£€])\s?(\d(?:[\d,]*\d)?(?:\.\d+)?)(?:\s*(million|billion|thousand)\b)?",
+                  lambda m: f"{m.group(2)} {m.group(3) + ' ' if m.group(3) else ''}{_CURRENCY[m.group(1)]}", text)
+    text = re.sub(r"(\d[\d,]*(?:\.\d+)?)\s?%", r"\1 percent", text)
+    for u in sorted(_UNITS, key=len, reverse=True):                            # 165 km h (after the / split) / 165 km/h
+        pat = re.escape(u).replace("/", "[ /]")
+        text = re.sub(rf"(\d)\s?{pat}\b", rf"\1 {_UNITS[u]}", text)
+    text = re.sub(r"\b(\d+)(st|nd|rd|th)\b", lambda m: _num(int(m.group(1)), "ordinal"), text)
+    text = re.sub(r"\b\d{1,3}(?:,\d{3})+(?:\.\d+)?\b", lambda m: _num(float(m.group(0).replace(",", "")))
+                  if "." in m.group(0) else _num(int(m.group(0).replace(",", ""))), text)
+    text = re.sub(r"\b\d+\.\d+\b", lambda m: _num(float(m.group(0))), text)
+    text = re.sub(r"\b\d{5,}\b", lambda m: _num(int(m.group(0))), text)         # g2p_en reads long digit runs badly
+    cmu = _g2p().cmu  # acronyms CMUdict does not know (IOC, BBC): letter by letter; NASA, UNESCO keep their entry
+    text = re.sub(r"\b[A-Z]{2,5}s?\b", lambda m: m.group(0) if m.group(0).lower() in cmu
+                  else " ".join(m.group(0).rstrip("s")) + ("'s" if m.group(0).endswith("s") and len(m.group(0)) > 2 else ""), text)
+    return text
+
+
+def normalize(text: str) -> str:
+    text = re.sub(r"[\"“”‘’`]", lambda m: "'" if m.group(0) in "‘’`" else "", text)
+    text = _expand(text)  # first: it reads digit-dash-digit as a range before dashes become pauses
+    text = text.replace("—", ", ").replace("–", ", ").replace("…", ". ")
+    text = re.sub(r"\s+", " ", text).strip()
+    return text
+
+
+def to_phonemes(text: str):
+    """Returns (ids, word_idx, words). Word boundaries become <sp> tokens with word_idx -1;
+    punctuation tokens get word_idx -1 as well. A trailing <sp> pads the end. Inline sound events ([laugh], [dog])
+    become one token each, between <sp> tokens, with word_idx -1; unknown [...] markers are dropped."""
+    text = normalize(text)
+    if "[" in text:
+        parts = _EVENT_RE.split(text)  # text, name, text, name, ...
+        if len(parts) > 1:
+            ids, widx, words, nw = [SP], [-1], [], 0
+            for k, p in enumerate(parts):
+                if k % 2:
+                    tok = f"[{p.lower()}]"
+                    if tok in SYM2ID:
+                        ids += [SYM2ID[tok], SP]; widx += [-1, -1]
+                elif p.strip(" []"):
+                    i2, w2, wd = to_phonemes(p.replace("[", " ").replace("]", " "))
+                    ids += i2[1:]; widx += [w + nw if w >= 0 else -1 for w in w2[1:]]
+                    words += wd; nw = max([w for w in widx if w >= 0], default=-1) + 1
+            return ids, widx, words
+    raw = _g2p()(text)
+    ids, widx, words = [SP], [-1], []
+    cur, wi = [], -1
+    for tok in raw + [" "]:
+        if tok == " ":
+            if cur:
+                wi += 1
+                ids += [SYM2ID[p] for p in cur]
+                widx += [wi] * len(cur)
+                cur = []
+                ids.append(SP); widx.append(-1)
+            continue
+        if tok in SYM2ID and tok not in ("<pad>", "<sp>"):
+            if tok in _PUNCT:
+                if cur:
+                    wi += 1
+                    ids += [SYM2ID[p] for p in cur]
+                    widx += [wi] * len(cur)
+                    cur = []
+                ids.append(SYM2ID[tok]); widx.append(-1)
+            else:
+                cur.append(tok)
+    # words list re-derived from the normalised text for display (best effort alignment)
+    words = re.findall(r"[A-Za-z0-9']+", text)
+    return ids, widx, words

config.json

@@ -0,0 +1,2954 @@
+{
+ "model_type": "steer-tts",
+ "release": "steertts-v1",
+ "sample_rate": 24000,
+ "language": "en",
+ "default_voice": "voice_1361",
+ "voices_order": [
+  "voice_0001",
+  "voice_0002",
+  "voice_0003",
+  "voice_0004",
+  "voice_0005",
+  "voice_0006",
+  "voice_0007",
+  "voice_0008",
+  "voice_0009",
+  "voice_0010",
+  "voice_0011",
+  "voice_0012",
+  "voice_0013",
+  "voice_0014",
+  "voice_0015",
+  "voice_0016",
+  "voice_0017",
+  "voice_0018",
+  "voice_0019",
+  "voice_0020",
+  "voice_0021",
+  "voice_0022",
+  "voice_0023",
+  "voice_0024",
+  "voice_0025",
+  "voice_0026",
+  "voice_0027",
+  "voice_0028",
+  "voice_0029",
+  "voice_0030",
+  "voice_0031",
+  "voice_0032",
+  "voice_0033",
+  "voice_0034",
+  "voice_0035",
+  "voice_0036",
+  "voice_0037",
+  "voice_0038",
+  "voice_0039",
+  "voice_0040",
+  "voice_0041",
+  "voice_0042",
+  "voice_0043",
+  "voice_0044",
+  "voice_0045",
+  "voice_0046",
+  "voice_0047",
+  "voice_0048",
+  "voice_0049",
+  "voice_0050",
+  "voice_0051",
+  "voice_0052",
+  "voice_0053",
+  "voice_0054",
+  "voice_0055",
+  "voice_0056",
+  "voice_0057",
+  "voice_0058",
+  "voice_0059",
+  "voice_0060",
+  "voice_0061",
+  "voice_0062",
+  "voice_0063",
+  "voice_0064",
+  "voice_0065",
+  "voice_0066",
+  "voice_0067",
+  "voice_0068",
+  "voice_0069",
+  "voice_0070",
+  "voice_0071",
+  "voice_0072",
+  "voice_0073",
+  "voice_0074",
+  "voice_0075",
+  "voice_0076",
+  "voice_0077",
+  "voice_0078",
+  "voice_0079",
+  "voice_0080",
+  "voice_0081",
+  "voice_0082",
+  "voice_0083",
+  "voice_0084",
+  "voice_0085",
+  "voice_0086",
+  "voice_0087",
+  "voice_0088",
+  "voice_0089",
+  "voice_0090",
+  "voice_0091",
+  "voice_0092",
+  "voice_0093",
+  "voice_0094",
+  "voice_0095",
+  "voice_0096",
+  "voice_0097",
+  "voice_0098",
+  "voice_0099",
+  "voice_0100",
+  "voice_0101",
+  "voice_0102",
+  "voice_0103",
+  "voice_0104",
+  "voice_0105",
+  "voice_0106",
+  "voice_0107",
+  "voice_0108",
+  "voice_0109",
+  "voice_0110",
+  "voice_0111",
+  "voice_0112",
+  "voice_0113",
+  "voice_0114",
+  "voice_0115",
+  "voice_0116",
+  "voice_0117",
+  "voice_0118",
+  "voice_0119",
+  "voice_0120",
+  "voice_0121",
+  "voice_0122",
+  "voice_0123",
+  "voice_0124",
+  "voice_0125",
+  "voice_0126",
+  "voice_0127",
+  "voice_0128",
+  "voice_0129",
+  "voice_0130",
+  "voice_0131",
+  "voice_0132",
+  "voice_0133",
+  "voice_0134",
+  "voice_0135",
+  "voice_0136",
+  "voice_0137",
+  "voice_0138",
+  "voice_0139",
+  "voice_0140",
+  "voice_0141",
+  "voice_0142",
+  "voice_0143",
+  "voice_0144",
+  "voice_0145",
+  "voice_0146",
+  "voice_0147",
+  "voice_0148",
+  "voice_0149",
+  "voice_0150",
+  "voice_0151",
+  "voice_0152",
+  "voice_0153",
+  "voice_0154",
+  "voice_0155",
+  "voice_0156",
+  "voice_0157",
+  "voice_0158",
+  "voice_0159",
+  "voice_0160",
+  "voice_0161",
+  "voice_0162",
+  "voice_0163",
+  "voice_0164",
+  "voice_0165",
+  "voice_0166",
+  "voice_0167",
+  "voice_0168",
+  "voice_0169",
+  "voice_0170",
+  "voice_0171",
+  "voice_0172",
+  "voice_0173",
+  "voice_0174",
+  "voice_0175",
+  "voice_0176",
+  "voice_0177",
+  "voice_0178",
+  "voice_0179",
+  "voice_0180",
+  "voice_0181",
+  "voice_0182",
+  "voice_0183",
+  "voice_0184",
+  "voice_0185",
+  "voice_0186",
+  "voice_0187",
+  "voice_0188",
+  "voice_0189",
+  "voice_0190",
+  "voice_0191",
+  "voice_0192",
+  "voice_0193",
+  "voice_0194",
+  "voice_0195",
+  "voice_0196",
+  "voice_0197",
+  "voice_0198",
+  "voice_0199",
+  "voice_0200",
+  "voice_0201",
+  "voice_0202",
+  "voice_0203",
+  "voice_0204",
+  "voice_0205",
+  "voice_0206",
+  "voice_0207",
+  "voice_0208",
+  "voice_0209",
+  "voice_0210",
+  "voice_0211",
+  "voice_0212",
+  "voice_0213",
+  "voice_0214",
+  "voice_0215",
+  "voice_0216",
+  "voice_0217",
+  "voice_0218",
+  "voice_0219",
+  "voice_0220",
+  "voice_0221",
+  "voice_0222",
+  "voice_0223",
+  "voice_0224",
+  "voice_0225",
+  "voice_0226",
+  "voice_0227",
+  "voice_0228",
+  "voice_0229",
+  "voice_0230",
+  "voice_0231",
+  "voice_0232",
+  "voice_0233",
+  "voice_0234",
+  "voice_0235",
+  "voice_0236",
+  "voice_0237",
+  "voice_0238",
+  "voice_0239",
+  "voice_0240",
+  "voice_0241",
+  "voice_0242",
+  "voice_0243",
+  "voice_0244",
+  "voice_0245",
+  "voice_0246",
+  "voice_0247",
+  "voice_0248",
+  "voice_0249",
+  "voice_0250",
+  "voice_0251",
+  "voice_0252",
+  "voice_0253",
+  "voice_0254",
+  "voice_0255",
+  "voice_0256",
+  "voice_0257",
+  "voice_0258",
+  "voice_0259",
+  "voice_0260",
+  "voice_0261",
+  "voice_0262",
+  "voice_0263",
+  "voice_0264",
+  "voice_0265",
+  "voice_0266",
+  "voice_0267",
+  "voice_0268",
+  "voice_0269",
+  "voice_0270",
+  "voice_0271",
+  "voice_0272",
+  "voice_0273",
+  "voice_0274",
+  "voice_0275",
+  "voice_0276",
+  "voice_0277",
+  "voice_0278",
+  "voice_0279",
+  "voice_0280",
+  "voice_0281",
+  "voice_0282",
+  "voice_0283",
+  "voice_0284",
+  "voice_0285",
+  "voice_0286",
+  "voice_0287",
+  "voice_0288",
+  "voice_0289",
+  "voice_0290",
+  "voice_0291",
+  "voice_0292",
+  "voice_0293",
+  "voice_0294",
+  "voice_0295",
+  "voice_0296",
+  "voice_0297",
+  "voice_0298",
+  "voice_0299",
+  "voice_0300",
+  "voice_0301",
+  "voice_0302",
+  "voice_0303",
+  "voice_0304",
+  "voice_0305",
+  "voice_0306",
+  "voice_0307",
+  "voice_0308",
+  "voice_0309",
+  "voice_0310",
+  "voice_0311",
+  "voice_0312",
+  "voice_0313",
+  "voice_0314",
+  "voice_0315",
+  "voice_0316",
+  "voice_0317",
+  "voice_0318",
+  "voice_0319",
+  "voice_0320",
+  "voice_0321",
+  "voice_0322",
+  "voice_0323",
+  "voice_0324",
+  "voice_0325",
+  "voice_0326",
+  "voice_0327",
+  "voice_0328",
+  "voice_0329",
+  "voice_0330",
+  "voice_0331",
+  "voice_0332",
+  "voice_0333",
+  "voice_0334",
+  "voice_0335",
+  "voice_0336",
+  "voice_0337",
+  "voice_0338",
+  "voice_0339",
+  "voice_0340",
+  "voice_0341",
+  "voice_0342",
+  "voice_0343",
+  "voice_0344",
+  "voice_0345",
+  "voice_0346",
+  "voice_0347",
+  "voice_0348",
+  "voice_0349",
+  "voice_0350",
+  "voice_0351",
+  "voice_0352",
+  "voice_0353",
+  "voice_0354",
+  "voice_0355",
+  "voice_0356",
+  "voice_0357",
+  "voice_0358",
+  "voice_0359",
+  "voice_0360",
+  "voice_0361",
+  "voice_0362",
+  "voice_0363",
+  "voice_0364",
+  "voice_0365",
+  "voice_0366",
+  "voice_0367",
+  "voice_0368",
+  "voice_0369",
+  "voice_0370",
+  "voice_0371",
+  "voice_0372",
+  "voice_0373",
+  "voice_0374",
+  "voice_0375",
+  "voice_0376",
+  "voice_0377",
+  "voice_0378",
+  "voice_0379",
+  "voice_0380",
+  "voice_0381",
+  "voice_0382",
+  "voice_0383",
+  "voice_0384",
+  "voice_0385",
+  "voice_0386",
+  "voice_0387",
+  "voice_0388",
+  "voice_0389",
+  "voice_0390",
+  "voice_0391",
+  "voice_0392",
+  "voice_0393",
+  "voice_0394",
+  "voice_0395",
+  "voice_0396",
+  "voice_0397",
+  "voice_0398",
+  "voice_0399",
+  "voice_0400",
+  "voice_0401",
+  "voice_0402",
+  "voice_0403",
+  "voice_0404",
+  "voice_0405",
+  "voice_0406",
+  "voice_0407",
+  "voice_0408",
+  "voice_0409",
+  "voice_0410",
+  "voice_0411",
+  "voice_0412",
+  "voice_0413",
+  "voice_0414",
+  "voice_0415",
+  "voice_0416",
+  "voice_0417",
+  "voice_0418",
+  "voice_0419",
+  "voice_0420",
+  "voice_0421",
+  "voice_0422",
+  "voice_0423",
+  "voice_0424",
+  "voice_0425",
+  "voice_0426",
+  "voice_0427",
+  "voice_0428",
+  "voice_0429",
+  "voice_0430",
+  "voice_0431",
+  "voice_0432",
+  "voice_0433",
+  "voice_0434",
+  "voice_0435",
+  "voice_0436",
+  "voice_0437",
+  "voice_0438",
+  "voice_0439",
+  "voice_0440",
+  "voice_0441",
+  "voice_0442",
+  "voice_0443",
+  "voice_0444",
+  "voice_0445",
+  "voice_0446",
+  "voice_0447",
+  "voice_0448",
+  "voice_0449",
+  "voice_0450",
+  "voice_0451",
+  "voice_0452",
+  "voice_0453",
+  "voice_0454",
+  "voice_0455",
+  "voice_0456",
+  "voice_0457",
+  "voice_0458",
+  "voice_0459",
+  "voice_0460",
+  "voice_0461",
+  "voice_0462",
+  "voice_0463",
+  "voice_0464",
+  "voice_0465",
+  "voice_0466",
+  "voice_0467",
+  "voice_0468",
+  "voice_0469",
+  "voice_0470",
+  "voice_0471",
+  "voice_0472",
+  "voice_0473",
+  "voice_0474",
+  "voice_0475",
+  "voice_0476",
+  "voice_0477",
+  "voice_0478",
+  "voice_0479",
+  "voice_0480",
+  "voice_0481",
+  "voice_0482",
+  "voice_0483",
+  "voice_0484",
+  "voice_0485",
+  "voice_0486",
+  "voice_0487",
+  "voice_0488",
+  "voice_0489",
+  "voice_0490",
+  "voice_0491",
+  "voice_0492",
+  "voice_0493",
+  "voice_0494",
+  "voice_0495",
+  "voice_0496",
+  "voice_0497",
+  "voice_0498",
+  "voice_0499",
+  "voice_0500",
+  "voice_0501",
+  "voice_0502",
+  "voice_0503",
+  "voice_0504",
+  "voice_0505",
+  "voice_0506",
+  "voice_0507",
+  "voice_0508",
+  "voice_0509",
+  "voice_0510",
+  "voice_0511",
+  "voice_0512",
+  "voice_0513",
+  "voice_0514",
+  "voice_0515",
+  "voice_0516",
+  "voice_0517",
+  "voice_0518",
+  "voice_0519",
+  "voice_0520",
+  "voice_0521",
+  "voice_0522",
+  "voice_0523",
+  "voice_0524",
+  "voice_0525",
+  "voice_0526",
+  "voice_0527",
+  "voice_0528",
+  "voice_0529",
+  "voice_0530",
+  "voice_0531",
+  "voice_0532",
+  "voice_0533",
+  "voice_0534",
+  "voice_0535",
+  "voice_0536",
+  "voice_0537",
+  "voice_0538",
+  "voice_0539",
+  "voice_0540",
+  "voice_0541",
+  "voice_0542",
+  "voice_0543",
+  "voice_0544",
+  "voice_0545",
+  "voice_0546",
+  "voice_0547",
+  "voice_0548",
+  "voice_0549",
+  "voice_0550",
+  "voice_0551",
+  "voice_0552",
+  "voice_0553",
+  "voice_0554",
+  "voice_0555",
+  "voice_0556",
+  "voice_0557",
+  "voice_0558",
+  "voice_0559",
+  "voice_0560",
+  "voice_0561",
+  "voice_0562",
+  "voice_0563",
+  "voice_0564",
+  "voice_0565",
+  "voice_0566",
+  "voice_0567",
+  "voice_0568",
+  "voice_0569",
+  "voice_0570",
+  "voice_0571",
+  "voice_0572",
+  "voice_0573",
+  "voice_0574",
+  "voice_0575",
+  "voice_0576",
+  "voice_0577",
+  "voice_0578",
+  "voice_0579",
+  "voice_0580",
+  "voice_0581",
+  "voice_0582",
+  "voice_0583",
+  "voice_0584",
+  "voice_0585",
+  "voice_0586",
+  "voice_0587",
+  "voice_0588",
+  "voice_0589",
+  "voice_0590",
+  "voice_0591",
+  "voice_0592",
+  "voice_0593",
+  "voice_0594",
+  "voice_0595",
+  "voice_0596",
+  "voice_0597",
+  "voice_0598",
+  "voice_0599",
+  "voice_0600",
+  "voice_0601",
+  "voice_0602",
+  "voice_0603",
+  "voice_0604",
+  "voice_0605",
+  "voice_0606",
+  "voice_0607",
+  "voice_0608",
+  "voice_0609",
+  "voice_0610",
+  "voice_0611",
+  "voice_0612",
+  "voice_0613",
+  "voice_0614",
+  "voice_0615",
+  "voice_0616",
+  "voice_0617",
+  "voice_0618",
+  "voice_0619",
+  "voice_0620",
+  "voice_0621",
+  "voice_0622",
+  "voice_0623",
+  "voice_0624",
+  "voice_0625",
+  "voice_0626",
+  "voice_0627",
+  "voice_0628",
+  "voice_0629",
+  "voice_0630",
+  "voice_0631",
+  "voice_0632",
+  "voice_0633",
+  "voice_0634",
+  "voice_0635",
+  "voice_0636",
+  "voice_0637",
+  "voice_0638",
+  "voice_0639",
+  "voice_0640",
+  "voice_0641",
+  "voice_0642",
+  "voice_0643",
+  "voice_0644",
+  "voice_0645",
+  "voice_0646",
+  "voice_0647",
+  "voice_0648",
+  "voice_0649",
+  "voice_0650",
+  "voice_0651",
+  "voice_0652",
+  "voice_0653",
+  "voice_0654",
+  "voice_0655",
+  "voice_0656",
+  "voice_0657",
+  "voice_0658",
+  "voice_0659",
+  "voice_0660",
+  "voice_0661",
+  "voice_0662",
+  "voice_0663",
+  "voice_0664",
+  "voice_0665",
+  "voice_0666",
+  "voice_0667",
+  "voice_0668",
+  "voice_0669",
+  "voice_0670",
+  "voice_0671",
+  "voice_0672",
+  "voice_0673",
+  "voice_0674",
+  "voice_0675",
+  "voice_0676",
+  "voice_0677",
+  "voice_0678",
+  "voice_0679",
+  "voice_0680",
+  "voice_0681",
+  "voice_0682",
+  "voice_0683",
+  "voice_0684",
+  "voice_0685",
+  "voice_0686",
+  "voice_0687",
+  "voice_0688",
+  "voice_0689",
+  "voice_0690",
+  "voice_0691",
+  "voice_0692",
+  "voice_0693",
+  "voice_0694",
+  "voice_0695",
+  "voice_0696",
+  "voice_0697",
+  "voice_0698",
+  "voice_0699",
+  "voice_0700",
+  "voice_0701",
+  "voice_0702",
+  "voice_0703",
+  "voice_0704",
+  "voice_0705",
+  "voice_0706",
+  "voice_0707",
+  "voice_0708",
+  "voice_0709",
+  "voice_0710",
+  "voice_0711",
+  "voice_0712",
+  "voice_0713",
+  "voice_0714",
+  "voice_0715",
+  "voice_0716",
+  "voice_0717",
+  "voice_0718",
+  "voice_0719",
+  "voice_0720",
+  "voice_0721",
+  "voice_0722",
+  "voice_0723",
+  "voice_0724",
+  "voice_0725",
+  "voice_0726",
+  "voice_0727",
+  "voice_0728",
+  "voice_0729",
+  "voice_0730",
+  "voice_0731",
+  "voice_0732",
+  "voice_0733",
+  "voice_0734",
+  "voice_0735",
+  "voice_0736",
+  "voice_0737",
+  "voice_0738",
+  "voice_0739",
+  "voice_0740",
+  "voice_0741",
+  "voice_0742",
+  "voice_0743",
+  "voice_0744",
+  "voice_0745",
+  "voice_0746",
+  "voice_0747",
+  "voice_0748",
+  "voice_0749",
+  "voice_0750",
+  "voice_0751",
+  "voice_0752",
+  "voice_0753",
+  "voice_0754",
+  "voice_0755",
+  "voice_0756",
+  "voice_0757",
+  "voice_0758",
+  "voice_0759",
+  "voice_0760",
+  "voice_0761",
+  "voice_0762",
+  "voice_0763",
+  "voice_0764",
+  "voice_0765",
+  "voice_0766",
+  "voice_0767",
+  "voice_0768",
+  "voice_0769",
+  "voice_0770",
+  "voice_0771",
+  "voice_0772",
+  "voice_0773",
+  "voice_0774",
+  "voice_0775",
+  "voice_0776",
+  "voice_0777",
+  "voice_0778",
+  "voice_0779",
+  "voice_0780",
+  "voice_0781",
+  "voice_0782",
+  "voice_0783",
+  "voice_0784",
+  "voice_0785",
+  "voice_0786",
+  "voice_0787",
+  "voice_0788",
+  "voice_0789",
+  "voice_0790",
+  "voice_0791",
+  "voice_0792",
+  "voice_0793",
+  "voice_0794",
+  "voice_0795",
+  "voice_0796",
+  "voice_0797",
+  "voice_0798",
+  "voice_0799",
+  "voice_0800",
+  "voice_0801",
+  "voice_0802",
+  "voice_0803",
+  "voice_0804",
+  "voice_0805",
+  "voice_0806",
+  "voice_0807",
+  "voice_0808",
+  "voice_0809",
+  "voice_0810",
+  "voice_0811",
+  "voice_0812",
+  "voice_0813",
+  "voice_0814",
+  "voice_0815",
+  "voice_0816",
+  "voice_0817",
+  "voice_0818",
+  "voice_0819",
+  "voice_0820",
+  "voice_0821",
+  "voice_0822",
+  "voice_0823",
+  "voice_0824",
+  "voice_0825",
+  "voice_0826",
+  "voice_0827",
+  "voice_0828",
+  "voice_0829",
+  "voice_0830",
+  "voice_0831",
+  "voice_0832",
+  "voice_0833",
+  "voice_0834",
+  "voice_0835",
+  "voice_0836",
+  "voice_0837",
+  "voice_0838",
+  "voice_0839",
+  "voice_0840",
+  "voice_0841",
+  "voice_0842",
+  "voice_0843",
+  "voice_0844",
+  "voice_0845",
+  "voice_0846",
+  "voice_0847",
+  "voice_0848",
+  "voice_0849",
+  "voice_0850",
+  "voice_0851",
+  "voice_0852",
+  "voice_0853",
+  "voice_0854",
+  "voice_0855",
+  "voice_0856",
+  "voice_0857",
+  "voice_0858",
+  "voice_0859",
+  "voice_0860",
+  "voice_0861",
+  "voice_0862",
+  "voice_0863",
+  "voice_0864",
+  "voice_0865",
+  "voice_0866",
+  "voice_0867",
+  "voice_0868",
+  "voice_0869",
+  "voice_0870",
+  "voice_0871",
+  "voice_0872",
+  "voice_0873",
+  "voice_0874",
+  "voice_0875",
+  "voice_0876",
+  "voice_0877",
+  "voice_0878",
+  "voice_0879",
+  "voice_0880",
+  "voice_0881",
+  "voice_0882",
+  "voice_0883",
+  "voice_0884",
+  "voice_0885",
+  "voice_0886",
+  "voice_0887",
+  "voice_0888",
+  "voice_0889",
+  "voice_0890",
+  "voice_0891",
+  "voice_0892",
+  "voice_0893",
+  "voice_0894",
+  "voice_0895",
+  "voice_0896",
+  "voice_0897",
+  "voice_0898",
+  "voice_0899",
+  "voice_0900",
+  "voice_0901",
+  "voice_0902",
+  "voice_0903",
+  "voice_0904",
+  "voice_0905",
+  "voice_0906",
+  "voice_0907",
+  "voice_0908",
+  "voice_0909",
+  "voice_0910",
+  "voice_0911",
+  "voice_0912",
+  "voice_0913",
+  "voice_0914",
+  "voice_0915",
+  "voice_0916",
+  "voice_0917",
+  "voice_0918",
+  "voice_0919",
+  "voice_0920",
+  "voice_0921",
+  "voice_0922",
+  "voice_0923",
+  "voice_0924",
+  "voice_0925",
+  "voice_0926",
+  "voice_0927",
+  "voice_0928",
+  "voice_0929",
+  "voice_0930",
+  "voice_0931",
+  "voice_0932",
+  "voice_0933",
+  "voice_0934",
+  "voice_0935",
+  "voice_0936",
+  "voice_0937",
+  "voice_0938",
+  "voice_0939",
+  "voice_0940",
+  "voice_0941",
+  "voice_0942",
+  "voice_0943",
+  "voice_0944",
+  "voice_0945",
+  "voice_0946",
+  "voice_0947",
+  "voice_0948",
+  "voice_0949",
+  "voice_0950",
+  "voice_0951",
+  "voice_0952",
+  "voice_0953",
+  "voice_0954",
+  "voice_0955",
+  "voice_0956",
+  "voice_0957",
+  "voice_0958",
+  "voice_0959",
+  "voice_0960",
+  "voice_0961",
+  "voice_0962",
+  "voice_0963",
+  "voice_0964",
+  "voice_0965",
+  "voice_0966",
+  "voice_0967",
+  "voice_0968",
+  "voice_0969",
+  "voice_0970",
+  "voice_0971",
+  "voice_0972",
+  "voice_0973",
+  "voice_0974",
+  "voice_0975",
+  "voice_0976",
+  "voice_0977",
+  "voice_0978",
+  "voice_0979",
+  "voice_0980",
+  "voice_0981",
+  "voice_0982",
+  "voice_0983",
+  "voice_0984",
+  "voice_0985",
+  "voice_0986",
+  "voice_0987",
+  "voice_0988",
+  "voice_0989",
+  "voice_0990",
+  "voice_0991",
+  "voice_0992",
+  "voice_0993",
+  "voice_0994",
+  "voice_0995",
+  "voice_0996",
+  "voice_0997",
+  "voice_0998",
+  "voice_0999",
+  "voice_1000",
+  "voice_1001",
+  "voice_1002",
+  "voice_1003",
+  "voice_1004",
+  "voice_1005",
+  "voice_1006",
+  "voice_1007",
+  "voice_1008",
+  "voice_1009",
+  "voice_1010",
+  "voice_1011",
+  "voice_1012",
+  "voice_1013",
+  "voice_1014",
+  "voice_1015",
+  "voice_1016",
+  "voice_1017",
+  "voice_1018",
+  "voice_1019",
+  "voice_1020",
+  "voice_1021",
+  "voice_1022",
+  "voice_1023",
+  "voice_1024",
+  "voice_1025",
+  "voice_1026",
+  "voice_1027",
+  "voice_1028",
+  "voice_1029",
+  "voice_1030",
+  "voice_1031",
+  "voice_1032",
+  "voice_1033",
+  "voice_1034",
+  "voice_1035",
+  "voice_1036",
+  "voice_1037",
+  "voice_1038",
+  "voice_1039",
+  "voice_1040",
+  "voice_1041",
+  "voice_1042",
+  "voice_1043",
+  "voice_1044",
+  "voice_1045",
+  "voice_1046",
+  "voice_1047",
+  "voice_1048",
+  "voice_1049",
+  "voice_1050",
+  "voice_1051",
+  "voice_1052",
+  "voice_1053",
+  "voice_1054",
+  "voice_1055",
+  "voice_1056",
+  "voice_1057",
+  "voice_1058",
+  "voice_1059",
+  "voice_1060",
+  "voice_1061",
+  "voice_1062",
+  "voice_1063",
+  "voice_1064",
+  "voice_1065",
+  "voice_1066",
+  "voice_1067",
+  "voice_1068",
+  "voice_1069",
+  "voice_1070",
+  "voice_1071",
+  "voice_1072",
+  "voice_1073",
+  "voice_1074",
+  "voice_1075",
+  "voice_1076",
+  "voice_1077",
+  "voice_1078",
+  "voice_1079",
+  "voice_1080",
+  "voice_1081",
+  "voice_1082",
+  "voice_1083",
+  "voice_1084",
+  "voice_1085",
+  "voice_1086",
+  "voice_1087",
+  "voice_1088",
+  "voice_1089",
+  "voice_1090",
+  "voice_1091",
+  "voice_1092",
+  "voice_1093",
+  "voice_1094",
+  "voice_1095",
+  "voice_1096",
+  "voice_1097",
+  "voice_1098",
+  "voice_1099",
+  "voice_1100",
+  "voice_1101",
+  "voice_1102",
+  "voice_1103",
+  "voice_1104",
+  "voice_1105",
+  "voice_1106",
+  "voice_1107",
+  "voice_1108",
+  "voice_1109",
+  "voice_1110",
+  "voice_1111",
+  "voice_1112",
+  "voice_1113",
+  "voice_1114",
+  "voice_1115",
+  "voice_1116",
+  "voice_1117",
+  "voice_1118",
+  "voice_1119",
+  "voice_1120",
+  "voice_1121",
+  "voice_1122",
+  "voice_1123",
+  "voice_1124",
+  "voice_1125",
+  "voice_1126",
+  "voice_1127",
+  "voice_1128",
+  "voice_1129",
+  "voice_1130",
+  "voice_1131",
+  "voice_1132",
+  "voice_1133",
+  "voice_1134",
+  "voice_1135",
+  "voice_1136",
+  "voice_1137",
+  "voice_1138",
+  "voice_1139",
+  "voice_1140",
+  "voice_1141",
+  "voice_1142",
+  "voice_1143",
+  "voice_1144",
+  "voice_1145",
+  "voice_1146",
+  "voice_1147",
+  "voice_1148",
+  "voice_1149",
+  "voice_1150",
+  "voice_1151",
+  "voice_1152",
+  "voice_1153",
+  "voice_1154",
+  "voice_1155",
+  "voice_1156",
+  "voice_1157",
+  "voice_1158",
+  "voice_1159",
+  "voice_1160",
+  "voice_1161",
+  "voice_1162",
+  "voice_1163",
+  "voice_1164",
+  "voice_1165",
+  "voice_1166",
+  "voice_1167",
+  "voice_1168",
+  "voice_1169",
+  "voice_1170",
+  "voice_1171",
+  "voice_1172",
+  "voice_1173",
+  "voice_1174",
+  "voice_1175",
+  "voice_1176",
+  "voice_1177",
+  "voice_1178",
+  "voice_1179",
+  "voice_1180",
+  "voice_1181",
+  "voice_1182",
+  "voice_1183",
+  "voice_1184",
+  "voice_1185",
+  "voice_1186",
+  "voice_1187",
+  "voice_1188",
+  "voice_1189",
+  "voice_1190",
+  "voice_1191",
+  "voice_1192",
+  "voice_1193",
+  "voice_1194",
+  "voice_1195",
+  "voice_1196",
+  "voice_1197",
+  "voice_1198",
+  "voice_1199",
+  "voice_1200",
+  "voice_1201",
+  "voice_1202",
+  "voice_1203",
+  "voice_1204",
+  "voice_1205",
+  "voice_1206",
+  "voice_1207",
+  "voice_1208",
+  "voice_1209",
+  "voice_1210",
+  "voice_1211",
+  "voice_1212",
+  "voice_1213",
+  "voice_1214",
+  "voice_1215",
+  "voice_1216",
+  "voice_1217",
+  "voice_1218",
+  "voice_1219",
+  "voice_1220",
+  "voice_1221",
+  "voice_1222",
+  "voice_1223",
+  "voice_1224",
+  "voice_1225",
+  "voice_1226",
+  "voice_1227",
+  "voice_1228",
+  "voice_1229",
+  "voice_1230",
+  "voice_1231",
+  "voice_1232",
+  "voice_1233",
+  "voice_1234",
+  "voice_1235",
+  "voice_1236",
+  "voice_1237",
+  "voice_1238",
+  "voice_1239",
+  "voice_1240",
+  "voice_1241",
+  "voice_1242",
+  "voice_1243",
+  "voice_1244",
+  "voice_1245",
+  "voice_1246",
+  "voice_1247",
+  "voice_1248",
+  "voice_1249",
+  "voice_1250",
+  "voice_1251",
+  "voice_1252",
+  "voice_1253",
+  "voice_1254",
+  "voice_1255",
+  "voice_1256",
+  "voice_1257",
+  "voice_1258",
+  "voice_1259",
+  "voice_1260",
+  "voice_1261",
+  "voice_1262",
+  "voice_1263",
+  "voice_1264",
+  "voice_1265",
+  "voice_1266",
+  "voice_1267",
+  "voice_1268",
+  "voice_1269",
+  "voice_1270",
+  "voice_1271",
+  "voice_1272",
+  "voice_1273",
+  "voice_1274",
+  "voice_1275",
+  "voice_1276",
+  "voice_1277",
+  "voice_1278",
+  "voice_1279",
+  "voice_1280",
+  "voice_1281",
+  "voice_1282",
+  "voice_1283",
+  "voice_1284",
+  "voice_1285",
+  "voice_1286",
+  "voice_1287",
+  "voice_1288",
+  "voice_1289",
+  "voice_1290",
+  "voice_1291",
+  "voice_1292",
+  "voice_1293",
+  "voice_1294",
+  "voice_1295",
+  "voice_1296",
+  "voice_1297",
+  "voice_1298",
+  "voice_1299",
+  "voice_1300",
+  "voice_1301",
+  "voice_1302",
+  "voice_1303",
+  "voice_1304",
+  "voice_1305",
+  "voice_1306",
+  "voice_1307",
+  "voice_1308",
+  "voice_1309",
+  "voice_1310",
+  "voice_1311",
+  "voice_1312",
+  "voice_1313",
+  "voice_1314",
+  "voice_1315",
+  "voice_1316",
+  "voice_1317",
+  "voice_1318",
+  "voice_1319",
+  "voice_1320",
+  "voice_1321",
+  "voice_1322",
+  "voice_1323",
+  "voice_1324",
+  "voice_1325",
+  "voice_1326",
+  "voice_1327",
+  "voice_1328",
+  "voice_1329",
+  "voice_1330",
+  "voice_1331",
+  "voice_1332",
+  "voice_1333",
+  "voice_1334",
+  "voice_1335",
+  "voice_1336",
+  "voice_1337",
+  "voice_1338",
+  "voice_1339",
+  "voice_1340",
+  "voice_1341",
+  "voice_1342",
+  "voice_1343",
+  "voice_1344",
+  "voice_1345",
+  "voice_1346",
+  "voice_1347",
+  "voice_1348",
+  "voice_1349",
+  "voice_1350",
+  "voice_1351",
+  "voice_1352",
+  "voice_1353",
+  "voice_1354",
+  "voice_1355",
+  "voice_1356",
+  "voice_1357",
+  "voice_1358",
+  "voice_1359",
+  "voice_1360",
+  "voice_1361",
+  "voice_1362",
+  "voice_1363",
+  "voice_1364",
+  "voice_1365",
+  "voice_1366",
+  "voice_1367",
+  "voice_1368",
+  "voice_1369",
+  "voice_1370",
+  "voice_1371",
+  "voice_1372",
+  "voice_1373",
+  "voice_1374",
+  "voice_1375",
+  "voice_1376",
+  "voice_1377",
+  "voice_1378",
+  "voice_1379",
+  "voice_1380",
+  "voice_1381",
+  "voice_1382",
+  "voice_1383",
+  "voice_1384",
+  "voice_1385",
+  "voice_1386",
+  "voice_1387",
+  "voice_1388",
+  "voice_1389",
+  "voice_1390",
+  "voice_1391",
+  "voice_1392",
+  "voice_1393",
+  "voice_1394",
+  "voice_1395",
+  "voice_1396",
+  "voice_1397",
+  "voice_1398",
+  "voice_1399",
+  "voice_1400",
+  "voice_1401",
+  "voice_1402",
+  "voice_1403",
+  "voice_1404",
+  "voice_1405",
+  "voice_1406",
+  "voice_1407",
+  "voice_1408",
+  "voice_1409",
+  "voice_1410",
+  "voice_1411",
+  "voice_1412",
+  "voice_1413",
+  "voice_1414",
+  "voice_1415",
+  "voice_1416",
+  "voice_1417",
+  "voice_1418",
+  "voice_1419",
+  "voice_1420",
+  "voice_1421",
+  "voice_1422",
+  "voice_1423",
+  "voice_1424",
+  "voice_1425",
+  "voice_1426",
+  "voice_1427",
+  "voice_1428",
+  "voice_1429",
+  "voice_1430",
+  "voice_1431",
+  "voice_1432",
+  "voice_1433",
+  "voice_1434",
+  "voice_1435",
+  "voice_1436",
+  "voice_1437",
+  "voice_1438",
+  "voice_1439",
+  "voice_1440",
+  "voice_1441",
+  "voice_1442",
+  "voice_1443",
+  "voice_1444",
+  "voice_1445",
+  "voice_1446",
+  "voice_1447",
+  "voice_1448",
+  "voice_1449",
+  "voice_1450",
+  "voice_1451",
+  "voice_1452",
+  "voice_1453",
+  "voice_1454",
+  "voice_1455",
+  "voice_1456",
+  "voice_1457",
+  "voice_1458",
+  "voice_1459",
+  "voice_1460",
+  "voice_1461",
+  "voice_1462",
+  "voice_1463",
+  "voice_1464",
+  "voice_1465",
+  "voice_1466",
+  "voice_1467",
+  "voice_1468",
+  "voice_1469",
+  "voice_1470",
+  "voice_1471",
+  "voice_1472",
+  "voice_1473",
+  "voice_1474",
+  "voice_1475",
+  "voice_1476",
+  "voice_1477",
+  "voice_1478",
+  "voice_1479",
+  "voice_1480",
+  "voice_1481",
+  "voice_1482",
+  "voice_1483",
+  "voice_1484",
+  "voice_1485",
+  "voice_1486",
+  "voice_1487",
+  "voice_1488",
+  "voice_1489",
+  "voice_1490",
+  "voice_1491",
+  "voice_1492",
+  "voice_1493",
+  "voice_1494",
+  "voice_1495",
+  "voice_1496",
+  "voice_1497",
+  "voice_1498",
+  "voice_1499",
+  "voice_1500",
+  "voice_1501",
+  "voice_1502",
+  "voice_1503",
+  "voice_1504",
+  "voice_1505",
+  "voice_1506",
+  "voice_1507",
+  "voice_1508",
+  "voice_1509",
+  "voice_1510",
+  "voice_1511",
+  "voice_1512",
+  "voice_1513",
+  "voice_1514",
+  "voice_1515",
+  "voice_1516",
+  "voice_1517",
+  "voice_1518",
+  "voice_1519",
+  "voice_1520",
+  "voice_1521",
+  "voice_1522",
+  "voice_1523",
+  "voice_1524",
+  "voice_1525",
+  "voice_1526",
+  "voice_1527",
+  "voice_1528",
+  "voice_1529",
+  "voice_1530",
+  "voice_1531",
+  "voice_1532",
+  "voice_1533",
+  "voice_1534",
+  "voice_1535",
+  "voice_1536",
+  "voice_1537",
+  "voice_1538",
+  "voice_1539",
+  "voice_1540",
+  "voice_1541",
+  "voice_1542",
+  "voice_1543",
+  "voice_1544",
+  "voice_1545",
+  "voice_1546",
+  "voice_1547",
+  "voice_1548",
+  "voice_1549",
+  "voice_1550",
+  "voice_1551",
+  "voice_1552",
+  "voice_1553",
+  "voice_1554",
+  "voice_1555",
+  "voice_1556",
+  "voice_1557",
+  "voice_1558",
+  "voice_1559",
+  "voice_1560",
+  "voice_1561",
+  "voice_1562",
+  "voice_1563",
+  "voice_1564",
+  "voice_1565",
+  "voice_1566",
+  "voice_1567",
+  "voice_1568",
+  "voice_1569",
+  "voice_1570",
+  "voice_1571",
+  "voice_1572",
+  "voice_1573",
+  "voice_1574",
+  "voice_1575",
+  "voice_1576",
+  "voice_1577",
+  "voice_1578",
+  "voice_1579",
+  "voice_1580",
+  "voice_1581",
+  "voice_1582",
+  "voice_1583",
+  "voice_1584",
+  "voice_1585",
+  "voice_1586",
+  "voice_1587",
+  "voice_1588",
+  "voice_1589",
+  "voice_1590",
+  "voice_1591",
+  "voice_1592",
+  "voice_1593",
+  "voice_1594",
+  "voice_1595",
+  "voice_1596",
+  "voice_1597",
+  "voice_1598",
+  "voice_1599",
+  "voice_1600",
+  "voice_1601",
+  "voice_1602",
+  "voice_1603",
+  "voice_1604",
+  "voice_1605",
+  "voice_1606",
+  "voice_1607",
+  "voice_1608",
+  "voice_1609",
+  "voice_1610",
+  "voice_1611",
+  "voice_1612",
+  "voice_1613",
+  "voice_1614",
+  "voice_1615",
+  "voice_1616",
+  "voice_1617",
+  "voice_1618",
+  "voice_1619",
+  "voice_1620",
+  "voice_1621",
+  "voice_1622",
+  "voice_1623",
+  "voice_1624",
+  "voice_1625",
+  "voice_1626",
+  "voice_1627",
+  "voice_1628",
+  "voice_1629",
+  "voice_1630",
+  "voice_1631",
+  "voice_1632",
+  "voice_1633",
+  "voice_1634",
+  "voice_1635",
+  "voice_1636",
+  "voice_1637",
+  "voice_1638",
+  "voice_1639",
+  "voice_1640",
+  "voice_1641",
+  "voice_1642",
+  "voice_1643",
+  "voice_1644",
+  "voice_1645",
+  "voice_1646",
+  "voice_1647",
+  "voice_1648",
+  "voice_1649",
+  "voice_1650",
+  "voice_1651",
+  "voice_1652",
+  "voice_1653",
+  "voice_1654",
+  "voice_1655",
+  "voice_1656",
+  "voice_1657",
+  "voice_1658",
+  "voice_1659",
+  "voice_1660",
+  "voice_1661",
+  "voice_1662",
+  "voice_1663",
+  "voice_1664",
+  "voice_1665",
+  "voice_1666",
+  "voice_1667",
+  "voice_1668",
+  "voice_1669",
+  "voice_1670",
+  "voice_1671",
+  "voice_1672",
+  "voice_1673",
+  "voice_1674",
+  "voice_1675",
+  "voice_1676",
+  "voice_1677",
+  "voice_1678",
+  "voice_1679",
+  "voice_1680",
+  "voice_1681",
+  "voice_1682",
+  "voice_1683",
+  "voice_1684",
+  "voice_1685",
+  "voice_1686",
+  "voice_1687",
+  "voice_1688",
+  "voice_1689",
+  "voice_1690",
+  "voice_1691",
+  "voice_1692",
+  "voice_1693",
+  "voice_1694",
+  "voice_1695",
+  "voice_1696",
+  "voice_1697",
+  "voice_1698",
+  "voice_1699",
+  "voice_1700",
+  "voice_1701",
+  "voice_1702",
+  "voice_1703",
+  "voice_1704",
+  "voice_1705",
+  "voice_1706",
+  "voice_1707",
+  "voice_1708",
+  "voice_1709",
+  "voice_1710",
+  "voice_1711",
+  "voice_1712",
+  "voice_1713",
+  "voice_1714",
+  "voice_1715",
+  "voice_1716",
+  "voice_1717",
+  "voice_1718",
+  "voice_1719",
+  "voice_1720",
+  "voice_1721",
+  "voice_1722",
+  "voice_1723",
+  "voice_1724",
+  "voice_1725",
+  "voice_1726",
+  "voice_1727",
+  "voice_1728",
+  "voice_1729",
+  "voice_1730",
+  "voice_1731",
+  "voice_1732",
+  "voice_1733",
+  "voice_1734",
+  "voice_1735",
+  "voice_1736",
+  "voice_1737",
+  "voice_1738",
+  "voice_1739",
+  "voice_1740",
+  "voice_1741",
+  "voice_1742",
+  "voice_1743",
+  "voice_1744",
+  "voice_1745",
+  "voice_1746",
+  "voice_1747",
+  "voice_1748",
+  "voice_1749",
+  "voice_1750",
+  "voice_1751",
+  "voice_1752",
+  "voice_1753",
+  "voice_1754",
+  "voice_1755",
+  "voice_1756",
+  "voice_1757",
+  "voice_1758",
+  "voice_1759",
+  "voice_1760",
+  "voice_1761",
+  "voice_1762",
+  "voice_1763",
+  "voice_1764",
+  "voice_1765",
+  "voice_1766",
+  "voice_1767",
+  "voice_1768",
+  "voice_1769",
+  "voice_1770",
+  "voice_1771",
+  "voice_1772",
+  "voice_1773",
+  "voice_1774",
+  "voice_1775",
+  "voice_1776",
+  "voice_1777",
+  "voice_1778",
+  "voice_1779",
+  "voice_1780",
+  "voice_1781",
+  "voice_1782",
+  "voice_1783",
+  "voice_1784",
+  "voice_1785",
+  "voice_1786",
+  "voice_1787",
+  "voice_1788",
+  "voice_1789",
+  "voice_1790",
+  "voice_1791",
+  "voice_1792",
+  "voice_1793",
+  "voice_1794",
+  "voice_1795",
+  "voice_1796",
+  "voice_1797",
+  "voice_1798",
+  "voice_1799",
+  "voice_1800",
+  "voice_1801",
+  "voice_1802",
+  "voice_1803",
+  "voice_1804",
+  "voice_1805",
+  "voice_1806",
+  "voice_1807",
+  "voice_1808",
+  "voice_1809",
+  "voice_1810",
+  "voice_1811",
+  "voice_1812",
+  "voice_1813",
+  "voice_1814",
+  "voice_1815",
+  "voice_1816",
+  "voice_1817",
+  "voice_1818",
+  "voice_1819",
+  "voice_1820",
+  "voice_1821",
+  "voice_1822",
+  "voice_1823",
+  "voice_1824",
+  "voice_1825",
+  "voice_1826",
+  "voice_1827",
+  "voice_1828",
+  "voice_1829",
+  "voice_1830",
+  "voice_1831",
+  "voice_1832",
+  "voice_1833",
+  "voice_1834",
+  "voice_1835",
+  "voice_1836",
+  "voice_1837",
+  "voice_1838",
+  "voice_1839",
+  "voice_1840",
+  "voice_1841",
+  "voice_1842",
+  "voice_1843",
+  "voice_1844",
+  "voice_1845",
+  "voice_1846",
+  "voice_1847",
+  "voice_1848",
+  "voice_1849",
+  "voice_1850",
+  "voice_1851",
+  "voice_1852",
+  "voice_1853",
+  "voice_1854",
+  "voice_1855",
+  "voice_1856",
+  "voice_1857",
+  "voice_1858",
+  "voice_1859",
+  "voice_1860",
+  "voice_1861",
+  "voice_1862",
+  "voice_1863",
+  "voice_1864",
+  "voice_1865",
+  "voice_1866",
+  "voice_1867",
+  "voice_1868",
+  "voice_1869",
+  "voice_1870",
+  "voice_1871",
+  "voice_1872",
+  "voice_1873",
+  "voice_1874",
+  "voice_1875",
+  "voice_1876",
+  "voice_1877",
+  "voice_1878",
+  "voice_1879",
+  "voice_1880",
+  "voice_1881",
+  "voice_1882",
+  "voice_1883",
+  "voice_1884",
+  "voice_1885",
+  "voice_1886",
+  "voice_1887",
+  "voice_1888",
+  "voice_1889",
+  "voice_1890",
+  "voice_1891",
+  "voice_1892",
+  "voice_1893",
+  "voice_1894",
+  "voice_1895",
+  "voice_1896",
+  "voice_1897",
+  "voice_1898",
+  "voice_1899",
+  "voice_1900",
+  "voice_1901",
+  "voice_1902",
+  "voice_1903",
+  "voice_1904",
+  "voice_1905",
+  "voice_1906",
+  "voice_1907",
+  "voice_1908",
+  "voice_1909",
+  "voice_1910",
+  "voice_1911",
+  "voice_1912",
+  "voice_1913",
+  "voice_1914",
+  "voice_1915",
+  "voice_1916",
+  "voice_1917",
+  "voice_1918",
+  "voice_1919",
+  "voice_1920",
+  "voice_1921",
+  "voice_1922",
+  "voice_1923",
+  "voice_1924",
+  "voice_1925",
+  "voice_1926",
+  "voice_1927",
+  "voice_1928",
+  "voice_1929",
+  "voice_1930",
+  "voice_1931",
+  "voice_1932",
+  "voice_1933",
+  "voice_1934",
+  "voice_1935",
+  "voice_1936",
+  "voice_1937",
+  "voice_1938",
+  "voice_1939",
+  "voice_1940",
+  "voice_1941",
+  "voice_1942",
+  "voice_1943",
+  "voice_1944",
+  "voice_1945",
+  "voice_1946",
+  "voice_1947",
+  "voice_1948",
+  "voice_1949",
+  "voice_1950",
+  "voice_1951",
+  "voice_1952",
+  "voice_1953",
+  "voice_1954",
+  "voice_1955",
+  "voice_1956",
+  "voice_1957",
+  "voice_1958",
+  "voice_1959",
+  "voice_1960",
+  "voice_1961",
+  "voice_1962",
+  "voice_1963",
+  "voice_1964",
+  "voice_1965",
+  "voice_1966",
+  "voice_1967",
+  "voice_1968",
+  "voice_1969",
+  "voice_1970",
+  "voice_1971",
+  "voice_1972",
+  "voice_1973",
+  "voice_1974",
+  "voice_1975",
+  "voice_1976",
+  "voice_1977",
+  "voice_1978",
+  "voice_1979",
+  "voice_1980",
+  "voice_1981",
+  "voice_1982",
+  "voice_1983",
+  "voice_1984",
+  "voice_1985",
+  "voice_1986",
+  "voice_1987",
+  "voice_1988",
+  "voice_1989",
+  "voice_1990",
+  "voice_1991",
+  "voice_1992",
+  "voice_1993",
+  "voice_1994",
+  "voice_1995",
+  "voice_1996",
+  "voice_1997",
+  "voice_1998",
+  "voice_1999",
+  "voice_2000",
+  "voice_2001",
+  "voice_2002",
+  "voice_2003",
+  "voice_2004",
+  "voice_2005",
+  "voice_2006",
+  "voice_2007",
+  "voice_2008",
+  "voice_2009",
+  "voice_2010",
+  "voice_2011",
+  "voice_2012",
+  "voice_2013",
+  "voice_2014",
+  "voice_2015",
+  "voice_2016",
+  "voice_2017",
+  "voice_2018",
+  "voice_2019",
+  "voice_2020",
+  "voice_2021",
+  "voice_2022",
+  "voice_2023",
+  "voice_2024",
+  "voice_2025",
+  "voice_2026",
+  "voice_2027",
+  "voice_2028",
+  "voice_2029",
+  "voice_2030",
+  "voice_2031",
+  "voice_2032",
+  "voice_2033",
+  "voice_2034",
+  "voice_2035",
+  "voice_2036",
+  "voice_2037",
+  "voice_2038",
+  "voice_2039",
+  "voice_2040",
+  "voice_2041",
+  "voice_2042",
+  "voice_2043",
+  "voice_2044",
+  "voice_2045",
+  "voice_2046",
+  "voice_2047",
+  "voice_2048",
+  "voice_2049",
+  "voice_2050",
+  "voice_2051",
+  "voice_2052",
+  "voice_2053",
+  "voice_2054",
+  "voice_2055",
+  "voice_2056",
+  "voice_2057",
+  "voice_2058",
+  "voice_2059",
+  "voice_2060",
+  "voice_2061",
+  "voice_2062",
+  "voice_2063",
+  "voice_2064",
+  "voice_2065",
+  "voice_2066",
+  "voice_2067",
+  "voice_2068",
+  "voice_2069",
+  "voice_2070",
+  "voice_2071",
+  "voice_2072",
+  "voice_2073",
+  "voice_2074",
+  "voice_2075",
+  "voice_2076",
+  "voice_2077",
+  "voice_2078",
+  "voice_2079",
+  "voice_2080",
+  "voice_2081",
+  "voice_2082",
+  "voice_2083",
+  "voice_2084",
+  "voice_2085",
+  "voice_2086",
+  "voice_2087",
+  "voice_2088",
+  "voice_2089",
+  "voice_2090",
+  "voice_2091",
+  "voice_2092",
+  "voice_2093",
+  "voice_2094",
+  "voice_2095",
+  "voice_2096",
+  "voice_2097",
+  "voice_2098",
+  "voice_2099",
+  "voice_2100",
+  "voice_2101",
+  "voice_2102",
+  "voice_2103",
+  "voice_2104",
+  "voice_2105",
+  "voice_2106",
+  "voice_2107",
+  "voice_2108",
+  "voice_2109",
+  "voice_2110",
+  "voice_2111",
+  "voice_2112",
+  "voice_2113",
+  "voice_2114",
+  "voice_2115",
+  "voice_2116",
+  "voice_2117",
+  "voice_2118",
+  "voice_2119",
+  "voice_2120",
+  "voice_2121",
+  "voice_2122",
+  "voice_2123",
+  "voice_2124",
+  "voice_2125",
+  "voice_2126",
+  "voice_2127",
+  "voice_2128",
+  "voice_2129",
+  "voice_2130",
+  "voice_2131",
+  "voice_2132",
+  "voice_2133",
+  "voice_2134",
+  "voice_2135",
+  "voice_2136",
+  "voice_2137",
+  "voice_2138",
+  "voice_2139",
+  "voice_2140",
+  "voice_2141",
+  "voice_2142",
+  "voice_2143",
+  "voice_2144",
+  "voice_2145",
+  "voice_2146",
+  "voice_2147",
+  "voice_2148",
+  "voice_2149",
+  "voice_2150",
+  "voice_2151",
+  "voice_2152",
+  "voice_2153",
+  "voice_2154",
+  "voice_2155",
+  "voice_2156",
+  "voice_2157",
+  "voice_2158",
+  "voice_2159",
+  "voice_2160",
+  "voice_2161",
+  "voice_2162",
+  "voice_2163",
+  "voice_2164",
+  "voice_2165",
+  "voice_2166",
+  "voice_2167",
+  "voice_2168",
+  "voice_2169",
+  "voice_2170",
+  "voice_2171",
+  "voice_2172",
+  "voice_2173",
+  "voice_2174",
+  "voice_2175",
+  "voice_2176",
+  "voice_2177",
+  "voice_2178",
+  "voice_2179",
+  "voice_2180",
+  "voice_2181",
+  "voice_2182",
+  "voice_2183",
+  "voice_2184",
+  "voice_2185",
+  "voice_2186",
+  "voice_2187",
+  "voice_2188",
+  "voice_2189",
+  "voice_2190",
+  "voice_2191",
+  "voice_2192",
+  "voice_2193",
+  "voice_2194",
+  "voice_2195",
+  "voice_2196",
+  "voice_2197",
+  "voice_2198",
+  "voice_2199",
+  "voice_2200",
+  "voice_2201",
+  "voice_2202",
+  "voice_2203",
+  "voice_2204",
+  "voice_2205",
+  "voice_2206",
+  "voice_2207",
+  "voice_2208",
+  "voice_2209",
+  "voice_2210",
+  "voice_2211",
+  "voice_2212",
+  "voice_2213",
+  "voice_2214",
+  "voice_2215",
+  "voice_2216",
+  "voice_2217",
+  "voice_2218",
+  "voice_2219",
+  "voice_2220",
+  "voice_2221",
+  "voice_2222",
+  "voice_2223",
+  "voice_2224",
+  "voice_2225",
+  "voice_2226",
+  "voice_2227",
+  "voice_2228",
+  "voice_2229",
+  "voice_2230",
+  "voice_2231",
+  "voice_2232",
+  "voice_2233",
+  "voice_2234",
+  "voice_2235",
+  "voice_2236",
+  "voice_2237",
+  "voice_2238",
+  "voice_2239",
+  "voice_2240",
+  "voice_2241",
+  "voice_2242",
+  "voice_2243",
+  "voice_2244",
+  "voice_2245",
+  "voice_2246",
+  "voice_2247",
+  "voice_2248",
+  "voice_2249",
+  "voice_2250",
+  "voice_2251",
+  "voice_2252",
+  "voice_2253",
+  "voice_2254",
+  "voice_2255",
+  "voice_2256",
+  "voice_2257",
+  "voice_2258",
+  "voice_2259",
+  "voice_2260",
+  "voice_2261",
+  "voice_2262",
+  "voice_2263",
+  "voice_2264",
+  "voice_2265",
+  "voice_2266",
+  "voice_2267",
+  "voice_2268",
+  "voice_2269",
+  "voice_2270",
+  "voice_2271",
+  "voice_2272",
+  "voice_2273",
+  "voice_2274",
+  "voice_2275",
+  "voice_2276",
+  "voice_2277",
+  "voice_2278",
+  "voice_2279",
+  "voice_2280",
+  "voice_2281",
+  "voice_2282",
+  "voice_2283",
+  "voice_2284",
+  "voice_2285",
+  "voice_2286",
+  "voice_2287",
+  "voice_2288",
+  "voice_2289",
+  "voice_2290",
+  "voice_2291",
+  "voice_2292",
+  "voice_2293",
+  "voice_2294",
+  "voice_2295",
+  "voice_2296",
+  "voice_2297",
+  "voice_2298",
+  "voice_2299",
+  "voice_2300",
+  "voice_2301",
+  "voice_2302",
+  "voice_2303",
+  "voice_2304",
+  "voice_2305",
+  "voice_2306",
+  "voice_2307",
+  "voice_2308",
+  "voice_2309",
+  "voice_2310",
+  "voice_2311",
+  "voice_2312",
+  "voice_2313",
+  "voice_2314",
+  "voice_2315",
+  "voice_2316",
+  "voice_2317",
+  "voice_2318",
+  "voice_2319",
+  "voice_2320",
+  "voice_2321",
+  "voice_2322",
+  "voice_2323",
+  "voice_2324",
+  "voice_2325",
+  "voice_2326",
+  "voice_2327",
+  "voice_2328",
+  "voice_2329",
+  "voice_2330",
+  "voice_2331",
+  "voice_2332",
+  "voice_2333",
+  "voice_2334",
+  "voice_2335",
+  "voice_2336",
+  "voice_2337",
+  "voice_2338",
+  "voice_2339",
+  "voice_2340",
+  "voice_2341",
+  "voice_2342",
+  "voice_2343",
+  "voice_2344",
+  "voice_2345",
+  "voice_2346",
+  "voice_2347",
+  "voice_2348",
+  "voice_2349",
+  "voice_2350",
+  "voice_2351",
+  "voice_2352",
+  "voice_2353",
+  "voice_2354",
+  "voice_2355",
+  "voice_2356",
+  "voice_2357",
+  "voice_2358",
+  "voice_2359",
+  "voice_2360",
+  "voice_2361",
+  "voice_2362",
+  "voice_2363",
+  "voice_2364",
+  "voice_2365",
+  "voice_2366",
+  "voice_2367",
+  "voice_2368",
+  "voice_2369",
+  "voice_2370",
+  "voice_2371",
+  "voice_2372",
+  "voice_2373",
+  "voice_2374",
+  "voice_2375",
+  "voice_2376",
+  "voice_2377",
+  "voice_2378",
+  "voice_2379",
+  "voice_2380",
+  "voice_2381",
+  "voice_2382",
+  "voice_2383",
+  "voice_2384",
+  "voice_2385",
+  "voice_2386",
+  "voice_2387",
+  "voice_2388",
+  "voice_2389",
+  "voice_2390",
+  "voice_2391",
+  "voice_2392",
+  "voice_2393",
+  "voice_2394",
+  "voice_2395",
+  "voice_2396",
+  "voice_2397",
+  "voice_2398",
+  "voice_2399",
+  "voice_2400",
+  "voice_2401",
+  "voice_2402",
+  "voice_2403",
+  "voice_2404",
+  "voice_2405",
+  "voice_2406",
+  "voice_2407",
+  "voice_2408",
+  "voice_2409",
+  "voice_2410",
+  "voice_2411",
+  "voice_2412",
+  "voice_2413",
+  "voice_2414",
+  "voice_2415",
+  "voice_2416",
+  "voice_2417",
+  "voice_2418",
+  "voice_2419",
+  "voice_2420",
+  "voice_2421",
+  "voice_2422",
+  "voice_2423",
+  "voice_2424",
+  "voice_2425",
+  "voice_2426",
+  "voice_2427",
+  "voice_2428",
+  "voice_2429",
+  "voice_2430",
+  "voice_2431",
+  "voice_2432",
+  "voice_2433",
+  "voice_2434",
+  "voice_2435",
+  "voice_2436",
+  "voice_2437",
+  "voice_2438",
+  "voice_2439",
+  "voice_2440",
+  "voice_2441",
+  "voice_2442",
+  "voice_2443",
+  "voice_2444",
+  "voice_2445",
+  "voice_2446",
+  "voice_2447",
+  "voice_2448",
+  "voice_2449",
+  "voice_2450",
+  "voice_2451",
+  "voice_2452",
+  "voice_2453",
+  "voice_2454",
+  "voice_2455",
+  "voice_2456",
+  "voice_2457",
+  "voice_2458",
+  "voice_2459",
+  "voice_2460",
+  "voice_2461",
+  "voice_2462",
+  "voice_2463",
+  "voice_2464",
+  "voice_2465",
+  "voice_2466",
+  "voice_2467",
+  "voice_2468",
+  "voice_2469",
+  "voice_2470",
+  "voice_2471",
+  "voice_2472",
+  "voice_2473",
+  "voice_2474",
+  "voice_2475",
+  "voice_2476",
+  "voice_2477",
+  "voice_2478",
+  "voice_2479",
+  "voice_2480",
+  "voice_2481",
+  "voice_2482",
+  "voice_2483",
+  "voice_2484",
+  "voice_2485",
+  "voice_2486",
+  "voice_2487",
+  "voice_2488",
+  "voice_2489",
+  "voice_2490",
+  "voice_2491",
+  "voice_2492",
+  "voice_2493",
+  "voice_2494",
+  "voice_2495",
+  "voice_2496",
+  "voice_2497",
+  "voice_2498",
+  "voice_2499",
+  "voice_2500",
+  "voice_2501",
+  "voice_2502",
+  "voice_2503",
+  "voice_2504",
+  "voice_2505",
+  "voice_2506",
+  "voice_2507",
+  "voice_2508",
+  "voice_2509",
+  "voice_2510",
+  "voice_2511",
+  "voice_2512",
+  "voice_2513",
+  "voice_2514",
+  "voice_2515",
+  "voice_2516",
+  "voice_2517",
+  "voice_2518",
+  "voice_2519",
+  "voice_2520",
+  "voice_2521",
+  "voice_2522",
+  "voice_2523",
+  "voice_2524",
+  "voice_2525",
+  "voice_2526",
+  "voice_2527",
+  "voice_2528",
+  "voice_2529",
+  "voice_2530",
+  "voice_2531",
+  "voice_2532",
+  "voice_2533",
+  "voice_2534",
+  "voice_2535",
+  "voice_2536",
+  "voice_2537",
+  "voice_2538",
+  "voice_2539",
+  "voice_2540",
+  "voice_2541",
+  "voice_2542",
+  "voice_2543",
+  "voice_2544",
+  "voice_2545",
+  "voice_2546",
+  "voice_2547",
+  "voice_2548",
+  "voice_2549",
+  "voice_2550",
+  "voice_2551",
+  "voice_2552",
+  "voice_2553",
+  "voice_2554",
+  "voice_2555",
+  "voice_2556",
+  "voice_2557",
+  "voice_2558",
+  "voice_2559",
+  "voice_2560",
+  "voice_2561",
+  "voice_2562",
+  "voice_2563",
+  "voice_2564",
+  "voice_2565",
+  "voice_2566",
+  "voice_2567",
+  "voice_2568",
+  "voice_2569",
+  "voice_2570",
+  "voice_2571",
+  "voice_2572",
+  "voice_2573",
+  "voice_2574",
+  "voice_2575",
+  "voice_2576",
+  "voice_2577",
+  "voice_2578",
+  "voice_2579",
+  "voice_2580",
+  "voice_2581",
+  "voice_2582",
+  "voice_2583",
+  "voice_2584",
+  "voice_2585",
+  "voice_2586",
+  "voice_2587",
+  "voice_2588",
+  "voice_2589",
+  "voice_2590",
+  "voice_2591",
+  "voice_2592",
+  "voice_2593",
+  "voice_2594",
+  "voice_2595",
+  "voice_2596",
+  "voice_2597",
+  "voice_2598",
+  "voice_2599",
+  "voice_2600",
+  "voice_2601",
+  "voice_2602",
+  "voice_2603",
+  "voice_2604",
+  "voice_2605",
+  "voice_2606",
+  "voice_2607",
+  "voice_2608",
+  "voice_2609",
+  "voice_2610",
+  "voice_2611",
+  "voice_2612",
+  "voice_2613",
+  "voice_2614",
+  "voice_2615",
+  "voice_2616",
+  "voice_2617",
+  "voice_2618",
+  "voice_2619",
+  "voice_2620",
+  "voice_2621",
+  "voice_2622",
+  "voice_2623",
+  "voice_2624",
+  "voice_2625",
+  "voice_2626",
+  "voice_2627",
+  "voice_2628",
+  "voice_2629",
+  "voice_2630",
+  "voice_2631",
+  "voice_2632",
+  "voice_2633",
+  "voice_2634",
+  "voice_2635",
+  "voice_2636",
+  "voice_2637",
+  "voice_2638",
+  "voice_2639",
+  "voice_2640",
+  "voice_2641",
+  "voice_2642",
+  "voice_2643",
+  "voice_2644",
+  "voice_2645",
+  "voice_2646",
+  "voice_2647",
+  "voice_2648",
+  "voice_2649",
+  "voice_2650",
+  "voice_2651",
+  "voice_2652",
+  "voice_2653",
+  "voice_2654",
+  "voice_2655",
+  "voice_2656",
+  "voice_2657",
+  "voice_2658",
+  "voice_2659",
+  "voice_2660",
+  "voice_2661",
+  "voice_2662",
+  "voice_2663",
+  "voice_2664",
+  "voice_2665",
+  "voice_2666",
+  "voice_2667",
+  "voice_2668",
+  "voice_2669",
+  "voice_2670",
+  "voice_2671",
+  "voice_2672",
+  "voice_2673",
+  "voice_2674",
+  "voice_2675",
+  "voice_2676",
+  "voice_2677",
+  "voice_2678",
+  "voice_2679",
+  "voice_2680",
+  "voice_2681",
+  "voice_2682",
+  "voice_2683",
+  "voice_2684",
+  "voice_2685",
+  "voice_2686",
+  "voice_2687",
+  "voice_2688",
+  "voice_2689",
+  "voice_2690",
+  "voice_2691",
+  "voice_2692",
+  "voice_2693",
+  "voice_2694",
+  "voice_2695",
+  "voice_2696",
+  "voice_2697",
+  "voice_2698",
+  "voice_2699",
+  "voice_2700",
+  "voice_2701",
+  "voice_2702",
+  "voice_2703",
+  "voice_2704",
+  "voice_2705",
+  "voice_2706",
+  "voice_2707",
+  "voice_2708",
+  "voice_2709",
+  "voice_2710",
+  "voice_2711",
+  "voice_2712",
+  "voice_2713",
+  "voice_2714",
+  "voice_2715",
+  "voice_2716",
+  "voice_2717",
+  "voice_2718",
+  "voice_2719",
+  "voice_2720",
+  "voice_2721",
+  "voice_2722",
+  "voice_2723",
+  "voice_2724",
+  "voice_2725",
+  "voice_2726",
+  "voice_2727",
+  "voice_2728",
+  "voice_2729",
+  "voice_2730",
+  "voice_2731",
+  "voice_2732",
+  "voice_2733",
+  "voice_2734",
+  "voice_2735",
+  "voice_2736",
+  "voice_2737",
+  "voice_2738",
+  "voice_2739",
+  "voice_2740",
+  "voice_2741",
+  "voice_2742",
+  "voice_2743",
+  "voice_2744",
+  "voice_2745",
+  "voice_2746",
+  "voice_2747",
+  "voice_2748",
+  "voice_2749",
+  "voice_2750",
+  "voice_2751",
+  "voice_2752",
+  "voice_2753",
+  "voice_2754",
+  "voice_2755",
+  "voice_2756",
+  "voice_2757",
+  "voice_2758",
+  "voice_2759",
+  "voice_2760",
+  "voice_2761",
+  "voice_2762",
+  "voice_2763",
+  "voice_2764",
+  "voice_2765",
+  "voice_2766",
+  "voice_2767",
+  "voice_2768",
+  "voice_2769",
+  "voice_2770",
+  "voice_2771",
+  "voice_2772",
+  "voice_2773",
+  "voice_2774",
+  "voice_2775",
+  "voice_2776",
+  "voice_2777",
+  "voice_2778",
+  "voice_2779",
+  "voice_2780",
+  "voice_2781",
+  "voice_2782",
+  "voice_2783",
+  "voice_2784",
+  "voice_2785",
+  "voice_2786",
+  "voice_2787",
+  "voice_2788",
+  "voice_2789",
+  "voice_2790",
+  "voice_2791",
+  "voice_2792",
+  "voice_2793",
+  "voice_2794",
+  "voice_2795",
+  "voice_2796",
+  "voice_2797",
+  "voice_2798",
+  "voice_2799",
+  "voice_2800",
+  "voice_2801",
+  "voice_2802",
+  "voice_2803",
+  "voice_2804",
+  "voice_2805",
+  "voice_2806",
+  "voice_2807",
+  "voice_2808",
+  "voice_2809",
+  "voice_2810",
+  "voice_2811",
+  "voice_2812",
+  "voice_2813",
+  "voice_2814",
+  "voice_2815",
+  "voice_2816",
+  "voice_2817",
+  "voice_2818",
+  "voice_2819",
+  "voice_2820",
+  "voice_2821",
+  "voice_3226",
+  "voice_3622",
+  "voice_3695",
+  "voice_4223"
+ ],
+ "hf_repo": null,
+ "files": {
+  "acoustic": "model.safetensors",
+  "speaker_vectors": "speaker_vectors.safetensors",
+  "voices": "voices.json",
+  "control_stats": "control_stats.json"
+ },
+ "acoustic": {
+  "step": 312500,
+  "n_speakers": 4919,
+  "n_tags": 51,
+  "arch": {
+   "enc_mixer": "hybrid",
+   "dec_mixer": "hybrid",
+   "ssm_state": 32,
+   "spk_enc_dim": 512,
+   "n_symbols": 118,
+   "n_env": 30
+  },
+  "params": 9585639
+ },
+ "refiner": {
+  "file": "flow.safetensors",
+  "cfg": {
+   "mean": -1.5826570987701416,
+   "std": 2.13140869140625,
+   "d": 384,
+   "n_layers": 12,
+   "g_dim": 128
+  },
+  "step": 100000,
+  "params": 17642598,
+  "steps": 10,
+  "temp": 0.4
+ },
+ "prompted_flow": {
+  "file": "pflow.safetensors",
+  "cfg": {
+   "mean": -1.5826570987701416,
+   "std": 2.13140869140625,
+   "d": 384,
+   "n_layers": 12,
+   "g_dim": 128,
+   "prompt": true,
+   "mu_cmn": true,
+   "attn_at": [
+    1,
+    3,
+    5,
+    7,
+    9,
+    11
+   ]
+  },
+  "step": 40000,
+  "params": 24674406,
+  "steps": 10,
+  "temp": 0.4
+ },
+ "prompt_adapter": {
+  "file": "prompt_adapter.safetensors",
+  "tags": [
+   "none",
+   "adoration",
+   "amazement",
+   "amusement",
+   "anger",
+   "anxiety",
+   "apologetic",
+   "bored",
+   "confidence",
+   "confusion",
+   "contempt",
+   "contentment",
+   "cuteness",
+   "desire",
+   "determined",
+   "disappointment",
+   "disgust",
+   "distress",
+   "ecstasy",
+   "embarrassment",
+   "emphasis",
+   "enunciated",
+   "excited",
+   "fast",
+   "fear",
+   "frustration",
+   "guilt",
+   "happiness",
+   "hesitant",
+   "high_pitch",
+   "interest",
+   "irritation",
+   "laughing",
+   "loud",
+   "low_pitch",
+   "mysterious",
+   "neutral",
+   "nostalgia",
+   "pain",
+   "panic",
+   "pride",
+   "realization",
+   "relief",
+   "sadness",
+   "sarcasm",
+   "serenity",
+   "skeptical",
+   "slow",
+   "sympathy",
+   "tired",
+   "whisper"
+  ],
+  "encoder": "sentence-transformers/all-MiniLM-L6-v2",
+  "params": 498251
+ },
+ "speaker_encoder": "microsoft/wavlm-base-plus-sv",
+ "vocoder": "charactr/vocos-mel-24khz",
+ "text_frontend": "g2p_en ARPAbet + rule-based normalisation (numbers, units, currency, acronyms)"
+}

flow.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:585ff2c931811c30d0ec953a947dc23ffb305979471f5be5133a1f2b708c1e90
+size 70580400

inference.py

@@ -0,0 +1,59 @@
+"""SteerTTS examples. Run from the model directory (or set STEERTTS_MODEL) after `pip install ".[all]"`.
+
+python inference.py                  # renders every example below into ./samples/
+python inference.py --device cpu
+"""
+import argparse
+import json
+import os
+from pathlib import Path
+
+from steertts import SteerTTS
+
+HERE = Path(__file__).resolve().parent
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default=os.environ.get("STEERTTS_MODEL", str(HERE)))
+    ap.add_argument("--device", default=None)
+    ap.add_argument("--out", default="samples")
+    ap.add_argument("--ref-wav", help="a consenting speaker's recording, for the cloning example")
+    a = ap.parse_args()
+    out = Path(a.out); out.mkdir(exist_ok=True)
+    tts = SteerTTS.from_pretrained(a.model, device=a.device)
+    print(tts, f"{len(tts.voices)} voices, default {tts.default_voice}")
+
+    # 1. plain text, default voice
+    tts.save(tts("The meeting has been moved to Thursday afternoon at three."), out / "01_default.wav")
+    # 2. explicit controls (speaker-relative z): faster and higher, rising at the end
+    tts.save(tts("Are you coming with us tonight?", ctrl={"rate": 1.0, "f0_med": 1.0, "f0_end": 2.0}), out / "02_ctrl.wav")
+    # 3. a style tag plus a control, stronger steering via guidance
+    tts.save(tts("I really thought it would work this time.", tag="sadness", ctrl={"loud": -1.0}, guidance=1.5),
+             out / "03_tag_guidance.wav")
+    # 4. another voice (see voices.json / `steertts voices`)
+    v = tts.voices[42]
+    print(v, tts.voice_info[v]["description"])
+    tts.save(tts("Good evening, and welcome to the program.", voice=v), out / "04_other_voice.wav")
+    # 5. a free-text direction -> the request it maps to (needs the `prompt` extra)
+    req = tts.resolve({"text": "Please keep this between the two of us.", "prompt": "whispering, slow and secretive"})
+    print(json.dumps(req, indent=1))
+    tts.save(tts.synthesize(req), out / "05_prompt.wav")
+    # 6. inline markup, streamed sentence by sentence
+    text = "Well, *that* was fast. [pause 400ms] <slow>Let me think about it.</slow> <whisper>Don't tell anyone.</whisper>"
+    chunks = []
+    for c in tts.stream(text):
+        print(f"  chunk {c.segment.tag:8s} {len(c.audio) / tts.sample_rate:.2f}s rendered in {c.synth_ms:.0f} ms")
+        chunks.append(c.audio)
+    import numpy as np
+    tts.save(np.concatenate(chunks), out / "06_markup_stream.wav")
+    # 7. per-word prosody without markup: word 1 ("never", 0-based) higher, louder and longer
+    tts.save(tts("I never said she stole it.", word_shift={1: {"pitch": 3, "energy": 4, "dur": 1.3}}), out / "07_word_shift.wav")
+    # 8. zero-shot cloning (needs the `clone` extra and a CONSENTING speaker's clip)
+    if a.ref_wav:
+        tts.save(tts("This is a cloned voice reading a new sentence.", ref_wav=a.ref_wav), out / "08_clone.wav")
+    print("wrote", sorted(p.name for p in out.glob("*.wav")))
+
+
+if __name__ == "__main__":
+    main()

leaderboard/steertts/Dockerfile

@@ -0,0 +1,12 @@
+# Stage-1 image for the Open TTS Leaderboard (a public Docker Space, e.g. WhissleAI/evals-steertts).
+# The eval scripts are injected at runtime; this only installs the model's dependencies.
+FROM pytorch/pytorch:2.7.1-cuda12.8-cudnn9-runtime
+RUN apt-get update && apt-get install -y --no-install-recommends libsndfile1 git && rm -rf /var/lib/apt/lists/*
+# torchaudio must match the image's torch (vocos imports it)
+RUN pip install --no-cache-dir "torchaudio==2.7.1" datasets tqdm \
+    "steertts[clone] @ git+https://huggingface.co/WhissleAI/steertts-v1"
+# warm the caches the first job would otherwise download (vocoder, speaker encoder, NLTK data)
+RUN python -c "from vocos import Vocos; Vocos.from_pretrained('charactr/vocos-mel-24khz')" && \
+    python -c "from steertts.text import to_phonemes; to_phonemes('warm up')" && \
+    python -c "from transformers import WavLMForXVector; WavLMForXVector.from_pretrained('microsoft/wavlm-base-plus-sv')"
+WORKDIR /app

leaderboard/steertts/run_eval.py

@@ -0,0 +1,138 @@
+"""
+TTS synthesis for the Open TTS Leaderboard (SteerTTS backend, stage 1).
+
+SteerTTS is a 9.6M-parameter acoustic model + Vocos (23.1M total; +17.6M flow refiner by default, or the 24.7M
+prompted flow when cloning), loaded in-process from the `steertts` package. English only, so only the `en` splits.
+Default voice = the pre-registered `voice_1361` (no test-set voice selection); `--voice_clone` clones each sample's
+`prompt_audio` (WavLM speaker vector + prompted flow; `prompt_text` is not used). Rows are synthesized one at a
+time (no batched API); generation is timed after `--warmup_steps` untimed samples. `--ttfa_probe` times the first
+sentence chunk of `SteerTTS.stream()` (sentence-level streaming).
+
+    PYTHONPATH=scripts python steertts/run_eval.py --model_id WhissleAI/steertts-v1 --dataset_path bezzam/seed_tts_eval \
+        --dataset tts --split en --device cuda:0 [--voice_clone] [--model_path /local/steertts-v1]
+"""
+
+import argparse
+import os
+import time
+
+import numpy as np
+import soundfile as sf
+import torch
+from tqdm import tqdm
+
+from steertts import SteerTTS
+from run_eval_utils import (
+    add_common_args, load_done_entries, load_tts_dataset, manifest_entry, open_manifest, output_paths,
+    pending_batches, print_model_size, print_next_steps, set_seed, warm_up, wav_rel_path, write_entry,
+)
+
+SAMPLING_RATE = 24_000
+
+
+def main(args):
+    set_seed(args.seed)
+    tts = SteerTTS.from_pretrained(args.model_path or args.model_id, device=args.device, flow=not args.no_flow)
+    voice = args.voice or tts.default_voice
+    print(f"Loaded {tts} voice={voice} voice_clone={args.voice_clone}")
+    print_model_size(tts.model, tts.vocos, tts.flow, tts.pflow if args.voice_clone else None)
+    is_cuda = str(tts.dev).startswith("cuda")
+
+    mode_suffix = "_voice_clone" if args.voice_clone else ""
+    paths = output_paths(args, mode_suffix)
+    model_dir, dataset_dir_name, output_dir = paths.model_dir, paths.dataset_dir_name, paths.output_dir
+    prompt_dir = os.path.join(output_dir, "prompts")
+    if args.voice_clone:
+        os.makedirs(prompt_dir, exist_ok=True)
+
+    def write_prompt(sample_id, prompt_audio, root):
+        prel = os.path.join(dataset_dir_name, "prompts", f"prompt_{sample_id}.wav")
+        ppath = os.path.join(root, prel)
+        os.makedirs(os.path.dirname(ppath), exist_ok=True)
+        if not os.path.exists(ppath):
+            sf.write(ppath, np.asarray(prompt_audio["array"], dtype=np.float32), prompt_audio["sampling_rate"])
+        return prel, ppath
+
+    def request(text, ref):
+        return dict(text=text, ref_wav=ref) if ref else dict(text=text, voice=voice)
+
+    def synth_one(text, ref=None):
+        if is_cuda:
+            torch.cuda.synchronize()
+        start = time.perf_counter()
+        audio = tts.synthesize(request(text, ref))
+        if is_cuda:
+            torch.cuda.synchronize()
+        return audio, time.perf_counter() - start
+
+    dataset = load_tts_dataset(args, extra_columns=("prompt_audio",) if args.voice_clone else ())
+
+    if args.ttfa_probe != 0:
+        from ttfa_probe import run_probe, sample_indices
+        import tempfile
+        tmp = tempfile.mkdtemp(prefix="ttfa_prompts_")
+        chunk_counts = []
+
+        def _gen(job, early_stop):
+            ref = write_prompt(job["id"][0], job["prompt_audio"][0], tmp)[1] if args.voice_clone else None
+            first, chunks = None, []
+            for c in tts.stream(request=request(job[args.text_column][0], ref)):
+                if first is None:
+                    first = time.perf_counter()
+                chunks.append(c.audio)
+                if early_stop:
+                    break
+            chunk_counts.append(len(chunks))
+            return (np.concatenate(chunks) if chunks else np.zeros(1, np.float32)), SAMPLING_RATE, first
+
+        columns = dataset.column_names
+        run_probe(_gen, [{c: [dataset[i][c]] for c in columns} for i in sample_indices(args.ttfa_probe, len(dataset))],
+                  model_id=args.model_id, dataset_path=args.dataset_path, dataset_config=args.dataset, split=args.split,
+                  out_dir=model_dir, mode_suffix=mode_suffix, n=args.ttfa_probe,
+                  extra={"device": args.device, "note": "batch 1, SteerTTS.stream() sentence chunks",
+                         "chunks_per_utterance": chunk_counts})
+        return
+
+    done_entries = load_done_entries(paths.manifest_path, args.resume)
+    manifest_file = open_manifest(paths.manifest_path, done_entries)
+    starts = pending_batches(dataset, 1, done_entries, dataset_dir_name)
+
+    def gen_batch(batch):
+        out = []
+        for k, text in enumerate(batch[args.text_column]):
+            ref = write_prompt(batch["id"][k], batch["prompt_audio"][k], model_dir)[1] if args.voice_clone else None
+            out.append(synth_one(text, ref))
+        return out
+
+    warm_up(gen_batch, dataset, starts, 1, args.warmup_steps)
+    set_seed(args.seed)
+    for i in tqdm(starts, desc="Generating"):
+        row = dataset[i]
+        prel = None
+        ref = None
+        if args.voice_clone:
+            prel, ref = write_prompt(row["id"], row["prompt_audio"], model_dir)  # I/O outside the timed region
+        audio, elapsed = synth_one(row[args.text_column], ref)
+        rel_path = wav_rel_path(dataset_dir_name, row["id"])
+        sf.write(os.path.join(model_dir, rel_path), audio, SAMPLING_RATE)
+        write_entry(manifest_file, manifest_entry(rel_path, len(audio) / SAMPLING_RATE, elapsed, row[args.text_column],
+                                                  prel if args.voice_clone else None))
+        manifest_file.flush()
+    manifest_file.close()
+    print("Results saved at path:", os.path.abspath(paths.manifest_path))
+    print_next_steps(args.voice_clone)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model_id", type=str, default="WhissleAI/steertts-v1",
+                        help="Name used for the results folder; also the HF repo loaded unless --model_path is set.")
+    parser.add_argument("--model_path", type=str, default=None, help="Local model directory (overrides --model_id).")
+    parser.add_argument("--voice", type=str, default=None, help="Built-in voice (default: the model's default_voice).")
+    parser.add_argument("--no_flow", action="store_true", help="Skip the flow refiner (faster, slightly hazier).")
+    add_common_args(parser, voice_clone=False)
+    args = parser.parse_args()
+    print("*" * 100)
+    print(f"Generating TTS with {args.model_id} on {args.dataset_path} / {args.dataset} / {args.split}")
+    print("*" * 100)
+    main(args)

leaderboard/steertts/submit_jobs.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# HF Jobs TTS eval — SteerTTS backend (generate → transcribe → SIM* → score; *voice-clone only).
+# Usage: HF_TOKEN=hf_... [VOICE_CLONE=true] bash steertts/submit_jobs.sh ["WhissleAI/steertts-v1" ...]
+# English only: SteerTTS has an English (ARPAbet) front end, so only the `en` splits are configured.
+source "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/scripts/tts_jobs_common.sh"
+
+TTS_SPACE="${TTS_SPACE:-WhissleAI/evals-steertts}"   # public Docker Space built from steertts/Dockerfile (TO CREATE)
+RUN_EVAL_INJECT="$(inject_run_eval)"
+SUPPORTS_VOICE_CLONE=true
+supports_language() { [[ "$1" == "en" ]]; }
+
+generate_stage() {
+    hf jobs run \
+        --flavor "${FLAVOR}" --timeout 8h --secrets HF_TOKEN ${NAMESPACE_ARG} \
+        --volume "hf://buckets/${RESULTS_BUCKET}:/results" "${TTS_IMAGE}" \
+        bash -c "
+            ${RUN_EVAL_INJECT}
+            cd /app/steertts && python run_eval.py \
+                --model_id=${MODEL_ID} \
+                --dataset_path=${DATASET_PATH} \
+                --dataset=${DATASET} \
+                --split=${SPLIT} \
+                --device=cuda:0 \
+                --max_eval_samples=${MAX_EVAL_SAMPLES} \
+                ${VOICE_CLONE_FLAG} &&
+            mkdir -p /results &&
+            cp -r results/${MODEL_SAFE} /results/
+        "
+}
+
+DATASET_CONFIGS=(
+    "tts en en"
+    "zero_shot en en ${CV3_EVAL_PATH:-bezzam/cv3_eval}"
+)
+
+MODEL_CONFIGS=("WhissleAI/steertts-v1")
+[[ $# -gt 0 ]] && MODEL_CONFIGS=("$@")
+
+run_pipeline

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adc11912871113e39d3dbbc2c5f7edfe0b9ec9bf7b98f0a926f7c9edfc07b17c
+size 38395628

pflow.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:877f81523dfaba5061823f6e6543b83edd88c72a3df724666ce3ee74de259327
+size 98711656

prompt_adapter.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aca34bf753456e469bae22f59217512f8cbe8efb326bfbede1529eeb9e817727
+size 1993780

pyproject.toml

@@ -0,0 +1,44 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "steertts"
+version = "1.0.0"
+description = "SteerTTS: a small, numerically steerable English TTS (hybrid attention + bidirectional Mamba-3)"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Code: MIT. Weights: CC BY-NC 4.0 (TBD by authors), see README.md" }
+# Versions are the ones the release was verified with (fresh env, TEST_LOG.md). torch is a range because the right
+# wheel depends on your CUDA / CPU setup; torchaudio (pulled in by vocos) must match the torch you install.
+dependencies = [
+  "torch>=2.4,<2.12",
+  "numpy>=1.24,<2.5",
+  "safetensors==0.8.0",
+  "huggingface_hub>=0.23",
+  "vocos==0.1.0",
+  "g2p_en==2.1.0",
+  "nltk==3.10.3",
+  "num2words==0.5.14",
+  "soundfile==0.14.0",
+  "soxr==1.1.0",
+]
+
+[project.optional-dependencies]
+prompt = ["transformers==5.19.0"]          # free-text directions (MiniLM sentence encoder)
+clone = ["transformers==5.19.0"]           # zero-shot cloning (WavLM speaker encoder)
+measure = ["praat-parselmouth==0.4.7"]     # read the 12 controls back from audio
+all = ["transformers==5.19.0", "praat-parselmouth==0.4.7"]
+test = ["transformers==5.19.0", "praat-parselmouth==0.4.7", "pytest>=8", "jsonschema>=4.20"]
+
+[project.scripts]
+steertts = "steertts.cli:main"
+
+[tool.setuptools]
+packages = ["steertts", "steertts.mamba3"]
+
+[tool.setuptools.package-data]
+steertts = ["request.schema.json"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

request.schema.json

@@ -0,0 +1,280 @@
+{
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
+ "$id": "urn:steertts:request-schema:v1",
+ "title": "SteerTTS synthesis request",
+ "description": "One request = one rendered waveform (24 kHz mono float32). Every field except `text` is optional. Controls are SPEAKER-RELATIVE z-scores: 0 = the chosen voice's own neutral speech, +-1 = one pooled within-speaker standard deviation (raw units per dim: control_stats.json pooled_sd). Mask semantics: a control named in `ctrl` is requested (value + availability mask 1); every control NOT named is unspecified (mask 0) and the model chooses it from voice, tag and text. `{\"rate\": 0}` therefore asks for the voice's neutral rate, while omitting `rate` lets the tag/text decide. SteerTTS.resolve(request) returns the fully resolved request (defaults filled, `prompt` mapped to `tag` + `ctrl`, recorded in `prompt_mapping`), which is itself a valid request.",
+ "type": "object",
+ "required": [
+  "text"
+ ],
+ "additionalProperties": false,
+ "properties": {
+  "$schema": {
+   "type": "string"
+  },
+  "text": {
+   "type": "string",
+   "minLength": 1,
+   "maxLength": 5000,
+   "description": "English text. Numbers, units, currency, ordinals, ranges and unknown acronyms are spelled out by the front end. With `markup: true` it may contain inline markup (see `markup`). Event tokens such as [laugh] or [dog] are accepted: the model reserves a short stretch of time for them but renders no sound."
+  },
+  "voice": {
+   "type": "string",
+   "pattern": "^voice_[0-9]{4}$",
+   "description": "Built-in voice (anonymised public name, see voices.json). Default: the model's default_voice (voice_1361). Mutually exclusive with ref_wav."
+  },
+  "ref_wav": {
+   "type": "string",
+   "description": "Path to a reference recording (any sample rate, first 8 s used, >= 0.5 s) for zero-shot cloning: the voice is taken from its WavLM speaker vector and, when flow is on, the prompted flow uses the clip's own mel as context. Mutually exclusive with voice. Only clone voices whose owners consented."
+  },
+  "prompt": {
+   "type": "string",
+   "description": "Free-text voice direction ('soft and sad', 'excited, a bit faster'). Mapped by the prompt adapter to a style tag and a subset of controls. An explicit `tag` overrides the prompt's tag; explicit `ctrl` dims override the prompt's values for those dims."
+  },
+  "tag": {
+   "type": "string",
+   "enum": [
+    "none",
+    "adoration",
+    "amazement",
+    "amusement",
+    "anger",
+    "anxiety",
+    "apologetic",
+    "bored",
+    "confidence",
+    "confusion",
+    "contempt",
+    "contentment",
+    "cuteness",
+    "desire",
+    "determined",
+    "disappointment",
+    "disgust",
+    "distress",
+    "ecstasy",
+    "embarrassment",
+    "emphasis",
+    "enunciated",
+    "excited",
+    "fast",
+    "fear",
+    "frustration",
+    "guilt",
+    "happiness",
+    "hesitant",
+    "high_pitch",
+    "interest",
+    "irritation",
+    "laughing",
+    "loud",
+    "low_pitch",
+    "mysterious",
+    "neutral",
+    "nostalgia",
+    "pain",
+    "panic",
+    "pride",
+    "realization",
+    "relief",
+    "sadness",
+    "sarcasm",
+    "serenity",
+    "skeptical",
+    "slow",
+    "sympathy",
+    "tired",
+    "whisper"
+   ],
+   "default": "none",
+   "description": "Style tag; 'none' = unspecified. Tags act as style switches: voice-quality tags such as whisper change the voice strongly, but an independent emotion classifier recognises only sadness among the emotion tags, and about half of the tags are acoustically close to neutral (see the model card). For a specific acoustic change prefer ctrl."
+  },
+  "ctrl": {
+   "type": "object",
+   "additionalProperties": false,
+   "properties": {
+    "rate": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "articulation rate (phones per second of speech, pauses excluded). + faster"
+    },
+    "loud": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "mean speech level (dB). + louder"
+    },
+    "f0_med": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "median pitch (semitones). + higher"
+    },
+    "f0_iqr": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "pitch range, interquartile (semitones). + livelier, - monotone"
+    },
+    "f0_end": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "pitch slope over the last 300 ms of voicing (st/s). + rising (question-like), - falling"
+    },
+    "pause": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "share of the speech span spent in silent pauses >= 100 ms. + more pauses"
+    },
+    "tilt": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral tilt, energy 1-5 kHz vs 50 Hz-1 kHz (dB). + brighter / pressed"
+    },
+    "voiced": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "voiced share of speech frames. - whispery"
+    },
+    "flat": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral flatness (log). + breathier / noisier"
+    },
+    "centroid": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "spectral centroid (log Hz). + brighter"
+    },
+    "cpps": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "cepstral peak prominence (dB), periodicity. - breathier / rougher"
+    },
+    "h1h2": {
+     "type": "number",
+     "minimum": -5,
+     "maximum": 5,
+     "description": "H1-H2 (dB). + breathier"
+    }
+   },
+   "default": {},
+   "description": "Requested controls {dim: z}. Values beyond +-3 are outside most training data."
+  },
+  "pace": {
+   "type": "number",
+   "minimum": 0.25,
+   "maximum": 4.0,
+   "default": 1.0,
+   "description": "Global duration divisor applied after prediction (>1 faster). Unlike ctrl.rate it does not change the style, only the timing."
+  },
+  "guidance": {
+   "type": "number",
+   "minimum": 0.0,
+   "maximum": 4.0,
+   "default": 1.0,
+   "description": "Steering scale s: out = out(no request) + s * (out(request) - out(no request)), on durations, pitch, energy and mel. s > 1 strengthens the requested change (control labels are noisy, so the learned effect is attenuated) at the cost of more leakage into other dims and some quality; only applied when a tag or ctrl is given."
+  },
+  "word_shift": {
+   "type": "object",
+   "description": "Per-word prosody edits {word_index: {pitch: semitones, energy: dB, dur: factor}} (word_index counts words of the normalised text from 0). Not combinable with markup (use *emph*, <slow> etc. there).",
+   "patternProperties": {
+    "^[0-9]+$": {
+     "type": "object",
+     "additionalProperties": false,
+     "properties": {
+      "pitch": {
+       "type": "number",
+       "minimum": -12,
+       "maximum": 12
+      },
+      "energy": {
+       "type": "number",
+       "minimum": -20,
+       "maximum": 20
+      },
+      "dur": {
+       "type": "number",
+       "minimum": 0.25,
+       "maximum": 4
+      }
+     }
+    }
+   },
+   "additionalProperties": false
+  },
+  "markup": {
+   "type": "boolean",
+   "default": false,
+   "description": "Parse inline markup in text: *emphasis* / <emph>..</emph>; per-word <slow> <fast> <high> <low> <loud> <soft>; [pause], [pause 300ms], [pause 1.5s]; span styles <whisper>..</whisper> (any tag) and <style prompt=\"...\">..</style>. The text is rendered sentence by sentence (and per span), joined with 10 ms fades. stream() always parses markup."
+  },
+  "flow": {
+   "type": "boolean",
+   "default": true,
+   "description": "Refine the mel with the flow-matching decoder (refiner for built-in voices, prompted flow for ref_wav). false = the acoustic model's mel straight into Vocos (faster, slightly hazier)."
+  },
+  "seed": {
+   "type": "integer",
+   "minimum": 0,
+   "default": 0,
+   "description": "Noise seed of the flow sampler (renders are deterministic per device for a given seed)."
+  },
+  "prompt_mapping": {
+   "type": "object",
+   "description": "Written by resolve(): what `prompt` mapped to {tag, ctrl, top_tags}. Informational; ignored on input.",
+   "properties": {
+    "tag": {
+     "type": "string"
+    },
+    "ctrl": {
+     "type": "object"
+    },
+    "top_tags": {
+     "type": "array"
+    }
+   }
+  }
+ },
+ "examples": [
+  {
+   "text": "The meeting has been moved to Thursday afternoon at three.",
+   "voice": "voice_1361"
+  },
+  {
+   "text": "I can't believe you did that!",
+   "voice": "voice_1361",
+   "tag": "anger",
+   "ctrl": {
+    "loud": 1.5,
+    "f0_med": 1.0
+   }
+  },
+  {
+   "text": "Are you serious?",
+   "ctrl": {
+    "f0_end": 2.0,
+    "f0_iqr": 1.5
+   }
+  },
+  {
+   "text": "Please keep this between us.",
+   "prompt": "whispering, slow and secretive"
+  },
+  {
+   "text": "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>",
+   "markup": true
+  },
+  {
+   "text": "Now in a cloned voice.",
+   "ref_wav": "reference.wav"
+  }
+ ]
+}

speaker_vectors.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edcabfd02dd4a5203309e9b1d6d6adb84d31a18285916238541cf93a0764d4ae
+size 5785680

steertts.egg-info/PKG-INFO

@@ -0,0 +1,397 @@
+Metadata-Version: 2.4
+Name: steertts
+Version: 1.0.0
+Summary: SteerTTS: a small, numerically steerable English TTS (hybrid attention + bidirectional Mamba-3)
+License: Code: MIT. Weights: CC BY-NC 4.0 (TBD by authors), see README.md
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: torch<2.12,>=2.4
+Requires-Dist: numpy<2.5,>=1.24
+Requires-Dist: safetensors==0.8.0
+Requires-Dist: huggingface_hub>=0.23
+Requires-Dist: vocos==0.1.0
+Requires-Dist: g2p_en==2.1.0
+Requires-Dist: nltk==3.10.3
+Requires-Dist: num2words==0.5.14
+Requires-Dist: soundfile==0.14.0
+Requires-Dist: soxr==1.1.0
+Provides-Extra: prompt
+Requires-Dist: transformers==5.19.0; extra == "prompt"
+Provides-Extra: clone
+Requires-Dist: transformers==5.19.0; extra == "clone"
+Provides-Extra: measure
+Requires-Dist: praat-parselmouth==0.4.7; extra == "measure"
+Provides-Extra: all
+Requires-Dist: transformers==5.19.0; extra == "all"
+Requires-Dist: praat-parselmouth==0.4.7; extra == "all"
+Provides-Extra: test
+Requires-Dist: transformers==5.19.0; extra == "test"
+Requires-Dist: praat-parselmouth==0.4.7; extra == "test"
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: jsonschema>=4.20; extra == "test"
+Dynamic: license-file
+
+---
+language:
+- en
+license: cc-by-nc-4.0
+library_name: steertts
+pipeline_tag: text-to-speech
+tags:
+- text-to-speech
+- tts
+- speech-synthesis
+- english
+- controllable-tts
+- steerable-tts
+- prosody-control
+- expressive-tts
+- style-control
+- inline-markup
+- word-emphasis
+- zero-shot-voice-cloning
+- voice-cloning
+- interpretable
+- non-autoregressive
+- fastpitch
+- mamba
+- mamba-3
+- state-space-model
+- hybrid-attention
+- flow-matching
+- vocos
+- small-model
+- on-device
+- streaming
+- non-commercial
+---
+
+# SteerTTS v1
+
+A small English text-to-speech model that you steer with numbers. Each request can set any subset of **12 acoustic
+controls**, for example speaking rate, loudness, median pitch, pitch range, final rise or fall, pauses, spectral tilt
+and breathiness. Each control is a z-score relative to the chosen voice's own neutral speech. You can also pick a
+**style tag**, give a **free-text direction** (which a small adapter turns into a request you can inspect), or use
+**inline markup**. The model can **clone** a voice from a short reference clip.
+
+| | |
+|---|---|
+| Acoustic model | **9.59M** parameters (encoder/decoder: attention + bidirectional Mamba-3, AdaLN conditioning) |
+| + Vocos vocoder | 13.53M (`charactr/vocos-mel-24khz`, frozen), **23.1M** total |
+| Flow-matching refiner (optional, on by default) | 17.64M → 40.8M total |
+| Prompted flow (cloning only) | 24.67M, used in place of the refiner when `ref_wav` is given → 47.8M total |
+| Frozen external encoders | WavLM-base-plus-sv 101M (cloning only), all-MiniLM-L6-v2 22.7M (free-text prompts only) |
+| Output | 24 kHz mono, fp32, CPU or CUDA |
+| Voices | 2,825 built-in voices under anonymised names (`voice_0001` …), default `voice_1361` |
+
+> **Licence: weights CC BY-NC 4.0 (non-commercial use only); package code MIT.** The weights were trained partly on
+> non-commercial data (see [Training data](#training-data)).
+
+## Quick start
+
+```bash
+git clone https://huggingface.co/WhissleAI/steer-tts && cd steer-tts      # or huggingface_hub.snapshot_download("WhissleAI/steer-tts")
+pip install ".[all]"          # core: torch, vocos, g2p_en …; [all] adds transformers (prompts, cloning) + parselmouth (measuring)
+steertts say "Hello there, nice to meet you." -o hello.wav
+steertts say "I can't believe it worked." --tag happiness --ctrl rate=1,f0_iqr=1.5 -o happy.wav
+steertts say "Please keep this quiet." --prompt "whispering, slow and secretive" -o secret.wav   # prints the resolved request
+steertts stream "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" -o markup.wav
+steertts say "Now in your voice." --ref-wav consenting_speaker.wav -o clone.wav
+steertts voices --gender female | head
+```
+
+```python
+from steertts import SteerTTS
+
+tts = SteerTTS.from_pretrained("WhissleAI/steer-tts")   # HF repo id or a local dir; device: cuda if available, else cpu
+wav = tts("The meeting moved to Thursday.")       # float32 numpy, 24 kHz, default voice voice_1361
+wav = tts("Are you serious?", voice="voice_0042", ctrl={"f0_end": 2.0, "f0_iqr": 1.5})   # rising, lively
+wav = tts("It is over.", tag="sadness", ctrl={"loud": -1}, guidance=1.5)
+
+req = tts.resolve({"text": "See you soon.", "prompt": "excited, a bit faster"})
+print(req)    # {'text': ..., 'voice': 'voice_1361', 'tag': 'excited', 'ctrl': {'rate': 0.29}, 'prompt_mapping': {...}, ...}
+wav = tts.synthesize(req)                         # a resolved request renders exactly as the prompt would
+
+for chunk in tts.stream("First sentence. Second sentence."):   # sentence-level streaming
+    play(chunk.audio)
+tts.save(wav, "out.wav")
+```
+
+`inference.py` renders one example of each feature. `python -m steertts.cli --help` lists the CLI.
+
+## The request
+
+Every synthesis is one JSON object. It is documented field by field in [`request.schema.json`](request.schema.json)
+(JSON Schema 2020-12; `steertts schema` prints it). Only `text` is required.
+
+| Field | Meaning |
+|---|---|
+| `text` | English text. Numbers, units, currency, ordinals, ranges and unknown acronyms are spelled out. |
+| `voice` / `ref_wav` | A built-in voice, **or** a reference clip to clone (first 8 s, at least 0.5 s). |
+| `tag` | One of 51 style tags (`none`, `whisper`, `sadness`, `anger`, `fast`, `high_pitch`, …). |
+| `ctrl` | `{control: z}` over the 12 controls below, each in [-5, 5]. |
+| `prompt` | Free-text direction. `resolve()` maps it to `tag` + `ctrl`; an explicit `tag` or `ctrl` dim wins. |
+| `guidance` | Steering scale s (default 1). See [Guidance](#how-well-it-steers). |
+| `pace` | Global tempo divisor (>1 is faster). It does not change the style. |
+| `word_shift` | `{word_index: {pitch: st, energy: dB, dur: ×}}`, per-word prosody edits. |
+| `markup` | Parse inline markup in `text` (see below). `stream()` always parses it. |
+| `flow`, `seed` | Use the flow refiner (default true) and set the sampler seed. Renders are deterministic per device. |
+
+**Controls and the mask.** The 12 controls are utterance-level acoustic measurements:
+
+- `rate`: articulation rate
+- `loud`: speech level
+- `f0_med`: median pitch
+- `f0_iqr`: pitch range
+- `f0_end`: final pitch slope
+- `pause`: pause share
+- `tilt`: spectral tilt
+- `voiced`: voiced share
+- `flat`: spectral flatness
+- `centroid`: spectral centroid
+- `cpps`: cepstral peak prominence
+- `h1h2`: H1–H2
+
+Each value is a z-score against the chosen voice's own neutral speech, in units of the pooled within-speaker SD
+(`control_stats.json`). The model also receives an availability mask:
+
+- A dim named in `ctrl` is requested.
+- A dim left out is unspecified, and the model chooses it from voice, tag and text.
+- `{"rate": 0}` asks for this voice's neutral rate. Omitting `rate` lets a `fast` tag or the text decide.
+
+`steertts measure out.wav --text "…" --voice voice_1361` reads the controls back with the same measurement code
+that labelled the training data.
+
+**Markup** (`markup: true` or `stream()`):
+
+| Markup | Effect |
+|---|---|
+| `*word*` or `<emph>…</emph>` | Emphasis: +2.5 st, +4 dB, ×1.2 duration |
+| `<slow> <fast> <high> <low> <loud> <soft>` | Per-word changes; the sentence is still rendered in one pass |
+| `[pause]`, `[pause 300ms]`, `[pause 1.5s]` | Exact silence |
+| `<whisper>…</whisper>` (any tag), `<style prompt="…">…</style>` | The span is rendered as its own segment |
+
+Text is rendered sentence by sentence, so the first sentence can play while the rest renders.
+
+## Architecture
+
+FastPitch-style explicit-variance TTS:
+
+1. **Front end.** g2p_en ARPAbet phonemes with word indices.
+2. **Encoder.** 4 blocks.
+3. **Variance predictors.** Per-phone duration, pitch and energy.
+4. **Length regulator.**
+5. **Decoder.** 4 blocks.
+6. **Output.** A 100-band log-mel, rendered by Vocos.
+
+Each encoder/decoder block is a **hybrid**: multi-head self-attention, then a **bidirectional Mamba-3** branch (two
+Mamba-3 layers run forward and backward and summed; complex-valued state, trapezoidal discretisation; 3.70M
+parameters in total), then a conv FFN.
+
+Every LayerNorm in every block and predictor is an **adaptive LayerNorm**. Its scale and shift are predicted from
+g = MLP([speaker vector, tag embedding, ctrl ⊙ mask, mask]). The speaker slot is a linear projection of a frozen
+512-d WavLM-base-plus-sv x-vector. Built-in voices use their stored mean vector; cloning uses the vector of the
+reference clip.
+
+**Guidance** renders the request and the uninstructed request, then extrapolates durations, pitch, energy and mel:
+out = out₀ + s·(out − out₀).
+
+Optional decoders:
+
+- **Flow refiner.** A conditional flow-matching ConvNeXt network (OT-CFM, 10 midpoint steps, temperature 0.4). It
+  samples mel detail given the acoustic model's mel and g, so every control still comes from the acoustic model.
+- **Prompted flow.** The same network plus attention, given the reference clip's own mel as context
+  (Voicebox-style infilling). Used for cloning.
+
+The `steertts` package here is self-contained. It is the inference subset of the training code, operation for
+operation, and its outputs are bit-identical to the training repository's synthesis on the same device (see
+[Release verification](#release-verification)).
+
+## How well it steers
+
+The numbers below are **held-out**: 16 LibriTTS-R test-clean speakers who share no speaker, chapter or recording
+with the training data (8 female, 8 male by a median-F0 rule; the corpus has no gender labels on disk), each
+conditioned on the speaker vector of one recording, × 8 texts. They are for the acoustic model at guidance 1 **without the flow refiner**.
+Steering through the refiner was not measured in the paper; the release check below confirms only the direction.
+
+Columns:
+
+- **Hit**: share of renders that move more than 0.5 SD the requested way.
+- **Δz**: achieved change for a request of ±2.
+- **ρ**: Spearman correlation between request and readback (by our extractor / by Praat).
+
+| Tier | Control | Hit at −2 / +2 | Δz at −2 / +2 | ρ (ours / Praat) |
+|---|---|---|---|---|
+| **Reliable** | loud | 1.00 / 1.00 | −1.57 / +1.81 | .97 / .97 |
+| | f0_med | .98 / 1.00 | −1.15 / +1.18 | .95 / .95 |
+| | h1h2 | 1.00 / 1.00 | −1.59 / +1.49 | .96 / .96 |
+| | tilt | 1.00 / .99 | −1.39 / +0.97 | .96 / .87 |
+| | centroid | .78 / .94 | −0.79 / +1.20 | .92 / .92 |
+| | f0_iqr | .66 / .91 | −0.63 / +1.09 | .88 / .88 |
+| **Usable** | rate | 1.00 / 1.00 | −1.61 / +1.30 | .96 / .66 |
+| | f0_end | .96 / .99 | −0.85 / +1.31 | .94 / .83 |
+| | flat | .62 / .41 | −0.64 / +0.51 | .85 / .68 |
+| **Weak** | pause | .38 / .64 | −0.37 / +0.71 | .76 / .70 |
+| | voiced | .39 / .12 | −0.44 / +0.26 | .79 / .60 |
+| | cpps | .56 / .35 | −0.45 / +0.30 | .58 / .46 |
+
+- **Tiers.** Reliable: ρ ≥ 0.85 under both extractors. Usable: under one. Weak: under neither.
+- **Gain.** The mean gain (achieved / requested) is about 0.5. `rate` and `f0_med` at +1 hit only .48 / .42.
+- **Cross-talk.** Mean leakage into the other 11 dims is 0.15–0.30 SD. The off-diagonal / diagonal ratio is 0.10.
+- **CPPS** is not a working control: a request moves pitch and level about as much as CPPS itself.
+- **Intelligibility.** WER (Whisper-large-v3) is 0.94 % uninstructed and 0.84–1.01 % across all requests.
+
+**Guidance** (pooled over loud, rate, f0_med, f0_iqr and tilt; ±2 requests; n = 1,280):
+
+| s | Steer ratio (Δz / requested) | Hit | Leak | WER |
+|---|---|---|---|---|
+| 1 | 0.65 | .95 | .20 | 0.9 % |
+| 2 | 1.35 | .99 | .34 | 1.0 % |
+| 3 | 1.97 | .98 | .48 | 1.0 % |
+
+- s ≈ 1.5 roughly delivers the requested size, at the cost of more leakage.
+- Guidance on tags is riskier: `whisper` at s = 3 raises WER to 16 %.
+
+**Markup** (8 held-out voices × 6 sentences × 2 words):
+
+- Emphasis achieved +2.58 st and +3.92 dB, against +2.5 st and +4 dB requested.
+- `<high>` / `<low>` gave +3.18 / −2.96 st for ±3.
+- `<loud>` / `<soft>` gave +4.84 / −5.54 dB for ±6. The decoder attenuates level by 10–20 %.
+- `<slow>` / `<fast>` and pauses are exact by construction.
+- Neighbouring words move by at most 0.19 st and 0.16 dB.
+
+**Free-text prompts** (161 hand-written prompts):
+
+- Which-dims precision / recall: .85 / .66. Sign accuracy: .88.
+- Tag top-1 / top-3: .56 / .69.
+- Composition (.23) and negation (.22) mostly fail. "Not fast" can come out fast.
+- Absolute-register prompts ("a deep male voice") are not supported. Pick a voice instead (`voices.json`).
+- **Always look at the resolved request.**
+
+## Quality
+
+Results under the Open TTS Leaderboard protocol (run locally on an RTX 5070, not an official submission;
+Qwen3-ASR-1.7B WER, WavLM-large SIM):
+
+| Setting | Seed-TTS en | CV3-Eval en | Avg |
+|---|---|---|---|
+| Default voice `voice_1361`, acoustic model + Vocos, WER | 1.00 % | 2.63 % | **1.81 %** |
+| Cloning (prompted flow), WER / SIM ×100 | 1.01 % / 37.4 | 2.73 % / 28.7 | 1.87 % / 33.1 |
+
+- The flow refiner was scored with an earlier acoustic checkpoint: 1.04 % / 2.70 % WER (no change).
+- **Cloning similarity is low.** The board's lowest cloning SIM is about 37, and its median is about 65. Do not
+  expect a convincing likeness.
+
+## Speed
+
+Measured on an RTX 5070 and on a shared, heavily loaded 8-thread CPU, so treat CPU figures as lower bounds:
+
+- **GPU.** Batch-1 RTFx 61–81 for the acoustic model + Vocos (leaderboard runs).
+- **CPU, 8 threads.** RTFx 3.6–7.9 for the acoustic model + Vocos. The flow refiner adds 12× the compute and
+  drops RTFx from 6.7 to 0.8 (0.4 with 1 thread). Use `flow=False` / `--no-flow` for real-time CPU use.
+- **Cloning on CPU.** RTFx 0.2 with the prompted flow, including the WavLM speaker encoder.
+- **Streaming on CPU.** First audio after about 1 s.
+- See [Release verification](#release-verification) for the numbers measured on this exact package.
+
+## Limitations
+
+- **English only.** The front end is ARPAbet. Single speaker per request, and the controls are utterance-level.
+- **Emotion tags are style switches, not emotions.** An independent emotion classifier (emotion2vec+ large)
+  recognises only `sadness` (0.16 top-1 vs 0.01 uninstructed). Every other emotion tag scores at most 0.02. The same
+  classifier recognises 7 of 10 of these emotions on real EARS recordings. 27 of 49 tags are acoustically within 0.5
+  of `neutral`. Voice-quality tags such as `whisper` act strongly (voiced share −9.9 SD).
+- **Event tokens only reserve time.** `[laugh]`, `[cough]`, `[dog]`, … add about 0.12 s of near-silence. An audio
+  tagger detects 0 of 1,728 renders. The project's demo mixed recorded sounds in, but that mixer is not part of this
+  release.
+- **The acoustic-environment input is untrained.** The weights contain an environment projection, but the final
+  training stage never saw a non-zero environment vector, so this package does not expose it.
+- **Cloning similarity is low** (see above), and cloning speed on CPU is slow with the prompted flow.
+- **The controls are acoustic measurements, not percepts.** `voiced`, `flat` and `cpps` barely move. `rate` and
+  `tilt` depend on how they are measured. z = 0 is the voice's typical render, not its real recordings.
+- **Test-set-informed choices.** The prompted-flow checkpoint was chosen on subsets of the Seed-TTS / CV3 test
+  sets, and the text-normalisation rules came from CV3 error analysis. The default voice `voice_1361` is the
+  pre-registered one: a sweep picked a lower-WER voice on the test sets, but it is not the default here.
+- **No human evaluation.** Every number above is from automatic metrics, with one training seed.
+
+## Intended use and misuse
+
+**Intended uses:**
+
+- Research on controllable / interpretable speech synthesis.
+- Accessibility and prototyping, where explicit, inspectable control matters more than peak naturalness.
+- On-device experiments with a ~23M-parameter TTS.
+
+**Voice cloning requires consent.** Clone only a voice whose owner has agreed, and do not use this model to:
+
+- impersonate real people;
+- produce deceptive or fraudulent audio;
+- bypass voice authentication;
+- create non-consensual content.
+
+Disclose synthetic speech to listeners. There is no watermark yet.
+
+**Built-in voices are anonymised.** Built-in voices are training speakers under opaque names. Speaker IDs from the
+source corpora are not included. Speakers from a celebrity-interview corpus (VoxCeleb, via NonverbalTTS) were
+excluded from the voice list.
+
+## Training data
+
+410,566 utterances (659 h) from 4,919 speakers:
+
+| Source | Hours | Licence |
+|---|---|---|
+| LibriTTS-R (clean-100, clean-360, other-500) | 516 | CC BY 4.0 |
+| EARS (read + freeform) | 78 | CC BY-NC 4.0 |
+| Expresso | 11 | CC BY-NC 4.0 |
+| NonverbalTTS (VoxCeleb and Expresso sources) | 12.5 | Source terms apply |
+| Synthetic speech from a commercial TTS API | 43 | The provider's terms of service, incl. a competing-model clause |
+
+Control labels are measured from the audio. Style tags come from the corpora's own style/emotion labels.
+
+**Licence: CC BY-NC 4.0. Non-commercial use only.**
+
+## Files
+
+| File | Contents |
+|---|---|
+| `model.safetensors` | Acoustic model (fp32) |
+| `flow.safetensors`, `pflow.safetensors` | Flow refiner and prompted flow |
+| `prompt_adapter.safetensors` | Free-text → request adapter (0.5M) |
+| `speaker_vectors.safetensors` | 512-d vector per built-in voice, in `config.json:voices_order` |
+| `voices.json` | Measured descriptions (see below) |
+| `control_stats.json` | Pooled SDs, tag list, and each voice's neutral control means (for `measure_z`) |
+| `config.json` | Architecture and file map |
+| `request.schema.json` | The request schema |
+| `steertts/` | The package (incl. `steertts/mamba3` with its own invariant tests) |
+| `tests/` | pytest suite |
+| `leaderboard/` | Open TTS Leaderboard backend |
+| `TEST_LOG.md` | Release verification log |
+| `LICENSE.md` | Code (MIT) and weights (CC BY-NC 4.0) |
+
+`voices.json` descriptions are measured, not copied from corpus metadata:
+
+- **Gender.** A probe on the speaker vector; 99.3 % 5-fold accuracy on 534 self-labelled speakers.
+- **Pitch register.** Median F0 within that gender.
+- **Pace.** Articulation rate.
+
+## Release verification
+
+The package was verified in a fresh CPU environment; `TEST_LOG.md` has the full log.
+
+- **Tests.** The pytest suite passes (16 tests).
+- **Parity.** Seven renders are bit-identical to the training code's synthesis on the same device: acoustic
+  only, controls, tag + guidance, per-word shifts, the flow refiner, and cloning through the prompted flow.
+- **Speed.** On this package, CPU with 4 threads on a loaded machine: RTFx 4.9 without the refiner, 0.7 with it.
+  First streamed audio arrives after 0.5 s.
+
+## Citation
+
+```bibtex
+@misc{steertts2026,
+  title  = {SteerTTS: A Tiny, Measurably Steerable Speech Synthesizer},
+  author = {{Whissle AI}},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/WhissleAI/steer-tts}}
+}
+```

steertts.egg-info/SOURCES.txt

@@ -0,0 +1,33 @@
+LICENSE.md
+README.md
+pyproject.toml
+steertts/__init__.py
+steertts/__main__.py
+steertts/audio.py
+steertts/cli.py
+steertts/config.py
+steertts/flow.py
+steertts/markup.py
+steertts/measure.py
+steertts/model.py
+steertts/pipeline.py
+steertts/prompt.py
+steertts/request.py
+steertts/request.schema.json
+steertts/speaker.py
+steertts/text.py
+steertts.egg-info/PKG-INFO
+steertts.egg-info/SOURCES.txt
+steertts.egg-info/dependency_links.txt
+steertts.egg-info/entry_points.txt
+steertts.egg-info/requires.txt
+steertts.egg-info/top_level.txt
+steertts/mamba3/__init__.py
+steertts/mamba3/layer.py
+steertts/mamba3/reference.py
+steertts/mamba3/scan.py
+steertts/mamba3/ssd.py
+steertts/mamba3/tests.py
+tests/test_mamba3.py
+tests/test_synthesis.py
+tests/test_text_request.py

steertts.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

steertts.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+steertts = steertts.cli:main

steertts.egg-info/requires.txt

@@ -0,0 +1,29 @@
+torch<2.12,>=2.4
+numpy<2.5,>=1.24
+safetensors==0.8.0
+huggingface_hub>=0.23
+vocos==0.1.0
+g2p_en==2.1.0
+nltk==3.10.3
+num2words==0.5.14
+soundfile==0.14.0
+soxr==1.1.0
+
+[all]
+transformers==5.19.0
+praat-parselmouth==0.4.7
+
+[clone]
+transformers==5.19.0
+
+[measure]
+praat-parselmouth==0.4.7
+
+[prompt]
+transformers==5.19.0
+
+[test]
+transformers==5.19.0
+praat-parselmouth==0.4.7
+pytest>=8
+jsonschema>=4.20

steertts.egg-info/top_level.txt

@@ -0,0 +1 @@
+steertts

steertts/__init__.py

@@ -0,0 +1,10 @@
+"""SteerTTS: a small, numerically steerable English TTS (hybrid attention + bidirectional Mamba-3, AdaLN on a
+masked 12-dim control request), with optional flow-matching refiners, a free-text prompt adapter and zero-shot
+cloning from a reference clip."""
+from .config import CONTROL_DIMS, DEFAULT_VOICE, SR
+from .pipeline import SteerTTS, SteerTTSPipeline, resolve_model_dir
+from .request import normalize_request, schema
+
+__all__ = ["SteerTTS", "SteerTTSPipeline", "CONTROL_DIMS", "DEFAULT_VOICE", "SR", "normalize_request", "schema",
+           "resolve_model_dir"]
+__version__ = "1.0.0"

steertts/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+main()

steertts/audio.py

@@ -0,0 +1,36 @@
+"""Waveform I/O and the Vocos mel front end (24 kHz, 100 bands, hop 256)."""
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from .config import HOP, N_FFT, N_MELS, SR
+
+_mel_fe = None
+
+
+def mel_extractor():
+    global _mel_fe
+    if _mel_fe is None:
+        from vocos.feature_extractors import MelSpectrogramFeatures
+        _mel_fe = MelSpectrogramFeatures(sample_rate=SR, n_fft=N_FFT, hop_length=HOP, n_mels=N_MELS, padding="center")
+    return _mel_fe
+
+
+def load_wav(x, sr=None):
+    """path or float array (at `sr`, default 24 kHz) -> (mono float32 at 24 kHz, original sample rate)."""
+    if isinstance(x, (str, Path)):
+        import soundfile as sf
+        y, sr = sf.read(str(x), dtype="float32")
+    else:
+        y, sr = np.asarray(x, np.float32), (sr or SR)
+    y = y.mean(1) if y.ndim > 1 else y
+    if sr != SR:
+        import soxr
+        y = soxr.resample(y, sr, SR)
+    return np.asarray(y, np.float32), sr
+
+
+def save_wav(wav, path):
+    import soundfile as sf
+    sf.write(str(path), np.asarray(wav, np.float32), SR)

steertts/cli.py

@@ -0,0 +1,157 @@
+"""steertts command line.
+
+  steertts say "Hello there." -o hello.wav                                    # default voice, default style
+  steertts say "I can't believe it." --voice voice_1361 --tag anger --ctrl loud=1.5,f0_med=1 -o angry.wav
+  steertts say "Keep it down." --prompt "whispering and slow" --show-request -o soft.wav
+  steertts say "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" --markup -o m.wav
+  steertts say "Now in your voice." --ref-wav me.wav -o clone.wav
+  steertts say --request req.json -o out.wav                                  # a JSON request (request.schema.json)
+  steertts stream "First sentence. Second sentence." -o s.wav                 # prints per-chunk timing
+  steertts voices [--gender female] | steertts tags | steertts schema | steertts prompt "soft and sad"
+  steertts measure out.wav --text "..." --voice voice_1361                    # read the 12 controls back (z)
+
+The model directory comes from --model, else $STEERTTS_MODEL, else the current directory if it holds config.json.
+"""
+import argparse
+import json
+import os
+import sys
+import time
+
+
+def _model_dir(a):
+    if a.model:
+        return a.model
+    if os.environ.get("STEERTTS_MODEL"):
+        return os.environ["STEERTTS_MODEL"]
+    if os.path.exists("config.json") and os.path.exists("voices.json"):
+        return "."
+    sys.exit("steertts: no model directory; pass --model DIR (or an HF repo id) or set STEERTTS_MODEL")
+
+
+def _ctrl(s):
+    out = {}
+    for kv in filter(None, (s or "").split(",")):
+        k, _, v = kv.partition("=")
+        out[k.strip()] = float(v)
+    return out
+
+
+def _load(a, **kw):
+    from . import SteerTTS
+    t0 = time.time()
+    tts = SteerTTS.from_pretrained(_model_dir(a), device=a.device, flow=not a.no_flow, **kw)
+    print(f"loaded {tts} in {time.time() - t0:.1f}s", file=sys.stderr)
+    return tts
+
+
+def _request(a):
+    req = json.load(open(a.request)) if a.request else {}
+    for k, v in dict(text=a.text, voice=a.voice, ref_wav=a.ref_wav, prompt=a.prompt, tag=a.tag,
+                     pace=a.pace, guidance=a.guidance, seed=a.seed).items():
+        if v is not None:
+            req[k] = v
+    if a.ctrl:
+        req["ctrl"] = {**req.get("ctrl", {}), **_ctrl(a.ctrl)}
+    if getattr(a, "markup", False):
+        req["markup"] = True
+    if a.no_flow:
+        req["flow"] = False
+    return req
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog="steertts", description="SteerTTS: steerable English TTS")
+    ap.add_argument("--model", help="model directory or HF repo id (default: $STEERTTS_MODEL or .)")
+    ap.add_argument("--device", help="cpu | cuda (default: cuda if available)")
+    ap.add_argument("--no-flow", action="store_true", help="skip the flow-matching refiner")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    for name in ("say", "stream"):
+        p = sub.add_parser(name)
+        p.add_argument("text", nargs="?")
+        p.add_argument("--request", help="JSON request file (fields as in request.schema.json)")
+        p.add_argument("--voice")
+        p.add_argument("--ref-wav", dest="ref_wav", help="clone this reference clip (consent required)")
+        p.add_argument("--prompt", help="free-text direction, e.g. 'soft and sad'")
+        p.add_argument("--tag")
+        p.add_argument("--ctrl", help="comma list dim=z, e.g. rate=1.5,f0_med=-1")
+        p.add_argument("--pace", type=float)
+        p.add_argument("--guidance", type=float)
+        p.add_argument("--seed", type=int)
+        p.add_argument("--markup", action="store_true")
+        p.add_argument("--show-request", action="store_true", help="print the resolved request JSON")
+        p.add_argument("-o", "--out", default="out.wav")
+    p = sub.add_parser("voices")
+    p.add_argument("--gender", choices=["female", "male"])
+    p.add_argument("--json", action="store_true")
+    sub.add_parser("tags")
+    sub.add_parser("schema")
+    p = sub.add_parser("prompt")
+    p.add_argument("prompt")
+    p = sub.add_parser("measure")
+    p.add_argument("wav")
+    p.add_argument("--text")
+    p.add_argument("--voice")
+    a = ap.parse_args(argv)
+
+    if a.cmd == "schema":
+        from .request import schema
+        print(json.dumps(schema(), indent=1))
+        return
+    if a.cmd in ("voices", "tags"):
+        from .pipeline import resolve_model_dir
+        d = resolve_model_dir(_model_dir(a))
+        if a.cmd == "tags":
+            print(" ".join(json.load(open(d / "control_stats.json"))["tags"]))
+            return
+        vs = json.load(open(d / "voices.json"))["voices"]
+        vs = [v for v in vs if not a.gender or v.get("gender") == a.gender]
+        if a.json:
+            print(json.dumps(vs, indent=1))
+        else:
+            for v in vs:
+                print(f"{v['name']}  {v['description']}")
+        return
+    if a.cmd == "measure":
+        from .audio import load_wav
+        from .measure import measure, measure_z, n_phones
+        from .pipeline import resolve_model_dir
+        y, _ = load_wav(a.wav)
+        if a.voice:
+            st = json.load(open(resolve_model_dir(_model_dir(a)) / "control_stats.json"))
+            z = measure_z(y, a.text, st["spk_mean"][a.voice], st["pooled_sd"])
+            print(json.dumps({k: round(v, 2) for k, v in z.items()}))
+        else:
+            from .config import CONTROL_DIMS
+            raw = measure(y, n_phones=n_phones(a.text) if a.text else None, want_mel=False)["controls"]
+            print(json.dumps({k: round(float(v), 3) for k, v in zip(CONTROL_DIMS, raw)}))
+        return
+    tts = _load(a)
+    if a.cmd == "prompt":
+        print(json.dumps(tts.map_prompt(a.prompt), indent=1))
+        return
+    req = _request(a)
+    if not req.get("text"):
+        sys.exit("steertts: no text (positional argument or --request)")
+    req = tts.resolve(req)
+    if getattr(a, "show_request", False) or "prompt_mapping" in req:
+        print(json.dumps(req, indent=1), file=sys.stderr)
+    import numpy as np
+    if a.cmd == "say":
+        t0 = time.time()
+        wav = tts.synthesize(req)
+        dt = time.time() - t0
+    else:
+        t0, chunks = time.time(), []
+        for i, c in enumerate(tts.stream(request=req)):
+            chunks.append(c.audio)
+            print(f"chunk {i}: {len(c.audio) / tts.sample_rate:.2f}s audio at +{(time.time() - t0) * 1000:.0f} ms "
+                  f"({c.segment.text or ('pause %d ms' % c.segment.pause_ms)!r})", file=sys.stderr)
+        wav = np.concatenate(chunks) if chunks else np.zeros(1, np.float32)
+        dt = time.time() - t0
+    tts.save(wav, a.out)
+    print(f"{a.out}: {len(wav) / tts.sample_rate:.2f}s audio in {dt:.2f}s", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

steertts/config.py

@@ -0,0 +1,27 @@
+"""Audio / control constants. They must match training: the charactr/vocos-mel-24khz mel front end."""
+SR = 24000
+N_FFT = 1024
+HOP = 256
+N_MELS = 100
+FPS = SR / HOP  # 93.75 mel frames per second
+PITCH_SCALE = 3.0  # predicted token pitch is semitones re the speaker median / 3; word_shift pitch is given in st
+
+# Utterance-level controls. Each is a SPEAKER-RELATIVE z-score: 0 = this voice's own neutral speech, +-1 = one
+# pooled within-speaker standard deviation (pooled_sd in control_stats.json gives the raw unit per dim).
+CONTROL_DIMS = [
+    "rate",      # articulation rate, phones per second of speech (pauses excluded)
+    "loud",      # mean RMS level of speech frames, dB
+    "f0_med",    # median F0, semitones
+    "f0_iqr",    # F0 interquartile range, semitones (pitch range / liveliness)
+    "f0_end",    # F0 slope over the final 300 ms of voicing, st/s (+ rising, question-like)
+    "pause",     # share of the speech span that is silent pauses >= 100 ms
+    "tilt",      # alpha ratio: energy 1-5 kHz vs 50 Hz-1 kHz, dB (+ brighter / pressed)
+    "voiced",    # voiced share of speech frames (whisper -> low)
+    "flat",      # spectral flatness of speech frames, log (breathy / whisper -> high)
+    "centroid",  # spectral centroid of speech frames, log Hz (brightness)
+    "cpps",      # smoothed cepstral peak prominence, dB (periodicity; breathy / tense -> low)
+    "h1h2",      # H1-H2, dB (breathy -> high)
+]
+CONTROL_RANGE = (-5.0, 5.0)  # accepted request range; training labels rarely exceed +-3
+
+DEFAULT_VOICE = "voice_1361"  # pre-registered default (chosen before any test-set voice sweep)

steertts/flow.py

@@ -0,0 +1,278 @@
+"""Flow-matching mel decoders (optional), inference subset of recipes/steer_tts/steer/flow.py.
+
+refiner (flow.safetensors): a conditional flow (OT-CFM, as the Matcha-TTS decoder) that samples a detailed mel
+    given the acoustic model's mel mu and its conditioning g. Durations, pitch, energy, the coarse spectrum and
+    every control still come from the acoustic model; the flow adds texture (sharper harmonics / noise).
+prompted flow (pflow.safetensors): the same with an audio prompt - the reference clip's own mel is given as
+    context before the target (Voicebox / F5-TTS-style infilling). Used only for zero-shot cloning.
+"""
+import math
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from .config import N_MELS
+
+SIGMA = 1e-4
+
+class TimeEmb(nn.Module):
+    def __init__(self, d):
+        super().__init__()
+        self.d = d
+        self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.SiLU(), nn.Linear(4 * d, d))
+
+    def forward(self, t):  # [B] in [0, 1]
+        half = self.d // 2
+        f = torch.exp(-math.log(10000) * torch.arange(half, device=t.device) / half)
+        a = 1000 * t[:, None] * f[None]
+        return self.mlp(torch.cat([a.sin(), a.cos()], -1))
+
+
+class Block(nn.Module):
+    """ConvNeXt block (depthwise dilated conv -> LN -> MLP) with AdaLN-zero conditioning on (t, g)."""
+
+    def __init__(self, d, c, k=7, dil=1, mult=3):
+        super().__init__()
+        self.dw = nn.Conv1d(d, d, k, padding=dil * (k // 2), dilation=dil, groups=d)
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.pw1, self.pw2 = nn.Linear(d, mult * d), nn.Linear(mult * d, d)
+
+    def forward(self, x, m, c):  # x [B,d,T], m [B,1,T], c [B,c]
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.dw(x * m).transpose(1, 2)
+        h = self.ln(h) * (1 + sc) + sh
+        h = self.pw2(F.gelu(self.pw1(h))) * gate
+        return x + h.transpose(1, 2)
+
+
+class AttnBlock(nn.Module):
+    """Full self-attention over the sequence with AdaLN-zero conditioning: lets the audio prompt (prompted flow)
+    reach every target frame, beyond the conv receptive field (~170 frames each side)."""
+
+    def __init__(self, d, c, heads=6):
+        super().__init__()
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.att = nn.MultiheadAttention(d, heads, batch_first=True)
+
+    def forward(self, x, m, c):
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.ln(x.transpose(1, 2)) * (1 + sc) + sh
+        h, _ = self.att(h, h, h, key_padding_mask=~m[:, 0].bool(), need_weights=False)
+        return x + (h * gate).transpose(1, 2) * m
+
+
+def rope(x):  # x [B,H,T,dh]: rotary position embedding (Su et al. 2021) over the frame axis
+    T, dh = x.shape[-2], x.shape[-1]
+    f = torch.exp(-math.log(10000) * torch.arange(0, dh, 2, device=x.device, dtype=torch.float32) / dh)
+    a = torch.arange(T, device=x.device, dtype=torch.float32)[:, None] * f[None]
+    cos, sin = a.cos().to(x.dtype), a.sin().to(x.dtype)
+    x1, x2 = x[..., 0::2], x[..., 1::2]
+    return torch.stack([x1 * cos - x2 * sin, x1 * sin + x2 * cos], -1).flatten(-2)
+
+
+class PromptPool(nn.Module):
+    """Perceiver-resampler pooling of the prompt (XTTS; SupertonicTTS's learnable timbre queries): n learnable
+    queries cross-attend to the encoded prompt frames -> n global voice tokens. A learned null key is always
+    present, so an item without a prompt still gets defined (prompt-free) tokens."""
+
+    def __init__(self, d, n, heads=6, layers=2):
+        super().__init__()
+        self.q = nn.Parameter(torch.randn(n, d) * 0.02)
+        self.null = nn.Parameter(torch.zeros(1, 1, d))
+        self.heads = heads
+        self.layers = nn.ModuleList([nn.ModuleDict(dict(
+            lq=nn.LayerNorm(d), lk=nn.LayerNorm(d), q=nn.Linear(d, d), kv=nn.Linear(d, 2 * d), o=nn.Linear(d, d),
+            lf=nn.LayerNorm(d), f=nn.Sequential(nn.Linear(d, 2 * d), nn.GELU(), nn.Linear(2 * d, d))))
+            for _ in range(layers)])
+
+    def forward(self, e, cm):  # e [B,d,T] encoded prompt (zero off-prompt), cm [B,1,T]
+        B, d, T = e.shape
+        kv_in = torch.cat([self.null.expand(B, 1, d), e.transpose(1, 2)], 1)
+        keep = torch.cat([torch.ones(B, 1, device=e.device, dtype=torch.bool), cm[:, 0].bool()], 1)[:, None, None]
+        q = self.q[None].expand(B, -1, -1)
+        H = self.heads
+        for l in self.layers:
+            k, v = l["kv"](l["lk"](kv_in)).chunk(2, -1)
+            qq = l["q"](l["lq"](q))
+            sp = lambda z: z.view(B, -1, H, d // H).transpose(1, 2)
+            h = F.scaled_dot_product_attention(sp(qq), sp(k), sp(v), attn_mask=keep)
+            q = q + l["o"](h.transpose(1, 2).reshape(B, -1, d))
+            q = q + l["f"](l["lf"](q))
+        return q  # [B,n,d]
+
+
+class RAttnBlock(nn.Module):
+    """Self-attention with RoPE positions (frames), plus optional cross-attention to the pooled prompt tokens; both
+    zero-gated (AdaLN-zero gate / zero output projection), so adding these blocks to a trained net changes nothing
+    at init."""
+
+    def __init__(self, d, c, heads=6, xattn=False):
+        super().__init__()
+        self.heads = heads
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.mod = nn.Linear(c, 3 * d)
+        nn.init.zeros_(self.mod.weight); nn.init.zeros_(self.mod.bias)
+        self.qkv, self.o = nn.Linear(d, 3 * d), nn.Linear(d, d)
+        self.xattn = xattn
+        if xattn:
+            self.lx = nn.LayerNorm(d)
+            self.xq, self.xkv, self.xo = nn.Linear(d, d), nn.Linear(d, 2 * d), nn.Linear(d, d)
+            nn.init.zeros_(self.xo.weight); nn.init.zeros_(self.xo.bias)
+
+    def forward(self, x, m, c, tok=None):  # x [B,d,T], tok [B,n,d]
+        B, d, T = x.shape
+        H = self.heads
+        sp = lambda z: z.view(B, -1, H, d // H).transpose(1, 2)
+        sc, sh, gate = self.mod(c)[:, None].chunk(3, -1)
+        h = self.ln(x.transpose(1, 2)) * (1 + sc) + sh
+        q, k, v = (sp(z) for z in self.qkv(h).chunk(3, -1))
+        h = F.scaled_dot_product_attention(rope(q), rope(k), v, attn_mask=m[:, 0].bool()[:, None, None])
+        h = self.o(h.transpose(1, 2).reshape(B, T, d)) * gate
+        if self.xattn and tok is not None:
+            hx = self.lx(x.transpose(1, 2) + h)
+            k2, v2 = self.xkv(tok).chunk(2, -1)
+            h = h + self.xo(F.scaled_dot_product_attention(sp(self.xq(hx)), sp(k2), sp(v2)).transpose(1, 2).reshape(B, T, d))
+        return x + h.transpose(1, 2) * m
+
+
+class FlowNet(nn.Module):
+    """prompt=True (prompted flow): also takes a context mel ctx with mask cm (1 = real audio given: the reference
+    clip's frames), as Voicebox / F5-TTS infilling. The context enters the input stack, a masked mean of it is added
+    to the conditioning (global voice), and attention layers at attn_at carry it across the whole sequence.
+
+    v5 additions (all zero-initialised, so a v4 checkpoint loads into a v5 net unchanged):
+      rope_at   layers that get an extra RoPE self-attention block (RAttnBlock)
+      n_ptok    pooled prompt tokens (PromptPool) that the RoPE blocks cross-attend to
+      skips     E2-TTS-style long skips, block i -> block n-1-i
+      align_at  layers whose target-frame mean is mapped to a speaker-verification vector (TLA-SA-style feature
+                alignment loss; training only)"""
+
+    def __init__(self, n_mel=N_MELS, d=384, n_layers=12, g_dim=128, dils=(1, 2, 4, 1, 2, 4, 1, 2, 4, 1, 2, 4),
+                 prompt=False, attn_at=(3, 7, 11), rope_at=(), n_ptok=0, skips=False, align_at=(), align_dim=512):
+        super().__init__()
+        self.cfg = dict(n_mel=n_mel, d=d, n_layers=n_layers, g_dim=g_dim, dils=tuple(dils), prompt=prompt,
+                        attn_at=tuple(attn_at), rope_at=tuple(rope_at), n_ptok=n_ptok, skips=skips,
+                        align_at=tuple(align_at), align_dim=align_dim)
+        self.prompt = prompt
+        self.inp = nn.Conv1d((3 * n_mel + 1) if prompt else 2 * n_mel, d, 3, padding=1)
+        self.temb = TimeEmb(d)
+        self.gemb = nn.Sequential(nn.Linear(g_dim, d), nn.SiLU(), nn.Linear(d, d))
+        self.blocks = nn.ModuleList([Block(d, d, dil=dils[i % len(dils)]) for i in range(n_layers)])
+        if prompt:
+            self.attn = nn.ModuleDict({str(i): AttnBlock(d, d) for i in attn_at})
+            self.ctx_enc = nn.Sequential(nn.Conv1d(n_mel, d, 3, padding=1), nn.SiLU(), nn.Conv1d(d, d, 3, padding=1))
+            self.ctx_out = nn.Linear(d, d)
+            nn.init.zeros_(self.ctx_out.weight); nn.init.zeros_(self.ctx_out.bias)
+            self.rattn = nn.ModuleDict({str(i): RAttnBlock(d, d, xattn=n_ptok > 0) for i in rope_at})
+            self.pool = PromptPool(d, n_ptok) if n_ptok else None
+        self.skips = skips
+        if skips:
+            self.skip = nn.ModuleDict({str(i): nn.Linear(d, d) for i in range(n_layers - n_layers // 2, n_layers)})
+            for l in self.skip.values():
+                nn.init.zeros_(l.weight); nn.init.zeros_(l.bias)
+        self.align_at = tuple(align_at)
+        if align_at:
+            self.align = nn.ModuleDict({str(i): nn.Sequential(nn.Linear(d, d), nn.SiLU(), nn.Linear(d, align_dim))
+                                        for i in align_at})
+        self.ln = nn.LayerNorm(d, elementwise_affine=False)
+        self.out = nn.Conv1d(d, n_mel, 1)
+        nn.init.zeros_(self.out.weight); nn.init.zeros_(self.out.bias)
+
+    def forward(self, x, t, mu, g, m, ctx=None, cm=None, feats=False):  # normalised mels [B,100,T]; m, cm [B,1,T]
+        """feats=True also returns {layer: speaker vector predicted from the target-frame mean} for align_at."""
+        c = self.temb(t) + self.gemb(g)
+        tok = None
+        if self.prompt:
+            if cm is None:
+                ctx, cm = torch.zeros_like(x), torch.zeros_like(m)
+            ctx = ctx * cm
+            e = self.ctx_enc(ctx) * cm
+            c = c + self.ctx_out(e.sum(-1) / cm.sum(-1).clamp_min(1))
+            if getattr(self, "pool", None) is not None:
+                tok = self.pool(e, cm)
+            h = self.inp(torch.cat([x, mu, ctx, cm], 1) * m)
+        else:
+            h = self.inp(torch.cat([x, mu], 1) * m)
+        rattn = getattr(self, "rattn", {})
+        stack, out = [], {}
+        n = len(self.blocks)
+        for i, b in enumerate(self.blocks):
+            if self.skips and str(i) in self.skip:
+                h = h + self.skip[str(i)](stack.pop().transpose(1, 2)).transpose(1, 2) * m
+            h = b(h, m, c)
+            if self.prompt and str(i) in self.attn:
+                h = self.attn[str(i)](h, m, c)
+            if str(i) in rattn:
+                h = rattn[str(i)](h, m, c, tok)
+            if self.skips and i < n // 2:
+                stack.append(h)
+            if feats and i in self.align_at:
+                w = m * (1 - cm) if cm is not None else m
+                out[i] = self.align[str(i)](((h * w).sum(-1) / w.sum(-1).clamp_min(1)).float())
+        v = self.out(self.ln(h.transpose(1, 2)).transpose(1, 2)) * m
+        return (v, out) if feats else v
+
+
+class Flow(nn.Module):
+    """FlowNet + mel normalisation; sample() maps a base mel to a refined mel (log-mel units, same frames)."""
+
+    def __init__(self, mean=-5.0, std=2.5, mu_cmn=False, **kw):
+        """mu_cmn (prompted flow v2): remove the per-band time-mean of the base mel (its static spectral envelope:
+        the long-term voice colouring the base model guessed from a 512-d speaker vector) so the flow must take the
+        voice from the audio prompt (or g); timing / pitch / energy / phonetic detail of mu are kept. Without it, mu
+        made from the TRUE speaker vector already carries the training speaker's colouring and the prompt adds
+        little (v3p @20k: Seed-TTS SIM 31.2 -> 33.2 only)."""
+        super().__init__()
+        self.net = FlowNet(**kw)
+        self.mu_cmn = mu_cmn
+        self.register_buffer("mean", torch.tensor(float(mean)))
+        self.register_buffer("std", torch.tensor(float(std)))
+
+    def norm(self, m):
+        return (m - self.mean) / self.std
+
+    def _cmn(self, mu_n, w):
+        """mu_n minus its per-band mean over the frames w marks (the target frames)."""
+        if not self.mu_cmn:
+            return mu_n
+        return mu_n - (mu_n * w).sum(-1, keepdim=True) / w.sum(-1, keepdim=True).clamp_min(1)
+
+    @torch.no_grad()
+    def sample(self, mu, g, m=None, steps=10, temp=0.8, method="midpoint", seed=None, ctx=None, cm=None, cfg=0.0):
+        """ctx / cm (prompted flow): real mel on the context frames (log-mel units), mask 1 there; mu is replaced by
+        the context on those frames, as in training. Returns every frame (callers slice off the context).
+        cfg (prompted flow): classifier-free guidance on the prompt, v + cfg * (v - v_no_prompt); the no-prompt
+        velocity is the same net with the context removed (trained on p_none of the items)."""
+        m = torch.ones_like(mu[:, :1]) if m is None else m
+        gen = None if seed is None else torch.Generator(device=mu.device).manual_seed(seed)
+        mu_n = self._cmn(self.norm(mu), m if cm is None else m * (1 - cm))
+        if cm is not None:
+            ctx = self.norm(ctx) * cm
+            mu_n = torch.where(cm.bool(), ctx, mu_n)
+        x = temp * torch.randn(mu_n.shape, device=mu.device, generator=gen) * m
+        ts = torch.linspace(0, 1, steps + 1, device=mu.device)
+
+        def vel(x, t):
+            v = self.net(x, t.expand(x.shape[0]), mu_n, g, m, ctx, cm)
+            if cfg and cm is not None:
+                v = v + cfg * (v - self.net(x, t.expand(x.shape[0]), mu_n, g, m, None, None))
+            return v
+
+        for i in range(steps):
+            t0, dt = ts[i], ts[i + 1] - ts[i]
+            v = vel(x, t0)
+            if method == "midpoint":
+                v = vel(x + 0.5 * dt * v, t0 + 0.5 * dt)
+            x = x + dt * v
+        return x * self.std + self.mean
+
+    @staticmethod
+    def from_state(state, cfg, device="cpu"):
+        f = Flow(**cfg)
+        f.load_state_dict(state)
+        return f.to(device).eval()

steertts/mamba3/__init__.py

@@ -0,0 +1,25 @@
+"""Mamba-3: a paper-faithful, verified implementation of Lahoti et al. (arXiv 2603.15569).
+
+Ported from the MMSE research repo (drowsiness-uta-rldd/mamba3, commit c5280e6+), without the MMSE
+multi-stream extension or the LM tooling. Checked fwd+bwd at float64 against the recurrence:
+  - exponential-trapezoidal discretization with DATA-DEPENDENT lambda_t (Prop 1)
+  - complex state as real block-diagonal rotations / data-dependent RoPE (Prop 2-4, eq 11)
+  - MIMO rank-R (Sec 3.3)
+  - BCNorm/QKNorm on B,C + learnable data-independent B,C biases
+  - output gating
+Recorded deviations: data-independent A (Remark 1 says equivalent); MIMO input uses a full DPR
+projection rather than the paper's DP+PR (R=1 identical).
+
+Paths: `Mamba3Layer.forward` (O(T) recurrence, reference), `scan_forward` (chunked scan),
+`scan_forward_ssd` (SSD reduction, the fast path for training). Run `python -m
+steertts.mamba3.tests` after any change.
+"""
+from .layer import Mamba3Layer, rotate_pairs, ssm_scan
+from .scan import scan_forward, chunked_scan, rotate_by, cumulative_angles
+from .ssd import scan_forward_ssd, ssd_inputs, ssd_pytorch
+
+__all__ = [
+    "Mamba3Layer", "rotate_pairs", "ssm_scan",
+    "scan_forward", "chunked_scan", "rotate_by", "cumulative_angles",
+    "scan_forward_ssd", "ssd_inputs", "ssd_pytorch",
+]