WhissleAI/steer-tts-edge

SteerTTS v1 edge

The 12 MB on-device build of SteerTTS v1, a small English text-to-speech model that you steer with numbers. It runs four ways from this one repo, all rendering the same audio:

Runtime	What an app ships	Peak RAM	Code
C library: plain C11, no dependencies (no Python, no ONNX Runtime)	13.1 MB of model files + a 1.4 MB static binary	27 MB	GitHub c/
Native C++ on ONNX Runtime: no Python	13.9 MB of model files + the 25 MB ONNX Runtime library	98 MB	GitHub native/
Python + ONNX Runtime (OnnxTTS, default since 1.1.0)	12.6 MB	127 MB	steertts_edge/ here
Python + PyTorch (EdgeTTS, the reference)	11.85 MB	369 MB	steertts_edge/ here

The model itself:

• Acoustic model: the SteerTTS v1 acoustic model (9.25M parameters; attention + bidirectional Mamba-3, AdaLN conditioning), quantization-aware trained to int4, with int8 kept for the conditioning path.
• Vocoder: a Vocos-style vocoder (6.2M parameters) distilled from charactr/vocos-mel-24khz, also int4 QAT.
• Text front end: pure numpy. A residual CMUdict lexicon, an int8 copy of g2p_en's neural G2P for unknown words, and number / currency / unit normalisation. No NLTK, g2p_en, num2words, Vocos or torchaudio at runtime.

It keeps the steering of the full model:

• 200 built-in voices (voice_0005 … voice_4223, a subset of the full model's 2,825; default voice_1688).
• 50 style tags (whisper, sadness, anger, fast, high_pitch, …).
• 12 numeric controls (rate, loudness, median pitch, pitch range, final rise, pauses, tilt, breathiness, …), each a z-score relative to the chosen voice.
• Inline markup: *emphasis*, [pause 400ms], <slow>, <whisper>…</whisper>, …

Not on device. Use the full model, WhissleAI/steer-tts, on a host for these:

• free-text prompts ("whispering, slow and secretive"): the host resolves them into tag + ctrl, and those you can send to the device;
• voice cloning: the WavLM speaker encoder is not included;
• the flow-matching refiner: the edge build is acoustic model + vocoder only.

Licence: weights CC BY-NC 4.0 (non-commercial use only); package code MIT. The weights are distilled from SteerTTS v1, which was trained partly on non-commercial data. See LICENSE.md.

Listen

Raw output of this package (int4 acoustic model + int4 vocoder, default voice voice_1688), rendered on a CPU with the steertts-edge CLI (tools/render_samples.py).

Request	Audio
"Hi, thanks for calling. I can help you reschedule your appointment today." (no controls)
"Are you coming with us tonight?" ctrl={"f0_end": 2, "f0_iqr": 1.5}: rising, livelier
"The weather was cold, but the view from the top was worth every step." ctrl={"rate": -1.5}
Same text, ctrl={"rate": 1.5}
"Please keep this between the two of us." tag="whisper"
"I really thought it would work this time." tag="sadness", guidance=1.5
Markup: Well, *that* was fast. [pause 400ms] <slow>Let me think about it.</slow> <whisper>Don't tell anyone.</whisper>

Size

The on-device payload, as counted by tools/sync_package.py (weights, voices, front end, runtime code, config, notices; docs, tests and tooling excluded):

Part	Bytes
acoustic.safetensors	6,648,886	Acoustic model, int4 QAT (+ int8 / fp16 for sensitive tensors)
vocoder.safetensors	3,862,640	Vocoder, int4 QAT (Vocos-style, dim 384, 8 ConvNeXt layers)
front/	1,150,249	Text front end: residual CMUdict lexicon, int8 GRU G2P, normaliser tables
steertts_edge/	133,800	Runtime code (Python)
voices.safetensors	30,544	200 built-in voices (64-d conditioning slots)
voices.json	19,720	Voice names and descriptors
NOTICE	1,947	Third-party notices
config.json	1,170	Config (tags, control scales)
Total	11,848,956	11.85 MB, budget 12,000,000 bytes

That is the PyTorch runtime's payload. The ONNX Runtime payload swaps the two safetensors for onnx/enc.onnx, dec.onnx and voc.onnx (11.28 MB, the same int4 / int8 codes plus graph structure): 12.62 MB with the Python front end and code, 13.89 MB for the native runtime (native/front/ is the uncompressed C++ form of front/).

For comparison, the full SteerTTS v1 install is about 765 MB of weights (fp32, including the external encoders), or 193 MB in its int8 variant.

Quick start

git clone https://huggingface.co/WhissleAI/steer-tts-edge && cd steer-tts-edge
pip install .                    # onnxruntime, numpy, safetensors; [hub] adds from_pretrained, [torch] the PyTorch runtime
steertts-edge "Hello there, nice to meet you." -o hello.wav
steertts-edge "I can't believe it worked." --voice voice_0047 --tag happiness --ctrl f0_iqr=1.5 --rate 1 -o happy.wav
steertts-edge "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" --markup -o markup.wav
steertts-edge --list voices      # or: tags, controls

The CLI uses --model DIR_OR_REPO, else $STEERTTS_EDGE_MODEL, else the current directory if it holds the package, else downloads WhissleAI/steer-tts-edge. --runtime onnx|torch picks the runtime (default: ONNX Runtime when it is installed). For the PyTorch runtime a CPU-only torch wheel is enough: pip install torch --index-url https://download.pytorch.org/whl/cpu.

from steertts_edge import OnnxTTS            # or EdgeTTS (PyTorch): same API, same audio up to float rounding

tts = OnnxTTS("path/to/steer-tts-edge", threads=1)            # a local directory, CPU
# tts = OnnxTTS.from_pretrained("WhissleAI/steer-tts-edge")   # downloads only what this runtime reads (needs huggingface_hub)
wav = tts("The meeting moved to Thursday.")                    # float32 numpy, 24 kHz, voice_1688
wav = tts("Are you serious?", voice="voice_0047", ctrl={"f0_end": 2.0, "f0_iqr": 1.5})   # rising, lively
wav = tts("It is over.", tag="sadness", ctrl={"loud": -1}, guidance=1.5)
wav = tts("Please keep this quiet.", tag="whisper", ctrl={"rate": -1})
wav = tts("Well, *that* was fast. [pause 400ms] <whisper>Don't tell anyone.</whisper>", markup=True)
tts.save(wav, "out.wav")                                       # 16-bit PCM WAV

for chunk in tts.stream("First sentence. Second sentence. <slow>Third.</slow>"):   # sentence-level streaming
    play(chunk.audio)                                          # chunk.segment has the text, tag and controls

Renders are deterministic: the same request gives the same samples, on the same device and thread count.

C library. The GitHub repo has a dependency-free C11 library (libsteertts, one header), a steertts CLI, examples and a ctypes binding. It reads config.json, voices.*, acoustic.safetensors, vocoder.safetensors and native/front/ from this repo and keeps the int4 weights packed in memory:

git clone https://github.com/WhissleAI/steer-tts && cd steer-tts/c && make     # or cmake -B build && cmake --build build
tools/get_models.sh edge                       # curl only; export HF_TOKEN=<read token> if this repo asks you to accept
build/steertts --model models/steer-tts-edge "Hello from C." --stream -o - | aplay -f S16_LE -r 24000 -c 1

Same audio as the Python runtimes (218 Seed-TTS sentences: identical lengths, ~88 dB median SNR to EdgeTTS, same WER), bit-identical at any thread count, and an ARM64 build that renders the same audio on an emulated Raspberry Pi 4 / 5 (estimated 2.2–4.1× real time on one Pi 4 core). Details: docs/c.md.

Native C++ on ONNX Runtime. The GitHub repo has a C++17 library, a C API and the same steertts-edge CLI without Python, built with CMake against the ONNX Runtime release (x86-64 and ARM64 Linux). It reads config.json, voices.*, onnx/ and native/front/ from this repo:

git clone https://github.com/WhissleAI/steer-tts && cd steer-tts/native && cmake -B build && cmake --build build -j8
huggingface-cli download WhissleAI/steer-tts-edge --local-dir model --include config.json --include "voices.*" --include "onnx/*" --include "native/*"
./build/steertts-edge --model model "Hello from C++." --stream -o - | aplay -f S16_LE -r 24000 -c 1

The C++ text front end matches the Python one exactly (6,343 texts, 610 markup inputs, 33,265 G2P words), and the native audio matches the Python ONNX runtime at ~140 dB SNR. An ARM64 build renders the same audio under an emulated Raspberry Pi 4 and Pi 5; speed on real Pi hardware is estimated, not yet measured (details).

The request

Argument	Meaning
text	English text. Numbers, currency, units, ordinals, ranges and unknown acronyms are spelled out. Text with nothing to say (empty, punctuation only, non-Latin script) returns an empty array.
voice	One of the 200 built-in voices (tts.voices); default voice_1688. The leaderboard rows use voice_1361.
tag	One of 50 style tags (tts.tags, plus none).
ctrl	{control: z} over the 12 controls, each in [−5, 5].
guidance	Steering scale (default 1). Above 1 it extrapolates away from the uninstructed render.
pace	Global tempo divisor (> 1 is faster).
word_shift	{word_index: {"pitch": st, "energy": dB, "dur": ×}} per-word edits (not with markup / stream()).
markup	Parse inline markup (see below). stream() always does.
spk_vec	Advanced: a 512-d WavLM-base-plus-sv speaker vector computed on a host by the full model, in place of voice. Not evaluated on the edge build.

prompt= and ref_wav= raise an error that points to the full model. flow= and seed= are accepted and ignored, so code written for steertts.SteerTTS(flow=False) runs unchanged.

Controls. Each is a z-score against the chosen voice's own neutral speech, in units of the pooled within-speaker SD: rate (articulation rate), loud (speech level), f0_med (median pitch), f0_iqr (pitch range), f0_end (final pitch slope), pause (pause share), tilt (spectral tilt), voiced (voiced share), flat (spectral flatness), centroid (spectral centroid), cpps (cepstral peak prominence), h1h2 (H1–H2). A control you leave out is chosen by the model from voice, tag and text; {"rate": 0} asks for the voice's neutral rate.

Style tags. adoration, amazement, amusement, anger, anxiety, apologetic, bored, confidence, confusion, contempt, contentment, cuteness, desire, determined, disappointment, disgust, distress, ecstasy, embarrassment, emphasis, enunciated, excited, fast, fear, frustration, guilt, happiness, hesitant, high_pitch, interest, irritation, laughing, loud, low_pitch, mysterious, neutral, nostalgia, pain, panic, pride, realization, relief, sadness, sarcasm, serenity, skeptical, slow, sympathy, tired, whisper.

Markup (markup=True or stream()):

Markup	Effect
*word* or <emph>…</emph>	Emphasis: +2.5 st, +4 dB, ×1.2 duration
<slow> <fast> <high> <low> <loud> <soft>	Per-word duration / pitch / energy changes; the sentence is still one pass
[pause], [pause 300ms], [pause 1.5s]	Exact silence (default 400 ms, max 5 s)
<whisper>…</whisper>, <sadness>…</sadness>, any style tag	The span is rendered as its own segment with that tag

Unknown markup tags are dropped and their text is spoken normally. <style prompt="…"> needs the host-side prompt adapter and raises an error on the device.

Quality

Build	Size	Seed-TTS en WER	Seed-TTS en UTMOS	CV3-Eval en WER	CV3-Eval en UTMOS
SteerTTS v1 fp32, flow refiner off	~765 MB	0.96 %	4.20	2.62 %	4.17
steer-tts-edge (this repo)	11.84 MB	0.96 %	4.11	2.67 %	4.08

Open TTS Leaderboard protocol, run locally (not an official submission): Qwen3-ASR-1.7B WER, UTMOS22-strong, voice voice_1361, Seed-TTS-eval English (1,088 sentences) and CV3-Eval English (500 sentences). The fp32 row is the full model's acoustic model + Vocos without the flow refiner, the configuration the edge build reproduces.

• Intelligibility is unchanged (+0.00 / +0.05 WER points).
• UTMOS drops by 0.09 (Seed) / 0.10 (CV3). Almost all of it comes from the 6M-parameter int4 vocoder, which replaces the 13.5M-parameter fp32 Vocos. The int4 acoustic model with the fp32 Vocos scores 4.20 / 4.17 UTMOS (fp32: 4.20 / 4.17) and 0.95 % / 2.66 % WER. An earlier vocoder of the same size (the first release candidate) scored 4.09 / 4.06.

How well it steers

Readback of the controls from the rendered audio (the measurement code that labelled the training data), 8 voices × 6 sentences, requests of −2 and +2, on CPU. Δz is the mean achieved change in z; hit is the share of renders that move more than 0.5 SD the requested way. The fp32 columns are the full model without the refiner, measured the same way on the same voices and sentences.

Control	fp32 Δz at −2 / +2	edge Δz at −2 / +2	edge / fp32 (worse side)	edge hit at −2 / +2
loud	-1.44 / +1.83	-1.43 / +1.81	0.99	1.00 / 1.00
f0_med	-1.15 / +1.51	-1.17 / +1.48	0.98	1.00 / 1.00
rate	-1.50 / +1.38	-1.51 / +1.36	0.99	1.00 / 1.00
h1h2	-1.77 / +1.57	-1.71 / +1.51	0.96	1.00 / 1.00
tilt	-1.51 / +0.97	-1.50 / +0.97	0.99	1.00 / 1.00
f0_iqr	-0.65 / +0.87	-0.62 / +0.85	0.95	0.65 / 0.79

Tag (readback)	fp32 hit	edge hit	Δ hit	fp32 Δz	edge Δz
whisper (voiced −)	1.00	1.00	+0.00	-9.47	-9.06
loud (loud +)	1.00	1.00	+0.00	+1.42	+1.35
fast (rate +)	0.50	0.44	-0.06	+0.50	+0.46
slow (rate −)	0.98	1.00	+0.02	-1.03	-1.07
high_pitch (f0_med +)	1.00	1.00	+0.00	+2.26	+2.32
anger (loud +)	0.12	0.23	+0.10	+0.24	+0.26
sadness (loud −)	0.79	0.73	-0.06	-0.78	-0.75

• Every control keeps at least 95 % of the fp32 effect (worst: f0_iqr at −2, 0.96×).
• Tags move the same way as in fp32, slightly less for two of them. fast (Δz +0.46 vs +0.50) and sadness (−0.75 vs −0.78) each lose 3 of 48 hits at the 0.5 SD threshold (0.50 → 0.44 and 0.79 → 0.73).
• The pitch-range control (f0_iqr) and the fast/anger tags are weak in fp32 too.

Speed

Runtime	CPU threads	RTF (20 sentences)	RTFx	Time to first audio (short sentence)	First streamed chunk	Load	Peak RSS
C library (steertts binary, no dependencies)	1	0.083	12.0×	95 ms	105 ms	0.03 s	27 MB
native C++ + ONNX Runtime (steertts-edge)	1	0.147	6.8×	412 ms	241 ms	1.55 s	96 MB
Python + ONNX Runtime (OnnxTTS)	1	0.147	6.8×	275 ms	236 ms	1.64 s	129 MB
Python + PyTorch (EdgeTTS)	1	0.156	6.4×	328 ms	258 ms	3.37 s	361 MB
C library (steertts binary, no dependencies)	4	0.063	15.8×	113 ms	100 ms	0.04 s	27 MB
native C++ + ONNX Runtime (steertts-edge)	4	0.137	7.3×	283 ms	312 ms	0.84 s	95 MB
Python + ONNX Runtime (OnnxTTS)	4	0.119	8.4×	213 ms	200 ms	1.15 s	128 MB
Python + PyTorch (EdgeTTS)	4	0.144	6.9×	354 ms	349 ms	3.17 s	368 MB

Measured with tools/bench_speed.py --runtime c native onnx torch on this package, the four runtimes back to back: CPU only (AMD Ryzen Threadripper 2970WX, a 2018 desktop part at 3.0 GHz) on a shared machine with a load average of about 30 from other jobs, so treat these as lower bounds; on a quiet machine the C library measured RTF 0.061 with one thread and 0.028 with four. RTF is synthesis time ÷ audio duration over 20 sentences (65.6 s of audio); time to first audio is a full call on a 4-word sentence ("Hello, how are you?"), warm, and the first streamed chunk is that sentence as the first segment of a two-sentence stream. Peak RSS is the whole process: PyTorch alone is about 200 MB before the model loads and unpacks the int4 weights to fp32 (59 MB); ONNX Runtime keeps them packed (MatMulNBits) but brings its own 25 MB library and allocator; the C library keeps the packed weights memory-mapped and dequantizes them panel by panel inside its matrix multiply. Streaming a 53 s paragraph sentence by sentence peaks at 107 MB (native) / 136 MB (Python + ONNX Runtime). Rendering a long paragraph in one call costs more, since attention memory grows with the square of the segment length; use stream().

Architecture and quantization

The acoustic model is SteerTTS v1 (FastPitch-style: phonemes → encoder → per-phone duration, pitch and energy → length regulator → decoder → 100-band log-mel). Every block is attention followed by a bidirectional Mamba-3 branch, and every LayerNorm is adaptive, conditioned on [voice slot, tag, controls ⊙ mask, mask]. See the full model card for details.

• Acoustic model. Quantization-aware training from the released v1 checkpoint (distillation + a control-delta loss), then packed: 6.49M weights int4 (groups of 32 along the input dimension, one fp16 scale per group), 2.68M int8 (embeddings, the conditioning MLP, AdaLN projections, the variance heads, Mamba-3 step-size projections; one fp16 scale per row), 85k fp16. The built-in voices are stored as their 64-d conditioning slots.
• Vocoder. A Vocos-style ConvNeXt + iSTFT vocoder (dim 384, intermediate 896, 8 layers) distilled from charactr/vocos-mel-24khz, then quantization-aware trained: 5.51M weights int4, 0.68M int8, the rest fp16.
• Compute. PyTorch runtime: weights are dequantized to fp32 at load (steertts_edge/packed.py has the format). ONNX graphs (onnx/): every Linear, Conv1d (as im2col) and AdaLN projection is an ONNX Runtime MatMulNBits node holding the same int4 / int8 codes and scales as the safetensors (nothing re-quantized), unpacked block by block inside the matmul; compute is fp32. tools/export_onnx.py + tools/pack_onnx.py rebuild them. The Mamba-3 scan is exported as plain batched matmuls, and sequences are padded to the 64-step scan chunk with masks that make padding inert, so the graphs take any length.
• Front end. The same symbols and rules as the full model's front end (g2p_en semantics, text normalisation), re-implemented on compact assets: CMUdict entries that the G2P model reproduces are stored without phones, the G2P GRU is int8, and heteronyms are resolved by a small context classifier instead of the NLTK tagger.

Limitations

• Audio quality is a step below the full model. UTMOS −0.09 / −0.10 (Seed / CV3) vs fp32 without the refiner, and further below the full model with its flow refiner (UTMOS 4.27 / 4.24). The cause is the small int4 vocoder.
• English only. ARPAbet front end, one speaker per request, utterance-level controls.
• Unknown words go through a small G2P model. Unusual names and foreign words can be mispronounced. Non-English letters are reduced to ASCII (café → cafe), and text in other scripts is dropped.
• Text normalisation has the full model's blind spots. The edge front end reproduces the v1 front end (identical phonemes on 99.6 % of Seed-TTS and 98.2 % of CV3-Eval sentences; phoneme error rate vs v1 below 0.1 %), including its mistakes: Dr. is read "drive", a phone number like 555-0198 as a range ("555 to 0198"), and clock times come out garbled (5:30 becomes one made-up word). Spell such text out when it matters.
• No cloning or free-text prompts on the device, and no flow refiner (see the top of this card). Passing a host-computed spk_vec works mechanically but was not evaluated.
• Emotion tags are style switches, not emotions (as in the full model). Voice-quality tags such as whisper act strongly; most emotion tags are acoustically close to neutral.
• The controls are acoustic measurements, not percepts. voiced, flat, cpps and pause barely move in the full model either.
• Test-set-informed choices. The vocoder was chosen between two candidates of the same architecture by their UTMOS on the leaderboard sets (Seed-TTS and CV3-Eval), so its numbers here are slightly optimistic. The leaderboard rows use voice_1361, the full model's pre-registered v1.0.0 default.
• No human evaluation. Every number here is from automatic metrics.

Intended use and misuse

Intended uses:

• Offline, on-device speech output where size matters: apps, embedded and edge devices, accessibility tools.
• Research on controllable, interpretable and compressed speech synthesis.

Do not use this model to impersonate real people, produce deceptive or fraudulent audio, bypass voice authentication or create non-consensual content. Disclose synthetic speech to listeners. There is no watermark.

Built-in voices are anonymised. They are training speakers under the full model's opaque public names. No speaker or clip identifiers from the source corpora are included.

Training data

The acoustic model and voices are distilled from SteerTTS v1, and the vocoder from charactr/vocos-mel-24khz. The full model's training data and terms apply: see Training data on its card.

Licence: CC BY-NC 4.0. Non-commercial use only.

Files

File	Contents
acoustic.safetensors	Acoustic model, packed int4 / int8 / fp16 (layout in the safetensors metadata)
vocoder.safetensors	Vocoder, packed int4 / int8 / fp16
voices.safetensors	Per voice: 64-d conditioning slot and neutral control means
voices.json	Voice names and measured descriptors (gender probe, pitch register, pace, median F0)
config.json	Tags, control scales, architecture, file map
front/	Text front end assets: residual lexicon, int8 G2P, heteronym and normaliser tables
onnx/	enc.onnx (encoder + variance predictors), dec.onnx (decoder), voc.onnx (vocoder network), int4 / int8 MatMulNBits; spkproj.safetensors (512-d speaker vector → voice slot, for host-computed clone vectors)
native/front/	The same front end assets re-encoded for the C++ runtime (no xz / npz / JSON parsing on device)
steertts_edge/	The runtime package (pip install .): OnnxTTS (ort.py), EdgeTTS (pipeline.py)
tests/	pytest suite
samples/*.wav	The clips in Listen (not part of the device payload)
tools/	sync_package.py (copy a built package in), export_onnx.py + pack_onnx.py (+ ssd_mm.py): build onnx/, bench_speed.py, render_tables.py, render_samples.py, release_numbers.json
TEST_LOG.md	Release verification log
LICENSE.md, NOTICE	Code MIT, weights CC BY-NC 4.0; third-party notices

Release verification

Verified in a fresh Python 3.11 venv with the CPU torch wheel, the package installed with pip and tested from outside the source tree; TEST_LOG.md has the full log.

• Tests. The pytest suite passes: all 200 voices, all 50 tags, each control at −2 / 0 / +2, markup, streaming, text edge cases, error messages, determinism, the CLI, and no NLTK / g2p_en / Vocos / torchaudio / transformers import at runtime.
• Parity. Renders are bit-identical to the evaluated build, including across torch 2.11 and 2.14.
• 1.1.0 runtimes. The suite runs every functional test on both Python runtimes (114 passed). ONNX vs PyTorch: identical lengths, mel max difference 3·10⁻⁵, waveforms at 52–90 dB SNR. Native C++ vs Python ONNX: 13 requests at ~140 dB SNR; the C++ front end is exact on its golden corpus; an aarch64 build matches under emulated Cortex-A72 and ARMv8.2 CPUs.
• Steering gates vs fp32. Every control keeps ≥ 0.9× its fp32 effect (worst 0.96×). Tag hit rates stay within 0.05 of fp32 except fast and sadness (−0.06 each, one render past the gate; see How well it steers).

Changelog

• 1.1.1 (2026-10-23): no file changes. The new stand-alone C library runs this package as is (27 MB peak RAM, fastest of the four runtimes); card updated with it.
• 1.1.0 (2026-10-22): ONNX Runtime runtime (OnnxTTS, now the default; 127 MB peak RAM instead of 369 MB) and the files for the native C/C++ runtime on GitHub (onnx/, native/front/). Same weights, same audio: ONNX vs PyTorch renders have identical lengths and 52–90 dB SNR. pip install . no longer pulls torch (use [torch]). CLI --runtime. steertts_edge imports neither runtime until one is used.
• 1.0.0 (2026-10-22): first release.

Citation

@misc{steertts2026,
  title  = {SteerTTS: A Tiny, Measurably Steerable Speech Synthesizer},
  author = {{Whissle AI}},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/WhissleAI/steer-tts}}
}