README.md

---
language: pl
license: cc-by-sa-4.0
library_name: onnx
pipeline_tag: text-to-speech
base_model: machinekind/Matcha-TTS-PL
tags: [tts, text-to-speech, polish, matcha-tts, flow-matching, hifigan, onnx, onnxruntime, cpu, edge, raspberry-pi, jetson, int8, quantized, streaming, low-latency, voxcpm2]
datasets: [machinekind/polish-synthetic-speech-voxcpm2]
---

# Matcha-TTS-PL-Edge — Polish TTS for CPUs: fp32 · int8 · streaming

> **Newer version:** [Matcha-TTS-PL-Edge-v2](https://huggingface.co/machinekind/Matcha-TTS-PL-Edge-v2).

The voices of [Matcha-TTS-PL](https://huggingface.co/machinekind/Matcha-TTS-PL) for Raspberry Pi / Jetson-class CPUs, in one
repo: **fp32 and pre-quantised int8** ONNX graphs, a **15× smaller vocoder**, and **streaming** (first audio in ~80 ms on one
Pi-class core). One inference file (`tts.py`, onnxruntime + numpy) does whole sentences and streaming.

Voices: `male_voxcpm_2`, `female_voxcpm_2` (synthetic, designed with VoxCPM2 from a text description — not recordings or clones
of real people). Modern Polish with English words read in English ("Po meetingu wyślę ci feedback").

**Family:** [Matcha-TTS-PL](https://huggingface.co/machinekind/Matcha-TTS-PL) (4 voices, large vocoder, best quality on a
GPU / desktop) · **Matcha-TTS-PL-Edge** (this repo) · [Matcha-TTS-PL-Pretrain](https://huggingface.co/machinekind/Matcha-TTS-PL-Pretrain)
(base for fine-tuning) · [dataset](https://huggingface.co/datasets/machinekind/polish-synthetic-speech-voxcpm2) ·
[demo Space](https://huggingface.co/spaces/machinekind/matcha-tts-pl) (runs these graphs in the browser, streaming included).

## Pick your files

| you want | precision · steps · mode | files of a voice (`<voice>/`) |
|---|---|---|
| a robot / Raspberry Pi 5 / Jetson: lowest latency | **int8 · 2 · streamed** (the default) | `encoder_int8.onnx`, `decoder_t2_int8.onnx`, `vocoder_fp16.onnx`, `voices.json` |
| a desktop / server CPU: best quality | fp32 · 4 · whole or streamed | `encoder.onnx`, `decoder_t4.onnx`, `vocoder.onnx`, `voices.json` |
| fine-tuning, PyTorch | — | `pytorch/` (Matcha checkpoint + HiFi-GAN V2 generator) |

Plus, at the root: `tts.py`, `frontend/`, `requirements.txt`. Download only what you need:

```bash
hf download machinekind/Matcha-TTS-PL-Edge --include "tts.py" "requirements.txt" "frontend/*" \
    "male_voxcpm_2/voices.json" "male_voxcpm_2/*_int8.onnx" "male_voxcpm_2/vocoder_fp16.onnx" --local-dir Matcha-TTS-PL-Edge
```

## Quick start

```bash
pip install -r requirements.txt        # onnxruntime, numpy, phonemizer, wordfreq (+ sounddevice for --play)
brew install espeak-ng                 # or: sudo apt install espeak-ng   (the phonemizer's backend)
python tts.py "Dzień dobry! W czym mogę dziś pomóc?" -o out.wav          # int8, 2 steps, streamed
python tts.py "Dzień dobry! W czym mogę dziś pomóc?" --play              # hear it while it is generated
python tts.py "…" --precision fp32 --steps 4 --no-stream -o best.wav     # best quality
```

```python
from tts import EdgeTTS
tts = EdgeTTS(voice="male_voxcpm_2", precision="int8", steps=2)
tts.warmup()                                   # once at start-up: loads the phonemizer and runs every graph
for audio in tts.stream("Dzień dobry! Mam dla ciebie kilka informacji na dziś."):
    play(audio)                                # float32 pieces, 22 050 Hz mono
wav = tts.synthesize("Dzień dobry!", stream=False)
```

## Streaming

Each sentence goes through the text encoder whole (so intonation is planned for the sentence); the flow-matching decoder then
runs on chunks of the mel, and the vocoder turns each piece into audio as soon as it is ready. Text is split into sentences and
the next one is prepared while the previous one plays. The defaults below (also in each `voices.json` under `"stream"`) are the
sweet spot of a sweep over 210 settings: smaller chunks starve playback on Pi-class CPUs, larger ones only add latency.

| parameter (`tts.py` flag) | default | suggested | what it does |
|---|---|---|---|
| `chunk` (`--chunk`) | 43 frames (0.5 s) | 43–86 | mel frames per chunk (11.6 ms each) |
| `ode_ctx` (`--ode-ctx`) | 16 | 8–32 | decoder context frames on each side of a chunk; below 8 the seams are audible |
| `voc_ctx` (`--voc-ctx`) | 12 | 8–12 | vocoder context frames; 12 = identical to whole-sentence vocoding, 8 = ~7 % less compute |
| `min_sentence_chars` | 40 | — | a shorter sentence is merged with the next (very short inputs synthesise less stably) |
| `first_chunk` (`--first-chunk`) | = chunk | — | a smaller first chunk starts ~45 ms earlier but stutters on slow CPUs |

Time to first audio, **short** / **long** input:

- short: *Dzień dobry, w czym mogę dziś pomóc?* — 7 words, 9 syllables, 1.8 s of speech
- long: *Robot nie zawsze ma dostęp do stabilnego połączenia z internetem. Musi też dzielić moc obliczeniową między wiele zadań.
  Dlatego potrzebuje modelu, który zapewnia dobrą jakość mowy bez nadmiernego obciążania urządzenia.* — 3 sentences, 30 words,
  72 syllables, 11.5 s of speech, given as one input (no sentence split: with the default split, first audio depends only on the
  first sentence, so it is lower)


| setting | Cortex-A725, 1 thread | Cortex-A725, 4 threads | Cortex-X925, 4 threads |
|---|---|---|---|
| int8 · 2 steps · streamed | 84 / 174 ms | 53 / 86 ms | 30 / 48 ms |
| int8 · 2 steps · whole sentence | 230 / 1518 ms | 96 / 554 ms | 58 / 314 ms |
| int8 · 4 steps · streamed | 104 / 196 ms | 66 / 99 ms | 38 / 56 ms |
| int8 · 4 steps · whole sentence | 273 / 1791 ms | 122 / 656 ms | 75 / 361 ms |
| fp32 · 2 steps · streamed | 151 / 418 ms | 54 / 132 ms | 27 / 58 ms |
| fp32 · 2 steps · whole sentence | 357 / 2352 ms | 115 / 694 ms | 54 / 278 ms |
| fp32 · 4 steps · streamed | 194 / 464 ms | 72 / 150 ms | 38 / 67 ms |
| fp32 · 4 steps · whole sentence | 460 / 3034 ms | 153 / 913 ms | 70 / 367 ms |

Compute per second of audio (real-time factor) of the long input, streamed / whole:

| setting | Cortex-A725, 1 thread | Cortex-A725, 4 threads | Cortex-X925, 4 threads |
|---|---|---|---|
| int8 · 2 steps | 0.20 / 0.13 | 0.09 / 0.05 | 0.06 / 0.03 |
| int8 · 4 steps | 0.25 / 0.16 | 0.12 / 0.06 | 0.08 / 0.03 |
| fp32 · 2 steps | 0.31 / 0.20 | 0.11 / 0.06 | 0.05 / 0.02 |
| fp32 · 4 steps | 0.42 / 0.26 | 0.15 / 0.08 | 0.07 / 0.03 |

Wall clock of `tts.py` on an NVIDIA DGX Spark (GB10) CPU pinned to one core type, ONNX Runtime 1.30, `male_voxcpm_2` (the voices
share the architecture), after `warmup()`; text front end excluded (< 1 ms per sentence). The Cortex-A725 is the closest
available stand-in for a Raspberry Pi 5 (Cortex-A76) or a Jetson Orin CPU (Cortex-A78AE) — expect those to be roughly 1.5–3×
slower. Streaming computes the context frames twice, so its total is higher; what it buys is the first audio.

## Quality

UTMOS ↑ / Whisper WER ↓ / speaker similarity ↑ on 12 held-out sentences (temperature 0.8, tempo 0.8, neutral style):

| setting | `male_voxcpm_2` | `female_voxcpm_2` |
|---|---|---|
| int8 · 2 steps · streamed (default) | 3.46 / 3.4 % / 0.864 | 3.45 / 2.7 % / 0.910 |
| int8 · 2 steps · whole sentence | 3.42 / 2.7 % / 0.866 | 3.45 / 2.7 % / 0.911 |
| fp32 · 2 steps · streamed | 3.49 / 2.7 % / 0.860 | 3.48 / 2.7 % / 0.918 |
| fp32 · 2 steps · whole sentence | 3.52 / 4.0 % / 0.860 | 3.53 / 2.7 % / 0.915 |
| fp32 · 4 steps · whole sentence | 3.58 / 2.7 % / 0.862 | 3.47 / 3.4 % / 0.917 |
| *Matcha-TTS-PL as released (large HiFi-GAN, 4 steps)* | 3.42 / 2.7 % / 0.853 | 3.27 / 3.4 % / 0.910 |

UTMOS and similarity move by ±0.1 between runs (the model samples noise); streamed and whole-sentence synthesis are equal within that.

## What is different from Matcha-TTS-PL

1. **Small vocoder.** HiFi-GAN V2 (0.93 M parameters instead of 13.9 M; ~1.7 instead of ~26.5 GMAC per second of audio), the
   generator config of the official [jik876/hifi-gan](https://github.com/jik876/hifi-gan) V2, started from its VCTK checkpoint
   and fine-tuned 70k steps per voice on the voice's recordings paired with mels predicted by the acoustic model.
2. **Mel level fixed.** Matcha-TTS-PL's checkpoints decode their mels with normalisation statistics inherited from a warm start
   (Matcha's VCTK model), ~1.7 log units (~15 dB) too quiet, which each voice's own vocoder there compensates. Here the
   statistics are the training data's (`mel_mean` −4.8672, `mel_std` 2.5093), so the mel has the level of real speech.
3. **Graphs split for streaming**: text encoder + durations, decoder (any length: the whole mel or a chunk), vocoder. Run on the
   whole mel they give the same output as one graph, at the same speed.
4. **int8**: static QDQ quantisation (per-channel weights, activations calibrated on 40 sentences of the voice) of the encoder
   and decoder; the vocoder is fp16 in the int8 setting (a fully int8 vocoder was ~0.3 UTMOS worse).
5. **Faster graphs**: Mish computed with exponentials instead of ONNX Runtime's slow `Softplus` kernel; in the int8 decoders every
   ODE step reads the same per-channel int8 weights (15–18 % faster in fp32, 40–47 % in int8, same output up to rounding).

## Files

- `<voice>/encoder.onnx`, `encoder_int8.onnx` — `x` [1,T] int64, `x_lengths`, `scales` [temperature, length scale], `spk_emb` →
  `mu` [1,80,F] (normalised mel prior), `mel_lengths`
- `<voice>/decoder_t2.onnx`, `decoder_t4.onnx` (+ `_int8`) — `mu`, `mask` [1,1,L], `z` (noise) [1,80,L], `spk_emb`, `temperature` →
  `mel` [1,80,L]; L a multiple of 4, padded frames masked; 2 or 4 ODE steps
- `<voice>/vocoder.onnx`, `vocoder_fp16.onnx` — `mel` [1,80,F] → 22.05 kHz waveform
- `<voice>/voices.json` — speaker vector, style vectors (8 = neutral), defaults, graph names, streaming defaults; `voices_index.json`
- `<voice>/pytorch/` — Matcha checkpoint (mel statistics fixed) and HiFi-GAN V2 generator + config (for the Matcha-TTS code)
- `<voice>/samples/eNN.wav` — the 12 evaluation sentences, default setting (`texts.txt`)
- `tts.py` — inference (whole / streamed, fp32 / int8, CLI and `EdgeTTS` class); `frontend/` — text → phoneme ids: Polish espeak-ng
  phonemisation with English words (curated list + rules, `english_detect.py`) read by espeak-ng en-us; you can also mark English
  words yourself: `"Wrzuć to na {Slack}a."`

## Licence and attribution

CC BY-SA 4.0 (`LICENSE`), like Matcha-TTS-PL and its training data. The vocoder starts from the HiFi-GAN V2 VCTK checkpoint by
Jungil Kong et al. (code and weights MIT), trained on the VCTK corpus (CC BY 4.0, University of Edinburgh, CSTR).
`frontend/text.py` contains the text cleaners and symbol table of Matcha-TTS (MIT, Shivam Mehta et al.) with our Polish additions.
espeak-ng (the runtime phonemiser, installed separately) is GPL-3.0. Full credits, including every source text of the training
data: `ATTRIBUTION.md`.