| --- |
| language: |
| - yue |
| - zh |
| - nl |
| - en |
| - fr |
| - de |
| - it |
| - ja |
| - ko |
| - pl |
| - es |
| license: apache-2.0 |
| base_model: |
| - Audio8/Audio8-TTS-Preview-0.6b |
| tags: |
| - text-to-speech |
| - tts |
| - multilingual |
| - onnx |
| - phoonnx |
| - arktts |
| - voice-cloning |
| pipeline_tag: text-to-speech |
| library_name: phoonnx |
| --- |
| |
| # phoonnx-audio8-tts |
|
|
| ONNX export of **[`Audio8/Audio8-TTS-Preview-0.6b`](https://huggingface.co/Audio8/Audio8-TTS-Preview-0.6b)** |
| — an 11-language text-to-speech model — packaged for the |
| [phoonnx](https://github.com/OpenVoiceOS/phoonnx) `arktts` engine. |
|
|
| Upstream ships PyTorch only. These graphs were exported with |
| [`scripts/conversion/arktts/export_arktts_onnx.py`](https://github.com/OpenVoiceOS/phoonnx/tree/dev/scripts/conversion/arktts), |
| using the same contract and the same tensor names as the official export of the model's |
| Basque fine-tune, [`itzune/zortzi-tts-onnx`](https://huggingface.co/itzune/zortzi-tts-onnx), |
| so graphs from either source are interchangeable. |
|
|
| Languages: Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, |
| Polish, Spanish. |
|
|
| ## Usage |
|
|
| ```python |
| from phoonnx.model_manager import TTSModelManager |
| from phoonnx.voice import TTSVoice |
| |
| voice = TTSVoice.load(TTSModelManager().get_voice("arktts/audio8-maider/en")) |
| audio = voice.synthesize("The quick brown fox jumps over the lazy dog.") |
| ``` |
|
|
| Voice ids are `arktts/audio8-{maider,antton}/{yue,zh,nl,en,fr,de,it,ja,ko,pl,es}`. |
|
|
| ## Files |
|
|
| ``` |
| slow_ar_fp16.onnx 24-layer backbone, KV-cached (1.35 GB) |
| fast_ar_fp16.onnx 4-layer depth transformer (134 MB) |
| codec_decoder_fp16.onnx 10 codebooks -> 44.1 kHz waveform (278 MB) |
| tokenizer.json the model's own Qwen2 subword BPE |
| voices/maider.json reference codes + transcription |
| voices/antton.json reference codes + transcription |
| runtime_manifest.json precision and architecture metadata |
| ``` |
|
|
| ## Voices, and where they come from |
|
|
| Upstream ships **no reference voices** — Audio8 is a cloning model, and a voice is the codec |
| codes of a short clip rather than an entry in a speaker table. To make it usable as a |
| preset-voice engine, this mirror carries two voices encoded from the reference clips |
| published in [`itzune/zortzi-tts`](https://huggingface.co/itzune/zortzi-tts): |
| `voices/maider.wav` and `voices/antton.wav`. |
|
|
| Those clips are from the **HiTZ-Aholab Basque TTS dataset** (CC BY 4.0). They condition |
| **timbre, not language** — the model infers the language from the text — so both voices are |
| offered for all eleven languages, and the WER table below is what they actually sound like |
| in each. The clips are Basque, which is not one of Audio8's languages; that is fine for |
| conditioning and is stated here so nobody has to work it out. |
|
|
| The two checkpoints carry a byte-identical `codec.pth`, so re-encoding those clips with |
| Audio8's own codec reproduces the codes upstream published for Zortzi: codebook 0 matches |
| exactly, and all ten codebooks agree on 99.2 to 99.6 % of frames. The published Zortzi codes |
| are shipped here for that reason — they are the reproducible artifact. |
|
|
| To add your own voice, encode a clip offline with |
| `scripts/conversion/arktts/mint_voice.py`. This mirror ships the codec **decoder** only; |
| cloning at synthesis time would need the encoder, which is not exported. |
|
|
| ## What was verified |
|
|
| Checked against the PyTorch checkpoint (float32) with |
| `scripts/conversion/arktts/verify_parity.py`, over 24 lockstep greedy decode steps: |
|
|
| | Tensor | max abs diff | greedy agreement | note | |
| |---|---|---|---| |
| | prompt `[1, 11, T]` | — | exact | identical to upstream's own processor output | |
| | slow AR logits | 0.118 | 24/24 | | |
| | slow AR hidden | 0.039 | — | | |
| | fast AR logits | 1.36 | 208/216 | every miss is a tie; worst margin 0.225 | |
| | codec decoder | 6.3e-4 | — | correlation 0.999999 | |
|
|
| The fast AR is noisier here than in the official Zortzi export because the half-precision |
| graph accumulates its RMS norms in half precision rather than single — the explicit |
| float32 round-trip upstream writes cannot survive the fp16 converter. It changes no greedy |
| decision that was not already a tie. |
|
|
| Intelligibility, one sentence per language per voice, transcribed with |
| `onnx-community/whisper-large-v3-turbo` on CPU: |
|
|
| | Language | WER | Note | |
| |---|---|---| |
| | en | 0.000 | | |
| | de | 0.000 | | |
| | es | 0.000 | | |
| | it | 0.000 | | |
| | nl | 0.125 | one clip, one recogniser slip | |
| | pl | 0.143 | the recogniser wrote "10" for "dziesięć" | |
| | fr | 0.333 | the recogniser heard a plural subject in both clips | |
|
|
| Cantonese, Chinese, Japanese and Korean are **not** covered by this run. Whisper is the only |
| model in `onnx-asr` that claims `yue` at all, and a Whisper-only Cantonese number would say |
| more about the recogniser than about the voice; when those are measured they should be |
| scored by character (CER), not by word. |
|
|
| CPU real-time factor on twelve cores: **13.1x** (Maider), **14.3x** (Antton) — slower than |
| the official Zortzi graphs at 8x, because this export carries the tied embedding matrix |
| twice and is not otherwise optimised. This is not a real-time model on CPU. |
|
|
| ## Licence and attribution |
|
|
| Apache-2.0, from the base model. The model is by **Audio8**; this repository only exports |
| and repackages it. |
|
|
| The reference clips that carry the two voices are from the **HiTZ-Aholab Basque TTS |
| dataset** (CC BY 4.0, [10.5281/zenodo.17952596](https://doi.org/10.5281/zenodo.17952596)), |
| by way of [`itzune/zortzi-tts`](https://huggingface.co/itzune/zortzi-tts). That licence |
| conditions the voices, so redistributing them *or audio generated with them* carries the |
| attribution below. |
|
|
| > The HiTZ-Aholab speech synthesis dataset was developed with funding from the Ministerio |
| > para la Transformación Digital y de la Función Pública and Plan de Recuperación, |
| > Transformación y Resiliencia — Funded by EU — NextGenerationEU within the framework of the |
| > project ILENIA (ref. 2022/TL22/00215335), and by a grant from the Department of Culture |
| > and Language Policy of the Basque Government (IKER-GAITU project). |
|
|
| ```bibtex |
| @dataset{navas_hernaez_2025_17952596, |
| author = {Navas, Eva and Hernaez Rioja, Inmaculada and Saratxaga, Ibon and |
| Sanchez, Jon and García Romillo, Víctor and Flores Ríos, Mariana and |
| Bellanco, Aitor}, |
| title = {{HiTZ-Aholab speech synthesis dataset in Basque}}, |
| month = dec, year = 2025, publisher = {Zenodo}, version = {1.0}, |
| doi = {10.5281/zenodo.17952596}, |
| url = {https://doi.org/10.5281/zenodo.17952596} |
| } |
| ``` |
|
|
| ## Known limitations |
|
|
| Upstream's, carried over unchanged: |
|
|
| - **Numbers are mispronounced** — spell them out in the text. |
| - **No text normalization** — expand acronyms yourself ("TTS" as "te te ese"). |
| - **Sampling is required** — greedy decoding never reaches end-of-speech. The engine's |
| defaults (temperature 0.8, top-p 0.95) are upstream's. |
|
|
| Specific to this export: |
|
|
| - **Half precision only.** The codec decoder is the fp16 graph from |
| `itzune/zortzi-tts-onnx`, which is valid here because both checkpoints carry the same |
| `codec.pth`; it is verified above against *this* model's PyTorch codec. |
| - **No codec encoder**, so cloning is an offline step. |
| - **`slow_ar_fp16.onnx` is about 270 MB larger than it needs to be** — the tracer |
| materialises the tied output projection separately from the embedding table. |
|
|
