README.md

---
language:
  - yue
  - zh
  - nl
  - en
  - fr
  - de
  - it
  - ja
  - ko
  - pl
  - es
license: apache-2.0
base_model:
  - Audio8/Audio8-TTS-Preview-0.6b
tags:
  - text-to-speech
  - tts
  - multilingual
  - onnx
  - phoonnx
  - arktts
  - voice-cloning
pipeline_tag: text-to-speech
library_name: phoonnx
---

# phoonnx-audio8-tts

ONNX export of **[`Audio8/Audio8-TTS-Preview-0.6b`](https://huggingface.co/Audio8/Audio8-TTS-Preview-0.6b)**
— an 11-language text-to-speech model — packaged for the
[phoonnx](https://github.com/OpenVoiceOS/phoonnx) `arktts` engine.

Upstream ships PyTorch only. These graphs were exported with
[`scripts/conversion/arktts/export_arktts_onnx.py`](https://github.com/OpenVoiceOS/phoonnx/tree/dev/scripts/conversion/arktts),
using the same contract and the same tensor names as the official export of the model's
Basque fine-tune, [`itzune/zortzi-tts-onnx`](https://huggingface.co/itzune/zortzi-tts-onnx),
so graphs from either source are interchangeable.

Languages: Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean,
Polish, Spanish.

## Usage

```python
from phoonnx.model_manager import TTSModelManager
from phoonnx.voice import TTSVoice

voice = TTSVoice.load(TTSModelManager().get_voice("arktts/audio8-maider/en"))
audio = voice.synthesize("The quick brown fox jumps over the lazy dog.")
```

Voice ids are `arktts/audio8-{maider,antton}/{yue,zh,nl,en,fr,de,it,ja,ko,pl,es}`.

## Files

```
slow_ar_fp16.onnx          24-layer backbone, KV-cached          (1.35 GB)
fast_ar_fp16.onnx          4-layer depth transformer              (134 MB)
codec_decoder_fp16.onnx    10 codebooks -> 44.1 kHz waveform      (278 MB)
tokenizer.json             the model's own Qwen2 subword BPE
voices/maider.json         reference codes + transcription
voices/antton.json         reference codes + transcription
runtime_manifest.json      precision and architecture metadata
```

## Voices, and where they come from

Upstream ships **no reference voices** — Audio8 is a cloning model, and a voice is the codec
codes of a short clip rather than an entry in a speaker table. To make it usable as a
preset-voice engine, this mirror carries two voices encoded from the reference clips
published in [`itzune/zortzi-tts`](https://huggingface.co/itzune/zortzi-tts):
`voices/maider.wav` and `voices/antton.wav`.

Those clips are from the **HiTZ-Aholab Basque TTS dataset** (CC BY 4.0). They condition
**timbre, not language** — the model infers the language from the text — so both voices are
offered for all eleven languages, and the WER table below is what they actually sound like
in each. The clips are Basque, which is not one of Audio8's languages; that is fine for
conditioning and is stated here so nobody has to work it out.

The two checkpoints carry a byte-identical `codec.pth`, so re-encoding those clips with
Audio8's own codec reproduces the codes upstream published for Zortzi: codebook 0 matches
exactly, and all ten codebooks agree on 99.2 to 99.6 % of frames. The published Zortzi codes
are shipped here for that reason — they are the reproducible artifact.

To add your own voice, encode a clip offline with
`scripts/conversion/arktts/mint_voice.py`. This mirror ships the codec **decoder** only;
cloning at synthesis time would need the encoder, which is not exported.

## What was verified

Checked against the PyTorch checkpoint (float32) with
`scripts/conversion/arktts/verify_parity.py`, over 24 lockstep greedy decode steps:

| Tensor | max abs diff | greedy agreement | note |
|---|---|---|---|
| prompt `[1, 11, T]` | — | exact | identical to upstream's own processor output |
| slow AR logits | 0.118 | 24/24 | |
| slow AR hidden | 0.039 | — | |
| fast AR logits | 1.36 | 208/216 | every miss is a tie; worst margin 0.225 |
| codec decoder | 6.3e-4 | — | correlation 0.999999 |

The fast AR is noisier here than in the official Zortzi export because the half-precision
graph accumulates its RMS norms in half precision rather than single — the explicit
float32 round-trip upstream writes cannot survive the fp16 converter. It changes no greedy
decision that was not already a tie.

Intelligibility, one sentence per language per voice, transcribed with
`onnx-community/whisper-large-v3-turbo` on CPU:

| Language | WER | Note |
|---|---|---|
| en | 0.000 | |
| de | 0.000 | |
| es | 0.000 | |
| it | 0.000 | |
| nl | 0.125 | one clip, one recogniser slip |
| pl | 0.143 | the recogniser wrote "10" for "dziesięć" |
| fr | 0.333 | the recogniser heard a plural subject in both clips |

Cantonese, Chinese, Japanese and Korean are **not** covered by this run. Whisper is the only
model in `onnx-asr` that claims `yue` at all, and a Whisper-only Cantonese number would say
more about the recogniser than about the voice; when those are measured they should be
scored by character (CER), not by word.

CPU real-time factor on twelve cores: **13.1x** (Maider), **14.3x** (Antton) — slower than
the official Zortzi graphs at 8x, because this export carries the tied embedding matrix
twice and is not otherwise optimised. This is not a real-time model on CPU.

## Licence and attribution

Apache-2.0, from the base model. The model is by **Audio8**; this repository only exports
and repackages it.

The reference clips that carry the two voices are from the **HiTZ-Aholab Basque TTS
dataset** (CC BY 4.0, [10.5281/zenodo.17952596](https://doi.org/10.5281/zenodo.17952596)),
by way of [`itzune/zortzi-tts`](https://huggingface.co/itzune/zortzi-tts). That licence
conditions the voices, so redistributing them *or audio generated with them* carries the
attribution below.

> The HiTZ-Aholab speech synthesis dataset was developed with funding from the Ministerio
> para la Transformación Digital y de la Función Pública and Plan de Recuperación,
> Transformación y Resiliencia — Funded by EU — NextGenerationEU within the framework of the
> project ILENIA (ref. 2022/TL22/00215335), and by a grant from the Department of Culture
> and Language Policy of the Basque Government (IKER-GAITU project).

```bibtex
@dataset{navas_hernaez_2025_17952596,
    author    = {Navas, Eva and Hernaez Rioja, Inmaculada and Saratxaga, Ibon and
                 Sanchez, Jon and García Romillo, Víctor and Flores Ríos, Mariana and
                 Bellanco, Aitor},
    title     = {{HiTZ-Aholab speech synthesis dataset in Basque}},
    month     = dec, year = 2025, publisher = {Zenodo}, version = {1.0},
    doi       = {10.5281/zenodo.17952596},
    url       = {https://doi.org/10.5281/zenodo.17952596}
}
```

## Known limitations

Upstream's, carried over unchanged:

- **Numbers are mispronounced** — spell them out in the text.
- **No text normalization** — expand acronyms yourself ("TTS" as "te te ese").
- **Sampling is required** — greedy decoding never reaches end-of-speech. The engine's
  defaults (temperature 0.8, top-p 0.95) are upstream's.

Specific to this export:

- **Half precision only.** The codec decoder is the fp16 graph from
  `itzune/zortzi-tts-onnx`, which is valid here because both checkpoints carry the same
  `codec.pth`; it is verified above against *this* model's PyTorch codec.
- **No codec encoder**, so cloning is an offline step.
- **`slow_ar_fp16.onnx` is about 270 MB larger than it needs to be** — the tracer
  materialises the tied output projection separately from the embedding table.