README.md

---
license: mit
language:
  - vi
library_name: onnx
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - tts
  - vietnamese
  - onnx
  - onnxruntime
  - zero-shot
  - speech-synthesis
  - voice-cloning
  - vietnamese-tts
  - tieng-viet
# NOTE deliberately NO `datasets:` field. It is the only thing that populates
# the Hub's cross-link, but the Hub renders it as "Models trained or fine-tuned
# on <dataset>" — which for our own held-out benchmark reads as train/test
# contamination. ZeroBench-TTS is EVALUATION data; every voice in it is held
# out of training. The `model-index` block below states that correctly, and the
# body links the benchmark in prose.
metrics:
  - wer
model-index:
  - name: ZeroTTS
    results:
      - task:
          type: text-to-speech
          name: Zero-Shot Text-to-Speech
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS
          split: test
        metrics:
          - type: wer
            value: 0.56
            name: WER (%) — normalized text
          - type: utmos
            value: 2.91
            name: UTMOSv2 naturalness MOS
          - type: speaker_similarity
            value: 0.936
            name: Speaker similarity (WavLM-SV cosine)
          - type: excess_silence
            value: 0.029
            name: Excess silence (s)
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — monolingual Vietnamese
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (vietnamese)
          config: vietnamese
          split: test
        metrics:
          - type: wer
            value: 0.21
            name: WER (%) — normalized text
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — Vietnamese/English code-switching
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (code_switch)
          config: code_switch
          split: test
        metrics:
          - type: wer
            value: 0.95
            name: WER (%) — normalized text
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — cross-lingual voice prompt
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (cross_lingual)
          config: cross_lingual
          split: test
        metrics:
          - type: wer
            value: 0.38
            name: WER (%) — normalized text
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — acronyms, dates, numbers
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (challenging)
          config: challenging
          split: test
        metrics:
          - type: wer
            value: 0.61
            name: WER (%) — normalized text
---

<img src="banner.png" alt="ZeroTTS — Vietnamese zero-shot text-to-speech" width="100%">

# ZeroTTS

### Ultra-natural Vietnamese speech, cloned from seconds of audio — streaming, real-time on a CPU

**The most accurate open Vietnamese TTS we know of — 13× fewer word errors than
the next open model**, and it runs faster than real time on a laptop CPU.

* 🎯 **Ultra-natural** — 2.91 UTMOS, ~0.5 MOS above every other open Vietnamese
  system, with near-zero dead air (0.029 s vs 0.23–0.53 s).
* 🗣️ **Zero-shot voice cloning** — a voice is a small latent array; drop it in
  and the model speaks in it. No fine-tuning, no per-speaker training.
* ⚡ **Real-time on CPU, streaming** — first audio chunk in ~100 ms, then chunks
  ramp up. No GPU required.
* 🇻🇳 **Built for Vietnamese** — tones, code-switched English, and a built-in
  normalizer that reads `31/12/2025` and `1.250 tỷ` the way a person would.
* 📊 **Measured, not asserted** — every number below comes from
  [ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)'s
  own public scorer, on 59 held-out voices.

* Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS**
* Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS**

```python
pip install zerotts
```

```python
from zerotts import ZeroTTS

tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS")
audio = tts.synthesize("Xin chào các bạn, mình là ZeroTTS.", voice="arya")
tts.save_audio(audio, "out.wav")
```

Streaming, with first audio in roughly 100 ms:

```python
for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voice="arya"):
    play(chunk)   # (1, n) float32 at 48 kHz
```

## Benchmarks

Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)** —
137 items, 59 held-out reference voices × 4 subsets — against the two public
Vietnamese XTTS-v2 finetunes. 137/137 scored, 0 empty generations.

**Scored by the benchmark, not by us.** ZeroTTS synthesizes the clips and hands
them to `zerobench_eval`, the official scorer published inside the benchmark
dataset repo. Nothing in this repo computes a metric.

### Headline

Every system reads **normalized text** — dates, numbers and acronyms already
spoken out, from the benchmark's own curated reading. Every system gets exactly
the same input, so the comparison is like-for-like.

This is the condition a Vietnamese TTS system meets in production, where a text
frontend runs ahead of the model. ZeroTTS ships one — `normalize_vi_text`,
applied by default (see the [GitHub README](https://github.com/zeroweight-ai/ZeroTTS#usage)) — which reaches the benchmark's reading
on 27 of the 35 items that need normalization; the remaining gaps are date
separators and alphanumeric codes. Neither baseline ships a Vietnamese frontend
at all, which is why the raw-text table below is so much harsher on them.

| | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
|---|:-:|:-:|:-:|
| **WER** ↓ | **0.56 %** | 7.27 % | 8.61 % |
| **Naturalness** (UTMOS) ↑ | **2.91** | 2.43 | 2.35 |
| **Voice similarity** (SSIM) ↑ | 0.936 | **0.940** | 0.935 |
| **Dead air** (excess silence) ↓ | **0.029 s** | 0.532 s | 0.233 s |

**13× fewer word errors**, ~0.5 MOS more natural, an order of magnitude less
dead air. Median WER is **0.00 %** on all four subsets — the typical generation
is transcribed exactly.

### WER — normalized text

The headline condition: numbers and dates already spoken out, as the shipped
normalizer produces.

| Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
|---|---|:-:|:-:|:-:|
| `vietnamese` | monolingual Vietnamese | **0.21 %** | 7.21 % | 7.54 % |
| `code_switch` | Vietnamese + embedded English | **0.95 %** | 10.14 % | 5.86 % |
| `cross_lingual` | foreign voice prompt → Vietnamese | **0.38 %** | 4.94 % | 6.61 % |
| `challenging` | acronyms, dates, %, currency | **0.61 %** | 5.63 % | 13.44 % |
| **overall** | | **0.56 %** | **7.27 %** | **8.61 %** |

### WER — raw text

The harder condition: the model is handed `31/12/2025` and `ChatGPT` verbatim
and has to read them itself, with no normalizer in front. This is what a system
with no Vietnamese text frontend faces.

| Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
|---|---|:-:|:-:|:-:|
| `vietnamese` | monolingual Vietnamese | **0.16 %** | 7.92 % | 9.56 % |
| `code_switch` | Vietnamese + embedded English | **0.97 %** | 10.94 % | 9.25 % |
| `cross_lingual` | foreign voice prompt → Vietnamese | **1.42 %** | 21.37 % | 27.27 % |
| `challenging` | acronyms, dates, %, currency | **1.75 %** | 27.86 % | 31.85 % |
| **overall** | | **1.03 %** | **16.42 %** | **18.40 %** |

**Reading these fairly:**

* **Normalization is where the baselines gain most, and we still win.** Their
  tokenizers genuinely have no Vietnamese number expansion, so raw text punishes
  them hard (`challenging` 27.86 %) and the normalized column is the fairest
  comparison available — it improves XTTS 2.3× and viXTTS 2.1×, against 1.8× for
  us. The gap narrows from 16× to 13× and stops there, because what remains is
  the acoustic model.
* **`vietnamese` barely moves for anyone** (0.16 % → 0.21 % for ZeroTTS). It has
  no digits or acronyms, so there is nothing to normalize — which is the control
  showing the other subsets' gains are real and not a scoring artifact.
* **Voice similarity is a tie, not a win.** 0.936 / 0.940 / 0.935 is within
  noise. On `cross_lingual` ZeroTTS is genuinely behind (0.911 vs ~0.935): it
  carries a foreign speaker's timbre into Vietnamese slightly less faithfully,
  while winning that subset's WER by 15×.
* **The WER definition matters more than the WER.** ZeroBench scores every clip
  with **two ASRs** (`whisper-large-v3` + `PhoWhisper-large`, min taken — neither
  can judge Vietnamese code-switch TTS alone) against **every acceptable
  reading** of the target text. Its test suite pins the policy in both
  directions: format artifacts must score 0, real defects must still cost.
* **Our remaining errors are published, not hidden.** Every item scoring above
  0.00 is audited in [docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md) — mostly voiced
  leading zeros in dates and `W`/`H` acronym letter names.

Reproduce, or score your own system:

```bash
pip install "zerotts[eval]"
SYNTH_FROM=text_normalized OUT_DIR=./eval/norm ./evaluation/run_benchmark.sh
./evaluation/run_benchmark.sh                                   # raw text
```

Not using ZeroTTS? The scorer stands alone — bring wavs from any system:

```bash
huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS
cd ZeroBench-TTS && pip install -r zerobench_eval/requirements.txt
python -m zerobench_eval manifest --out manifest.jsonl    # what to synthesize
python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel
```

## Voices, and voice cloning

A voice is a small array of speaker latents, `(1, n_voice_queries, d_model)`,
shipped as a `.npz` under `voices/`. That array is the entire speaker
conditioning — no reference transcript, no audio prompt.

> **Voice cloning is not available in this release.** Those latents come from a
> voice encoder that reads a reference clip, and that encoder is not published.
> This repository ships ready-to-use voices; it cannot create new ones from
> audio.
>
> To get latents for your own speaker, see **[zeroweight.ai](https://zeroweight.ai)**
> or get in touch.

Because a voice is just an array, latents obtained that way drop into
`voices/<name>/voice.npz` and work with no code change.

## Repository layout

```
config.json                          runtime config
tokenizer.json                       BPE tokenizer
null_voice_emb.npy                   learned unconditional voice prefix
onnx/text_encoder.onnx               text → encoder states (once per utterance)
onnx/prefix_step.onnx                global transformer step (once per frame)
onnx/local_frame_decode.onnx         frame decode + sampling (once per frame)
onnx/codec/                          MOSS-Audio-Tokenizer-Nano decoder (Apache-2.0)
voices/<name>/voice.npz              speaker latents
```

fp32, not quantized: ~900 MB total. Two ONNX Runtime calls per audio frame;
frames are produced at 12.5 Hz and decoded to 48 kHz.

The model architecture, training code, and the ONNX export script are not
published, and the voice encoder is not included.

## Intended use and limitations

Built for **Vietnamese**. It handles English words embedded in Vietnamese text
(`code_switch`), but it is not an English TTS system and is not evaluated as one.

Do not use it to impersonate a real person, to generate speech attributed to
someone without their consent, or to produce audio intended to deceive. The
shipped voices are for evaluation and demos.

Synthetic speech should be disclosed as synthetic wherever a listener might
reasonably assume otherwise.

## Credits

Speech codec: **MOSS-Audio-Tokenizer-Nano** by the OpenMOSS team, Apache-2.0.
Its ONNX **decoder** graphs are redistributed under `onnx/codec/` so ZeroTTS has
no external runtime dependency; the encoder is not included. See
`onnx/codec/LICENSE-Apache-2.0.txt`.

```bibtex
@misc{gong2026mossaudiotokenizerscalingaudiotokenizers,
  title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models},
  author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen
          and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li
          and Qingyuan Cheng and Shimin Li and Xipeng Qiu},
  year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD}
}
```

## License

ZeroTTS weights and code: **MIT**.
Bundled MOSS codec decoder: **Apache-2.0**.

The ZeroBench-TTS *dataset* is CC-BY-NC-4.0 because it redistributes reference
audio from VIVOS, viVoice, phoaudiobook and Emilia. That license applies to the
benchmark dataset only — **not** to these weights.