| --- |
| license: mit |
| language: |
| - vi |
| library_name: onnx |
| pipeline_tag: text-to-speech |
| tags: |
| - text-to-speech |
| - tts |
| - vietnamese |
| - onnx |
| - onnxruntime |
| - zero-shot |
| - speech-synthesis |
| - voice-cloning |
| - vietnamese-tts |
| - tieng-viet |
| datasets: |
| - zeroweight-ai/ZeroBench-TTS |
| metrics: |
| - wer |
| model-index: |
| - name: ZeroTTS |
| results: |
| - task: |
| type: text-to-speech |
| name: Zero-Shot Text-to-Speech |
| dataset: |
| type: zeroweight-ai/ZeroBench-TTS |
| name: ZeroBench-TTS |
| split: test |
| metrics: |
| - type: wer |
| value: 1.03 |
| name: WER (%) |
| - type: utmos |
| value: 2.91 |
| name: UTMOSv2 naturalness MOS |
| - type: speaker_similarity |
| value: 0.936 |
| name: Speaker similarity (WavLM-SV cosine) |
| - type: excess_silence |
| value: 0.029 |
| name: Excess silence (s) |
| - task: |
| type: text-to-speech |
| name: Zero-Shot TTS — monolingual Vietnamese |
| dataset: |
| type: zeroweight-ai/ZeroBench-TTS |
| name: ZeroBench-TTS (vietnamese) |
| config: vietnamese |
| split: test |
| metrics: |
| - type: wer |
| value: 0.16 |
| name: WER (%) |
| - task: |
| type: text-to-speech |
| name: Zero-Shot TTS — Vietnamese/English code-switching |
| dataset: |
| type: zeroweight-ai/ZeroBench-TTS |
| name: ZeroBench-TTS (code_switch) |
| config: code_switch |
| split: test |
| metrics: |
| - type: wer |
| value: 0.97 |
| name: WER (%) |
| - task: |
| type: text-to-speech |
| name: Zero-Shot TTS — cross-lingual voice prompt |
| dataset: |
| type: zeroweight-ai/ZeroBench-TTS |
| name: ZeroBench-TTS (cross_lingual) |
| config: cross_lingual |
| split: test |
| metrics: |
| - type: wer |
| value: 1.42 |
| name: WER (%) |
| - task: |
| type: text-to-speech |
| name: Zero-Shot TTS — acronyms, dates, numbers |
| dataset: |
| type: zeroweight-ai/ZeroBench-TTS |
| name: ZeroBench-TTS (challenging) |
| config: challenging |
| split: test |
| metrics: |
| - type: wer |
| value: 1.75 |
| name: WER (%) |
| --- |
| |
| # ZeroTTS |
|
|
| Vietnamese text-to-speech. The whole inference path is **numpy + ONNX Runtime** — |
| no PyTorch, no CUDA — so it runs on a laptop CPU or in a browser. |
|
|
| * Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS** |
| * Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS** |
|
|
| ```python |
| pip install zerotts |
| ``` |
|
|
| ```python |
| from zerotts import ZeroTTS |
| |
| tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS") |
| audio = tts.synthesize("Xin chào các bạn, mình là ZeroTTS.", voice="arya") |
| tts.save_audio(audio, "out.wav") |
| ``` |
|
|
| Streaming, with first audio in roughly 100 ms: |
|
|
| ```python |
| for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voice="arya"): |
| play(chunk) # (1, n) float32 at 48 kHz |
| ``` |
|
|
| ## Benchmarks |
|
|
| Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)** |
| — 137 items, 59 held-out reference voices, 4 subsets — against the two public |
| Vietnamese XTTS-v2 finetunes. 137/137 scored, 0 empty generations. |
|
|
| Scored by the benchmark's own published scorer (`zerobench_eval`), not by us. |
| WER is the minimum over two ASRs (`whisper-large-v3` + `PhoWhisper-large`) and |
| over every acceptable reading of the target text — see the |
| [benchmark README](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS) |
| for why both matter. |
|
|
| | | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS | |
| |---|:-:|:-:|:-:| |
| | **WER** ↓ | **1.03%** | 16.42% | 18.40% | |
| | **Naturalness** (UTMOS) ↑ | **2.91** | 2.43 | 2.35 | |
| | **Voice similarity** (SSIM) ↑ | 0.936 | **0.940** | 0.935 | |
| | **Dead air** (excess silence) ↓ | **0.029 s** | 0.532 s | 0.233 s | |
|
|
| **16× fewer word errors**, ~0.5 MOS more natural, an order of magnitude less |
| dead air. Median WER is **0.00%** on all four subsets. |
|
|
| WER by subset: |
|
|
| | Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS | |
| |---|---|:-:|:-:|:-:| |
| | `vietnamese` | plain Vietnamese | **0.16%** | 7.92% | 9.56% | |
| | `code_switch` | Vietnamese + embedded English | **0.97%** | 10.94% | 9.25% | |
| | `cross_lingual` | foreign voice prompt → Vietnamese | **1.42%** | 21.37% | 27.27% | |
| | `challenging` | acronyms, dates, %, currency | **1.75%** | 27.86% | 31.85% | |
|
|
| Reading these fairly: |
|
|
| * **Voice similarity is a tie, not a win** — 0.936 / 0.939 / 0.935 is within |
| noise. On `cross_lingual` ZeroTTS is genuinely behind (0.911 vs ~0.935): it |
| carries a foreign speaker's timbre into Vietnamese less faithfully than the |
| XTTS backbone, while winning that subset's WER by 15×. |
| * **"The baselines just need text normalization" is testable, and fails.** |
| Given the spoken-out text instead of raw orthography, XTTS improves to 7.27% |
| and viXTTS to 8.61% — big gains, confirming their tokenizers lack Vietnamese |
| number expansion — but ZeroTTS improves to 0.56%, so the gap stays 13–15×. |
| * **ZeroTTS's own remaining errors are known and listed**, not hidden: |
| [evaluation/HIGH_WER_ANALYSIS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/HIGH_WER_ANALYSIS.md) |
| audits every item above 0.00 — mostly voiced leading zeros in dates and |
| `W`/`H` acronym letter names. |
|
|
| Score your own model on the same benchmark: |
|
|
| ```bash |
| huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS |
| cd ZeroBench-TTS && pip install -r zerobench_eval/requirements.txt |
| python -m zerobench_eval manifest --out manifest.jsonl # what to synthesize |
| python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel |
| ``` |
|
|
| Full tables and reproduction commands: |
| [evaluation/RESULTS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/RESULTS.md). |
|
|
| ## Voices, and voice cloning |
|
|
| A voice is a small array of speaker latents, `(1, n_voice_queries, d_model)`, |
| shipped as a `.npz` under `voices/`. That array is the entire speaker |
| conditioning — no reference transcript, no audio prompt. |
|
|
| > **Voice cloning is not available in this release.** Those latents come from a |
| > voice encoder that reads a reference clip, and that encoder is not published. |
| > This repository ships ready-to-use voices; it cannot create new ones from |
| > audio. |
| > |
| > To get latents for your own speaker, see **[zeroweight.ai](https://zeroweight.ai)** |
| > or get in touch. |
|
|
| Because a voice is just an array, latents obtained that way drop into |
| `voices/<name>/voice.npz` and work with no code change. |
|
|
| ## Repository layout |
|
|
| ``` |
| config.json runtime config |
| tokenizer.json BPE tokenizer |
| null_voice_emb.npy learned unconditional voice prefix |
| onnx/text_encoder.onnx text → encoder states (once per utterance) |
| onnx/prefix_step.onnx global transformer step (once per frame) |
| onnx/local_frame_decode.onnx frame decode + sampling (once per frame) |
| onnx/codec/ MOSS-Audio-Tokenizer-Nano decoder (Apache-2.0) |
| voices/<name>/voice.npz speaker latents |
| ``` |
|
|
| fp32, not quantized: ~900 MB total. Two ONNX Runtime calls per audio frame; |
| frames are produced at 12.5 Hz and decoded to 48 kHz. |
|
|
| The model architecture, training code, and the ONNX export script are not |
| published, and the voice encoder is not included. |
|
|
| ## Intended use and limitations |
|
|
| Built for **Vietnamese**. It handles English words embedded in Vietnamese text |
| (`code_switch`), but it is not an English TTS system and is not evaluated as one. |
|
|
| Do not use it to impersonate a real person, to generate speech attributed to |
| someone without their consent, or to produce audio intended to deceive. The |
| shipped voices are for evaluation and demos. |
|
|
| Synthetic speech should be disclosed as synthetic wherever a listener might |
| reasonably assume otherwise. |
|
|
| ## Credits |
|
|
| Speech codec: **MOSS-Audio-Tokenizer-Nano** by the OpenMOSS team, Apache-2.0. |
| Its ONNX **decoder** graphs are redistributed under `onnx/codec/` so ZeroTTS has |
| no external runtime dependency; the encoder is not included. See |
| `onnx/codec/LICENSE-Apache-2.0.txt`. |
|
|
| ```bibtex |
| @misc{gong2026mossaudiotokenizerscalingaudiotokenizers, |
| title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models}, |
| author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen |
| and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li |
| and Qingyuan Cheng and Shimin Li and Xipeng Qiu}, |
| year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD} |
| } |
| ``` |
|
|
| ## License |
|
|
| ZeroTTS weights and code: **MIT**. |
| Bundled MOSS codec decoder: **Apache-2.0**. |
|
|
| The ZeroBench-TTS *dataset* is CC-BY-NC-4.0 because it redistributes reference |
| audio from VIVOS, viVoice, phoaudiobook and Emilia. That license applies to the |
| benchmark dataset only — **not** to these weights. |
|
|
