--- license: mit language: - vi library_name: onnx pipeline_tag: text-to-speech tags: - text-to-speech - tts - vietnamese - onnx - onnxruntime - zero-shot - speech-synthesis - voice-cloning - vietnamese-tts - tieng-viet datasets: - zeroweight-ai/ZeroBench-TTS metrics: - wer model-index: - name: ZeroTTS results: - task: type: text-to-speech name: Zero-Shot Text-to-Speech dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS split: test metrics: - type: wer value: 1.03 name: WER (%) - type: utmos value: 2.91 name: UTMOSv2 naturalness MOS - type: speaker_similarity value: 0.936 name: Speaker similarity (WavLM-SV cosine) - type: excess_silence value: 0.029 name: Excess silence (s) - task: type: text-to-speech name: Zero-Shot TTS — monolingual Vietnamese dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (vietnamese) config: vietnamese split: test metrics: - type: wer value: 0.16 name: WER (%) - task: type: text-to-speech name: Zero-Shot TTS — Vietnamese/English code-switching dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (code_switch) config: code_switch split: test metrics: - type: wer value: 0.97 name: WER (%) - task: type: text-to-speech name: Zero-Shot TTS — cross-lingual voice prompt dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (cross_lingual) config: cross_lingual split: test metrics: - type: wer value: 1.42 name: WER (%) - task: type: text-to-speech name: Zero-Shot TTS — acronyms, dates, numbers dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (challenging) config: challenging split: test metrics: - type: wer value: 1.75 name: WER (%) --- # ZeroTTS Vietnamese text-to-speech. The whole inference path is **numpy + ONNX Runtime** — no PyTorch, no CUDA — so it runs on a laptop CPU or in a browser. * Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS** * Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS** ```python pip install zerotts ``` ```python from zerotts import ZeroTTS tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS") audio = tts.synthesize("Xin chào các bạn, mình là ZeroTTS.", voice="arya") tts.save_audio(audio, "out.wav") ``` Streaming, with first audio in roughly 100 ms: ```python for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voice="arya"): play(chunk) # (1, n) float32 at 48 kHz ``` ## Benchmarks Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)** — 137 items, 59 held-out reference voices, 4 subsets — against the two public Vietnamese XTTS-v2 finetunes. 137/137 scored, 0 empty generations. Scored by the benchmark's own published scorer (`zerobench_eval`), not by us. WER is the minimum over two ASRs (`whisper-large-v3` + `PhoWhisper-large`) and over every acceptable reading of the target text — see the [benchmark README](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS) for why both matter. | | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS | |---|:-:|:-:|:-:| | **WER** ↓ | **1.03%** | 16.42% | 18.40% | | **Naturalness** (UTMOS) ↑ | **2.91** | 2.43 | 2.35 | | **Voice similarity** (SSIM) ↑ | 0.936 | **0.940** | 0.935 | | **Dead air** (excess silence) ↓ | **0.029 s** | 0.532 s | 0.233 s | **16× fewer word errors**, ~0.5 MOS more natural, an order of magnitude less dead air. Median WER is **0.00%** on all four subsets. WER by subset: | Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS | |---|---|:-:|:-:|:-:| | `vietnamese` | plain Vietnamese | **0.16%** | 7.92% | 9.56% | | `code_switch` | Vietnamese + embedded English | **0.97%** | 10.94% | 9.25% | | `cross_lingual` | foreign voice prompt → Vietnamese | **1.42%** | 21.37% | 27.27% | | `challenging` | acronyms, dates, %, currency | **1.75%** | 27.86% | 31.85% | Reading these fairly: * **Voice similarity is a tie, not a win** — 0.936 / 0.939 / 0.935 is within noise. On `cross_lingual` ZeroTTS is genuinely behind (0.911 vs ~0.935): it carries a foreign speaker's timbre into Vietnamese less faithfully than the XTTS backbone, while winning that subset's WER by 15×. * **"The baselines just need text normalization" is testable, and fails.** Given the spoken-out text instead of raw orthography, XTTS improves to 7.27% and viXTTS to 8.61% — big gains, confirming their tokenizers lack Vietnamese number expansion — but ZeroTTS improves to 0.56%, so the gap stays 13–15×. * **ZeroTTS's own remaining errors are known and listed**, not hidden: [evaluation/HIGH_WER_ANALYSIS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/HIGH_WER_ANALYSIS.md) audits every item above 0.00 — mostly voiced leading zeros in dates and `W`/`H` acronym letter names. Score your own model on the same benchmark: ```bash huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS cd ZeroBench-TTS && pip install -r zerobench_eval/requirements.txt python -m zerobench_eval manifest --out manifest.jsonl # what to synthesize python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel ``` Full tables and reproduction commands: [evaluation/RESULTS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/RESULTS.md). ## Voices, and voice cloning A voice is a small array of speaker latents, `(1, n_voice_queries, d_model)`, shipped as a `.npz` under `voices/`. That array is the entire speaker conditioning — no reference transcript, no audio prompt. > **Voice cloning is not available in this release.** Those latents come from a > voice encoder that reads a reference clip, and that encoder is not published. > This repository ships ready-to-use voices; it cannot create new ones from > audio. > > To get latents for your own speaker, see **[zeroweight.ai](https://zeroweight.ai)** > or get in touch. Because a voice is just an array, latents obtained that way drop into `voices//voice.npz` and work with no code change. ## Repository layout ``` config.json runtime config tokenizer.json BPE tokenizer null_voice_emb.npy learned unconditional voice prefix onnx/text_encoder.onnx text → encoder states (once per utterance) onnx/prefix_step.onnx global transformer step (once per frame) onnx/local_frame_decode.onnx frame decode + sampling (once per frame) onnx/codec/ MOSS-Audio-Tokenizer-Nano decoder (Apache-2.0) voices//voice.npz speaker latents ``` fp32, not quantized: ~900 MB total. Two ONNX Runtime calls per audio frame; frames are produced at 12.5 Hz and decoded to 48 kHz. The model architecture, training code, and the ONNX export script are not published, and the voice encoder is not included. ## Intended use and limitations Built for **Vietnamese**. It handles English words embedded in Vietnamese text (`code_switch`), but it is not an English TTS system and is not evaluated as one. Do not use it to impersonate a real person, to generate speech attributed to someone without their consent, or to produce audio intended to deceive. The shipped voices are for evaluation and demos. Synthetic speech should be disclosed as synthetic wherever a listener might reasonably assume otherwise. ## Credits Speech codec: **MOSS-Audio-Tokenizer-Nano** by the OpenMOSS team, Apache-2.0. Its ONNX **decoder** graphs are redistributed under `onnx/codec/` so ZeroTTS has no external runtime dependency; the encoder is not included. See `onnx/codec/LICENSE-Apache-2.0.txt`. ```bibtex @misc{gong2026mossaudiotokenizerscalingaudiotokenizers, title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models}, author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li and Qingyuan Cheng and Shimin Li and Xipeng Qiu}, year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD} } ``` ## License ZeroTTS weights and code: **MIT**. Bundled MOSS codec decoder: **Apache-2.0**. The ZeroBench-TTS *dataset* is CC-BY-NC-4.0 because it redistributes reference audio from VIVOS, viVoice, phoaudiobook and Emilia. That license applies to the benchmark dataset only — **not** to these weights.