--- license: mit language: - vi library_name: onnx pipeline_tag: text-to-speech tags: - text-to-speech - tts - vietnamese - onnx - onnxruntime - zero-shot - speech-synthesis - voice-cloning - vietnamese-tts - tieng-viet # NOTE deliberately NO `datasets:` field. It is the only thing that populates # the Hub's cross-link, but the Hub renders it as "Models trained or fine-tuned # on " — which for our own held-out benchmark reads as train/test # contamination. ZeroBench-TTS is EVALUATION data; every voice in it is held # out of training. The `model-index` block below states that correctly, and the # body links the benchmark in prose. metrics: - wer model-index: - name: ZeroTTS results: - task: type: text-to-speech name: Zero-Shot Text-to-Speech dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS split: test metrics: - type: wer value: 0.56 name: WER (%) — normalized text - type: utmos value: 2.91 name: UTMOSv2 naturalness MOS - type: speaker_similarity value: 0.936 name: Speaker similarity (WavLM-SV cosine) - type: excess_silence value: 0.029 name: Excess silence (s) - task: type: text-to-speech name: Zero-Shot TTS — monolingual Vietnamese dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (vietnamese) config: vietnamese split: test metrics: - type: wer value: 0.21 name: WER (%) — normalized text - task: type: text-to-speech name: Zero-Shot TTS — Vietnamese/English code-switching dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (code_switch) config: code_switch split: test metrics: - type: wer value: 0.95 name: WER (%) — normalized text - task: type: text-to-speech name: Zero-Shot TTS — cross-lingual voice prompt dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (cross_lingual) config: cross_lingual split: test metrics: - type: wer value: 0.38 name: WER (%) — normalized text - task: type: text-to-speech name: Zero-Shot TTS — acronyms, dates, numbers dataset: type: zeroweight-ai/ZeroBench-TTS name: ZeroBench-TTS (challenging) config: challenging split: test metrics: - type: wer value: 0.61 name: WER (%) — normalized text --- ZeroTTS — Vietnamese zero-shot text-to-speech # ZeroTTS ### Vietnamese Zero-Shot Text-to-Speech (TTS) with real-time streaming and voice cloning from seconds of audio. Fast, natural, and optimised for CPU inference. **The most accurate open Vietnamese TTS we know of — 13× fewer word errors than the next open model**, and it runs faster than real time on a laptop CPU. * 🎯 **Ultra-natural** — 2.91 UTMOS, ~0.5 MOS above every other open Vietnamese system, with near-zero dead air (0.029 s vs 0.23–0.53 s). * 🗣️ **Zero-shot voice cloning** — a voice is a small latent array; drop it in and the model speaks in it. No fine-tuning, no per-speaker training. * ⚡ **Real-time on CPU, streaming** — first audio chunk in ~100 ms, then chunks ramp up. No GPU required. * 🇻🇳 **Built for Vietnamese** — tones, code-switched English, and a built-in normalizer that reads `31/12/2025` and `1.250 tỷ` the way a person would. * 📊 **Measured, not asserted** — every number below comes from [ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)'s own public scorer, on 59 held-out voices. * Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS** * Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS** ```python pip install zerotts ``` ```python from zerotts import ZeroTTS tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS") audio = tts.synthesize("Xin chào các bạn, mình là ZeroTTS.", voice="arya") tts.save_audio(audio, "out.wav") ``` Streaming, with first audio in roughly 100 ms: ```python for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voice="arya"): play(chunk) # (1, n) float32 at 48 kHz ``` ## Benchmarks Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)** — 137 items, 59 held-out reference voices × 4 subsets — against [OmniVoice](https://huggingface.co/k2-fsa/OmniVoice) and the two public Vietnamese XTTS-v2 finetunes. 137/137 scored for every system, 0 empty generations. OmniVoice is given its optional `language="vi"` hint, which its model card recommends and which measurably helps it. **Scored by the benchmark, not by us.** ZeroTTS synthesizes the clips and hands them to `zerobench_eval`, the official scorer published inside the benchmark dataset repo. Nothing in this repo computes a metric. ### Headline Every system reads **normalized text** — dates, numbers and acronyms already spoken out, from the benchmark's own curated reading. Every system gets exactly the same input, so the comparison is like-for-like. This is the condition a Vietnamese TTS system meets in production, where a text frontend runs ahead of the model. ZeroTTS ships one — `normalize_vi_text`, applied by default (see the [GitHub README](https://github.com/zeroweight-ai/ZeroTTS#usage)) — which reproduces the benchmark's reading on 34 of the 35 items that need normalization. Neither baseline ships a Vietnamese frontend at all, which is why the raw-text table below is so much harsher on them. | | **ZeroTTS** | OmniVoice | XTTS-v2-vietnamse | viXTTS | |---|:-:|:-:|:-:|:-:| | **WER** ↓ | **0.56 %** | 2.12 % | 7.27 % | 8.61 % | | **Naturalness** (UTMOS) ↑ | **2.91** | 2.75 | 2.49 | 2.34 | | **Voice similarity** (SSIM) ↑ | 0.938 | **0.951** | 0.941 | 0.935 | | **Dead air** (excess silence) ↓ | **0.029 s** | 0.386 s | 0.568 s | 0.215 s | | Size | **81 M**, CPU | 3.1 GB, GPU | 1.9 GB, GPU | 1.9 GB, GPU | **4× fewer word errors than the next-best system**, ~0.2 MOS more natural, an order of magnitude less dead air — from a model small enough to run real-time on a laptop CPU. Median WER is **0.00 %** on all four subsets: the typical generation is transcribed exactly. (Every figure is from the same normalized-text runs, so the rows are mutually consistent.) ### WER — normalized text The headline condition: numbers and dates already spoken out, as the shipped normalizer produces. | Subset | what it tests | **ZeroTTS** | OmniVoice | XTTS-v2-vietnamse | viXTTS | |---|---|:-:|:-:|:-:|:-:| | `vietnamese` | monolingual Vietnamese | **0.21 %** | 0.50 % | 7.21 % | 7.54 % | | `code_switch` | Vietnamese + embedded English | 0.95 % | **0.46 %** | 10.14 % | 5.86 % | | `cross_lingual` | foreign voice prompt → Vietnamese | **0.38 %** | 9.60 % | 4.94 % | 6.61 % | | `challenging` | acronyms, dates, %, currency | **0.61 %** | 1.56 % | 5.63 % | 13.44 % | | **overall** | | **0.56 %** | **2.12 %** | **7.27 %** | **8.61 %** | ### WER — raw text The harder condition: the model is handed `31/12/2025` and `ChatGPT` verbatim and has to read them itself, with no normalizer in front. This is what a system with no Vietnamese text frontend faces. | Subset | what it tests | **ZeroTTS** | OmniVoice | XTTS-v2-vietnamse | viXTTS | |---|---|:-:|:-:|:-:|:-:| | `vietnamese` | monolingual Vietnamese | **0.16 %** | 0.50 % | 7.92 % | 9.56 % | | `code_switch` | Vietnamese + embedded English | 0.97 % | **0.46 %** | 10.94 % | 9.25 % | | `cross_lingual` | foreign voice prompt → Vietnamese | **1.42 %** | 17.71 % | 21.37 % | 27.27 % | | `challenging` | acronyms, dates, %, currency | **1.75 %** | 4.46 % | 27.86 % | 31.85 % | | **overall** | | **1.03 %** | **4.13 %** | **16.42 %** | **18.40 %** | **Reading these fairly:** * **OmniVoice beats us on two things, and they are worth naming.** Its speaker similarity is the best of the four (0.951 vs our 0.938), and on `code_switch` it is roughly half our error rate (0.46 % vs 0.95 %). If cloning fidelity or English-in-Vietnamese is your priority, it is a genuinely strong option — at 3.1 GB on a GPU. * **OmniVoice's overall figure is dominated by one subset.** `cross_lingual` (foreign voice prompt, Vietnamese text) costs it 17.71 % raw against our 1.42 %, and it is language-dependent — German 0.00 %, Korean 0.13 %, Japanese 0.41 %. Excluding that subset it lands near 1.7 % raw. Both ASRs agree the audio genuinely degrades there, so it is the model, not the scorer. * **Normalization is where the weakest systems gain most, and the order does not change.** The XTTS tokenizers have no Vietnamese number expansion, so raw text punishes them hard (`challenging` 27.86 %) and the normalized column is the fairest comparison available — it improves XTTS 2.3× and viXTTS 2.1×, against 1.8× for us. What remains is the acoustic model. * **`vietnamese` barely moves for anyone** (0.16 % → 0.21 % for ZeroTTS). It has no digits or acronyms, so there is nothing to normalize — which is the control showing the other subsets' gains are real and not a scoring artifact. * **On `cross_lingual` our voice similarity is the weak spot** (0.911 vs ~0.935 for the others): ZeroTTS carries a foreign speaker's timbre into Vietnamese slightly less faithfully, while winning that subset's WER by 12×. * **The WER definition matters more than the WER.** ZeroBench scores every clip with **two ASRs** (`whisper-large-v3` + `PhoWhisper-large`, min taken — neither can judge Vietnamese code-switch TTS alone) against **every acceptable reading** of the target text, so a system is never charged for an ASR's choice between "31/12/2025" and "ba mươi mốt tháng mười hai". Its test suite pins that in both directions: format differences must score 0, real mispronunciations must still cost. * **Our remaining errors are published, not hidden.** Every item scoring above 0.00 is audited in [docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md). The two recurring ones: a leading zero read aloud (`18/04` → "tháng **không** tư"), and the letters `W` and `H` coming out wrong when an acronym has to be spelled — `WHO` should be spelled out letter by letter, and instead comes out as something like "Hall". Reproduce, or score your own system: ```bash pip install "zerotts[eval]" SYNTH_FROM=text_normalized OUT_DIR=./eval/norm ./evaluation/run_benchmark.sh ./evaluation/run_benchmark.sh # raw text ``` Not using ZeroTTS? The scorer stands alone — bring wavs from any system: ```bash huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS cd ZeroBench-TTS && pip install -r zerobench_eval/requirements.txt python -m zerobench_eval manifest --out manifest.jsonl # what to synthesize python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel ``` ## Voices, and voice cloning A voice is a small array of speaker latents, `(1, n_voice_queries, d_model)`, shipped as a `.npz` under `voices/`. That array is the entire speaker conditioning — no reference transcript, no audio prompt. > **Voice cloning is not available in this release.** Those latents come from a > voice encoder that reads a reference clip, and that encoder is not published. > This repository ships ready-to-use voices; it cannot create new ones from > audio. > > To get latents for your own speaker, see **[zeroweight.ai](https://zeroweight.ai)** > or get in touch. Because a voice is just an array, latents obtained that way drop into `voices//voice.npz` and work with no code change. ## Repository layout ``` config.json runtime config tokenizer.json BPE tokenizer null_voice_emb.npy learned unconditional voice prefix onnx/text_encoder.onnx text → encoder states (once per utterance) onnx/prefix_step.onnx global transformer step (once per frame) onnx/local_frame_decode.onnx frame decode + sampling (once per frame) onnx/codec/ MOSS-Audio-Tokenizer-Nano decoder (Apache-2.0) voices//voice.npz speaker latents ``` fp32, not quantized: ~900 MB total. Two ONNX Runtime calls per audio frame; frames are produced at 12.5 Hz and decoded to 48 kHz. The model architecture, training code, and the ONNX export script are not published, and the voice encoder is not included. ## Intended use and limitations Built for **Vietnamese**. It handles English words embedded in Vietnamese text (`code_switch`), but it is not an English TTS system and is not evaluated as one. Do not use it to impersonate a real person, to generate speech attributed to someone without their consent, or to produce audio intended to deceive. The shipped voices are for evaluation and demos. Synthetic speech should be disclosed as synthetic wherever a listener might reasonably assume otherwise. ## Credits Speech codec: **MOSS-Audio-Tokenizer-Nano** by the OpenMOSS team, Apache-2.0. Its ONNX **decoder** graphs are redistributed under `onnx/codec/` so ZeroTTS has no external runtime dependency; the encoder is not included. See `onnx/codec/LICENSE-Apache-2.0.txt`. ```bibtex @misc{gong2026mossaudiotokenizerscalingaudiotokenizers, title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models}, author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li and Qingyuan Cheng and Shimin Li and Xipeng Qiu}, year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD} } ``` ## License ZeroTTS weights and code: **MIT**. Bundled MOSS codec decoder: **Apache-2.0**. The ZeroBench-TTS *dataset* is CC-BY-NC-4.0 because it redistributes reference audio from VIVOS, viVoice, phoaudiobook and Emilia. That license applies to the benchmark dataset only — **not** to these weights.