Hugging Face's logo

zeroweight-ai
/
ZeroTTS

Text-to-Speech
ONNX
Vietnamese
tts
vietnamese
onnxruntime
zero-shot
speech-synthesis
voice-cloning
vietnamese-tts
tieng-viet
Eval Results (legacy)
Model card Files
xet
 Community
13
ZeroTTS / README.md
zeroweightai's picture
zeroweightai
Drop datasets: field — ZeroBench-TTS is eval data, not training data
143a7d0 verified
preview code
|
Raw
History Blame
9.65 kB
metadata 
license: mit
language:
  - vi
library_name: onnx
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - tts
  - vietnamese
  - onnx
  - onnxruntime
  - zero-shot
  - speech-synthesis
  - voice-cloning
  - vietnamese-tts
  - tieng-viet
metrics:
  - wer
model-index:
  - name: ZeroTTS
    results:
      - task:
          type: text-to-speech
          name: Zero-Shot Text-to-Speech
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS
          split: test
        metrics:
          - type: wer
            value: 1.03
            name: WER (%)
          - type: utmos
            value: 2.91
            name: UTMOSv2 naturalness MOS
          - type: speaker_similarity
            value: 0.936
            name: Speaker similarity (WavLM-SV cosine)
          - type: excess_silence
            value: 0.029
            name: Excess silence (s)
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — monolingual Vietnamese
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (vietnamese)
          config: vietnamese
          split: test
        metrics:
          - type: wer
            value: 0.16
            name: WER (%)
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — Vietnamese/English code-switching
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (code_switch)
          config: code_switch
          split: test
        metrics:
          - type: wer
            value: 0.97
            name: WER (%)
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — cross-lingual voice prompt
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (cross_lingual)
          config: cross_lingual
          split: test
        metrics:
          - type: wer
            value: 1.42
            name: WER (%)
      - task:
          type: text-to-speech
          name: Zero-Shot TTS — acronyms, dates, numbers
        dataset:
          type: zeroweight-ai/ZeroBench-TTS
          name: ZeroBench-TTS (challenging)
          config: challenging
          split: test
        metrics:
          - type: wer
            value: 1.75
            name: WER (%)

ZeroTTS

Vietnamese text-to-speech. The whole inference path is numpy + ONNX Runtime — no PyTorch, no CUDA — so it runs on a laptop CPU or in a browser.

pip install zerotts

from zerotts import ZeroTTS

tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS")
audio = tts.synthesize("Xin chào các bạn, mình là ZeroTTS.", voice="arya")
tts.save_audio(audio, "out.wav")

Streaming, with first audio in roughly 100 ms:

for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voice="arya"):
    play(chunk)   # (1, n) float32 at 48 kHz

Benchmarks

Measured on ZeroBench-TTS — 137 items, 59 held-out reference voices, 4 subsets — against the two public Vietnamese XTTS-v2 finetunes. 137/137 scored, 0 empty generations.

Scored by the benchmark's own published scorer (zerobench_eval), not by us. WER is the minimum over two ASRs (whisper-large-v3 + PhoWhisper-large) and over every acceptable reading of the target text — see the benchmark README for why both matter.

ZeroTTS XTTS-v2-vietnamse viXTTS
WER — raw text ↓ 1.03% 16.42% 18.40%
WER — pre-normalized text ↓ 0.56% 7.27% 8.61%
Naturalness (UTMOS) ↑ 2.91 2.43 2.35
Voice similarity (SSIM) ↑ 0.936 0.940 0.935
Dead air (excess silence) ↓ 0.029 s 0.532 s 0.233 s

16× fewer word errors on the real task, 13× fewer even after handing every model a perfect text frontend; ~0.5 MOS more natural, an order of magnitude less dead air. Median WER is 0.00% on all four subsets.

WER by subset:

Subset what it tests ZeroTTS XTTS-v2-vietnamse viXTTS
vietnamese plain Vietnamese 0.16% 7.92% 9.56%
code_switch Vietnamese + embedded English 0.97% 10.94% 9.25%
cross_lingual foreign voice prompt → Vietnamese 1.42% 21.37% 27.27%
challenging acronyms, dates, %, currency 1.75% 27.86% 31.85%

Reading these fairly:

  • Voice similarity is a tie, not a win — 0.936 / 0.939 / 0.935 is within noise. On cross_lingual ZeroTTS is genuinely behind (0.911 vs ~0.935): it carries a foreign speaker's timbre into Vietnamese less faithfully than the XTTS backbone, while winning that subset's WER by 15×.
  • "The baselines just need text normalization" is testable, and fails. Given the spoken-out text instead of raw orthography, XTTS improves to 7.27% and viXTTS to 8.61% — big gains, confirming their tokenizers lack Vietnamese number expansion — but ZeroTTS improves to 0.56%, so the gap stays 13–15×.
  • ZeroTTS's own remaining errors are known and listed, not hidden: evaluation/HIGH_WER_ANALYSIS.md audits every item above 0.00 — mostly voiced leading zeros in dates and W/H acronym letter names.

Score your own model on the same benchmark:

huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS
cd ZeroBench-TTS && pip install -r zerobench_eval/requirements.txt
python -m zerobench_eval manifest --out manifest.jsonl    # what to synthesize
python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel

Full tables and reproduction commands: evaluation/RESULTS.md.

Voices, and voice cloning

A voice is a small array of speaker latents, (1, n_voice_queries, d_model), shipped as a .npz under voices/. That array is the entire speaker conditioning — no reference transcript, no audio prompt.

Voice cloning is not available in this release. Those latents come from a voice encoder that reads a reference clip, and that encoder is not published. This repository ships ready-to-use voices; it cannot create new ones from audio.

To get latents for your own speaker, see zeroweight.ai or get in touch.

Because a voice is just an array, latents obtained that way drop into voices/<name>/voice.npz and work with no code change.

Repository layout 

config.json                          runtime config
tokenizer.json                       BPE tokenizer
null_voice_emb.npy                   learned unconditional voice prefix
onnx/text_encoder.onnx               text → encoder states (once per utterance)
onnx/prefix_step.onnx                global transformer step (once per frame)
onnx/local_frame_decode.onnx         frame decode + sampling (once per frame)
onnx/codec/                          MOSS-Audio-Tokenizer-Nano decoder (Apache-2.0)
voices/<name>/voice.npz              speaker latents

fp32, not quantized: ~900 MB total. Two ONNX Runtime calls per audio frame; frames are produced at 12.5 Hz and decoded to 48 kHz.

The model architecture, training code, and the ONNX export script are not published, and the voice encoder is not included.

Intended use and limitations

Built for Vietnamese. It handles English words embedded in Vietnamese text (code_switch), but it is not an English TTS system and is not evaluated as one.

Do not use it to impersonate a real person, to generate speech attributed to someone without their consent, or to produce audio intended to deceive. The shipped voices are for evaluation and demos.

Synthetic speech should be disclosed as synthetic wherever a listener might reasonably assume otherwise.

Credits

Speech codec: MOSS-Audio-Tokenizer-Nano by the OpenMOSS team, Apache-2.0. Its ONNX decoder graphs are redistributed under onnx/codec/ so ZeroTTS has no external runtime dependency; the encoder is not included. See onnx/codec/LICENSE-Apache-2.0.txt.

@misc{gong2026mossaudiotokenizerscalingaudiotokenizers,
  title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models},
  author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen
          and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li
          and Qingyuan Cheng and Shimin Li and Xipeng Qiu},
  year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD}
}

License

ZeroTTS weights and code: MIT. Bundled MOSS codec decoder: Apache-2.0.

The ZeroBench-TTS dataset is CC-BY-NC-4.0 because it redistributes reference audio from VIVOS, viVoice, phoaudiobook and Emilia. That license applies to the benchmark dataset only — not to these weights.