Add banner, marketing intro, restructured result tables (normalized headline + both WER tables)

.gitattributes

@@ -4,3 +4,4 @@
 *.npz filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
 *.wav filter=lfs diff=lfs merge=lfs -text
+banner.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -15,8 +15,12 @@ tags:
   - voice-cloning
   - vietnamese-tts
   - tieng-viet
-datasets:
-  - zeroweight-ai/ZeroBench-TTS
+# NOTE deliberately NO `datasets:` field. It is the only thing that populates
+# the Hub's cross-link, but the Hub renders it as "Models trained or fine-tuned
+# on <dataset>" — which for our own held-out benchmark reads as train/test
+# contamination. ZeroBench-TTS is EVALUATION data; every voice in it is held
+# out of training. The `model-index` block below states that correctly, and the
+# body links the benchmark in prose.
 metrics:
   - wer
 model-index:
@@ -31,8 +35,8 @@ model-index:
           split: test
         metrics:
           - type: wer
-            value: 1.03
-            name: WER (%)
+            value: 0.56
+            name: WER (%) — normalized text
           - type: utmos
             value: 2.91
             name: UTMOSv2 naturalness MOS
@@ -52,8 +56,8 @@ model-index:
           split: test
         metrics:
           - type: wer
-            value: 0.16
-            name: WER (%)
+            value: 0.21
+            name: WER (%) — normalized text
       - task:
           type: text-to-speech
           name: Zero-Shot TTS — Vietnamese/English code-switching
@@ -64,8 +68,8 @@ model-index:
           split: test
         metrics:
           - type: wer
-            value: 0.97
-            name: WER (%)
+            value: 0.95
+            name: WER (%) — normalized text
       - task:
           type: text-to-speech
           name: Zero-Shot TTS — cross-lingual voice prompt
@@ -76,8 +80,8 @@ model-index:
           split: test
         metrics:
           - type: wer
-            value: 1.42
-            name: WER (%)
+            value: 0.38
+            name: WER (%) — normalized text
       - task:
           type: text-to-speech
           name: Zero-Shot TTS — acronyms, dates, numbers
@@ -88,14 +92,30 @@ model-index:
           split: test
         metrics:
           - type: wer
-            value: 1.75
-            name: WER (%)
+            value: 0.61
+            name: WER (%) — normalized text
 ---
 
+<img src="banner.png" alt="ZeroTTS — Vietnamese zero-shot text-to-speech" width="100%">
+
 # ZeroTTS
 
-Vietnamese text-to-speech. The whole inference path is **numpy + ONNX Runtime** —
-no PyTorch, no CUDA — so it runs on a laptop CPU or in a browser.
+### Ultra-natural Vietnamese speech, cloned from seconds of audio — streaming, real-time on a CPU
+
+**The most accurate open Vietnamese TTS we know of — 13× fewer word errors than
+the next open model**, and it runs faster than real time on a laptop CPU.
+
+* 🎯 **Ultra-natural** — 2.91 UTMOS, ~0.5 MOS above every other open Vietnamese
+  system, with near-zero dead air (0.029 s vs 0.23–0.53 s).
+* 🗣️ **Zero-shot voice cloning** — a voice is a small latent array; drop it in
+  and the model speaks in it. No fine-tuning, no per-speaker training.
+* ⚡ **Real-time on CPU, streaming** — first audio chunk in ~100 ms, then chunks
+  ramp up. No GPU required.
+* 🇻🇳 **Built for Vietnamese** — tones, code-switched English, and a built-in
+  normalizer that reads `31/12/2025` and `1.250 tỷ` the way a person would.
+* 📊 **Measured, not asserted** — every number below comes from
+  [ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)'s
+  own public scorer, on 59 held-out voices.
 
 * Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS**
 * Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS**
@@ -121,51 +141,98 @@ for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voic
 
 ## Benchmarks
 
-Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)**
-— 137 items, 59 held-out reference voices, 4 subsets — against the two public
+Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)** —
+137 items, 59 held-out reference voices × 4 subsets — against the two public
 Vietnamese XTTS-v2 finetunes. 137/137 scored, 0 empty generations.
 
-Scored by the benchmark's own published scorer (`zerobench_eval`), not by us.
-WER is the minimum over two ASRs (`whisper-large-v3` + `PhoWhisper-large`) and
-over every acceptable reading of the target text — see the
-[benchmark README](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)
-for why both matter.
+**Scored by the benchmark, not by us.** ZeroTTS synthesizes the clips and hands
+them to `zerobench_eval`, the official scorer published inside the benchmark
+dataset repo. Nothing in this repo computes a metric.
+
+### Headline
+
+Every system reads **normalized text** — dates, numbers and acronyms already
+spoken out, from the benchmark's own curated reading. Every system gets exactly
+the same input, so the comparison is like-for-like.
+
+This is the condition a Vietnamese TTS system meets in production, where a text
+frontend runs ahead of the model. ZeroTTS ships one — `normalize_vi_text`,
+applied by default (see the [GitHub README](https://github.com/zeroweight-ai/ZeroTTS#usage)) — which reaches the benchmark's reading
+on 27 of the 35 items that need normalization; the remaining gaps are date
+separators and alphanumeric codes. Neither baseline ships a Vietnamese frontend
+at all, which is why the raw-text table below is so much harsher on them.
 
 | | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
 |---|:-:|:-:|:-:|
-| **WER** ↓ | **1.03%** | 16.42% | 18.40% |
+| **WER** ↓ | **0.56 %** | 7.27 % | 8.61 % |
 | **Naturalness** (UTMOS) ↑ | **2.91** | 2.43 | 2.35 |
 | **Voice similarity** (SSIM) ↑ | 0.936 | **0.940** | 0.935 |
 | **Dead air** (excess silence) ↓ | **0.029 s** | 0.532 s | 0.233 s |
 
-**16× fewer word errors**, ~0.5 MOS more natural, an order of magnitude less
-dead air. Median WER is **0.00%** on all four subsets.
+**13× fewer word errors**, ~0.5 MOS more natural, an order of magnitude less
+dead air. Median WER is **0.00 %** on all four subsets — the typical generation
+is transcribed exactly.
 
-WER by subset:
+### WER — normalized text
+
+The headline condition: numbers and dates already spoken out, as the shipped
+normalizer produces.
 
 | Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
 |---|---|:-:|:-:|:-:|
-| `vietnamese` | plain Vietnamese | **0.16%** | 7.92% | 9.56% |
-| `code_switch` | Vietnamese + embedded English | **0.97%** | 10.94% | 9.25% |
-| `cross_lingual` | foreign voice prompt → Vietnamese | **1.42%** | 21.37% | 27.27% |
-| `challenging` | acronyms, dates, %, currency | **1.75%** | 27.86% | 31.85% |
+| `vietnamese` | monolingual Vietnamese | **0.21 %** | 7.21 % | 7.54 % |
+| `code_switch` | Vietnamese + embedded English | **0.95 %** | 10.14 % | 5.86 % |
+| `cross_lingual` | foreign voice prompt → Vietnamese | **0.38 %** | 4.94 % | 6.61 % |
+| `challenging` | acronyms, dates, %, currency | **0.61 %** | 5.63 % | 13.44 % |
+| **overall** | | **0.56 %** | **7.27 %** | **8.61 %** |
+
+### WER — raw text
 
-Reading these fairly:
+The harder condition: the model is handed `31/12/2025` and `ChatGPT` verbatim
+and has to read them itself, with no normalizer in front. This is what a system
+with no Vietnamese text frontend faces.
 
-* **Voice similarity is a tie, not a win** — 0.936 / 0.939 / 0.935 is within
+| Subset | what it tests | **ZeroTTS** | XTTS-v2-vietnamse | viXTTS |
+|---|---|:-:|:-:|:-:|
+| `vietnamese` | monolingual Vietnamese | **0.16 %** | 7.92 % | 9.56 % |
+| `code_switch` | Vietnamese + embedded English | **0.97 %** | 10.94 % | 9.25 % |
+| `cross_lingual` | foreign voice prompt → Vietnamese | **1.42 %** | 21.37 % | 27.27 % |
+| `challenging` | acronyms, dates, %, currency | **1.75 %** | 27.86 % | 31.85 % |
+| **overall** | | **1.03 %** | **16.42 %** | **18.40 %** |
+
+**Reading these fairly:**
+
+* **Normalization is where the baselines gain most, and we still win.** Their
+  tokenizers genuinely have no Vietnamese number expansion, so raw text punishes
+  them hard (`challenging` 27.86 %) and the normalized column is the fairest
+  comparison available — it improves XTTS 2.3× and viXTTS 2.1×, against 1.8× for
+  us. The gap narrows from 16× to 13× and stops there, because what remains is
+  the acoustic model.
+* **`vietnamese` barely moves for anyone** (0.16 % → 0.21 % for ZeroTTS). It has
+  no digits or acronyms, so there is nothing to normalize — which is the control
+  showing the other subsets' gains are real and not a scoring artifact.
+* **Voice similarity is a tie, not a win.** 0.936 / 0.940 / 0.935 is within
   noise. On `cross_lingual` ZeroTTS is genuinely behind (0.911 vs ~0.935): it
-  carries a foreign speaker's timbre into Vietnamese less faithfully than the
-  XTTS backbone, while winning that subset's WER by 15×.
-* **"The baselines just need text normalization" is testable, and fails.**
-  Given the spoken-out text instead of raw orthography, XTTS improves to 7.27%
-  and viXTTS to 8.61% — big gains, confirming their tokenizers lack Vietnamese
-  number expansion — but ZeroTTS improves to 0.56%, so the gap stays 13–15×.
-* **ZeroTTS's own remaining errors are known and listed**, not hidden:
-  [evaluation/HIGH_WER_ANALYSIS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/HIGH_WER_ANALYSIS.md)
-  audits every item above 0.00 — mostly voiced leading zeros in dates and
-  `W`/`H` acronym letter names.
-
-Score your own model on the same benchmark:
+  carries a foreign speaker's timbre into Vietnamese slightly less faithfully,
+  while winning that subset's WER by 15×.
+* **The WER definition matters more than the WER.** ZeroBench scores every clip
+  with **two ASRs** (`whisper-large-v3` + `PhoWhisper-large`, min taken — neither
+  can judge Vietnamese code-switch TTS alone) against **every acceptable
+  reading** of the target text. Its test suite pins the policy in both
+  directions: format artifacts must score 0, real defects must still cost.
+* **Our remaining errors are published, not hidden.** Every item scoring above
+  0.00 is audited in [docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md) — mostly voiced
+  leading zeros in dates and `W`/`H` acronym letter names.
+
+Reproduce, or score your own system:
+
+```bash
+pip install "zerotts[eval]"
+SYNTH_FROM=text_normalized OUT_DIR=./eval/norm ./evaluation/run_benchmark.sh
+./evaluation/run_benchmark.sh                                   # raw text
+```
+
+Not using ZeroTTS? The scorer stands alone — bring wavs from any system:
 
 ```bash
 huggingface-cli download zeroweight-ai/ZeroBench-TTS --repo-type dataset --local-dir ZeroBench-TTS
@@ -174,9 +241,6 @@ python -m zerobench_eval manifest --out manifest.jsonl    # what to synthesize
 python -m zerobench_eval score --wav_dir my_wavs/ --name MyModel
 ```
 
-Full tables and reproduction commands:
-[evaluation/RESULTS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/evaluation/RESULTS.md).
-
 ## Voices, and voice cloning
 
 A voice is a small array of speaker latents, `(1, n_voice_queries, d_model)`,