Update README benchmarks section with full comparison and CPU speed tables

#8
Files changed (1) hide show
  1. README.md +40 -6
README.md CHANGED
@@ -134,15 +134,49 @@ for chunk in tts.synthesize_stream("Một đoạn văn bản dài hơn…", voic
134
  ## Benchmarks
135
 
136
  Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)**
137
- against OmniVoice, XTTS-v2-vietnamse and viXTTS, on normalized text (dates,
138
- numbers and acronyms already spoken out): **0.56 % WER** (4× fewer errors than
139
- the next-best system, median 0.00 % on all four subsets), **2.91 UTMOS**, and
140
- **0.029 s** of dead air. On CPU it runs ~2× faster than real time (RTF 0.5×)
141
- with first audio in ~70 ms of streaming.
 
 
 
 
 
 
 
 
 
 
 
 
142
 
143
  Full comparison tables, per-subset breakdowns, and CPU speed methodology:
144
  **[docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md)**
145
- on GitHub.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
 
147
  ## Voices, and voice cloning
148
 
 
134
  ## Benchmarks
135
 
136
  Measured on **[ZeroBench-TTS](https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS)**
137
+
138
+ Every system reads **normalized text** dates, numbers and acronyms already
139
+ spoken out, from the benchmark's own curated reading.
140
+
141
+ | | **ZeroTTS** | OmniVoice | XTTS-v2-vietnamse | viXTTS |
142
+ |---|:-:|:-:|:-:|:-:|
143
+ | **WER** ↓ | **0.56 %** | 2.12 % | 7.27 % | 8.61 % |
144
+ | **Naturalness** (UTMOS) ↑ | **2.91** | 2.75 | 2.49 | 2.34 |
145
+ | **Voice similarity** (SSIM) ↑ | 0.938 | **0.951** | 0.941 | 0.935 |
146
+ | **Dead air** (excess silence) ↓ | **0.029 s** | 0.386 s | 0.568 s | 0.215 s |
147
+ | **RTF, CPU** ↓ | **0.50×** | 6.12× | 0.71× | 0.73× |
148
+ | **Time to first audio, CPU** ↓ | **~70 ms** | ~34 s | ~6.1 s | ~5.1 s |
149
+ | Size | **81 M**, CPU | 3.1 GB, GPU | 1.9 GB, GPU | 1.9 GB, GPU |
150
+
151
+ **4× fewer word errors than the next-best system**, and the only one of the four
152
+ that's actually fast on CPU — the others are sized and tuned for a GPU, and it
153
+ shows.
154
 
155
  Full comparison tables, per-subset breakdowns, and CPU speed methodology:
156
  **[docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md)**
157
+
158
+ ### Speed — CPU
159
+
160
+ RTF (realtime factor, wall-clock synthesis time ÷ output audio duration — lower
161
+ is faster; below 1× is faster than real time) and time-to-first-audio, all
162
+ measured **on CPU**, single request, 8 inference threads pinned to a dedicated
163
+ core pool (no other synthesis running concurrently). Three Vietnamese samples —
164
+ short (26 chars), medium (77 chars), long (227 chars) — each run 6 times with
165
+ the first 2 (cold-cache) discarded; figures below are the mean of the
166
+ remaining 4.
167
+
168
+ | | **ZeroTTS** | OmniVoice | XTTS-v2-vietnamse | viXTTS |
169
+ |---|:-:|:-:|:-:|:-:|
170
+ | RTF — short | **0.51×** | 10.87× | 0.70× | 0.71× |
171
+ | RTF — medium | **0.47×** | 4.82× | 0.70× | 0.70× |
172
+ | RTF — long | **0.53×** | 2.67× | 0.71× | 0.78× |
173
+ | TTFA — short | **53 ms** | 21.7 s | 4.02 s | 2.45 s |
174
+ | TTFA — medium | **66 ms** | 28.9 s | 4.02 s | 3.72 s |
175
+ | TTFA — long | **89 ms** | 52.3 s | 10.3 s | 9.22 s |
176
+
177
+ ZeroTTS's time-to-first-audio comes from its real streaming path — first audio frame,
178
+ not first full utterance. The three baselines have no working CPU streaming
179
+ path, so their TTFA is the time to the complete utterance.
180
 
181
  ## Voices, and voice cloning
182