Update README
#12
by zeroweightai - opened
- .gitattributes +1 -0
- .gitignore +1 -0
- README.md +55 -9
- voices/thanhtruc/preview.wav → samples/conversation.mp3 +2 -2
- voices/thanhtruc/voice.npz → samples/cross-lingual-english.mp3 +2 -2
- voices/thanhtruc/voice.bin → samples/cross-lingual-reference-vi.mp3 +2 -2
- samples/news-code-switch.mp3 +3 -0
- samples/storytelling.mp3 +3 -0
- voices/hamy/preview.wav +2 -2
- voices/hamy/voice.bin +1 -1
- voices/hamy/voice.npz +1 -1
- voices/index.json +1 -21
- voices/maichi/preview.wav +2 -2
- voices/thanhtruc/meta.json +0 -15
.gitattributes
CHANGED
|
@@ -4,4 +4,5 @@
|
|
| 4 |
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 5 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 6 |
*.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 7 |
banner.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 4 |
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 5 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 6 |
*.wav filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.mp3 filter=lfs diff=lfs merge=lfs -text
|
| 8 |
banner.png filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
*.DS_Store
|
README.md
CHANGED
|
@@ -96,8 +96,8 @@ model-index:
|
|
| 96 |
|
| 97 |
### Vietnamese Zero-Shot Text-to-Speech (TTS) with real-time streaming and voice cloning from seconds of audio. Fast, natural, and optimised for CPU inference.
|
| 98 |
|
| 99 |
-
**The most accurate open Vietnamese TTS we know of —
|
| 100 |
-
the next
|
| 101 |
|
| 102 |
* 🎯 **Ultra-natural** — 2.91 UTMOS, ~0.5 MOS above every other open Vietnamese
|
| 103 |
system, with near-zero dead air (0.029 s vs 0.23–0.53 s).
|
|
@@ -107,11 +107,30 @@ the next open model**, and it runs faster than real time on a laptop CPU.
|
|
| 107 |
* ⚡ **Real-time on CPU, streaming** — ~2× faster than real time (RTF 0.5×),
|
| 108 |
first audio chunk in ~70 ms. No GPU required.
|
| 109 |
* 🇻🇳 **Built for Vietnamese** — tones, code-switched English, and a built-in
|
| 110 |
-
normalizer that reads `31/12/2025` and `
|
| 111 |
|
| 112 |
* Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS**
|
| 113 |
* Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS**
|
| 114 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
```python
|
| 116 |
pip install zerotts
|
| 117 |
```
|
|
@@ -127,8 +146,33 @@ tts.save_audio(audio, "out.wav")
|
|
| 127 |
Streaming, with first audio in roughly 70 ms:
|
| 128 |
|
| 129 |
```python
|
| 130 |
-
|
| 131 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
```
|
| 133 |
|
| 134 |
## Benchmarks
|
|
@@ -146,11 +190,13 @@ spoken out, from the benchmark's own curated reading.
|
|
| 146 |
| **Dead air** (excess silence) ↓ | **0.029 s** | 0.386 s | 0.568 s | 0.215 s |
|
| 147 |
| **RTF, CPU** ↓ | **0.50×** | 6.12× | 0.71× | 0.73× |
|
| 148 |
| **Time to first audio, CPU** ↓ | **~70 ms** | ~34 s | ~6.1 s | ~5.1 s |
|
| 149 |
-
| Size | **
|
| 150 |
|
| 151 |
-
**4× fewer word errors than the next-best system**, and the
|
| 152 |
-
|
| 153 |
-
|
|
|
|
|
|
|
| 154 |
|
| 155 |
Full comparison tables, per-subset breakdowns, and CPU speed methodology:
|
| 156 |
**[docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md)**
|
|
|
|
| 96 |
|
| 97 |
### Vietnamese Zero-Shot Text-to-Speech (TTS) with real-time streaming and voice cloning from seconds of audio. Fast, natural, and optimised for CPU inference.
|
| 98 |
|
| 99 |
+
**The most accurate open Vietnamese TTS we know of — 4× fewer word errors than
|
| 100 |
+
the next best model**, and it runs faster than real time on a laptop CPU.
|
| 101 |
|
| 102 |
* 🎯 **Ultra-natural** — 2.91 UTMOS, ~0.5 MOS above every other open Vietnamese
|
| 103 |
system, with near-zero dead air (0.029 s vs 0.23–0.53 s).
|
|
|
|
| 107 |
* ⚡ **Real-time on CPU, streaming** — ~2× faster than real time (RTF 0.5×),
|
| 108 |
first audio chunk in ~70 ms. No GPU required.
|
| 109 |
* 🇻🇳 **Built for Vietnamese** — tones, code-switched English, and a built-in
|
| 110 |
+
normalizer that reads `31/12/2025` and `ZeroTTS` the way a person would.
|
| 111 |
|
| 112 |
* Code, examples, browser demo: **https://github.com/zeroweight-ai/ZeroTTS**
|
| 113 |
* Benchmark dataset: **https://huggingface.co/datasets/zeroweight-ai/ZeroBench-TTS**
|
| 114 |
|
| 115 |
+
## Samples
|
| 116 |
+
|
| 117 |
+
**Hội thoại hai giọng**
|
| 118 |
+
|
| 119 |
+
<audio controls><source src="https://huggingface.co/zeroweight-ai/ZeroTTS/resolve/main/samples/conversation.mp3" type="audio/mpeg"></audio>
|
| 120 |
+
|
| 121 |
+
**Kể chuyện, dạng dài**
|
| 122 |
+
|
| 123 |
+
<audio controls><source src="https://huggingface.co/zeroweight-ai/ZeroTTS/resolve/main/samples/storytelling.mp3" type="audio/mpeg"></audio>
|
| 124 |
+
|
| 125 |
+
**Bản tin, chèn tiếng Anh**
|
| 126 |
+
|
| 127 |
+
<audio controls><source src="https://huggingface.co/zeroweight-ai/ZeroTTS/resolve/main/samples/news-code-switch.mp3" type="audio/mpeg"></audio>
|
| 128 |
+
|
| 129 |
+
**Xuyên ngôn ngữ — giọng tham chiếu tiếng Việt, đọc tiếng Anh**
|
| 130 |
+
|
| 131 |
+
<audio controls><source src="https://huggingface.co/zeroweight-ai/ZeroTTS/resolve/main/samples/cross-lingual-reference-vi.mp3" type="audio/mpeg"></audio>
|
| 132 |
+
<audio controls><source src="https://huggingface.co/zeroweight-ai/ZeroTTS/resolve/main/samples/cross-lingual-english.mp3" type="audio/mpeg"></audio>
|
| 133 |
+
|
| 134 |
```python
|
| 135 |
pip install zerotts
|
| 136 |
```
|
|
|
|
| 146 |
Streaming, with first audio in roughly 70 ms:
|
| 147 |
|
| 148 |
```python
|
| 149 |
+
import queue
|
| 150 |
+
|
| 151 |
+
import numpy as np
|
| 152 |
+
import sounddevice as sd # pip install sounddevice
|
| 153 |
+
|
| 154 |
+
TEXT = ("Đây là chế độ phát trực tuyến. Âm thanh được tạo ra và phát ngay lập tức, "
|
| 155 |
+
"không cần chờ toàn bộ đoạn văn hoàn thành. Nhờ vậy, người nghe chỉ mất "
|
| 156 |
+
"khoảng 70 mili giây là đã nghe thấy câu đầu tiên, ngay cả khi mô "
|
| 157 |
+
"hình đang chạy trên CPU của một chiếc laptop bình thường.")
|
| 158 |
+
|
| 159 |
+
pending, tail = queue.Queue(), np.zeros(0, dtype="float32")
|
| 160 |
+
|
| 161 |
+
def feed(outdata, frames, _time, _status):
|
| 162 |
+
global tail
|
| 163 |
+
while len(tail) < frames and not pending.empty():
|
| 164 |
+
tail = np.concatenate([tail, pending.get_nowait()])
|
| 165 |
+
n = min(frames, len(tail))
|
| 166 |
+
outdata[:n, 0] = tail[:n]
|
| 167 |
+
outdata[n:] = 0
|
| 168 |
+
tail = tail[n:]
|
| 169 |
+
|
| 170 |
+
with sd.OutputStream(samplerate=tts.sample_rate, channels=1,
|
| 171 |
+
dtype="float32", callback=feed):
|
| 172 |
+
for chunk in tts.synthesize_stream(TEXT, voice="maichi"):
|
| 173 |
+
pending.put(chunk.reshape(-1)) # chunk is (1, n) float32 at 48 kHz
|
| 174 |
+
while not pending.empty() or len(tail):
|
| 175 |
+
sd.sleep(50) # let the buffer drain before closing
|
| 176 |
```
|
| 177 |
|
| 178 |
## Benchmarks
|
|
|
|
| 190 |
| **Dead air** (excess silence) ↓ | **0.029 s** | 0.386 s | 0.568 s | 0.215 s |
|
| 191 |
| **RTF, CPU** ↓ | **0.50×** | 6.12× | 0.71× | 0.73× |
|
| 192 |
| **Time to first audio, CPU** ↓ | **~70 ms** | ~34 s | ~6.1 s | ~5.1 s |
|
| 193 |
+
| Size | **202 M** params, 0.86 GB fp32, CPU | 3.1 GB, GPU | 1.9 GB, GPU | 1.9 GB, GPU |
|
| 194 |
|
| 195 |
+
**4× fewer word errors than the next-best system**, and the fastest of the four
|
| 196 |
+
on CPU. The gap is much wider in latency than in throughput: the two XTTS
|
| 197 |
+
fine-tunes also beat real time (0.71×) but need seconds to emit their first
|
| 198 |
+
sample, while OmniVoice is 6× *slower* than real time. All three are sized and
|
| 199 |
+
tuned for a GPU, and it shows.
|
| 200 |
|
| 201 |
Full comparison tables, per-subset breakdowns, and CPU speed methodology:
|
| 202 |
**[docs/BENCHMARKS.md](https://github.com/zeroweight-ai/ZeroTTS/blob/main/docs/BENCHMARKS.md)**
|
voices/thanhtruc/preview.wav → samples/conversation.mp3
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac110a35c09efbdfd91354c53330df511d9eaf9faf21a378c94fc4f15f1efc34
|
| 3 |
+
size 188588
|
voices/thanhtruc/voice.npz → samples/cross-lingual-english.mp3
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3a318f85ed126ab59f17f22ac76552a42c5240f5c445270892630513a2db62c7
|
| 3 |
+
size 263468
|
voices/thanhtruc/voice.bin → samples/cross-lingual-reference-vi.mp3
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b019bb424b186aaf26c721eb2a7ed68eeb3ca9cff8f3c3fe062151fbaaea003
|
| 3 |
+
size 34220
|
samples/news-code-switch.mp3
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be58f3701e6169d33ee23502b0b4674be269f03c6ae113035b671a3d20f9783a
|
| 3 |
+
size 544172
|
samples/storytelling.mp3
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f0ab526814b46d0bd6c9fdf1026338dc84089cf7e7d9da5552d334ba972d7775
|
| 3 |
+
size 971564
|
voices/hamy/preview.wav
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4bc33835023d7786bc75cb6c5c2681b1aec1ca3fb01300044b30d2161b29a171
|
| 3 |
+
size 460844
|
voices/hamy/voice.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 30720
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e137f9544c20c491b3d139aeebb1d8549f8bd6f84f035a2df78830d3baec025d
|
| 3 |
size 30720
|
voices/hamy/voice.npz
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 31262
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a864c127386f74976154c840875f6ad615d74c377768bc6edcb75e3278f228b7
|
| 3 |
size 31262
|
voices/index.json
CHANGED
|
@@ -159,26 +159,6 @@
|
|
| 159 |
10,
|
| 160 |
768
|
| 161 |
]
|
| 162 |
-
},
|
| 163 |
-
{
|
| 164 |
-
"name": "thanhtruc",
|
| 165 |
-
"display_name": "Thanh Trúc",
|
| 166 |
-
"language": "vi",
|
| 167 |
-
"gender": "nữ",
|
| 168 |
-
"description": "nữ, trẻ, dịu dàng, thủ thỉ, nhẹ",
|
| 169 |
-
"tags": [
|
| 170 |
-
"nữ",
|
| 171 |
-
"trẻ",
|
| 172 |
-
"dịu dàng",
|
| 173 |
-
"thủ thỉ",
|
| 174 |
-
"nhẹ"
|
| 175 |
-
],
|
| 176 |
-
"n_voice_queries": 10,
|
| 177 |
-
"shape": [
|
| 178 |
-
1,
|
| 179 |
-
10,
|
| 180 |
-
768
|
| 181 |
-
]
|
| 182 |
}
|
| 183 |
]
|
| 184 |
-
}
|
|
|
|
| 159 |
10,
|
| 160 |
768
|
| 161 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
}
|
| 163 |
]
|
| 164 |
+
}
|
voices/maichi/preview.wav
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:987b84f2287607d58d08b4979f0bf5387a8235bc2eff28263065de34481caa67
|
| 3 |
+
size 491564
|
voices/thanhtruc/meta.json
DELETED
|
@@ -1,15 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"name": "thanhtruc",
|
| 3 |
-
"display_name": "Thanh Trúc",
|
| 4 |
-
"language": "vi",
|
| 5 |
-
"gender": "nữ",
|
| 6 |
-
"description": "nữ, trẻ, dịu dàng, thủ thỉ, nhẹ",
|
| 7 |
-
"tags": [
|
| 8 |
-
"nữ",
|
| 9 |
-
"trẻ",
|
| 10 |
-
"dịu dàng",
|
| 11 |
-
"thủ thỉ",
|
| 12 |
-
"nhẹ"
|
| 13 |
-
],
|
| 14 |
-
"n_voice_queries": 10
|
| 15 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|