File size: 2,582 Bytes
f34bd6d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | # PrimeTTS v2-Stream — token-level streaming zh-TW/English TTS
A **streaming** MB-iSTFT-VITS (Xinran voice, 16 kHz): band-attention encoder +
causal FFN + **causal MB-iSTFT vocoder**, so audio can be emitted incrementally
in fixed chunks. Bit-exact between whole-utterance and chunked inference.
Streaming params: **chunk = 24 frames (~384 ms), left cache = 64, right lookahead = 4**
(1 frame = 256 samples @ 16 kHz). First-audio ≈ enc(once) + one chunk.
> This is a **streaming-capable** variant of PrimeTTS v2 (a distinct checkpoint,
> not v2 run in a streaming loop). The causal + limited-lookahead architecture
> trades some quality for streamability vs the non-streaming v2.
## Files
| file | purpose |
|---|---|
| `v2stream_enc.onnx` | text (x,tone,lang,x_lengths,noise_scale,length_scale) → z[1,192,T]. Run **once**. |
| `v2stream_dec.onnx` | z[1,192,Tc] → wav[1,1,Tc·256]. Run **per chunk** (overlap-save). |
| `primetts_v2stream_xinran_f32.gguf` | single-graph gguf for RapidSpeech.cpp ggml (`MBISTFT_STREAM=1`). |
| `onnx_stream.py` | reference streaming runner (ORT). |
| `meta.json` | params + frontend convention. |
## Run A — ONNX Runtime (Python, torch-free)
```python
from onnx_stream import StreamingTTS # in this repo
tts = StreamingTTS("v2stream_enc.onnx", "v2stream_dec.onnx")
z = tts.encode(phone_ids, tone_ids, lang_ids) # once
for pcm_chunk in tts.stream(z): # per 24-frame chunk
play(pcm_chunk) # 16 kHz float32
```
`phone_ids/tone_ids/lang_ids` come from the **frontend** (text → ids): g2pw
(bopomofo, Taiwan readings) + g2p_en (arpabet), 88 syms / 6 tones / 2 langs,
with `add_blank` (0 interleaved between tokens). See `frontend_bopomofo` in the
[demo Space](https://huggingface.co/spaces/Luigi/PrimeTTS-Streaming).
## Run B — sherpa-onnx fork (C++/ORT)
`OfflineTtsMbistftStreamModel(enc=..., dec=..., num_threads=2).generate(x,tone,lang,
callback=...)` emits audio per chunk via the callback. Branch:
[vieenrose/sherpa-onnx @ mbistft-streaming-tts](https://github.com/vieenrose/sherpa-onnx/tree/mbistft-streaming-tts).
## Run C — RapidSpeech.cpp (ggml, GPU/Jetson)
`MBISTFT_STREAM=1 MBISTFT_STREAM_LOOKAHEAD=5 mbistft-parity primetts_v2stream_xinran_f32.gguf
inputs.json out --stream-chunks`. Branch:
[vieenrose/RapidSpeech.cpp @ jetson-nano-gen1](https://github.com/vieenrose/RapidSpeech.cpp).
Validated on Jetson Nano (Tegra X1): parity cosine > 0.99, first-audio ~285 ms.
License: Apache-2.0. Part of the [Luigi/PrimeTTS](https://huggingface.co/Luigi/PrimeTTS) family.
|