v2stream_streaming/README.md

# PrimeTTS v2-Stream — token-level streaming zh-TW/English TTS

A **streaming** MB-iSTFT-VITS (Xinran voice, 16 kHz): band-attention encoder +
causal FFN + **causal MB-iSTFT vocoder**, so audio can be emitted incrementally
in fixed chunks. Bit-exact between whole-utterance and chunked inference.

Streaming params: **chunk = 24 frames (~384 ms), left cache = 64, right lookahead = 4**
(1 frame = 256 samples @ 16 kHz). First-audio ≈ enc(once) + one chunk.

> This is a **streaming-capable** variant of PrimeTTS v2 (a distinct checkpoint,
> not v2 run in a streaming loop). The causal + limited-lookahead architecture
> trades some quality for streamability vs the non-streaming v2.

## Files
| file | purpose |
|---|---|
| `v2stream_enc.onnx` | text (x,tone,lang,x_lengths,noise_scale,length_scale) → z[1,192,T]. Run **once**. |
| `v2stream_dec.onnx` | z[1,192,Tc] → wav[1,1,Tc·256]. Run **per chunk** (overlap-save). |
| `primetts_v2stream_xinran_f32.gguf` | single-graph gguf for RapidSpeech.cpp ggml (`MBISTFT_STREAM=1`). |
| `onnx_stream.py` | reference streaming runner (ORT). |
| `meta.json` | params + frontend convention. |

## Run A — ONNX Runtime (Python, torch-free)
```python
from onnx_stream import StreamingTTS      # in this repo
tts = StreamingTTS("v2stream_enc.onnx", "v2stream_dec.onnx")
z = tts.encode(phone_ids, tone_ids, lang_ids)   # once
for pcm_chunk in tts.stream(z):                  # per 24-frame chunk
    play(pcm_chunk)                              # 16 kHz float32
```
`phone_ids/tone_ids/lang_ids` come from the **frontend** (text → ids): g2pw
(bopomofo, Taiwan readings) + g2p_en (arpabet), 88 syms / 6 tones / 2 langs,
with `add_blank` (0 interleaved between tokens). See `frontend_bopomofo` in the
[demo Space](https://huggingface.co/spaces/Luigi/PrimeTTS-Streaming).

## Run B — sherpa-onnx fork (C++/ORT)
`OfflineTtsMbistftStreamModel(enc=..., dec=..., num_threads=2).generate(x,tone,lang,
callback=...)` emits audio per chunk via the callback. Branch:
[vieenrose/sherpa-onnx @ mbistft-streaming-tts](https://github.com/vieenrose/sherpa-onnx/tree/mbistft-streaming-tts).

## Run C — RapidSpeech.cpp (ggml, GPU/Jetson)
`MBISTFT_STREAM=1 MBISTFT_STREAM_LOOKAHEAD=5 mbistft-parity primetts_v2stream_xinran_f32.gguf
inputs.json out --stream-chunks`. Branch:
[vieenrose/RapidSpeech.cpp @ jetson-nano-gen1](https://github.com/vieenrose/RapidSpeech.cpp).
Validated on Jetson Nano (Tegra X1): parity cosine > 0.99, first-audio ~285 ms.

License: Apache-2.0. Part of the [Luigi/PrimeTTS](https://huggingface.co/Luigi/PrimeTTS) family.