| --- |
| license: mit |
| library_name: onnx |
| tags: |
| - text-to-speech |
| - tts |
| - hebrew |
| - multilingual |
| - flow-matching |
| - onnx |
| - onnxruntime |
| pipeline_tag: text-to-speech |
| --- |
| |
| # BlueV3 — ONNX Runtime Bundle |
|
|
| ONNX export of BlueV3 TTS for CPU / CUDA / TensorRT inference. Includes the **vocoder (codec decoder)**. |
|
|
| **TTS version:** `v1.7.3` · **Sample rate:** 44.1 kHz · Exported from PyTorch `ckpt_step_767000` + AE `ae_541000` |
|
|
| ## Contents |
|
|
| | File | Role | |
| |------|------| |
| | `text_encoder.onnx` | Phoneme IDs + style → text embedding | |
| | `vector_estimator.onnx` | Flow-matching Euler step (CFG baked in) | |
| | `vocoder.onnx` | Latent → 44.1 kHz waveform (**codec**) | |
| | `duration_predictor.onnx` | Text + `style_dp` → duration (seconds) | |
| | `stats.npz` | Latent `mean` / `std` / `normalizer_scale` | |
| | `uncond.npz` | Unconditional tokens (CFG / debugging) | |
| | `tts.json` | Runtime config | |
|
|
| ## Companion repo |
|
|
| PyTorch weights (no codec): [`notmax123/BlueV3`](https://huggingface.co/notmax123/BlueV3) |
|
|
| ## Download |
|
|
| ```bash |
| hf download notmax123/BlueV3-onnx --local-dir ./onnx_models |
| ``` |
|
|
| ## Inference contract |
|
|
| 1. **Style** — provide `style_ttl` `[1, 50, 256]` and `style_dp` `[1, 8, 16]` (from a style JSON / reference encoder). |
| 2. **Text** — IPA / phoneme string → `text_ids`, `text_mask`. |
| 3. **Duration** — `duration_predictor` → seconds; divide by `speed`; convert to latent length with `base_chunk_size=512`, `chunk_compress_factor=6`. |
| 4. **Encode** — `text_encoder(text_ids, style_ttl, text_mask)` → `text_emb`. |
| 5. **Sample** — loop `vector_estimator` for `N` steps (e.g. 8). Output is the next latent state (CFG is **inside** the graph; do not apply CFG again). |
| 6. **Decode** — unnormalize with `stats.npz`: |
| ```text |
| z = (x / normalizer_scale) * std + mean # raw 144-d |
| ``` |
| optionally drop the last compressed frame, then `vocoder(latent=z)` → `wav_tts`. |
|
|
| `vocoder.onnx` expects **raw (unnormalized) 144-channel** latents (`normalizer_scale=1` inside the export). |
|
|
| ## Example (this codebase) |
|
|
| ```bash |
| # ONNX (ORT CUDA / CPU) |
| uv run python run_onnx_inference.py --onnx_dir ./onnx_models --speaker netsiga --steps 8 |
| |
| # TensorRT (after building engines from these ONNX files) |
| uv run python create_tensorrt.py --onnx_dir ./onnx_models --engine_dir trt_engines |
| uv run python benchmark_trt.py --style_json voice_styles/Rotem.json --steps 8 --out out.wav |
| ``` |
|
|
| ## I/O cheat sheet |
|
|
| **text_encoder:** `text_ids`, `style_ttl`, `text_mask` → `text_emb` |
| **vector_estimator:** `noisy_latent`, `text_emb`, `style_ttl`, `latent_mask`, `text_mask`, `current_step`, `total_step` → `denoised_latent` |
| **duration_predictor:** `text_ids`, `style_dp`, `text_mask` → `duration` |
| **vocoder:** `latent` `[B, 144, T]` → `wav_tts` |
|
|
| ## License |
|
|
| MIT (see frontmatter). |
|
|
