---
license: mit
library_name: onnx
tags:
  - text-to-speech
  - tts
  - hebrew
  - multilingual
  - flow-matching
  - onnx
  - onnxruntime
pipeline_tag: text-to-speech
---

# BlueV3 — ONNX Runtime Bundle

ONNX export of BlueV3 TTS for CPU / CUDA / TensorRT inference. Includes the **vocoder (codec decoder)**.

**TTS version:** `v1.7.3` · **Sample rate:** 44.1 kHz · Exported from PyTorch `ckpt_step_767000` + AE `ae_541000`

## Contents

| File | Role |
|------|------|
| `text_encoder.onnx` | Phoneme IDs + style → text embedding |
| `vector_estimator.onnx` | Flow-matching Euler step (CFG baked in) |
| `vocoder.onnx` | Latent → 44.1 kHz waveform (**codec**) |
| `duration_predictor.onnx` | Text + `style_dp` → duration (seconds) |
| `stats.npz` | Latent `mean` / `std` / `normalizer_scale` |
| `uncond.npz` | Unconditional tokens (CFG / debugging) |
| `tts.json` | Runtime config |

## Companion repo

PyTorch weights (no codec): [`notmax123/BlueV3`](https://huggingface.co/notmax123/BlueV3)

## Download

```bash
hf download notmax123/BlueV3-onnx --local-dir ./onnx_models
```

## Inference contract

1. **Style** — provide `style_ttl` `[1, 50, 256]` and `style_dp` `[1, 8, 16]` (from a style JSON / reference encoder).
2. **Text** — IPA / phoneme string → `text_ids`, `text_mask`.
3. **Duration** — `duration_predictor` → seconds; divide by `speed`; convert to latent length with `base_chunk_size=512`, `chunk_compress_factor=6`.
4. **Encode** — `text_encoder(text_ids, style_ttl, text_mask)` → `text_emb`.
5. **Sample** — loop `vector_estimator` for `N` steps (e.g. 8). Output is the next latent state (CFG is **inside** the graph; do not apply CFG again).
6. **Decode** — unnormalize with `stats.npz`:
   ```text
   z = (x / normalizer_scale) * std + mean   # raw 144-d
   ```
   optionally drop the last compressed frame, then `vocoder(latent=z)` → `wav_tts`.

`vocoder.onnx` expects **raw (unnormalized) 144-channel** latents (`normalizer_scale=1` inside the export).

## Example (this codebase)

```bash
# ONNX (ORT CUDA / CPU)
uv run python run_onnx_inference.py --onnx_dir ./onnx_models --speaker netsiga --steps 8

# TensorRT (after building engines from these ONNX files)
uv run python create_tensorrt.py --onnx_dir ./onnx_models --engine_dir trt_engines
uv run python benchmark_trt.py --style_json voice_styles/Rotem.json --steps 8 --out out.wav
```

## I/O cheat sheet

**text_encoder:** `text_ids`, `style_ttl`, `text_mask` → `text_emb`  
**vector_estimator:** `noisy_latent`, `text_emb`, `style_ttl`, `latent_mask`, `text_mask`, `current_step`, `total_step` → `denoised_latent`  
**duration_predictor:** `text_ids`, `style_dp`, `text_mask` → `duration`  
**vocoder:** `latent` `[B, 144, T]` → `wav_tts`

## License

MIT (see frontmatter).