README.md

---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license
language: [ar, de, en, es, fr, hi, it, ja, ko, pt, vi, zh]
library_name: phoonnx
pipeline_tag: text-to-speech
tags: [onnx, tts, magpie, nvidia, multilingual, phoonnx]
base_model: nvidia/magpie_tts_multilingual_357m
---

# phoonnx-magpie-tts

ONNX export of [nvidia/magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m)
(MagpieTTS Multilingual, v2607) for use with [phoonnx](https://github.com/TigreGotico/phoonnx).

The original model is a transformer encoder-decoder that predicts discrete audio codec
tokens autoregressively. It uses 8 codebooks, a frame stacking factor of 2, and a local
transformer that refines each stacked frame. A frozen
[NanoCodec](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps)
decodes the tokens into a 22.05 kHz waveform.

Weights and architecture are NVIDIA's. This repository only changes the file format.

## License and attribution

The model is released under the
[NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license).
That license applies to every file here. Developed by NVIDIA. See the
[original model card](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) for the
full model card, dataset list, and limitations.

## Contents

| File | Purpose |
|---|---|
| `text_encoder.onnx` | 6-layer causal text encoder. Text token ids to encoder states. |
| `cross_kv.onnx` | Per-layer cross-attention K/V, computed once from the encoder states. |
| `decoder_step.onnx` | 12-layer causal decoder with external KV cache. Returns codebook logits, decoder states, the updated cache, and per-layer cross-attention probabilities. |
| `local_step.onnx` | One step of the 2-layer local transformer that refines the 16 stacked codebook tokens. |
| `audio_embed.onnx` | Stacked codebook tokens to one decoder input embedding. |
| `lt_embed.onnx` | One codebook token to the next local-transformer input. |
| `codec_decoder.onnx` | NanoCodec decoder. Codes to waveform. |
| `assets/context_embeddings.npy` | Baked speaker context embeddings, shape (5, 217, 768). |
| `assets/tokenizer.json` | Aggregated tokenizer vocabulary and per-language offsets. |
| `assets/tokenizer/` | IPA dictionaries and heteronym lists shipped inside the original checkpoint. |
| `config.json` | Graph inventory and IO contract. |

All graphs are opset 18 and use external data files (`*.onnx.data`). Download the whole
repository, not single files.

## Two decode modes

NeMo ships `use_kv_cache_for_inference: false`. In that mode the newest attention prior is
re-applied over the whole history, so the decoder states must be recomputed every step. A
KV-cached decoder does not produce the same tokens.

`decoder_step.onnx` supports both:

- **exact** - pass the full sequence with an empty cache every step. Matches the NeMo
  default.
- **cached** - pass one frame plus the cache. Matches NeMo with
  `use_kv_cache_for_inference: true`. Roughly 2.5x faster.

Both reach 100% greedy token agreement with the matching NeMo reference across all 8
codebooks.

## Usage

```python
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
voice = manager.get_voice(magpie/Leo/en)
wav = voice.synthesize(Hello from Magpie running on ONNX.)
```

## Related

- [phoonnx](https://github.com/TigreGotico/phoonnx) - the ONNX TTS runtime that consumes this repository
- [nvidia/magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m) - the source checkpoint
- [nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps) - the codec