Export formats and quantization
Release formats
| Format | Release status | Notes |
|---|---|---|
| PyTorch FP32 | Supported | Canonical weights and the fully tested runtime. |
| PyTorch FP16/BF16 | Not released | Smaller in memory, but not validated for waveform quality or CPU compatibility. |
| ONNX FP32 | Supported | Separate official Inflect-Nano-v2-ONNX repository with verified dynamic-length graphs, a torch-free Python runner, and a reproducible exporter. |
| INT8 / INT4 | Not released | Naive weight quantization can audibly damage the integrated waveform decoder. |
| GGUF | Not applicable today | GGUF targets transformer/LLM runtimes and is not a drop-in container for this VITS-family convolutional waveform model. |
| Core ML / TFLite | Not released | Requires separate conversion and target-device validation. |
Supported ONNX package
The official
Inflect-Nano-v2-ONNX
package contains:
onnx/duration.onnx: token sequence to aligned acoustic distribution;onnx/decode.onnx: aligned distribution and seeded noise to waveform;onnx/inference_onnx.py: ONNX Runtime API and CLI;onnx/export_onnx.py: reproducible export from the canonical checkpoint;- machine-readable provenance, parity results, and SHA-256 checksums.
Dynamic text and waveform lengths are supported. The public wrapper implements seeded latent sampling, speed and variation controls, punctuation-aware long-text chunking, and CPU/CUDA/DirectML provider selection. eSpeak-ng remains the non-neural English frontend and is not embedded in the ONNX graphs.
See the Inflect-Nano-v2-ONNX model card
for installation and verified parity measurements.
Why only verified FP32 ships
The model is already small: 37.53 MB for Micro and 15.97 MB for Nano. A format is useful only if it is smaller or faster and preserves pronunciation, stability, and waveform quality. An export that loads but produces degraded speech is not release-ready.
The canonical model.pth remains the source of truth. Future quantized or
platform-specific exports must:
- reproduce deterministic fixed-seed outputs within a declared tolerance;
- pass matched semantic-WER and signal-diagnostic gates;
- complete a held-out listening comparison against FP32;
- report target hardware, runtime version, memory, latency, and file size;
- remain compatible with punctuation-aware long-text synthesis.
Practical deployment today
Use either the canonical PyTorch runtime or the verified FP32 ONNX Runtime package and keep one engine instance loaded. Nano is the recommended footprint-first deployment; Micro is the quality-first deployment. Choose ONNX for C/C++ bindings, browser work, DirectML, or other ONNX Runtime execution providers. Choose PyTorch when integrating with the original Python model code.