| # Export formats and quantization |
|
|
| ## Release formats |
|
|
| | Format | Release status | Notes | |
| | --- | --- | --- | |
| | PyTorch FP32 | Supported | Canonical weights and the fully tested runtime. | |
| | PyTorch FP16/BF16 | Not released | Smaller in memory, but not validated for waveform quality or CPU compatibility. | |
| | ONNX FP32 | Supported | Separate official [`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) repository with verified dynamic-length graphs, a torch-free Python runner, and a reproducible exporter. | |
| | INT8 / INT4 | Not released | Naive weight quantization can audibly damage the integrated waveform decoder. | |
| | GGUF | Not applicable today | GGUF targets transformer/LLM runtimes and is not a drop-in container for this VITS-family convolutional waveform model. | |
| | Core ML / TFLite | Not released | Requires separate conversion and target-device validation. | |
|
|
| ## Supported ONNX package |
|
|
| The official |
| [`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) |
| package contains: |
|
|
| - `onnx/duration.onnx`: token sequence to aligned acoustic distribution; |
| - `onnx/decode.onnx`: aligned distribution and seeded noise to waveform; |
| - `onnx/inference_onnx.py`: ONNX Runtime API and CLI; |
| - `onnx/export_onnx.py`: reproducible export from the canonical checkpoint; |
| - machine-readable provenance, parity results, and SHA-256 checksums. |
|
|
| Dynamic text and waveform lengths are supported. The public wrapper implements |
| seeded latent sampling, speed and variation controls, punctuation-aware |
| long-text chunking, and CPU/CUDA/DirectML provider selection. eSpeak-ng remains |
| the non-neural English frontend and is not embedded in the ONNX graphs. |
|
|
| See the [`Inflect-Micro-v2-ONNX` model card](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) |
| for installation and verified parity measurements. |
|
|
| ## Why only verified FP32 ships |
|
|
| The model is already small: 37.53 MB for Micro and 15.97 MB for Nano. A format |
| is useful only if it is smaller or faster **and** preserves pronunciation, |
| stability, and waveform quality. An export that loads but produces degraded |
| speech is not release-ready. |
|
|
| The canonical `model.pth` remains the source of truth. Future quantized or |
| platform-specific exports must: |
|
|
| 1. reproduce deterministic fixed-seed outputs within a declared tolerance; |
| 2. pass matched semantic-WER and signal-diagnostic gates; |
| 3. complete a held-out listening comparison against FP32; |
| 4. report target hardware, runtime version, memory, latency, and file size; |
| 5. remain compatible with punctuation-aware long-text synthesis. |
|
|
| ## Practical deployment today |
|
|
| Use either the canonical PyTorch runtime or the verified FP32 ONNX Runtime |
| package and keep one engine instance loaded. Nano is the recommended |
| footprint-first deployment; Micro is the quality-first deployment. Choose ONNX |
| for C/C++ bindings, browser work, DirectML, or other ONNX Runtime execution |
| providers. Choose PyTorch when integrating with the original Python model code. |
|
|
