File size: 2,991 Bytes
d0385b5 132edf8 d0385b5 83d6c02 132edf8 83d6c02 132edf8 83d6c02 d0385b5 83d6c02 d0385b5 83d6c02 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | # Export formats and quantization
## Release formats
| Format | Release status | Notes |
| --- | --- | --- |
| PyTorch FP32 | Supported | Canonical weights and the fully tested runtime. |
| PyTorch FP16/BF16 | Not released | Smaller in memory, but not validated for waveform quality or CPU compatibility. |
| ONNX FP32 | Supported | Separate official [`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) repository with verified dynamic-length graphs, a torch-free Python runner, and a reproducible exporter. |
| INT8 / INT4 | Not released | Naive weight quantization can audibly damage the integrated waveform decoder. |
| GGUF | Not applicable today | GGUF targets transformer/LLM runtimes and is not a drop-in container for this VITS-family convolutional waveform model. |
| Core ML / TFLite | Not released | Requires separate conversion and target-device validation. |
## Supported ONNX package
The official
[`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX)
package contains:
- `onnx/duration.onnx`: token sequence to aligned acoustic distribution;
- `onnx/decode.onnx`: aligned distribution and seeded noise to waveform;
- `onnx/inference_onnx.py`: ONNX Runtime API and CLI;
- `onnx/export_onnx.py`: reproducible export from the canonical checkpoint;
- machine-readable provenance, parity results, and SHA-256 checksums.
Dynamic text and waveform lengths are supported. The public wrapper implements
seeded latent sampling, speed and variation controls, punctuation-aware
long-text chunking, and CPU/CUDA/DirectML provider selection. eSpeak-ng remains
the non-neural English frontend and is not embedded in the ONNX graphs.
See the [`Inflect-Micro-v2-ONNX` model card](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX)
for installation and verified parity measurements.
## Why only verified FP32 ships
The model is already small: 37.53 MB for Micro and 15.97 MB for Nano. A format
is useful only if it is smaller or faster **and** preserves pronunciation,
stability, and waveform quality. An export that loads but produces degraded
speech is not release-ready.
The canonical `model.pth` remains the source of truth. Future quantized or
platform-specific exports must:
1. reproduce deterministic fixed-seed outputs within a declared tolerance;
2. pass matched semantic-WER and signal-diagnostic gates;
3. complete a held-out listening comparison against FP32;
4. report target hardware, runtime version, memory, latency, and file size;
5. remain compatible with punctuation-aware long-text synthesis.
## Practical deployment today
Use either the canonical PyTorch runtime or the verified FP32 ONNX Runtime
package and keep one engine instance loaded. Nano is the recommended
footprint-first deployment; Micro is the quality-first deployment. Choose ONNX
for C/C++ bindings, browser work, DirectML, or other ONNX Runtime execution
providers. Choose PyTorch when integrating with the original Python model code.
|