docs/EXPORTS.md

# Export formats and quantization

## Release formats

| Format | Release status | Notes |
| --- | --- | --- |
| PyTorch FP32 | Supported | Canonical weights and the fully tested runtime. |
| PyTorch FP16/BF16 | Not released | Smaller in memory, but not validated for waveform quality or CPU compatibility. |
| ONNX FP32 | Supported | Separate official [`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX) repository with verified dynamic-length graphs, a torch-free Python runner, and a reproducible exporter. |
| INT8 / INT4 | Not released | Naive weight quantization can audibly damage the integrated waveform decoder. |
| GGUF | Not applicable today | GGUF targets transformer/LLM runtimes and is not a drop-in container for this VITS-family convolutional waveform model. |
| Core ML / TFLite | Not released | Requires separate conversion and target-device validation. |

## Supported ONNX package

The official
[`Inflect-Micro-v2-ONNX`](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX)
package contains:

- `onnx/duration.onnx`: token sequence to aligned acoustic distribution;
- `onnx/decode.onnx`: aligned distribution and seeded noise to waveform;
- `onnx/inference_onnx.py`: ONNX Runtime API and CLI;
- `onnx/export_onnx.py`: reproducible export from the canonical checkpoint;
- machine-readable provenance, parity results, and SHA-256 checksums.

Dynamic text and waveform lengths are supported. The public wrapper implements
seeded latent sampling, speed and variation controls, punctuation-aware
long-text chunking, and CPU/CUDA/DirectML provider selection. eSpeak-ng remains
the non-neural English frontend and is not embedded in the ONNX graphs.

See the [`Inflect-Micro-v2-ONNX` model card](https://huggingface.co/owensong/Inflect-Micro-v2-ONNX)
for installation and verified parity measurements.

## Why only verified FP32 ships

The model is already small: 37.53 MB for Micro and 15.97 MB for Nano. A format
is useful only if it is smaller or faster **and** preserves pronunciation,
stability, and waveform quality. An export that loads but produces degraded
speech is not release-ready.

The canonical `model.pth` remains the source of truth. Future quantized or
platform-specific exports must:

1. reproduce deterministic fixed-seed outputs within a declared tolerance;
2. pass matched semantic-WER and signal-diagnostic gates;
3. complete a held-out listening comparison against FP32;
4. report target hardware, runtime version, memory, latency, and file size;
5. remain compatible with punctuation-aware long-text synthesis.

## Practical deployment today

Use either the canonical PyTorch runtime or the verified FP32 ONNX Runtime
package and keep one engine instance loaded. Nano is the recommended
footprint-first deployment; Micro is the quality-first deployment. Choose ONNX
for C/C++ bindings, browser work, DirectML, or other ONNX Runtime execution
providers. Choose PyTorch when integrating with the original Python model code.