Inflect-Nano-v1-ONNX

ONNX export of owensong/Inflect-Nano-v1 — a 4.63M-parameter feed-forward English TTS (FastSpeech-style acoustic + Snake-activation HiFi-GAN vocoder, 24 kHz). This repo packages the model as ONNX graphs for torch-free, real-time CPU inference on edge devices (verified on a Jetson Nano gen1).

All credit for the model, training, and weights goes to the original author (owensong/Inflect-Nano-v1, Apache-2.0). This repository only adds the ONNX export + a reference onnxruntime runner.

Why ONNX

The original ships as PyTorch .pt. A single-graph trace is blocked by the acoustic model's dynamic length-regulator (Python .tolist() loops). This export uses the standard FastSpeech split — the neural parts are ONNX, the length regulator is a small vectorized NumPy step on the host:

phone/tone/lang/speaker ids
   │
   ▼  acoustic_encoder.onnx        (embeddings + Conv-FFN encoder + duration/pitch/energy heads)
conditioned[1,T,H], durations[1,T], pitch[1,T,2]
   │
   ▼  host length-regulator (NumPy, ~free)   — repeat-by-duration + frame meta + local context + abs-frame pos
frames / frame_meta / local_ctx_raw / abs_pos / pitch_frame / frame_mask
   │
   ▼  acoustic_decoder.onnx        (learned projections + Conv-FFN decoder + BiGRU + mel head + postnet)
mel[1,80,F]
   │
   ▼  vocoder.onnx                 (Snake HiFi-GAN)
wav[1,1,F*256]  @ 24 kHz

Parity vs the original torch pipeline: waveform max-abs-diff 2.1e-4 (mel 2.9e-6).

file	size	what
acoustic_encoder.onnx	5.6 MB	text ids → conditioned features + durations + pitch
acoustic_decoder.onnx	8.0 MB	regulated frames → mel
vocoder.onnx	4.7 MB	mel → 24 kHz waveform
inflect_onnx_infer.py	—	reference onnxruntime runner incl. the NumPy host_regulate

Opset 17. Dynamic axes on sequence/frame/sample length. Inputs are integer phoneme / tone / language ids produced by the original model's text frontend (run owensong/Inflect-Nano-v1's text_to_tokens, then feed the ids here).

Real-time on Jetson Nano gen1 (Tegra X1, 4× Cortex-A57, CPU-only)

onnxruntime 1.16.3, clocks unpinned, median of 3:

threads	RTF	peak RSS
4	0.51	142 MB
2	0.68	136 MB
1	1.11	139 MB

Real-time (RTF < 1) from 2 threads up; ~2× headroom at 4. ~142 MB RSS. On x86 CPU it runs at RTF ~0.03.

Usage — text → wav (saves output.wav you can play)

pip install onnxruntime soundfile numpy g2p_en transformers numba
# the base model provides the text frontend (text -> phoneme ids):
git clone https://huggingface.co/owensong/Inflect-Nano-v1
# run this script from the folder containing the .onnx files + inflect_onnx_infer.py

import sys, numpy as np, onnxruntime as ort, soundfile as sf
sys.path.insert(0, "Inflect-Nano-v1")                                  # base model frontend
sys.path.insert(0, "Inflect-Nano-v1/third_party/tiny_tts_frontend")
from inference import text_to_tokens          # owensong/Inflect-Nano-v1: text -> ids
from inflect_onnx_infer import host_regulate   # this repo: NumPy length-regulator

# 1) text -> phoneme / tone / language ids
phone, tone, lang = text_to_tokens("Hello, this is a tiny on-device text to speech model.")
phone, tone, lang = phone.numpy()[None], tone.numpy()[None], lang.numpy()[None]   # [1, T] int64
speaker = np.array([0], dtype=np.int64)

# 2) ONNX pipeline: encoder -> NumPy regulator -> decoder -> vocoder
sA = ort.InferenceSession("acoustic_encoder.onnx", providers=["CPUExecutionProvider"])
sB = ort.InferenceSession("acoustic_decoder.onnx", providers=["CPUExecutionProvider"])
sV = ort.InferenceSession("vocoder.onnx",          providers=["CPUExecutionProvider"])
cond, dur, pitch = sA.run(None, {"phone": phone, "tone": tone, "lang": lang, "speaker": speaker})
mel = sB.run(None, host_regulate(cond, dur, pitch))[0]
wav = sV.run(None, {"mel": mel.astype(np.float32)})[0].reshape(-1)     # 24 kHz mono float32

# 3) save to disk for inspection / playback
sf.write("output.wav", wav, 24000)
print(f"wrote output.wav  (24 kHz, {len(wav)/24000:.1f}s)")

Play / inspect the result:

aplay output.wav        # Linux
afplay output.wav       # macOS
# or open output.wav in any audio editor, or in a notebook:
#   from IPython.display import Audio; Audio("output.wav")

License & attribution

Apache-2.0, inherited from the base model. Please cite and follow the license of owensong/Inflect-Nano-v1. ONNX export contributed by the jetson-tts edge-TTS project.