Immortalizer/moonshine-streaming-medium-onnx

Moonshine Streaming Medium — merged-decoder int8 ONNX

int8-quantized ONNX export of Moonshine, the fast English speech-recognition model from Useful Sensors (paper: Moonshine: Speech Recognition for Live Transcription and Voice Commands, arXiv:2410.15608). All model weights, architecture, and accuracy are Useful Sensors' work — please cite and credit them as the source of this model.

This repo is only a packaging change: the decoder is shipped as a single use_cache_branch-gated graph instead of the usual two-file split decoder. Nothing about the math, weights, or quantization changes.

What's different from a standard streaming ONNX export

Streaming-Moonshine ONNX exports normally ship the decoder as two files — decoder_model_quantized.onnx (step 0, no KV cache) and decoder_with_past_model_quantized.onnx (steps 1..N, with cache) — whose weights are ~90% identical, so both sit resident in RAM at once. This repo merges that pair into one decoder_model_merged_quantized.onnx switched by a use_cache_branch input — the same single-graph shape the official onnx-community/moonshine-{tiny,base} exports already use.

file	size
encoder_model_quantized.onnx	~136 MB
decoder_model_merged_quantized.onnx	~228 MB
tokenizer.json	~3.6 MB

Result: ~209 MB smaller on disk and roughly half the resident decoder RAM, with no accuracy change — the merge operates on the already-int8 graphs (no re-quantization), and the merged decoder is bit-for-bit functionally identical to the split pair (same token sequence; logits agree to ~1e-5, pure graph-optimization reordering).

Lineage / attribution

1. Useful Sensors — Moonshine — the original model, weights, and architecture. The model is their work.
2. Mazino0/moonshine-streaming-medium-onnx — the int8 split-decoder ONNX export this repack is built from.
3. This repo — the split decoder merged into one use_cache_branch graph (encoder and tokenizer copied verbatim).

License

MIT, inherited from the upstream Moonshine release.

Usage

Built for my-voice, an offline English push-to-talk voice-typing daemon, but it's a standard onnxruntime-loadable encoder + merged-decoder + tokenizer.json layout.