SAVA β€” ONNX

SAVA is Inworld's streaming model that answers two questions about live audio every 40 ms: is someone speaking, and has their turn ended. The original runs in PyTorch. This repository is the same model exported to ONNX, in full precision and in an 8-bit version a quarter of the size, so it runs with onnxruntime and numpy alone.

This is an unofficial conversion. It is not made or endorsed by Inworld.

The graph takes raw audio: the log-mel front end is inside it. Feed one 640-sample frame of 16 kHz audio and the stream's state, and read three logits and the next state.

Model

Property Value
Parameters 25.5 M (Mamba-2 and sliding-window attention trunk, speech and end-of-turn heads)
Inputs pcm float32 [1, 640]: one 40 ms frame of 16 kHz mono audio in [-1, 1]
state float32 [1, 798707]: all zeros at the start of a stream
Outputs speech [1, 1]: logit, is this frame speech
lookahead [1, 1]: logit, was the frame 3 frames back speech (the more accurate one; -30 for a stream's first 3 frames)
turn [1, 1, 4]: logits, has the turn ended, at horizons of 0, 5, 10 and 20 frames
new_state [1, 798707]: pass back in with the next frame
Formats float32 (103 MB), int8 (26.5 MB)
Opset 20, standard operators only
Upstream inworld/sava at revision 1beae7c1dd0d

Files

File Size Description
sava.onnx 103 MB float32, same output as the PyTorch model
sava-int8.onnx 26.5 MB ONNX Runtime dynamic quantization of the 27 large linear layers: 8-bit weights, with their inputs quantized to 8 bits as it runs. The mel front end, the attention score and value products and the recurrence stay float32
sava_onnx.py β€” optional helper: push audio of any length, get per-frame probabilities
config.json β€” input and output contract, upstream revision, quantization details
scripts/ β€” the export, quantization and benchmark code that produced these files and numbers
LICENSE, NOTICE β€” Apache-2.0 license and Inworld's notice for the upstream weights

Performance

Scored following the upstream repository's method on two of its test sets. The ONNX files run each recording as a stream from its start, one frame per call; the PyTorch rows use the upstream package's run() on the whole recording, which gives the same outputs. "Phone line" is the same audio through a simulated telephone channel (8 kHz, 300–3400 Hz, G.711 mu-law; the code is scripts/telephony.py). Higher is better. All accuracy numbers were measured with native ONNX Runtime on Apple Silicon.

End of turn: all 31,527 clips of pipecat-ai/smart-turn-data-v3.2-test, score read at the clip's last frame, threshold 0.5.

Model Accuracy AUC Accuracy, phone line AUC, phone line
Upstream PyTorch 93.36% 0.9795 89.52% 0.9575
sava.onnx 93.36% 0.9795 89.52% 0.9575
sava-int8.onnx 93.41% 0.9793 89.36% 0.9561

Inworld publishes 93.35% accuracy and 0.9795 AUC for the PyTorch model on this set.

Speech detection: the 30 labeled recordings of the TEN-VAD test set, 6,570 frames.

Model AUC F1 AUC, phone line F1, phone line
Upstream PyTorch 0.9768 0.955 0.9298 0.882
sava.onnx 0.9768 0.955 0.9298 0.882
sava-int8.onnx 0.9771 0.956 0.9204 0.867

Inworld publishes 0.9769 AUC and 0.956 F1 for the PyTorch model on this set.

sava.onnx reproduces the reference logits shipped with the upstream tests to within 4e-5. It makes the same decision as the PyTorch model on every frame of the speech set and on every one of the 31,527 end-of-turn clips, clean and phone line; on the end-of-turn clips the largest difference in probability is 6e-6.

sava-int8.onnx changes the end-of-turn decision on 0.5% of clips, and on 1.1% of phone-line clips. Against the PyTorch model, paired over clips or recordings (95% intervals):

Clean Phone line
End-of-turn accuracy +0.05 points (-0.03 to +0.13) -0.16 points (-0.27 to -0.04)
Speech AUC +0.0004 (-0.0006 to +0.0014) -0.0094 (-0.0175 to -0.0024)

So the 8-bit file costs nothing measurable on wideband audio and a small amount on phone-line audio.

Speed: time to process one 40 ms frame on one CPU thread, averaged over the 23 s upstream test recording after two warm-up passes (scripts/speed.py on the Mac, scripts/ios-bench and scripts/web-bench on the phone and in the browser).

Device Runtime sava.onnx sava-int8.onnx
iPhone 17 Pro, iOS 27.2 ONNX Runtime 1.30, native app 2.2 ms 1.2 ms
Apple M4 Max ONNX Runtime 1.32, native 1.5 ms 1.2 ms
Apple M4 Max onnxruntime-web 1.31, WebAssembly, Chromium 6.0 ms 9.0 ms
iPhone 17 Pro, iOS 27.2 onnxruntime-web 1.31, WebAssembly, Arc (WebKit) 89 ms 146 ms

For comparison, the PyTorch model's own streaming step takes 2.3 ms on one thread of the M4 Max.

  • In a native app the 8-bit file is the one to use: on the iPhone it is the smaller file and runs 33 times faster than real time.
  • In WebAssembly it is the other way round: the full-precision file was faster on both devices, and neither file kept up with live audio in the one iPhone browser tried. The browser rows are single runs; WebGPU and multi-threaded WebAssembly were not tried.
  • In the native iPhone app both files made the same decision as on the Mac on every frame. In WebAssembly, on the iPhone and in Chromium on the Mac alike, the full-precision file matched too, but the 8-bit file's logits moved by up to 0.3 and 1 of the 573 frames changed decision.
  • Not measured on Android or on a single-board computer.

Usage

import numpy as np
import onnxruntime as ort

session = ort.InferenceSession("sava-int8.onnx", providers=["CPUExecutionProvider"])
state = np.zeros((1, 798707), dtype=np.float32)   # a new stream starts from zeros


def step(frame: np.ndarray) -> tuple[float, float]:
    """One 40 ms frame (640 float32 samples at 16 kHz) -> P(speech), P(turn ended)."""
    global state
    _, lookahead, turn, state = session.run(None, {"pcm": frame[None], "state": state})
    sigmoid = lambda x: 1.0 / (1.0 + np.exp(-x))
    return float(sigmoid(lookahead[0, 0])), float(sigmoid(turn[0, 0, 0]))

P(speech) from lookahead describes the frame 120 ms earlier; use speech for an on-time answer. P(turn ended) is on time. Open a turn when speech starts and close it when the turn probability crosses your threshold; the upstream repository documents a reference policy.

sava_onnx.py wraps this loop with the same push / flush interface as inworld.sava.Session.

How it was made

  • Export. torch.onnx.export on the model's own one-frame streaming step, with its streaming state packed into a single tensor. The frame counter is stored in the state's last slot as a float, which is exact for the first 2**24 frames (186 hours) of a stream. Start a new stream (state back to zeros) before then: past that point the outputs drift without any error.
  • Quantization. ONNX Runtime dynamic quantization, restricted to the 27 learned linear layers with at least 100,000 weights. Quantizing every matrix product, the default, also quantizes the mel front end's DFT and filterbank, and the model then answers "no speech" on every frame.
  • Checks. scripts/test_parity.py compares against the upstream reference logits; scripts/bench.py produced the tables above.

Limitations

  • Everything upstream says about the model applies: it responds to any voice, does not pick out a target speaker, and leaves interruptions and backchannels to the application.
  • The model is weaker on telephone-band audio than on wideband audio: end-of-turn accuracy falls from 93.4% to 89.5% and speech AUC from 0.977 to 0.930 (89.4% and 0.920 for the 8-bit file). Test on your own call audio before relying on it. A version fine-tuned on simulated phone lines, with its measurements, is in matthiastl/sava-telephony.
  • The phone line here is simulated; real calls add packet loss, noise and other codecs.
  • Single-threaded WebAssembly was too slow for live use on an iPhone. Plan on a native runtime there, or test your own browser setup first.
  • The 8-bit file was not tested on x86 CPUs. ONNX Runtime warns that this kind of 8-bit model can lose accuracy on x86 CPUs without VNNI instructions; check it there, or use sava.onnx.
  • A 4-bit version was tried and is not published: on a few minutes of other test audio it changed 4–7% of the frame-by-frame end-of-turn answers.

License and attribution

The upstream code and weights are Β© 2026 Inworld AI, released under Apache-2.0. These files are a converted and, for sava-int8.onnx, quantized form of those weights, distributed under the same license. LICENSE and NOTICE are the upstream files; NOTICE also lists what was changed.

Downloads last month
8
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for matthiastl/sava-onnx

Base model

inworld/sava
Quantized
(2)
this model