SAVA β ONNX
SAVA is Inworld's streaming model that answers two
questions about live audio every 40 ms: is someone speaking, and has their turn ended. The original
runs in PyTorch. This repository is the same model exported to ONNX, in full precision and in an
8-bit version a quarter of the size, so it runs with onnxruntime and numpy alone.
This is an unofficial conversion. It is not made or endorsed by Inworld.
The graph takes raw audio: the log-mel front end is inside it. Feed one 640-sample frame of 16 kHz audio and the stream's state, and read three logits and the next state.
Model
| Property | Value |
|---|---|
| Parameters | 25.5 M (Mamba-2 and sliding-window attention trunk, speech and end-of-turn heads) |
| Inputs | pcm float32 [1, 640]: one 40 ms frame of 16 kHz mono audio in [-1, 1]state float32 [1, 798707]: all zeros at the start of a stream |
| Outputs | speech [1, 1]: logit, is this frame speechlookahead [1, 1]: logit, was the frame 3 frames back speech (the more accurate one; -30 for a stream's first 3 frames)turn [1, 1, 4]: logits, has the turn ended, at horizons of 0, 5, 10 and 20 framesnew_state [1, 798707]: pass back in with the next frame |
| Formats | float32 (103 MB), int8 (26.5 MB) |
| Opset | 20, standard operators only |
| Upstream | inworld/sava at revision 1beae7c1dd0d |
Files
| File | Size | Description |
|---|---|---|
sava.onnx |
103 MB | float32, same output as the PyTorch model |
sava-int8.onnx |
26.5 MB | ONNX Runtime dynamic quantization of the 27 large linear layers: 8-bit weights, with their inputs quantized to 8 bits as it runs. The mel front end, the attention score and value products and the recurrence stay float32 |
sava_onnx.py |
β | optional helper: push audio of any length, get per-frame probabilities |
config.json |
β | input and output contract, upstream revision, quantization details |
scripts/ |
β | the export, quantization and benchmark code that produced these files and numbers |
LICENSE, NOTICE |
β | Apache-2.0 license and Inworld's notice for the upstream weights |
Performance
Scored following the upstream repository's method on two of its test sets. The ONNX files run
each recording as a stream from its start, one frame per call; the PyTorch rows use the upstream
package's run() on the whole recording, which gives the same outputs. "Phone line" is the same
audio through a simulated telephone channel (8 kHz, 300β3400 Hz, G.711 mu-law; the code is
scripts/telephony.py). Higher is better. All accuracy numbers were measured with native ONNX
Runtime on Apple Silicon.
End of turn: all 31,527 clips of pipecat-ai/smart-turn-data-v3.2-test, score read at the
clip's last frame, threshold 0.5.
| Model | Accuracy | AUC | Accuracy, phone line | AUC, phone line |
|---|---|---|---|---|
| Upstream PyTorch | 93.36% | 0.9795 | 89.52% | 0.9575 |
sava.onnx |
93.36% | 0.9795 | 89.52% | 0.9575 |
sava-int8.onnx |
93.41% | 0.9793 | 89.36% | 0.9561 |
Inworld publishes 93.35% accuracy and 0.9795 AUC for the PyTorch model on this set.
Speech detection: the 30 labeled recordings of the TEN-VAD test set, 6,570 frames.
| Model | AUC | F1 | AUC, phone line | F1, phone line |
|---|---|---|---|---|
| Upstream PyTorch | 0.9768 | 0.955 | 0.9298 | 0.882 |
sava.onnx |
0.9768 | 0.955 | 0.9298 | 0.882 |
sava-int8.onnx |
0.9771 | 0.956 | 0.9204 | 0.867 |
Inworld publishes 0.9769 AUC and 0.956 F1 for the PyTorch model on this set.
sava.onnx reproduces the reference logits shipped with the upstream tests to within 4e-5. It
makes the same decision as the PyTorch model on every frame of the speech set and on every one of
the 31,527 end-of-turn clips, clean and phone line; on the end-of-turn clips the largest difference
in probability is 6e-6.
sava-int8.onnx changes the end-of-turn decision on 0.5% of clips, and on 1.1% of phone-line
clips. Against the PyTorch model, paired over clips or recordings (95% intervals):
| Clean | Phone line | |
|---|---|---|
| End-of-turn accuracy | +0.05 points (-0.03 to +0.13) | -0.16 points (-0.27 to -0.04) |
| Speech AUC | +0.0004 (-0.0006 to +0.0014) | -0.0094 (-0.0175 to -0.0024) |
So the 8-bit file costs nothing measurable on wideband audio and a small amount on phone-line audio.
Speed: time to process one 40 ms frame on one CPU thread, averaged over the 23 s upstream test
recording after two warm-up passes (scripts/speed.py on the Mac, scripts/ios-bench and
scripts/web-bench on the phone and in the browser).
| Device | Runtime | sava.onnx |
sava-int8.onnx |
|---|---|---|---|
| iPhone 17 Pro, iOS 27.2 | ONNX Runtime 1.30, native app | 2.2 ms | 1.2 ms |
| Apple M4 Max | ONNX Runtime 1.32, native | 1.5 ms | 1.2 ms |
| Apple M4 Max | onnxruntime-web 1.31, WebAssembly, Chromium | 6.0 ms | 9.0 ms |
| iPhone 17 Pro, iOS 27.2 | onnxruntime-web 1.31, WebAssembly, Arc (WebKit) | 89 ms | 146 ms |
For comparison, the PyTorch model's own streaming step takes 2.3 ms on one thread of the M4 Max.
- In a native app the 8-bit file is the one to use: on the iPhone it is the smaller file and runs 33 times faster than real time.
- In WebAssembly it is the other way round: the full-precision file was faster on both devices, and neither file kept up with live audio in the one iPhone browser tried. The browser rows are single runs; WebGPU and multi-threaded WebAssembly were not tried.
- In the native iPhone app both files made the same decision as on the Mac on every frame. In WebAssembly, on the iPhone and in Chromium on the Mac alike, the full-precision file matched too, but the 8-bit file's logits moved by up to 0.3 and 1 of the 573 frames changed decision.
- Not measured on Android or on a single-board computer.
Usage
import numpy as np
import onnxruntime as ort
session = ort.InferenceSession("sava-int8.onnx", providers=["CPUExecutionProvider"])
state = np.zeros((1, 798707), dtype=np.float32) # a new stream starts from zeros
def step(frame: np.ndarray) -> tuple[float, float]:
"""One 40 ms frame (640 float32 samples at 16 kHz) -> P(speech), P(turn ended)."""
global state
_, lookahead, turn, state = session.run(None, {"pcm": frame[None], "state": state})
sigmoid = lambda x: 1.0 / (1.0 + np.exp(-x))
return float(sigmoid(lookahead[0, 0])), float(sigmoid(turn[0, 0, 0]))
P(speech) from lookahead describes the frame 120 ms earlier; use speech for an on-time
answer. P(turn ended) is on time. Open a turn when speech starts and close it when the turn
probability crosses your threshold; the upstream repository documents a reference policy.
sava_onnx.py wraps this loop with the same push / flush interface as inworld.sava.Session.
How it was made
- Export.
torch.onnx.exporton the model's own one-frame streaming step, with its streaming state packed into a single tensor. The frame counter is stored in the state's last slot as a float, which is exact for the first 2**24 frames (186 hours) of a stream. Start a new stream (state back to zeros) before then: past that point the outputs drift without any error. - Quantization. ONNX Runtime dynamic quantization, restricted to the 27 learned linear layers with at least 100,000 weights. Quantizing every matrix product, the default, also quantizes the mel front end's DFT and filterbank, and the model then answers "no speech" on every frame.
- Checks.
scripts/test_parity.pycompares against the upstream reference logits;scripts/bench.pyproduced the tables above.
Limitations
- Everything upstream says about the model applies: it responds to any voice, does not pick out a target speaker, and leaves interruptions and backchannels to the application.
- The model is weaker on telephone-band audio than on wideband audio: end-of-turn accuracy falls from 93.4% to 89.5% and speech AUC from 0.977 to 0.930 (89.4% and 0.920 for the 8-bit file). Test on your own call audio before relying on it. A version fine-tuned on simulated phone lines, with its measurements, is in matthiastl/sava-telephony.
- The phone line here is simulated; real calls add packet loss, noise and other codecs.
- Single-threaded WebAssembly was too slow for live use on an iPhone. Plan on a native runtime there, or test your own browser setup first.
- The 8-bit file was not tested on x86 CPUs. ONNX Runtime warns that this kind of 8-bit model can
lose accuracy on x86 CPUs without VNNI instructions; check it there, or use
sava.onnx. - A 4-bit version was tried and is not published: on a few minutes of other test audio it changed 4β7% of the frame-by-frame end-of-turn answers.
License and attribution
The upstream code and weights are Β© 2026 Inworld AI, released under Apache-2.0. These files are a
converted and, for sava-int8.onnx, quantized form of those weights, distributed under the same
license. LICENSE and NOTICE are the upstream files; NOTICE also lists what was changed.
- Downloads last month
- 8
Model tree for matthiastl/sava-onnx
Base model
inworld/sava