SAVA for phone lines
SAVA is Inworld's streaming model that tells, every 40 ms, whether someone is speaking and whether they have finished their turn. It gets worse on phone audio, so this is a fine-tune of it on simulated phone lines. End-of-turn accuracy goes from 89.5% to 93.0% on a simulated landline and from 88.5% to 92.4% on a GSM mobile codec, and stays the same on wideband audio (93.4% before, 93.5% after).
Summary
| Original SAVA | This fine-tune | Smart Turn v3.2 | |
|---|---|---|---|
| End-of-turn accuracy, wideband | 93.4% | 93.5% | 93.7% |
| End-of-turn accuracy, simulated landline | 89.5% | 93.0% | 91.7% |
| End-of-turn accuracy, simulated GSM codec | 88.5% | 92.4% | 90.5% |
| Speech missed on real phone calls | 28% | 15% | n/a |
| False alarms on real phone calls | 8% | 13% | n/a |
Bold is the best number in each row; for the last two rows lower is better. Smart Turn is Pipecat's end-of-turn model, shown for comparison. It does not detect speech.
Things to know before using it:
- The end-of-turn numbers come from simulated phone lines. End of turn on real calls could not be measured, because no real-call test set has labels for it (see "What could not be measured").
- On real calls, most of the speech-detection gain is a threshold effect. The original gets close if you lower its speech threshold from 0.5 to 0.17.
- The fine-tune does not teach the model anything new. Through a phone line the original goes quiet (on meeting recordings it misses 39% to 43% of the speech instead of 26%), and this version detects speech on the line about as well as the original does on wideband audio.
- It also says "ended" more readily at pauses. On the real calls, 0.4 s into a pause where the same person then goes on, it says "ended" 80% of the time against 63% for the original. In a voice agent that means more interruptions at the default threshold, so set the threshold and wait time on your own calls.
- The 8-bit ONNX file is a quarter of the size and scores the same.
Test it on your own call audio before relying on it.
This is an unofficial fine-tune. It is not made or endorsed by Inworld.
Model
| Property | Value |
|---|---|
| Parameters | 25.5 M, same architecture as inworld/sava (Mamba-2 and sliding-window attention trunk, speech and end-of-turn heads) |
| Audio in | 16 kHz mono float32 in [-1, 1]. Upsample 8 kHz call audio to 16 kHz first |
| ONNX inputs | pcm float32 [1, 640]: one 40 ms framestate float32 [1, 798707]: all zeros at the start of a stream |
| ONNX outputs | speech [1, 1]: logit, is this frame speechlookahead [1, 1]: logit, was the frame 3 frames back speech (the more accurate one; -30 for a stream's first 3 frames)turn [1, 1, 4]: logits, has the turn ended, at horizons of 0, 5, 10 and 20 framesnew_state [1, 798707]: pass back in with the next frame |
| Opset | 20, standard operators only |
| Upstream | inworld/sava at revision 1beae7c1dd0d |
Files
| File | Size | Description |
|---|---|---|
sava-telephony.safetensors |
102 MB | PyTorch weights, loads with the upstream inworld-sava package |
sava-telephony.onnx |
103 MB | float32 ONNX, same output as the PyTorch weights |
sava-telephony-int8.onnx |
27 MB | 8-bit weights (dynamic quantization) in the 27 large linear layers; front end, attention and recurrence in float32 |
sava_onnx.py |
β | optional helper: push audio of any length, get per-frame probabilities |
config.json |
β | input and output contract, upstream revision, fine-tuning and quantization details |
scripts/ |
β | the fine-tuning, export, quantization and benchmark code that produced these files and numbers |
results/ |
β | the training log and the scores behind the tables below |
LICENSE, NOTICE |
β | Apache-2.0 license and Inworld's notice for the upstream weights |
Performance
Each recording is run as a stream from its start, one frame per call. Three conditions:
- Wideband: the test audio as published, 16 kHz.
- Landline: the same audio at 8 kHz, 300β3400 Hz, through G.711 mu-law.
- Mobile: the same band through the GSM 06.10 full-rate codec. No GSM audio was used in fine-tuning.
Both lines are simulated. Higher is better.
End of turn: all 31,527 clips of pipecat-ai/smart-turn-data-v3.2-test, score read at the
clip's last frame, threshold 0.5. Accuracy, with AUC in brackets.
| Model | Wideband | Landline | Mobile |
|---|---|---|---|
inworld/sava (upstream) |
93.36% (0.9795) | 89.52% (0.9575) | 88.52% (0.9493) |
sava-telephony.safetensors |
93.50% (0.9797) | 93.03% (0.9772) | 92.42% (0.9740) |
sava-telephony.onnx |
93.50% (0.9797) | 93.03% (0.9772) | 92.42% (0.9740) |
sava-telephony-int8.onnx |
93.52% (0.9796) | 93.02% (0.9771) | 92.44% (0.9738) |
| Pipecat Smart Turn v3.2, for reference | 93.71% (0.9825) | 91.71% (0.9724) | 90.48% (0.9664) |
Differences in accuracy, paired over clips, in points with 95% intervals:
| Wideband | Landline | Mobile | |
|---|---|---|---|
| This model minus upstream | +0.15 (-0.00 to +0.28) | +3.51 (+3.23 to +3.81) | +3.90 (+3.58 to +4.21) |
| This model minus Smart Turn v3.2 | -0.20 (-0.48 to +0.08) | +1.33 (+1.01 to +1.64) | +1.94 (+1.59 to +2.27) |
| 8-bit file minus this model | +0.02 (-0.06 to +0.10) | -0.02 (-0.10 to +0.07) | +0.02 (-0.08 to +0.11) |
The intervals treat clips as independent. Many clips share a speaker or a source, so the true intervals are somewhat wider; the gains on the phone lines are large enough that this does not change them.
- A phone line now costs this model 0.5 points (landline) and 1.1 points (mobile), where it cost upstream 3.8 and 4.8 points and Smart Turn 2.0 and 3.2 points.
- On the test set's 5,863 real recordings (the rest is synthetic speech) this model scores 95.7%, 95.3% and 94.6%; upstream 95.8%, 90.2% and 89.5%; Smart Turn 94.8%, 93.1% and 92.3%.
- On wideband audio this model's accuracy is within noise of both others'; Smart Turn has the higher AUC.
- Smart Turn v3.2 (
smart-turn-v3.2-gpu.onnx) was scored the way its own inference code runs it, on the last 8 seconds of each clip.
Speech detection: the 30 labeled recordings of the TEN-VAD test set, 6,570 frames. AUC, with F1 in brackets.
| Model | Wideband | Landline | Mobile |
|---|---|---|---|
inworld/sava (upstream) |
0.9768 (0.955) | 0.9298 (0.882) | 0.9090 (0.871) |
sava-telephony.safetensors |
0.9792 (0.959) | 0.9782 (0.958) | 0.9737 (0.951) |
sava-telephony.onnx |
0.9792 (0.959) | 0.9782 (0.958) | 0.9737 (0.951) |
sava-telephony-int8.onnx |
0.9791 (0.959) | 0.9782 (0.958) | 0.9732 (0.950) |
AUC against upstream, paired over recordings: +0.002 (-0.001 to +0.008) wideband, +0.048 (+0.021 to +0.085) landline, +0.065 (+0.024 to +0.119) mobile. This set is small, so the intervals are wide.
sava-telephony.onnx makes the same decision as the PyTorch weights on every one of the 31,527
end-of-turn clips in all three conditions; the largest difference in probability is 6e-6.
sava-telephony-int8.onnx changes the decision on 0.5% to 0.7% of clips, about as often for the
better as for the worse.
Real phone calls: 28 calls, 3.9 hours, from CallHome English (23 with two speakers, 5 with three or four), with hand-marked times of who spoke when. Everyone is in one channel, and the audio is used as the dataset ships it, already at 16 kHz. Models run at their default threshold of 0.5.
Speech detection, 354,555 frames:
| Model | AUC | F1 | Accuracy | Speech missed | False alarms |
|---|---|---|---|---|---|
inworld/sava (upstream) |
0.907 | 0.829 | 74.4% | 28.3% | 8.0% |
sava-telephony.safetensors |
0.920 | 0.911 | 85.6% | 14.6% | 13.1% |
sava-telephony-int8.onnx |
0.920 | 0.911 | 85.5% | 14.7% | 13.0% |
Accuracy is a poor measure on these calls: 87% of the frames are marked as speech, so answering "speech" on every frame would score 86.6%, more than either model. The misses, the false alarms and the AUC say more. This model misses 14.6% of the speech instead of 28.3% and raises more false alarms, 13.1% instead of 8.0%. Its AUC, which does not depend on the threshold, is higher by 0.013 (0.009 to 0.017, paired over calls); that part is better detection.
The rest is the model saying "speech" more readily on phone audio, which lowering upstream's threshold also does. At a threshold of 0.17, chosen on these same calls so that upstream says "speech" as often as this model does at 0.5, upstream misses 15.1% and raises 16.2% false alarms. The marked times are approximate, and every model scores lower here than on the TEN-VAD set.
At pauses this model says "ended" more often than upstream, whoever speaks next. 0.4 s into a pause it says so at 93% of the pauses where the other person speaks next (upstream 77%) and at 80% of those where the same person goes on (upstream 63%), 16 points more for each (95% intervals 8 to 25 and 11 to 22, paired over calls).
Meetings, as recorded and through the simulated lines: 6 meetings, 3.0 hours, from the AMI corpus (headset microphones, wideband), with marked times of who spoke when. Here the same audio is on both sides of the comparison, so the table shows what a phone line does. The last four rows read the end-of-turn probability 0.4 s into each of 446 pauses of at least 0.5 s.
| At the default threshold of 0.5 | Wideband | Landline | Mobile |
|---|---|---|---|
| Speech missed, upstream | 25.7% | 39.0% | 42.9% |
| Speech missed, this model | 24.6% | 25.3% | 25.7% |
| Says "ended" when another person speaks next, upstream | 85% | 65% | 66% |
| Says "ended" when another person speaks next, this model | 86% | 84% | 89% |
| Says "ended" when the same person goes on, upstream | 71% | 53% | 49% |
| Says "ended" when the same person goes on, this model | 72% | 76% | 77% |
Through a phone line upstream goes quiet at its default thresholds. This model on a phone line detects speech as upstream does on wideband audio, and says "ended" a little more often where the same person goes on (76% and 77% against 71%).
What could not be measured: end of turn on real calls. No real-call test set we could use
says whether a turn has ended. An indirect test was tried on the real calls and on the meetings: at a pause,
does the end-of-turn probability tell "another person speaks next" from "the same person goes
on"? Every model scores low on it, and a phone line barely moves the score: on the meetings
upstream scores an AUC of 0.61 on wideband audio and 0.59 and 0.58 through the lines, this model
0.62, 0.61 and 0.61, Smart Turn 0.64, 0.62 and 0.65. On the real calls upstream scores 0.58, this
model 0.61 and Smart Turn 0.69, the one clear difference. A test on which the line costs
upstream almost nothing cannot show what this fine-tune recovers, so it is not used for any
claim here. Its numbers are in results/real-calls.json and results/meetings.json.
Speed: one 40 ms frame on one CPU thread of an Apple M4 Max with ONNX Runtime 1.32, averaged
over a 23 s recording after two warm-up passes (scripts/speed.py): 1.5 ms for
sava-telephony.onnx, 1.2 ms for sava-telephony-int8.onnx. The graph is the one in
matthiastl/sava-onnx with different weights; that
repository has iPhone and browser timings, which were not repeated for these files.
Usage
Phone audio arrives at 8 kHz. Decode it to linear PCM and upsample it to 16 kHz before the model;
the simulated lines it was trained and tested on are upsampled with
scipy.signal.resample_poly(x, 2, 1).
import numpy as np
import onnxruntime as ort
session = ort.InferenceSession("sava-telephony-int8.onnx", providers=["CPUExecutionProvider"])
state = np.zeros((1, 798707), dtype=np.float32) # a new stream starts from zeros
def step(frame: np.ndarray) -> tuple[float, float]:
"""One 40 ms frame (640 float32 samples at 16 kHz) -> P(speech), P(turn ended)."""
global state
_, lookahead, turn, state = session.run(None, {"pcm": frame[None], "state": state})
sigmoid = lambda x: 1.0 / (1.0 + np.exp(-x))
return float(sigmoid(lookahead[0, 0])), float(sigmoid(turn[0, 0, 0]))
P(speech) from lookahead describes the frame 120 ms earlier; use speech for an on-time
answer. P(turn ended) is on time. Open a turn when speech starts and close it when the turn
probability crosses your threshold; the upstream repository documents a reference policy.
sava_onnx.py wraps this loop with the same push / flush interface as inworld.sava.Session.
In PyTorch, with pip install inworld-sava:
from inworld import sava
model = sava.load("sava-telephony.safetensors")
session = sava.Session(model) # then push audio as with the upstream weights
How it was made
- Fine-tuning. Two copies of the upstream model. The frozen one hears each clip as published. The other hears the same clip through a random simulated phone line and is trained to give the frozen one's answers, frame by frame, on all three outputs. Training uses no labels; the labels of the tenth file below were only used to pick the checkpoint. One clip in four is left clean so the wideband behaviour stays put.
- Training lines. 8 kHz; band-pass with a low edge of 150β400 Hz and a high edge of 3000β3700 Hz; G.711 mu-law, A-law or no codec; input level 0 to -20 dB; white line noise on 30% of clips, at -70 to -30 dB relative to full scale. GSM was kept out of training to serve as the unseen line in the tests.
- Data. 29,385 clips from 9 of the 83 files of
pipecat-ai/smart-turn-data-v3.2-train, a set the upstream model was itself trained on, so no new speech data is involved. A tenth file chose the checkpoint, scored clean and through the test landline. 2,500 steps of batch 32 took 90 minutes on an Apple M4 Max GPU. Midway through, the step-500 checkpoint was scored once on 8,000 of the test clips to check that training was working; nothing was chosen from it. - Export and quantization. As in matthiastl/sava-onnx:
torch.onnx.exporton the one-frame streaming step, then ONNX Runtime dynamic quantization of the 27 learned linear layers with at least 100,000 weights. - Checks.
scripts/test_parity.pycompares the exports with the PyTorch weights;scripts/bench.pyandscripts/compare.pyproduced the tables above, andscripts/calls.pythe real-call ones.config.jsonhas the full settings andscripts/README.mdthe commands.
Limitations
- The end-of-turn gain is measured on simulated lines only. End of turn on real calls is not measured; see "What could not be measured".
- At the default threshold the model says "ended" 0.4 s into most pauses in conversation, whoever speaks next, and more often than upstream: on the real calls at 80% of the pauses where the same person goes on, against 63%. Pick your threshold and wait time on your own audio.
- Part of the real-call gain in speech detection is a matter of threshold: upstream with its speech threshold at 0.17, chosen on the same calls, comes close.
- The real-call test is small and old: 28 English calls from the 1990s with both speakers in one channel. It does not cover one caller's side alone, which is what a voice agent hears, nor packet loss, echo or current codecs (AMR, Opus, EVS). Test on your own call audio before relying on the model.
- The landline condition falls inside the family of lines used in training; the mobile condition does not, and it is the fairer estimate of an unseen line.
- The end-of-turn test set comes from the same collection as the fine-tuning audio, as it does for upstream and for Smart Turn.
- Fine-tuning copies upstream's wideband answers, so it cannot make the model better than upstream on wideband audio, and everything upstream says about the model still applies: it responds to any voice, does not pick out a target speaker, and leaves interruptions and backchannels to the application.
- One training run, on 11% of the training set. More data, more steps and real codecs in the training lines were not tried.
License and attribution
The upstream code and weights are Β© 2026 Inworld AI, released under Apache-2.0. These files are a
fine-tuned, converted and, for sava-telephony-int8.onnx, quantized form of those weights,
distributed under the same license. LICENSE and NOTICE are the upstream files; NOTICE also
lists what was changed. The fine-tuning audio is from
pipecat-ai/smart-turn-data-v3.2-train
(CC BY 4.0). The end-of-turn test set is
pipecat-ai/smart-turn-data-v3.2-test,
and the speech-detection test set is the testset/ folder of
TEN-framework/ten-vad.
The real-call test uses the first 28 calls of the English part of talkbank/callhome (CABank English CallHome Corpus, TalkBank, doi:10.21415/T5KP54, CC BY-NC-SA 4.0). It was used for testing only: the model was not trained on it, and none of its audio or annotation is in this repository.
The meeting test uses the first 6 meetings of the test split of diarizers-community/ami (AMI Meeting Corpus, headset mix, CC BY 4.0), also for testing only.
- Downloads last month
- -
Model tree for matthiastl/sava-telephony
Base model
inworld/sava