SAVA for phone lines

SAVA is Inworld's streaming model that tells, every 40 ms, whether someone is speaking and whether they have finished their turn. It gets worse on phone audio, so this is a fine-tune of it on simulated phone lines. End-of-turn accuracy goes from 89.5% to 93.0% on a simulated landline and from 88.5% to 92.4% on a GSM mobile codec, and stays the same on wideband audio (93.4% before, 93.5% after).

Summary

Original SAVA This fine-tune Smart Turn v3.2
End-of-turn accuracy, wideband 93.4% 93.5% 93.7%
End-of-turn accuracy, simulated landline 89.5% 93.0% 91.7%
End-of-turn accuracy, simulated GSM codec 88.5% 92.4% 90.5%
Speech missed on real phone calls 28% 15% n/a
False alarms on real phone calls 8% 13% n/a

Bold is the best number in each row; for the last two rows lower is better. Smart Turn is Pipecat's end-of-turn model, shown for comparison. It does not detect speech.

Things to know before using it:

  • The end-of-turn numbers come from simulated phone lines. End of turn on real calls could not be measured, because no real-call test set has labels for it (see "What could not be measured").
  • On real calls, most of the speech-detection gain is a threshold effect. The original gets close if you lower its speech threshold from 0.5 to 0.17.
  • The fine-tune does not teach the model anything new. Through a phone line the original goes quiet (on meeting recordings it misses 39% to 43% of the speech instead of 26%), and this version detects speech on the line about as well as the original does on wideband audio.
  • It also says "ended" more readily at pauses. On the real calls, 0.4 s into a pause where the same person then goes on, it says "ended" 80% of the time against 63% for the original. In a voice agent that means more interruptions at the default threshold, so set the threshold and wait time on your own calls.
  • The 8-bit ONNX file is a quarter of the size and scores the same.

Test it on your own call audio before relying on it.

This is an unofficial fine-tune. It is not made or endorsed by Inworld.

Model

Property Value
Parameters 25.5 M, same architecture as inworld/sava (Mamba-2 and sliding-window attention trunk, speech and end-of-turn heads)
Audio in 16 kHz mono float32 in [-1, 1]. Upsample 8 kHz call audio to 16 kHz first
ONNX inputs pcm float32 [1, 640]: one 40 ms frame
state float32 [1, 798707]: all zeros at the start of a stream
ONNX outputs speech [1, 1]: logit, is this frame speech
lookahead [1, 1]: logit, was the frame 3 frames back speech (the more accurate one; -30 for a stream's first 3 frames)
turn [1, 1, 4]: logits, has the turn ended, at horizons of 0, 5, 10 and 20 frames
new_state [1, 798707]: pass back in with the next frame
Opset 20, standard operators only
Upstream inworld/sava at revision 1beae7c1dd0d

Files

File Size Description
sava-telephony.safetensors 102 MB PyTorch weights, loads with the upstream inworld-sava package
sava-telephony.onnx 103 MB float32 ONNX, same output as the PyTorch weights
sava-telephony-int8.onnx 27 MB 8-bit weights (dynamic quantization) in the 27 large linear layers; front end, attention and recurrence in float32
sava_onnx.py β€” optional helper: push audio of any length, get per-frame probabilities
config.json β€” input and output contract, upstream revision, fine-tuning and quantization details
scripts/ β€” the fine-tuning, export, quantization and benchmark code that produced these files and numbers
results/ β€” the training log and the scores behind the tables below
LICENSE, NOTICE β€” Apache-2.0 license and Inworld's notice for the upstream weights

Performance

Each recording is run as a stream from its start, one frame per call. Three conditions:

  • Wideband: the test audio as published, 16 kHz.
  • Landline: the same audio at 8 kHz, 300–3400 Hz, through G.711 mu-law.
  • Mobile: the same band through the GSM 06.10 full-rate codec. No GSM audio was used in fine-tuning.

Both lines are simulated. Higher is better.

End of turn: all 31,527 clips of pipecat-ai/smart-turn-data-v3.2-test, score read at the clip's last frame, threshold 0.5. Accuracy, with AUC in brackets.

Model Wideband Landline Mobile
inworld/sava (upstream) 93.36% (0.9795) 89.52% (0.9575) 88.52% (0.9493)
sava-telephony.safetensors 93.50% (0.9797) 93.03% (0.9772) 92.42% (0.9740)
sava-telephony.onnx 93.50% (0.9797) 93.03% (0.9772) 92.42% (0.9740)
sava-telephony-int8.onnx 93.52% (0.9796) 93.02% (0.9771) 92.44% (0.9738)
Pipecat Smart Turn v3.2, for reference 93.71% (0.9825) 91.71% (0.9724) 90.48% (0.9664)

Differences in accuracy, paired over clips, in points with 95% intervals:

Wideband Landline Mobile
This model minus upstream +0.15 (-0.00 to +0.28) +3.51 (+3.23 to +3.81) +3.90 (+3.58 to +4.21)
This model minus Smart Turn v3.2 -0.20 (-0.48 to +0.08) +1.33 (+1.01 to +1.64) +1.94 (+1.59 to +2.27)
8-bit file minus this model +0.02 (-0.06 to +0.10) -0.02 (-0.10 to +0.07) +0.02 (-0.08 to +0.11)

The intervals treat clips as independent. Many clips share a speaker or a source, so the true intervals are somewhat wider; the gains on the phone lines are large enough that this does not change them.

  • A phone line now costs this model 0.5 points (landline) and 1.1 points (mobile), where it cost upstream 3.8 and 4.8 points and Smart Turn 2.0 and 3.2 points.
  • On the test set's 5,863 real recordings (the rest is synthetic speech) this model scores 95.7%, 95.3% and 94.6%; upstream 95.8%, 90.2% and 89.5%; Smart Turn 94.8%, 93.1% and 92.3%.
  • On wideband audio this model's accuracy is within noise of both others'; Smart Turn has the higher AUC.
  • Smart Turn v3.2 (smart-turn-v3.2-gpu.onnx) was scored the way its own inference code runs it, on the last 8 seconds of each clip.

Speech detection: the 30 labeled recordings of the TEN-VAD test set, 6,570 frames. AUC, with F1 in brackets.

Model Wideband Landline Mobile
inworld/sava (upstream) 0.9768 (0.955) 0.9298 (0.882) 0.9090 (0.871)
sava-telephony.safetensors 0.9792 (0.959) 0.9782 (0.958) 0.9737 (0.951)
sava-telephony.onnx 0.9792 (0.959) 0.9782 (0.958) 0.9737 (0.951)
sava-telephony-int8.onnx 0.9791 (0.959) 0.9782 (0.958) 0.9732 (0.950)

AUC against upstream, paired over recordings: +0.002 (-0.001 to +0.008) wideband, +0.048 (+0.021 to +0.085) landline, +0.065 (+0.024 to +0.119) mobile. This set is small, so the intervals are wide.

sava-telephony.onnx makes the same decision as the PyTorch weights on every one of the 31,527 end-of-turn clips in all three conditions; the largest difference in probability is 6e-6. sava-telephony-int8.onnx changes the decision on 0.5% to 0.7% of clips, about as often for the better as for the worse.

Real phone calls: 28 calls, 3.9 hours, from CallHome English (23 with two speakers, 5 with three or four), with hand-marked times of who spoke when. Everyone is in one channel, and the audio is used as the dataset ships it, already at 16 kHz. Models run at their default threshold of 0.5.

Speech detection, 354,555 frames:

Model AUC F1 Accuracy Speech missed False alarms
inworld/sava (upstream) 0.907 0.829 74.4% 28.3% 8.0%
sava-telephony.safetensors 0.920 0.911 85.6% 14.6% 13.1%
sava-telephony-int8.onnx 0.920 0.911 85.5% 14.7% 13.0%

Accuracy is a poor measure on these calls: 87% of the frames are marked as speech, so answering "speech" on every frame would score 86.6%, more than either model. The misses, the false alarms and the AUC say more. This model misses 14.6% of the speech instead of 28.3% and raises more false alarms, 13.1% instead of 8.0%. Its AUC, which does not depend on the threshold, is higher by 0.013 (0.009 to 0.017, paired over calls); that part is better detection.

The rest is the model saying "speech" more readily on phone audio, which lowering upstream's threshold also does. At a threshold of 0.17, chosen on these same calls so that upstream says "speech" as often as this model does at 0.5, upstream misses 15.1% and raises 16.2% false alarms. The marked times are approximate, and every model scores lower here than on the TEN-VAD set.

At pauses this model says "ended" more often than upstream, whoever speaks next. 0.4 s into a pause it says so at 93% of the pauses where the other person speaks next (upstream 77%) and at 80% of those where the same person goes on (upstream 63%), 16 points more for each (95% intervals 8 to 25 and 11 to 22, paired over calls).

Meetings, as recorded and through the simulated lines: 6 meetings, 3.0 hours, from the AMI corpus (headset microphones, wideband), with marked times of who spoke when. Here the same audio is on both sides of the comparison, so the table shows what a phone line does. The last four rows read the end-of-turn probability 0.4 s into each of 446 pauses of at least 0.5 s.

At the default threshold of 0.5 Wideband Landline Mobile
Speech missed, upstream 25.7% 39.0% 42.9%
Speech missed, this model 24.6% 25.3% 25.7%
Says "ended" when another person speaks next, upstream 85% 65% 66%
Says "ended" when another person speaks next, this model 86% 84% 89%
Says "ended" when the same person goes on, upstream 71% 53% 49%
Says "ended" when the same person goes on, this model 72% 76% 77%

Through a phone line upstream goes quiet at its default thresholds. This model on a phone line detects speech as upstream does on wideband audio, and says "ended" a little more often where the same person goes on (76% and 77% against 71%).

What could not be measured: end of turn on real calls. No real-call test set we could use says whether a turn has ended. An indirect test was tried on the real calls and on the meetings: at a pause, does the end-of-turn probability tell "another person speaks next" from "the same person goes on"? Every model scores low on it, and a phone line barely moves the score: on the meetings upstream scores an AUC of 0.61 on wideband audio and 0.59 and 0.58 through the lines, this model 0.62, 0.61 and 0.61, Smart Turn 0.64, 0.62 and 0.65. On the real calls upstream scores 0.58, this model 0.61 and Smart Turn 0.69, the one clear difference. A test on which the line costs upstream almost nothing cannot show what this fine-tune recovers, so it is not used for any claim here. Its numbers are in results/real-calls.json and results/meetings.json.

Speed: one 40 ms frame on one CPU thread of an Apple M4 Max with ONNX Runtime 1.32, averaged over a 23 s recording after two warm-up passes (scripts/speed.py): 1.5 ms for sava-telephony.onnx, 1.2 ms for sava-telephony-int8.onnx. The graph is the one in matthiastl/sava-onnx with different weights; that repository has iPhone and browser timings, which were not repeated for these files.

Usage

Phone audio arrives at 8 kHz. Decode it to linear PCM and upsample it to 16 kHz before the model; the simulated lines it was trained and tested on are upsampled with scipy.signal.resample_poly(x, 2, 1).

import numpy as np
import onnxruntime as ort

session = ort.InferenceSession("sava-telephony-int8.onnx", providers=["CPUExecutionProvider"])
state = np.zeros((1, 798707), dtype=np.float32)   # a new stream starts from zeros


def step(frame: np.ndarray) -> tuple[float, float]:
    """One 40 ms frame (640 float32 samples at 16 kHz) -> P(speech), P(turn ended)."""
    global state
    _, lookahead, turn, state = session.run(None, {"pcm": frame[None], "state": state})
    sigmoid = lambda x: 1.0 / (1.0 + np.exp(-x))
    return float(sigmoid(lookahead[0, 0])), float(sigmoid(turn[0, 0, 0]))

P(speech) from lookahead describes the frame 120 ms earlier; use speech for an on-time answer. P(turn ended) is on time. Open a turn when speech starts and close it when the turn probability crosses your threshold; the upstream repository documents a reference policy.

sava_onnx.py wraps this loop with the same push / flush interface as inworld.sava.Session.

In PyTorch, with pip install inworld-sava:

from inworld import sava

model = sava.load("sava-telephony.safetensors")
session = sava.Session(model)   # then push audio as with the upstream weights

How it was made

  • Fine-tuning. Two copies of the upstream model. The frozen one hears each clip as published. The other hears the same clip through a random simulated phone line and is trained to give the frozen one's answers, frame by frame, on all three outputs. Training uses no labels; the labels of the tenth file below were only used to pick the checkpoint. One clip in four is left clean so the wideband behaviour stays put.
  • Training lines. 8 kHz; band-pass with a low edge of 150–400 Hz and a high edge of 3000–3700 Hz; G.711 mu-law, A-law or no codec; input level 0 to -20 dB; white line noise on 30% of clips, at -70 to -30 dB relative to full scale. GSM was kept out of training to serve as the unseen line in the tests.
  • Data. 29,385 clips from 9 of the 83 files of pipecat-ai/smart-turn-data-v3.2-train, a set the upstream model was itself trained on, so no new speech data is involved. A tenth file chose the checkpoint, scored clean and through the test landline. 2,500 steps of batch 32 took 90 minutes on an Apple M4 Max GPU. Midway through, the step-500 checkpoint was scored once on 8,000 of the test clips to check that training was working; nothing was chosen from it.
  • Export and quantization. As in matthiastl/sava-onnx: torch.onnx.export on the one-frame streaming step, then ONNX Runtime dynamic quantization of the 27 learned linear layers with at least 100,000 weights.
  • Checks. scripts/test_parity.py compares the exports with the PyTorch weights; scripts/bench.py and scripts/compare.py produced the tables above, and scripts/calls.py the real-call ones. config.json has the full settings and scripts/README.md the commands.

Limitations

  • The end-of-turn gain is measured on simulated lines only. End of turn on real calls is not measured; see "What could not be measured".
  • At the default threshold the model says "ended" 0.4 s into most pauses in conversation, whoever speaks next, and more often than upstream: on the real calls at 80% of the pauses where the same person goes on, against 63%. Pick your threshold and wait time on your own audio.
  • Part of the real-call gain in speech detection is a matter of threshold: upstream with its speech threshold at 0.17, chosen on the same calls, comes close.
  • The real-call test is small and old: 28 English calls from the 1990s with both speakers in one channel. It does not cover one caller's side alone, which is what a voice agent hears, nor packet loss, echo or current codecs (AMR, Opus, EVS). Test on your own call audio before relying on the model.
  • The landline condition falls inside the family of lines used in training; the mobile condition does not, and it is the fairer estimate of an unseen line.
  • The end-of-turn test set comes from the same collection as the fine-tuning audio, as it does for upstream and for Smart Turn.
  • Fine-tuning copies upstream's wideband answers, so it cannot make the model better than upstream on wideband audio, and everything upstream says about the model still applies: it responds to any voice, does not pick out a target speaker, and leaves interruptions and backchannels to the application.
  • One training run, on 11% of the training set. More data, more steps and real codecs in the training lines were not tried.

License and attribution

The upstream code and weights are Β© 2026 Inworld AI, released under Apache-2.0. These files are a fine-tuned, converted and, for sava-telephony-int8.onnx, quantized form of those weights, distributed under the same license. LICENSE and NOTICE are the upstream files; NOTICE also lists what was changed. The fine-tuning audio is from pipecat-ai/smart-turn-data-v3.2-train (CC BY 4.0). The end-of-turn test set is pipecat-ai/smart-turn-data-v3.2-test, and the speech-detection test set is the testset/ folder of TEN-framework/ten-vad.

The real-call test uses the first 28 calls of the English part of talkbank/callhome (CABank English CallHome Corpus, TalkBank, doi:10.21415/T5KP54, CC BY-NC-SA 4.0). It was used for testing only: the model was not trained on it, and none of its audio or annotation is in this repository.

The meeting test uses the first 6 meetings of the test split of diarizers-community/ami (AMI Meeting Corpus, headset mix, CC BY 4.0), also for testing only.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for matthiastl/sava-telephony

Base model

inworld/sava
Finetuned
(1)
this model

Dataset used to train matthiastl/sava-telephony