Clearline-VAD

Streaming voice activity detection for LiveKit Agents, a drop-in for silero.VAD. 16 kHz, one probability every 16 ms, CPU only, for wideband and telephone audio. Two models: the release (55k parameters, the plugin default, chosen for turns) and a distilled variant (127k parameters with per-band level trackers, the better frame classifier, taught by a Whisper-encoder teacher what rooms full of other voices sound like). Both are trained to hear audio that a noise filter has been through, as a LiveKit agent runs one before its VAD.

2026-10-29: both defaults replaced. In a LiveKit agent the noise filter runs before the VAD, and behind one that removes the noise outright (Clearline-NC's quality mode, STT-API's GTCRN) the previous models lost more than Silero v6: they heard the filtered pauses as loud. The new defaults are the same graphs fine-tuned with half their training examples heard through Clearline-NC first, chosen by a rule fixed before the runs and judged behind GTCRN, which they never heard: mean frame ROC-AUC over the eight development sets behind it 0.932 (release) and 0.939 (distilled), from 0.925 and 0.932 (Silero v6 0.933). With no filter they are where the previous models were (13 test sets: 0.920 and 0.921). On FLEURS under DEMAND behind Clearline-NC's quality mode the distilled variant scores 0.916 against Silero v6's 0.897. The one loss is distant talkers behind a strong filter: AliMeeting dev behind the quality mode 0.891 (release) and 0.834 (distilled), from 0.903 and 0.894. The release keeps its operating point at 512-sample windows; the distilled variant's moves (below). The previous models are revision 8bd11bca.

Work in progress. Clearline is still being trained, and this repository is updated as better models pass our development-set rules; every upload is its own revision, so pin one (STT-API's plugin does). In progress now: the distilled variant on far-field meetings, where it misses distant talkers that the release keeps (AliMeeting), and heavy noise, where Silero v6 still leads by a little (FLEURS under DEMAND at 0 and -5 dB). On QUT-NOISE, a noise corpus recorded for testing voice activity detection (ten places: cafes, homes, streets, a car, reverberant rooms), the distilled variant leads Silero v6 at every SNR and through the phone codecs; its one weak place there is a cafe. Tried since, and not shipped because none passed the development-set rules fixed before it was scored: a telephone-band front end (it matches the distilled variant on the phone at the release's cost), causal time convolutions, 2D convolutions over time and band, a learnt filterbank and PCEN features (all level with the GRU or behind it); then, for the cafe, crowds of talkers and louder cafe noise in training, Silero VAD as a second teacher, bigger Whisper teachers, and model soups and task arithmetic over the results (the best a thousandth ahead of the distilled variant, where the bar is three). The four best of those are the first four candidates below. Last, both graphs trained from scratch on targets blended with Silero: they keep far-field meetings (AliMeeting dev up to 0.90, against 0.80 for the distilled variant) but stop short of the distilled variant on the other development sets, and on the release's graph they learn Silero's later turn ends with its frames; the best of them is the fifth candidate, for meetings. The numbers below the candidates are the models in this revision.

Candidates (not the defaults)

Five models from the last two passes sit beside the defaults in candidates/, for anyone who wants to try them. None passed the rule that would make it a default, and the plugin never loads them unless asked. Each was distilled, wholly or in part, with Silero VAD (MIT) as one of the teachers: it is causal, as our models are, and it is right where the Whisper teachers are wrong, on distant talkers above all.

• candidates/clearline_vad_release_silero.onnx (and _256.onnx, and candidates/pytorch/): the release's 64-unit graph fine-tuned on a target half whisper-small, half Silero v5. Better frames than the release, in meetings most, but it catches fewer turn onsets and ends at its own operating point, so the release rule scores it under the default.
• candidates/clearline_vad_distilled_silero.onnx (and the same): the distilled variant fine-tuned on whisper-small with 0.4 Silero v6. A little better than the distilled variant on AMI and VoxConverse, the same in cafes and on AliMeeting; at its own operating point it ends a few more turns with fewer false triggers.
• candidates/clearline_vad_release_mix.onnx: the release with a quarter of the first candidate averaged into its weights. Turns as the release's, a little better on frames and meetings.
• candidates/clearline_vad_distilled_soup.onnx: a model soup, the weights of the distilled candidate averaged with a run trained on crowds of talkers and cafe noise as loud as the speaker: the best in cafes, weaker on AliMeeting.
• candidates/clearline_vad_meetings.onnx (and _256.onnx, and candidates/pytorch/), for meetings: the distilled variant's graph trained from scratch (72k steps) on whisper-small with 0.4 Silero v6, so it never picked up the distilled variant's habit of treating a quiet voice as background. Ahead of both defaults on AMI dev with either microphone (0.965 close-talk and 0.950 distant, against 0.963 and 0.943 for the distilled variant; Silero v6 0.961 and 0.950), it keeps AliMeeting's distant talkers as the release does (0.868 against 0.869; the distilled variant 0.798), and at its own operating point its turns come close to the release's with fewer false triggers. Behind the distilled variant in noise and on the phone.

Development sets only (frame ROC-AUC; turns at each model's own operating point, LiveKit's state machine replayed on AMI dev and VoxConverse dev, wideband and on the phone):

model	file	eight dev sets	QUT's cafes	AliMeeting dev	turn onsets	turn ends	false/h	release rule	threshold / ExpFilter
Clearline-VAD (default)	clearline_vad.onnx	0.9423	0.910	0.853	0.936	0.882	32.6	0.9090	0.825 / 0.35
the previous default (before 2026-10-29)		0.9412	0.908	0.869	0.936	0.884	33.5	0.9101	0.825 / 0.35
release size, distilled with Silero v5	candidates/clearline_vad_release_silero.onnx	0.9440	0.908	0.886	0.928	0.872	31.6	0.8997	0.7 / 0.35
the release, 1/4 of that mixed in	candidates/clearline_vad_release_mix.onnx	0.9425	0.909	0.875	0.937	0.882	32.7	0.9094	0.8 / 0.35
Clearline-VAD, distilled	clearline_vad_distilled.onnx	0.9485	0.921	0.797	0.923	0.860	25.9	0.8919	0.825 / 0.35
the previous distilled variant		0.9483	0.919	0.798	0.917	0.847	24.5	0.8817	0.85 / 0.2
distilled, with Silero v6	candidates/clearline_vad_distilled_silero.onnx	0.9493	0.920	0.799	0.915	0.854	22.7	0.8842	0.825 / 0.2
soup: that and the crowd run	candidates/clearline_vad_distilled_soup.onnx	0.9495	0.922	0.779	0.918	0.855	23.6	0.8864	0.825 / 0.2
meetings: from scratch, with Silero v6	candidates/clearline_vad_meetings.onnx	0.9456	0.916	0.868	0.931	0.879	27.9	0.9048	0.825 / 0.2
Silero v6		0.9398	0.907	0.885	0.903	0.846	33.7	0.8748	0.7 / 0.35

from huggingface_hub import hf_hub_download
from clearline.livekit import VAD

path = hf_hub_download("Scicom-intl/Clearline-VAD", "candidates/clearline_vad_release_silero.onnx")
vad = VAD.load(onnx_file_path=path, activation_threshold=0.7, smoothing_alpha=0.35)  # the table's point, 512 samples

	Silero v6	Clearline-VAD	Clearline-VAD, distilled
model CPU per 32 ms, one core	103 µs	60 µs	83 µs
plugin CPU per stream, 16 kHz, 8 streams (% of a core)	1.19%	0.47%	not measured
plugin CPU per stream, 48 kHz, 8 streams	1.57%	1.03%	not measured
mean frame ROC-AUC, 13 test sets	0.912	0.920 (higher on 8)	0.921 (higher on 10)
LiveKit: turns detected / median onset	0.801 / 247 ms	0.835 / 261 ms	0.801 / 465 ms
LiveKit: turn ends on time	0.718	0.757	0.728
LiveKit: false triggers per hour, 10 sets	29.6	35.1	21.4
LiveKit: false triggers per hour, without the private telephone sets	28.1	23.6	14.3
telephone, frame ROC-AUC: FLEURS through 5 codecs	0.936	0.924	0.939
telephone, frame ROC-AUC: the dev sets on the phone	0.963	0.961	0.966
telephone, LiveKit on AMI dev: onsets / ends / false per hour	0.877 / 0.750 / 17.4	0.944 / 0.866 / 34.3	0.929 / 0.825 / 22.7

LiveKit rows: every turn replayed through LiveKit's own VAD state machine on 10 test sets, each model at the point it ships with, chosen on the four development sets (AMI dev and VoxConverse dev, wideband and through the phone); Silero v6 at its own best point on the same four (threshold 0.70). The median onset is the mean of the ten sets' medians, and the distilled variant's is mostly CHiME-6's dinner parties, where it catches 6% of turns, 2.7 s late (Silero v6 15%, at 0.5 s); over the other nine its onsets average 220 ms, as Silero's do. Use the release for meetings.

Languages

FLEURS test speech in 40 languages, 60 s streams from 30 to -5 dB SNR and through a phone path, under two noise families: DEMAND (homes, offices, public places, transport) and TUT urban scenes (airports, metro, streets, malls). Frame ROC-AUC per language, conditions pooled; "training h" is that language's hours in our simulated training pool.

language	training h	DEMAND: Silero v6	Clearline-VAD	distilled	TUT: Silero v6	Clearline-VAD	distilled
German	421	0.961	0.971	0.973	0.960	0.980	0.980
English	404	0.943	0.931	0.935	0.951	0.938	0.944
French	214	0.987	0.991	0.992	0.989	0.990	0.989
Spanish	200	0.956	0.956	0.961	0.958	0.951	0.959
Dutch	91	0.889	0.889	0.889	0.893	0.893	0.901
Hausa	88	0.947	0.940	0.941	0.949	0.922	0.953
Italian	71	0.969	0.965	0.968	0.967	0.957	0.963
Mandarin	54	0.946	0.951	0.954	0.932	0.922	0.934
Malay	49	0.929	0.903	0.942	0.919	0.939	0.948
Portuguese	40	0.923	0.924	0.927	0.904	0.909	0.903
Swahili	32	0.903	0.878	0.898	0.961	0.965	0.971
Polish	29	0.981	0.982	0.987	0.977	0.985	0.988
Yoruba	27	0.927	0.911	0.925	0.985	0.982	0.979
Bengali	27	0.960	0.952	0.961	0.958	0.956	0.965
Georgian	21	0.972	0.977	0.980	0.970	0.975	0.978
Japanese	21	0.857	0.872	0.853	0.878	0.885	0.881
Thai	14	0.869	0.857	0.891	0.975	0.971	0.978
Tamil	8	0.946	0.949	0.954	0.949	0.952	0.954
Cantonese	7	0.950	0.961	0.965	0.946	0.944	0.947
Russian	7	0.905	0.903	0.928	0.987	0.984	0.985
Korean	6	0.923	0.901	0.906	0.912	0.892	0.895
Ukrainian	4	0.860	0.804	0.857	0.939	0.940	0.951
Javanese	4	0.944	0.919	0.937	0.885	0.869	0.912
Telugu	4	0.974	0.969	0.976	0.973	0.970	0.981
Burmese	4	0.947	0.938	0.943	0.913	0.925	0.936
Turkish	4	0.873	0.839	0.869	0.913	0.923	0.933
Khmer	3	0.870	0.857	0.880	0.878	0.895	0.891
Urdu	3	0.921	0.912	0.937	0.986	0.989	0.991
Mongolian	3	0.906	0.876	0.898	0.937	0.939	0.944
Persian	2	0.941	0.926	0.935	0.937	0.923	0.931
Arabic	2	0.977	0.975	0.981	0.968	0.970	0.977
Greek	1	0.952	0.969	0.973	0.951	0.972	0.972
Hebrew	<1	0.936	0.952	0.957	0.917	0.927	0.938
Indonesian	<1	0.975	0.960	0.967	0.975	0.970	0.973
Hindi	<1	0.921	0.915	0.923	0.917	0.915	0.927
Amharic	<1	0.990	0.992	0.994	0.974	0.977	0.983
Vietnamese	<1	0.869	0.849	0.861	0.948	0.930	0.949
Lao	<1	0.966	0.962	0.976	0.954	0.939	0.959
Zulu	<1	0.957	0.957	0.959	0.962	0.964	0.964
Filipino	0	0.734	0.700	0.747	0.972	0.972	0.962
all 40, pooled		0.926	0.922	0.933	0.940	0.941	0.948

ahead of Silero v6 in (of 40)	noise	30 dB	10 dB	5 dB	0 dB	-5 dB	phone	pooled
Clearline-VAD	DEMAND	34	16	16	10	11	12	13
Clearline-VAD	TUT	36	22	15	16	23	17	22
distilled	DEMAND	39	32	22	14	14	26	24
distilled	TUT	37	28	25	21	29	25	31

In quiet Clearline-VAD leads in most languages. In noise it depends on the noise: level with Silero under TUT's urban scenes, behind it under DEMAND from 10 dB down. The distilled variant is the stronger of the two under both: ahead of Silero in most languages, and under DEMAND ahead from 5 dB up and a little behind at 0 and -5 dB.

Rooms full of other voices

DEMAND's cafeteria, restaurant and meeting room are where Silero kept its lead, so the distilled variant was chosen on a second unseen noise set with such rooms: FLEURS development sentences under CochlScene (13 scenes recorded on smartphones). Per scene, at 10, 5 and 0 dB, against Silero v6:

A third noise set, QUT-NOISE, was recorded for testing voice activity detection: ten places, a cafe, a food court, a kitchen, a living room, two streets, a car with its windows down and up, a reverberant pool and car park. Under each place's first session the distilled variant leads Silero v6 at every SNR and through the phone codecs. Place by place, at 10, 5 and 0 dB, what is left is the cafes:

On far-field meetings (AliMeeting) it misses distant talkers and fires in their pauses: it learnt that a voice well under the loudest one is someone else's, which in a cafe or behind a caller is true and in a meeting is not. Use the release there (AliMeeting: 0.908, level with Silero v6; the distilled variant 0.849). No model we trained keeps both. Every checkpoint we scored, on AliMeeting's development meetings (every voice is speech) against FLEURS under CochlScene (other people's voices are not):

Telephone

The same FLEURS streams at 10 dB through five codecs at 8 kHz (* never in training), and the two development sets through the training codecs.

frame ROC-AUC	Silero v6	TEN VAD	Clearline-VAD	Clearline-VAD, distilled
G.711	0.940	0.913	0.927	0.941
GSM	0.935	0.900	0.929	0.941
Opus	0.939	0.913	0.924	0.940
AMR-NB*	0.931	0.909	0.919	0.934
G.723.1*	0.934	0.912	0.921	0.939
FLEURS, all five	0.936	0.903	0.924	0.939
dev sets on the phone	0.963	0.944	0.961	0.966

On FLEURS through the five codecs the distilled variant is ahead of Silero on every codec and the release a little behind; Clearline-VAD catches more turns on the line, at more false triggers (first table).

Use

import sys
from huggingface_hub import snapshot_download

sys.path.insert(0, snapshot_download("Scicom-intl/Clearline-VAD"))
from clearline.livekit import VAD

def prewarm(proc):
    proc.userdata["vad"] = VAD.load()      # threshold 0.825 (0.875 with window=256), filter 0.35 by default

session = AgentSession(vad=ctx.proc.userdata["vad"], stt=..., llm=..., tts=...)

Raw ONNX: inputs x [1, 512] float32 at 16 kHz, ctx [1, 256], mean [1, 40] (start at the mean0 metadata value), h0, h1 [1, 1, 64]; outputs p [1, 2] and the new states. The distilled graphs add a floor [1, 40] (start at zero) and carry the peak tracker in mean [1, 80], with h0, h1 [1, 1, 96]; the code below reads the shapes from the graph, so it runs either.

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("clearline_vad.onnx", providers=["CPUExecutionProvider"])
names = [i.name for i in sess.get_inputs()][1:]
state = {i.name: np.zeros(i.shape, np.float32) for i in sess.get_inputs()[1:]}
state["mean"][:] = float(sess.get_modelmeta().custom_metadata_map["mean0"])

def step(chunk):                           # 512 float32 samples at 16 kHz
    global state
    p, *new = sess.run(None, {"x": chunk[None].astype(np.float32), **state})
    state = dict(zip(names, new))
    return p[0]                            # two probabilities, one per 16 ms

Files

file	what
clearline_vad.onnx, clearline_vad_256.onnx	release model, 512- or 256-sample chunks
clearline_vad_distilled.onnx, clearline_vad_distilled_256.onnx	distilled variant (96 units, level trackers)
pytorch/*.pt	PyTorch weights for clearline/vad_model.py
clearline/	the LiveKit plugin, release model bundled

Notes

• Default threshold 0.825, not 0.5 (0.875 with window=256), chosen on development data; smoothing_alpha is LiveKit's ExpFilter alpha (weight of the previous value, 0 = off).
• The distilled graphs have their own points: activation_threshold=0.825 with smoothing_alpha=0.35 at 512-sample windows, 0.75 with 0.5 at 256.
• CPU inside a real LiveKit agent (livekit-agents 1.3.11, one process per call, dummy STT, LLM and TTS so the VAD is the only model): a hundred simultaneous calls take 8.2 cores with Clearline-VAD and 10.2 with Silero v6, 5.6 of them LiveKit's own audio path; an eight-core worker holds about 73 calls at LiveKit's default load limit (0.7) with Clearline-VAD and 59 with Silero v6. With a non-streaming STT (a Whisper endpoint) LiveKit runs the VAD twice per call; a streaming STT halves the VAD's share.
• Feed it 16 kHz audio (AudioInputOptions(sample_rate=16000)): at 48 kHz LiveKit's resampler costs more CPU than the model.
• Weak spot: line silence on telephone legs (more false triggers than Silero); raise activation_threshold there.
• The distilled variant ranks frames best and is the one for noisy rooms and the phone; use it where frames matter more than turns, such as offline segmentation. It costs about 1.4 times the release per 32 ms (still under Silero), its 16 ms graph more than Silero, and on far-field meetings it misses distant talkers: use the release there.
• Training: simulated streams from 2,728 h of speech in 137+ languages, a telephone line with real codecs, and 792 h of labelled real conversation (podcasts, a private telephone dataset, NOTSOFAR-1). No test corpus is in training; checkpoints chosen on AMI dev and VoxConverse dev, wideband and through the phone.
• License not set yet: some training data is CC BY-NC (Expresso, EARS, WHAM!).

Husein Zolkepli. Clearline: voice activity detection and noise cancellation for voice agents, on one CPU core. Scicom (MSC) Berhad, 2026.