--- license: other license_name: mit-and-apache-2.0 license_link: https://opensource.org/licenses/MIT base_model: - pyannote/segmentation-3.0 base_model_relation: quantized pipeline_tag: voice-activity-detection library_name: onnx language: - zh - en tags: - speaker-diarization - pyannote - wespeaker - onnx - 42model --- # Speaker Diarization Models · Who Spoke When (Fully Local) [![License: MIT + Apache 2.0](https://img.shields.io/badge/License-MIT_%2B_Apache_2.0-blue.svg)](https://opensource.org/licenses/MIT) [![Runs Local](https://img.shields.io/badge/Runs-100%25_Local-brightgreen.svg)](https://42model.com) [![Platform](https://img.shields.io/badge/Platform-macOS_|_Windows_|_Linux-lightgrey.svg)](https://42model.com) [![ModelScope](https://img.shields.io/badge/ModelScope-42ailab-624AFF.svg)](https://modelscope.cn/models/42ailab/Speaker-Diarization-Models) **English** | [简体中文](./README_zh.md) · Also on [ModelScope](https://modelscope.cn/models/42ailab/Speaker-Diarization-Models) · [42ailab.com](https://42ailab.com) Give it a multi-speaker conversation recording and this model marks **who said each segment** (Speaker 1, Speaker 2, …) — entirely on your own computer. **No cloud, free, private.** > **The models were built by two open-source teams**: [pyannote](https://github.com/pyannote/pyannote-audio) (speech segmentation, MIT) and [WeSpeaker](https://github.com/wenet-e2e/wespeaker) (speaker embeddings, Apache-2.0); the ONNX deployment packaging follows [avencera/speakrs](https://github.com/avencera/speakrs) (Apache-2.0). **This repo is not a new model** — we converted these open models into deployment builds that **run offline on your computer** and verified their quality. It is an **enhancement model** and does not transcribe: transcription is still done by [Qwen3-ASR](https://huggingface.co/Qwen/Qwen3-ASR-0.6B), and this model adds the "who is speaking" information on top. ## 1. The problem it solves Speech-to-text tells you "what was said", but in a multi-speaker conversation it often cannot tell you **who said this line**. Many scenarios depend precisely on that distinction: - **Meeting minutes**: organize notes by speaker so each party's position is visible at a glance; - **Interviews and podcasts**: separate host from guest, making transcripts and quotations easier to prepare; - **Customer service and QA**: distinguish agent from customer and analyze each separately. Doing speaker diarization used to mean uploading the recording to a cloud service — with both privacy concerns and possible costs. This model keeps that step **entirely local**. ## 2. How it works It chains three steps into a "who spoke when" pipeline: - **Segmentation** (pyannote): first find which stretches of audio contain speech, and where the speaker changes; - **Speaker embeddings** (WeSpeaker): extract a "voiceprint" vector representing the voice characteristics of each short segment; - **Clustering**: group segments with similar voiceprints as the same person, and finally attach a speaker label to each segment. **What we did**: converted the upstream open weights into locally runnable deployment builds, verified that quality did not degrade in conversion, and made them **run offline** across macOS / Windows / Linux; we also selected the right acceleration path for Apple silicon (CoreML), NVIDIA GPUs (CUDA) and **plain CPU**, so machines without a discrete GPU can use it too. ## 3. How well it performs For the complete diarization-quality benchmarks (DER, lower is better), refer to upstream public evaluations — per [speakrs benchmarks](https://github.com/avencera/speakrs/tree/master/benchmarks), this pipeline (CoreML) achieves about **7.1% DER** on the VoxConverse **Dev** subset, essentially on par with pyannote community-1 (about 7.2%). Here we list only **our own verification**: | What we verified | Result | |---|---| | Correctness on a Chinese two-person conversation (5-minute real recording) | **Correctly separated 2 speakers**, with sensible speaker-change points | | Speed · Apple silicon (CoreML) | **~380× real time** (5 minutes of audio in about 0.8 s) | | Speed · plain CPU | **~1.9× real time** (5 minutes of audio in about 2.6 minutes), usable on machines with no GPU | | Memory use (plain CPU) | About 1.2 GB | > Note: "correctness", "speed" and "memory" above are our own measurements on an Apple M3 Max; "7.1% DER (VoxConverse Dev)" is a public figure from upstream **speakrs** (a third-party CoreML port's evaluation — neither pyannote official nor measured by us). ## 4. Limitations and what's next - **Requires a transcription model**: this model only answers "who is speaking" and does not transcribe; use it together with Qwen3-ASR. - **Plain-CPU speed**: extremely fast with Apple / NVIDIA acceleration; about 1.9× real time on plain CPU — usable, but clearly slower than the accelerated tiers, so process long recordings on an accelerated machine where possible. - **Overlapping speech**: where two people talk at once, speaker attribution may be imprecise (a shared difficulty of speaker diarization). - Next: keep improving plain-CPU speed and handling of overlapping segments. ## 5. How to download and use it This model is packaged for [42model](https://42model.com), which is the recommended way to get it: **Desktop app** 1. Open **Model Library → Transcription** and download **Qwen3-ASR Enhanced** (speaker diarization is built in); 2. When transcribing, turn on **Speaker diarization** in the parameter settings, and results will be segmented by speaker. ## Files and license | File | Role | Source · License | |---|---|---| | `segmentation-3.0.onnx` | Speech segmentation / speaker-change detection | pyannote/segmentation-3.0 · **MIT** | | `wespeaker-voxceleb-resnet34.onnx` | Speaker embedding extraction | WeSpeaker voxceleb ResNet34 · **Apache-2.0** | | `plda_lda.npy`, `plda_tr.npy`, `plda_mu.npy`, `plda_psi.npy`, `plda_mean1.npy`, `plda_mean2.npy` | Voiceprint clustering parameters (PLDA / VBx, 6 files) | speakrs pipeline · Apache-2.0 | | `manifest.json` | sha256 / size listing for each file (generated at release, for integrity self-verification) | Generated by this repo | Per-file sha256 values are in the bundled `manifest.json` and on the Files page, and can be verified independently. CoreML acceleration on Apple silicon is compiled automatically on first local run — no extra files needed. **License**: this repo contains local deployment builds of open models from several sources — speech segmentation is [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0) (© pyannote.audio, **MIT**; the upstream HF page requires accepting terms for access, but the license is MIT and redistribution is unrestricted — the ONNX here comes from the same weights under the same license); speaker embeddings are [WeSpeaker](https://github.com/wenet-e2e/wespeaker) voxceleb ResNet34 (© the WeNet community, **Apache-2.0**); the ONNX packaging and pipeline follow [avencera/speakrs](https://github.com/avencera/speakrs) (Apache-2.0). Each file is governed by its original license; by using them you agree to the respective upstream license terms. ## Citation For **the models themselves**, please cite upstream pyannote and WeSpeaker: ```bibtex @inproceedings{Plaquet23, author = {Alexis Plaquet and Herv{\'e} Bredin}, title = {{Powerset multi-class cross entropy loss for neural speaker diarization}}, booktitle = {Proc. INTERSPEECH 2023}, year = {2023}, } @inproceedings{Bredin23, author = {Herv{\'e} Bredin}, title = {{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}}, booktitle = {Proc. INTERSPEECH 2023}, year = {2023}, } @inproceedings{wang2023wespeaker, author = {Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin}, title = {{Wespeaker: A research and production oriented speaker embedding learning toolkit}}, booktitle = {ICASSP 2023}, year = {2023}, } ``` If **this repo's local deployment packaging** was useful to you, you may additionally cite: ```bibtex @misc{yang2026diarizationlocal, title = {Speaker Diarization Models: A Local Build of pyannote + WeSpeaker for On-Device Diarization}, author = {Yang, Zhiping}, year = {2026}, howpublished = {\url{https://huggingface.co/42ailab/Speaker-Diarization-Models}}, organization = {42ailab}, note = {Local deployment packaging (ONNX + cross-platform acceleration); the models themselves are pyannote/segmentation-3.0 (MIT) and WeSpeaker voxceleb ResNet34 (Apache-2.0). Contact: contact@42ailab.com} } ``` Contact us: **contact@42ailab.com** ## About us **[42ailab](https://42ailab.com)** — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike. **[42model](https://42model.com)** — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.