README.md

---
license: other
license_name: mit-and-apache-2.0
license_link: https://opensource.org/licenses/MIT
base_model:
  - pyannote/segmentation-3.0
base_model_relation: quantized
pipeline_tag: voice-activity-detection
library_name: onnx
language:
  - zh
  - en
tags:
  - speaker-diarization
  - pyannote
  - wespeaker
  - onnx
  - 42model
---

<!--
  README for the Hugging Face repo `42ailab/Speaker-Diarization-Models`, uploaded with the diarization model package.
  Editing this file changes the repo landing page. Multi-source models: pyannote segmentation (MIT) + WeSpeaker embeddings (Apache-2.0).
-->
# Speaker Diarization Models · Who Spoke When (Fully Local)

[![License: MIT + Apache 2.0](https://img.shields.io/badge/License-MIT_%2B_Apache_2.0-blue.svg)](https://opensource.org/licenses/MIT)
[![Runs Local](https://img.shields.io/badge/Runs-100%25_Local-brightgreen.svg)](https://42model.com)
[![Platform](https://img.shields.io/badge/Platform-macOS_|_Windows_|_Linux-lightgrey.svg)](https://42model.com)
[![ModelScope](https://img.shields.io/badge/ModelScope-42ailab-624AFF.svg)](https://modelscope.cn/models/42ailab/Speaker-Diarization-Models)

**English** | [简体中文](./README_zh.md) · Also on [ModelScope](https://modelscope.cn/models/42ailab/Speaker-Diarization-Models) · [42ailab.com](https://42ailab.com)

Give it a multi-speaker conversation recording and this model marks **who said each segment** (Speaker 1, Speaker 2, …) — entirely on your own computer. **No cloud, free, private.**

> **The models were built by two open-source teams**: [pyannote](https://github.com/pyannote/pyannote-audio) (speech segmentation, MIT) and [WeSpeaker](https://github.com/wenet-e2e/wespeaker) (speaker embeddings, Apache-2.0); the ONNX deployment packaging follows [avencera/speakrs](https://github.com/avencera/speakrs) (Apache-2.0). **This repo is not a new model** — we converted these open models into deployment builds that **run offline on your computer** and verified their quality. It is an **enhancement model** and does not transcribe: transcription is still done by [Qwen3-ASR](https://huggingface.co/Qwen/Qwen3-ASR-0.6B), and this model adds the "who is speaking" information on top.

## 1. The problem it solves

Speech-to-text tells you "what was said", but in a multi-speaker conversation it often cannot tell you **who said this line**. Many scenarios depend precisely on that distinction:

- **Meeting minutes**: organize notes by speaker so each party's position is visible at a glance;
- **Interviews and podcasts**: separate host from guest, making transcripts and quotations easier to prepare;
- **Customer service and QA**: distinguish agent from customer and analyze each separately.

Doing speaker diarization used to mean uploading the recording to a cloud service — with both privacy concerns and possible costs. This model keeps that step **entirely local**.

## 2. How it works

It chains three steps into a "who spoke when" pipeline:

- **Segmentation** (pyannote): first find which stretches of audio contain speech, and where the speaker changes;
- **Speaker embeddings** (WeSpeaker): extract a "voiceprint" vector representing the voice characteristics of each short segment;
- **Clustering**: group segments with similar voiceprints as the same person, and finally attach a speaker label to each segment.

**What we did**: converted the upstream open weights into locally runnable deployment builds, verified that quality did not degrade in conversion, and made them **run offline** across macOS / Windows / Linux; we also selected the right acceleration path for Apple silicon (CoreML), NVIDIA GPUs (CUDA) and **plain CPU**, so machines without a discrete GPU can use it too.

## 3. How well it performs

For the complete diarization-quality benchmarks (DER, lower is better), refer to upstream public evaluations — per [speakrs benchmarks](https://github.com/avencera/speakrs/tree/master/benchmarks), this pipeline (CoreML) achieves about **7.1% DER** on the VoxConverse **Dev** subset, essentially on par with pyannote community-1 (about 7.2%). Here we list only **our own verification**:

| What we verified | Result |
|---|---|
| Correctness on a Chinese two-person conversation (5-minute real recording) | **Correctly separated 2 speakers**, with sensible speaker-change points |
| Speed · Apple silicon (CoreML) | **~380× real time** (5 minutes of audio in about 0.8 s) |
| Speed · plain CPU | **~1.9× real time** (5 minutes of audio in about 2.6 minutes), usable on machines with no GPU |
| Memory use (plain CPU) | About 1.2 GB |

> Note: "correctness", "speed" and "memory" above are our own measurements on an Apple M3 Max; "7.1% DER (VoxConverse Dev)" is a public figure from upstream **speakrs** (a third-party CoreML port's evaluation — neither pyannote official nor measured by us).

## 4. Limitations and what's next

- **Requires a transcription model**: this model only answers "who is speaking" and does not transcribe; use it together with Qwen3-ASR.
- **Plain-CPU speed**: extremely fast with Apple / NVIDIA acceleration; about 1.9× real time on plain CPU — usable, but clearly slower than the accelerated tiers, so process long recordings on an accelerated machine where possible.
- **Overlapping speech**: where two people talk at once, speaker attribution may be imprecise (a shared difficulty of speaker diarization).
- Next: keep improving plain-CPU speed and handling of overlapping segments.

## 5. How to download and use it

This model is packaged for [42model](https://42model.com), which is the recommended way to get it:

**Desktop app**
1. Open **Model Library → Transcription** and download **Qwen3-ASR Enhanced** (speaker diarization is built in);
2. When transcribing, turn on **Speaker diarization** in the parameter settings, and results will be segmented by speaker.

## Files and license

| File | Role | Source · License |
|---|---|---|
| `segmentation-3.0.onnx` | Speech segmentation / speaker-change detection | pyannote/segmentation-3.0 · **MIT** |
| `wespeaker-voxceleb-resnet34.onnx` | Speaker embedding extraction | WeSpeaker voxceleb ResNet34 · **Apache-2.0** |
| `plda_lda.npy`, `plda_tr.npy`, `plda_mu.npy`, `plda_psi.npy`, `plda_mean1.npy`, `plda_mean2.npy` | Voiceprint clustering parameters (PLDA / VBx, 6 files) | speakrs pipeline · Apache-2.0 |
| `manifest.json` | sha256 / size listing for each file (generated at release, for integrity self-verification) | Generated by this repo |

Per-file sha256 values are in the bundled `manifest.json` and on the Files page, and can be verified independently. CoreML acceleration on Apple silicon is compiled automatically on first local run — no extra files needed.

**License**: this repo contains local deployment builds of open models from several sources — speech segmentation is [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0) (© pyannote.audio, **MIT**; the upstream HF page requires accepting terms for access, but the license is MIT and redistribution is unrestricted — the ONNX here comes from the same weights under the same license); speaker embeddings are [WeSpeaker](https://github.com/wenet-e2e/wespeaker) voxceleb ResNet34 (© the WeNet community, **Apache-2.0**); the ONNX packaging and pipeline follow [avencera/speakrs](https://github.com/avencera/speakrs) (Apache-2.0). Each file is governed by its original license; by using them you agree to the respective upstream license terms.

## Citation

For **the models themselves**, please cite upstream pyannote and WeSpeaker:

```bibtex
@inproceedings{Plaquet23,
  author    = {Alexis Plaquet and Herv{\'e} Bredin},
  title     = {{Powerset multi-class cross entropy loss for neural speaker diarization}},
  booktitle = {Proc. INTERSPEECH 2023},
  year      = {2023},
}
@inproceedings{Bredin23,
  author    = {Herv{\'e} Bredin},
  title     = {{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
  booktitle = {Proc. INTERSPEECH 2023},
  year      = {2023},
}
@inproceedings{wang2023wespeaker,
  author    = {Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},
  title     = {{Wespeaker: A research and production oriented speaker embedding learning toolkit}},
  booktitle = {ICASSP 2023},
  year      = {2023},
}
```

If **this repo's local deployment packaging** was useful to you, you may additionally cite:

```bibtex
@misc{yang2026diarizationlocal,
  title        = {Speaker Diarization Models: A Local Build of pyannote + WeSpeaker for On-Device Diarization},
  author       = {Yang, Zhiping},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/42ailab/Speaker-Diarization-Models}},
  organization = {42ailab},
  note         = {Local deployment packaging (ONNX + cross-platform acceleration); the models themselves are pyannote/segmentation-3.0 (MIT) and WeSpeaker voxceleb ResNet34 (Apache-2.0). Contact: contact@42ailab.com}
}
```

Contact us: **contact@42ailab.com**

## About us

**[42ailab](https://42ailab.com)** — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike.

**[42model](https://42model.com)** — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.