--- license: other license_name: mixed-mit-cc-by-4-apache-2 license_link: LICENSE language: - en - multilingual library_name: onnx tags: - speaker-diarization - diarization - pyannote - speaker-embedding - wespeaker - segmentation pipeline_tag: voice-activity-detection --- # dia-models — pyannote community-1 model bundle for the `dia` Rust crate A single-repo distribution of every model artifact the [`dia`](https://github.com/al8n/diarization) Rust crate needs to run end-to-end speaker diarization with **pyannote-community-1** parity: - The **segmentation-3.0** powerset speaker network (16 kHz audio → per-frame speaker activations). - The **WeSpeaker ResNet34-LM** speaker-embedding network, in three forms (external-data ONNX, single-file ONNX, TorchScript). - The **PLDA** whitening + LDA weights from the [`pyannote/speaker-diarization-community-1`](https://huggingface.co/pyannote/speaker-diarization-community-1) pipeline, in both `.npz` (build-time) and raw little-endian f64 `.bin` (runtime) form. `dia` already embeds the segmentation model and the PLDA weights into the compiled binary via `include_bytes!`; the **WeSpeaker** ONNX is the only artifact callers must download separately. This repo lets callers grab any individual model — or the whole bundle — without spelunking through the upstream pyannote / WeSpeaker repos. > **Attribution: this is a redistribution, not new model training.** > All weights come from upstream pyannote / WeSpeaker / BUT Speech@FIT. > The licenses below MUST be preserved by anyone redistributing. ## Files | File | Size | Format | License | |---|---:|---|---| | `segmentation-3.0.onnx` | 5.99 MiB | ONNX (single file) | MIT | | `wespeaker_resnet34_lm.onnx` | 256 KiB | ONNX header (external data) | Apache-2.0 | | `wespeaker_resnet34_lm.onnx.data` | 25.3 MiB | external-data weights | Apache-2.0 | | `wespeaker_resnet34_lm_packed.onnx` | 25.5 MiB | ONNX (single file, repacked) | Apache-2.0 | | `wespeaker_resnet34_lm.pt` | 25.6 MiB | TorchScript | Apache-2.0 | | `plda/eigenvectors_desc.bin` | 128 KiB | f64 (128×128 row-major) | CC-BY-4.0 | | `plda/lda.bin` | 256 KiB | f64 (256×128 row-major) | CC-BY-4.0 | | `plda/mean1.bin` | 2 KiB | f64 (256,) | CC-BY-4.0 | | `plda/mean2.bin` | 1 KiB | f64 (128,) | CC-BY-4.0 | | `plda/mu.bin` | 1 KiB | f64 (128,) | CC-BY-4.0 | | `plda/phi_desc.bin` | 1 KiB | f64 (128,) | CC-BY-4.0 | | `plda/psi.bin` | 1 KiB | f64 (128,) | CC-BY-4.0 | | `plda/tr.bin` | 128 KiB | f64 (128×128 row-major) | CC-BY-4.0 | | `plda/plda.npz` | 131 KiB | numpy (`mu`, `tr`, `psi`) | CC-BY-4.0 | | `plda/xvec_transform.npz` | 131 KiB | numpy (`mean1`, `mean2`, `lda`) | CC-BY-4.0 | ## Which file do I want? ### Segmentation Use `segmentation-3.0.onnx`. It feeds `dia::segment::SegmentModel` (or any pyannote-segmentation-compatible runtime). Single file, no external data, works on every ORT execution provider. ### Embedding (WeSpeaker) Three forms, same weights, pick by use case: - **`wespeaker_resnet34_lm.onnx` + `wespeaker_resnet34_lm.onnx.data`** — the default ONNX layout. Loads on CPU / TensorRT / CUDA / OpenVINO / DirectML. The `.onnx` and `.onnx.data` files MUST sit next to each other on disk; ORT resolves the external pointer by relative path. - **`wespeaker_resnet34_lm_packed.onnx`** — same model with all weights inlined into one file. Use this if you want a single-file artifact, or if the runtime is **CoreML** (Apple Silicon — Apple's graph optimizer chokes on external initializers and reports `model_path must not be empty`; the packed form sidesteps it). Otherwise functionally identical. - **`wespeaker_resnet34_lm.pt`** — TorchScript export for the `tch` backend. Bit-exact to upstream PyTorch on hard cases (heavy- overlap fixtures where the ONNX→ORT path can drift by O(1) per element). Pulls in libtorch (~600 MB shared library). ### PLDA The eight `.bin` files are the runtime data — raw little-endian f64 blobs that `dia::plda` embeds via `include_bytes!`. The two `.npz` files are the build-time sources (`xvec_transform.npz` exposes `mean1` / `mean2` / `lda`; `plda.npz` exposes `mu` / `tr` / `psi`); they are mirrored from the upstream pyannote-community-1 snapshot for traceability and so the `.bin` extraction can be re-run via `scripts/extract-plda-blobs.sh` in the dia repo. `eigenvectors_desc.bin` and `phi_desc.bin` are scipy-derived eigenvectors of the PLDA generalized eigenproblem `(B, W)` — pinned to avoid LAPACK eigenvector-sign indeterminism (which produced a 38% DER divergence on three-speaker fixtures when nalgebra and scipy disagreed on 67 of 128 column signs). See [`models/plda/SOURCE.md`](https://github.com/al8n/diarization/blob/main/models/plda/SOURCE.md) in the dia repo for the regeneration procedure. ## Provenance ### segmentation-3.0.onnx - **Upstream:** [`pyannote/segmentation-3.0`](https://huggingface.co/pyannote/segmentation-3.0) - **Original layout:** `pytorch_model.onnx` in the upstream HF repo. - **License:** MIT — Copyright (c) 2023 CNRS - **Author:** Hervé Bredin (CNRS / IRIT), pyannote.audio author and lead trainer. - **SHA-256:** `057ee564753071c0b09b5b611648b50ac188d50846bff5f01e9f7bbf1591ea25` ### wespeaker_resnet34_lm.onnx (+ .data) / .pt / _packed.onnx - **Upstream model architecture:** WeSpeaker ResNet34 with large-margin (LM) angular fine-tuning, trained on VoxCeleb-2. - **Upstream sources:** - [WeSpeaker project](https://github.com/wenet-e2e/wespeaker) (Apache-2.0) - [`onnx-community/wespeaker_resnet34_lm`](https://huggingface.co/onnx-community/wespeaker_resnet34_lm) for the ONNX export. - **License:** Apache-2.0. - **`_packed.onnx` derivative:** produced by loading `wespeaker_resnet34_lm.onnx` + `.onnx.data` via the `onnx` Python library (`onnx.load(path, load_external_data=True)`) and re-saving with `save_as_external_data=False`. Same weights, no external file. ### plda/ - **Upstream:** [`pyannote/speaker-diarization-community-1`](https://huggingface.co/pyannote/speaker-diarization-community-1) - **License:** CC-BY-4.0 - **Snapshot revision:** `3533c8cf8e369892e6b79ff1bf80f7b0286a54ee` - **Original layout in the upstream HF repo:** `plda/xvec_transform.npz` and `plda/plda.npz`. - **Attribution (per upstream `plda/README.md`):** PLDA model trained by [BUT Speech@FIT](https://speech.fit.vut.cz/); integration of VBx in pyannote.audio by Jiangyu Han and Petr Pálka. ## Usage ### From `dia` (Rust) ```rust use diarization::{ embed::EmbedModel, plda::PldaTransform, segment::SegmentModel, }; // Segmentation + PLDA are bundled by default — no download needed. let mut seg = SegmentModel::bundled()?; let plda = PldaTransform::new()?; // WeSpeaker is BYO; download from this repo. let mut emb = EmbedModel::from_file("wespeaker_resnet34_lm.onnx")?; # Ok::<(), Box>(()) ``` ### Direct download ```bash # whole bundle hf download FinDIT-Studio/dia-models --local-dir ./dia-models # just the embedding model (default ONNX form) hf download FinDIT-Studio/dia-models \ wespeaker_resnet34_lm.onnx wespeaker_resnet34_lm.onnx.data \ --local-dir ./models # CoreML-friendly single-file form hf download FinDIT-Studio/dia-models \ wespeaker_resnet34_lm_packed.onnx --local-dir ./models ``` ## Licenses This repository **redistributes** model artifacts under three different licenses. Each artifact retains its upstream license. By using this bundle you agree to comply with **all three**: - **MIT** for `segmentation-3.0.onnx` (Copyright © 2023 CNRS, Hervé Bredin). See `LICENSE.MIT`. - **Apache-2.0** for the WeSpeaker artifacts. See `LICENSE.APACHE-2.0`. - **CC-BY-4.0** for everything under `plda/`. See `LICENSE.CC-BY-4.0`. Required attribution: *PLDA model trained by BUT Speech@FIT; integration of VBx in pyannote.audio by Jiangyu Han and Petr Pálka.* The `dia` Rust crate that consumes these models is itself dual-licensed MIT OR Apache-2.0; that licensing applies to the source code, not to the model weights bundled here. ## Citation If you use these weights in academic work, please cite the upstream papers / model cards: - **Segmentation-3.0:** Hervé Bredin, *pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe*, Interspeech 2023. - **WeSpeaker:** Wang et al., *WeSpeaker: A research and production oriented speaker embedding learning toolkit*, ICASSP 2023. - **PLDA / VBx:** Landini et al., *Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks*, Computer Speech & Language, 2022. ## Issues / questions This repo is a **redistribution** of upstream artifacts. Please file issues against: - The dia Rust crate: - The pyannote.audio project: - The WeSpeaker project: