--- license: cc-by-4.0 pipeline_tag: voice-activity-detection library_name: coreml tags: - speaker-diarization - coreml - pyannote base_model: pyannote/speaker-diarization-community-1 --- # Speaker diarization (pyannote Community-1), Core ML Fluid Inference's Core ML build of [pyannote](https://github.com/pyannote/pyannote-audio)'s [speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1): the powerset segmentation model, the WeSpeaker ResNet34 speaker embedding model with its FBank frontend, and BUT Speech@FIT's PLDA. MacEpic hosts this copy so TextSniper can download it at a fixed revision; nothing here is MacEpic's own work, and no file of the model was changed. ## Source [FluidInference/speaker-diarization-coreml](https://huggingface.co/FluidInference/speaker-diarization-coreml) at commit [`7eab74f3463c736418d0eb10296261ae581146c9`](https://huggingface.co/FluidInference/speaker-diarization-coreml/tree/7eab74f3463c736418d0eb10296261ae581146c9). These 21 files are byte for byte the same as there (21,602,787 bytes): - `Segmentation.mlmodelc`, `FBank.mlmodelc`, `Embedding.mlmodelc`: the compiled Core ML models. - `plda-parameters.json`, `xvector-transform.json`: BUT Speech@FIT's PLDA as Float32 tensors. - `LICENSE`, `NOTICE.md`, `PROVENANCE.md`, `provenance.json`: the license, its scope and attribution, and where each file comes from. Changes from the source, as CC BY 4.0 asks them to be noted: `PLDA.mlmodelc`, `PldaRho.mlmodelc`, the legacy models, `mlpackages/`, the plots, `config.json` and Fluid Inference's README were left out, and this README and `bundle.json` (every file with its size and SHA-256) were added. ## License and attribution [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), as the source; the full text is in `LICENSE`, and `NOTICE.md` sets out which files it covers. The files are modified Core ML conversions of: - pyannote's speaker-diarization-community-1 pipeline (CNRS, pyannoteAI), CC BY 4.0; - WeSpeaker's VoxCeleb ResNet34 speaker embedding model, through pyannote, CC BY 4.0; - Brno University of Technology / BUT Speech@FIT's PLDA (`plda.npz`, `xvec_transform.npz`), CC BY 4.0; converted to Core ML by [Fluid Inference](https://huggingface.co/FluidInference), which introduced fixed and enumerated input shapes, mixed-precision storage, a separate FBank frontend, and compiled the models for Apple platforms. ## Citations ### Speaker segmentation ```bibtex @inproceedings{Plaquet23, author={Alexis Plaquet and Hervé Bredin}, title={{Powerset multi-class cross entropy loss for neural speaker diarization}}, year=2023, booktitle={Proc. INTERSPEECH 2023}, } ``` ### Speaker embedding ```bibtex @inproceedings{Wang2023, title={Wespeaker: A research and production oriented speaker embedding learning toolkit}, author={Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin}, booktitle={ICASSP 2023, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages={1--5}, year={2023}, organization={IEEE} } ``` ### Speaker clustering ```bibtex @article{Landini2022, author={Landini, Federico and Profant, J{\'a}n and Diez, Mireia and Burget, Luk{\'a}{\v{s}}}, title={{Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks}}, year={2022}, journal={Computer Speech & Language}, } ```