vboguckis's picture
TextSniper's copy of FluidInference/speaker-diarization-coreml at 7eab74f
114d3ec verified
|
Raw History Blame Contribute Delete
3.52 kB
---
license: cc-by-4.0
pipeline_tag: voice-activity-detection
library_name: coreml
tags:
- speaker-diarization
- coreml
- pyannote
base_model: pyannote/speaker-diarization-community-1
---
# Speaker diarization (pyannote Community-1), Core ML
Fluid Inference's Core ML build of [pyannote](https://github.com/pyannote/pyannote-audio)'s
[speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1): the powerset
segmentation model, the WeSpeaker ResNet34 speaker embedding model with its FBank frontend, and BUT Speech@FIT's PLDA.
MacEpic hosts this copy so TextSniper can download it at a fixed revision; nothing here is MacEpic's own work, and no
file of the model was changed.
## Source
[FluidInference/speaker-diarization-coreml](https://huggingface.co/FluidInference/speaker-diarization-coreml) at commit
[`7eab74f3463c736418d0eb10296261ae581146c9`](https://huggingface.co/FluidInference/speaker-diarization-coreml/tree/7eab74f3463c736418d0eb10296261ae581146c9). These 21 files are byte for byte the
same as there (21,602,787 bytes):
- `Segmentation.mlmodelc`, `FBank.mlmodelc`, `Embedding.mlmodelc`: the compiled Core ML models.
- `plda-parameters.json`, `xvector-transform.json`: BUT Speech@FIT's PLDA as Float32 tensors.
- `LICENSE`, `NOTICE.md`, `PROVENANCE.md`, `provenance.json`: the license, its scope and attribution, and where each
file comes from.
Changes from the source, as CC BY 4.0 asks them to be noted: `PLDA.mlmodelc`, `PldaRho.mlmodelc`, the legacy models,
`mlpackages/`, the plots, `config.json` and Fluid Inference's README were left out, and this README and `bundle.json`
(every file with its size and SHA-256) were added.
## License and attribution
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), as the source; the full text is in `LICENSE`, and
`NOTICE.md` sets out which files it covers.
The files are modified Core ML conversions of:
- pyannote's speaker-diarization-community-1 pipeline (CNRS, pyannoteAI), CC BY 4.0;
- WeSpeaker's VoxCeleb ResNet34 speaker embedding model, through pyannote, CC BY 4.0;
- Brno University of Technology / BUT Speech@FIT's PLDA (`plda.npz`, `xvec_transform.npz`), CC BY 4.0;
converted to Core ML by [Fluid Inference](https://huggingface.co/FluidInference), which introduced fixed and
enumerated input shapes, mixed-precision storage, a separate FBank frontend, and compiled the models for Apple
platforms.
## Citations
### Speaker segmentation
```bibtex
@inproceedings{Plaquet23,
author={Alexis Plaquet and Hervé Bredin},
title={{Powerset multi-class cross entropy loss for neural speaker diarization}},
year=2023,
booktitle={Proc. INTERSPEECH 2023},
}
```
### Speaker embedding
```bibtex
@inproceedings{Wang2023,
title={Wespeaker: A research and production oriented speaker embedding learning toolkit},
author={Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},
booktitle={ICASSP 2023, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={1--5},
year={2023},
organization={IEEE}
}
```
### Speaker clustering
```bibtex
@article{Landini2022,
author={Landini, Federico and Profant, J{\'a}n and Diez, Mireia and Burget, Luk{\'a}{\v{s}}},
title={{Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks}},
year={2022},
journal={Computer Speech & Language},
}
```