File size: 3,534 Bytes
3db6998 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
license: other
license_name: mixed-upstream
license_link: https://github.com/pyannote/pyannote-audio
tags:
- coreml
- speaker-diarization
- pyannote
- wespeaker
- apple-neural-engine
library_name: coremlit
---
# speakerkit-coreml — fp16-safe CoreML conversions for speaker diarization
CoreML conversions of the pyannote community-1 diarization pipeline's neural components,
re-converted with **fp16-survivable numerical guards** so they are correct on the Apple
Neural Engine and GPU — not just CPU.
Consumed by the `speakerkit` crate of the [coremlit](https://github.com/findit-studio/coremlit)
workspace (Rust, sync, sans-I/O), feeding the `dia` diarization pipeline's clustering.
## Why these conversions exist
Stock conversions of these models carry numerical guard epsilons **below fp16's smallest
subnormal (`2^-24`)**. When a graph runs on the ANE (which computes in fp16 regardless of
the declared dtype), those guards round to zero:
- the segmentation model's `softmax → log(eps≈0)` tail saturates `log(0)` to **≈ −45440**,
which in the downstream pipeline collapsed an 8-speaker recording to 5 detected speakers
(16.6% DER, 100% confusion) while the ONNX reference was frame-perfect;
- the embedder's pooling `real_div` guards (`1e-8`) vanish the same way.
Even a *naive re-export* reproduces the defect: `torch.log_softmax` still lowers to the
decomposed `softmax → log(eps=0)` under coremltools 9.0. These conversions use an
explicitly **fused `x − logsumexp(x)`** tail and raise every guard constant to `0x1p-24`.
## Contents
| artifact | form | what it is |
|---|---|---|
| `pyannote_segmentation.{mlpackage,mlmodelc}` | fp16 | pyannote **segmentation-3.0** (PyanNet), fused log-probability tail |
| `wespeaker.{mlpackage,mlmodelc}` | fp32 weights | **WeSpeaker ResNet34-LM** speaker embedder, pooling guards raised to `0x1p-24` (otherwise bit-identical numerics to the stock conversion) |
| `wespeaker_int8.{mlpackage,mlmodelc}` | 8-bit palettized | same fixed embedder, k-means per-tensor palettization (~3.7× smaller, the production default) |
`.mlpackage` is the canonical distributable; `.mlmodelc` is the compiled form the test
suite consumes directly. `CHECKSUMS.sha256` covers every file.
## Verification (summary)
- Every graph passes a **MIL-level fp16-guard audit** (every guard's *effective floor*
≥ `2^-24`, constants followed through `cast`, unresolvable guards fail the audit).
- Segmentation: worst per-window delta vs the ONNX oracle dropped from **45,422 → 0.07**
(ANE), argmax agreement 100%; the 8-speaker collapse is fixed at model level.
- Embedder: All-vs-CPU cosine 1.0 on real speech; int8 clusters identically to fp32 on the
multi-speaker parity corpus (speaker counts never differ).
- Conversion toolchain pinned in the recipe: coremltools 9.0, torch 2.5.1, onnx 1.22.0,
python 3.11.15.
## Upstream provenance & licensing
| component | upstream | license |
|---|---|---|
| segmentation | [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0) (weights via its ONNX export) | MIT (upstream repo is access-gated; the MIT license permits redistribution with attribution — please also respect pyannote's usage conditions and cite their papers) |
| embedder | WeSpeaker ResNet34-LM (voxceleb) | Apache-2.0 (WeSpeaker project) |
These are *derivative conversions*: same weights, corrected graph numerics. If you are the
upstream author of either model and want changes to this redistribution, open a discussion.
|