---
license: other
license_name: mixed-upstream
license_link: https://github.com/pyannote/pyannote-audio
tags:
  - coreml
  - speaker-diarization
  - pyannote
  - wespeaker
  - apple-neural-engine
library_name: coremlit
---

# speakerkit-coreml — fp16-safe CoreML conversions for speaker diarization

CoreML conversions of the pyannote community-1 diarization pipeline's neural components,
re-converted with **fp16-survivable numerical guards** so they are correct on the Apple
Neural Engine and GPU — not just CPU.

Consumed by the `speakerkit` crate of the [coremlit](https://github.com/findit-studio/coremlit)
workspace (Rust, sync, sans-I/O), feeding the `dia` diarization pipeline's clustering.

## Why these conversions exist

Stock conversions of these models carry numerical guard epsilons **below fp16's smallest
subnormal (`2^-24`)**. When a graph runs on the ANE (which computes in fp16 regardless of
the declared dtype), those guards round to zero:

- the segmentation model's `softmax → log(eps≈0)` tail saturates `log(0)` to **≈ −45440**,
  which in the downstream pipeline collapsed an 8-speaker recording to 5 detected speakers
  (16.6% DER, 100% confusion) while the ONNX reference was frame-perfect;
- the embedder's pooling `real_div` guards (`1e-8`) vanish the same way.

Even a *naive re-export* reproduces the defect: `torch.log_softmax` still lowers to the
decomposed `softmax → log(eps=0)` under coremltools 9.0. These conversions use an
explicitly **fused `x − logsumexp(x)`** tail and raise every guard constant to `0x1p-24`.

## Contents

| artifact | form | what it is |
|---|---|---|
| `pyannote_segmentation.{mlpackage,mlmodelc}` | fp16 | pyannote **segmentation-3.0** (PyanNet), fused log-probability tail |
| `wespeaker.{mlpackage,mlmodelc}` | fp32 weights | **WeSpeaker ResNet34-LM** speaker embedder, pooling guards raised to `0x1p-24` (otherwise bit-identical numerics to the stock conversion) |
| `wespeaker_int8.{mlpackage,mlmodelc}` | 8-bit palettized | same fixed embedder, k-means per-tensor palettization (~3.7× smaller, the production default) |

`.mlpackage` is the canonical distributable; `.mlmodelc` is the compiled form the test
suite consumes directly. `CHECKSUMS.sha256` covers every file.

## Verification (summary)

- Every graph passes a **MIL-level fp16-guard audit** (every guard's *effective floor*
  ≥ `2^-24`, constants followed through `cast`, unresolvable guards fail the audit).
- Segmentation: worst per-window delta vs the ONNX oracle dropped from **45,422 → 0.07**
  (ANE), argmax agreement 100%; the 8-speaker collapse is fixed at model level.
- Embedder: All-vs-CPU cosine 1.0 on real speech; int8 clusters identically to fp32 on the
  multi-speaker parity corpus (speaker counts never differ).
- Conversion toolchain pinned in the recipe: coremltools 9.0, torch 2.5.1, onnx 1.22.0,
  python 3.11.15.

## Upstream provenance & licensing

| component | upstream | license |
|---|---|---|
| segmentation | [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0) (weights via its ONNX export) | MIT (upstream repo is access-gated; the MIT license permits redistribution with attribution — please also respect pyannote's usage conditions and cite their papers) |
| embedder | WeSpeaker ResNet34-LM (voxceleb) | Apache-2.0 (WeSpeaker project) |

These are *derivative conversions*: same weights, corrected graph numerics. If you are the
upstream author of either model and want changes to this redistribution, open a discussion.