Sortformer CoreML — v3 (BNNS-fixed rebuild)
Rebuilt streaming Sortformer diarization models (NVIDIA diar_streaming_sortformer_4spk-v2.1 / -v2),
converted to CoreML. v3 fixes the BNNS graph-compile crash that affected the earlier root-level models.
Why v3
The earlier models had chunk_pre_encoder_embs_out as both an input and an output of the head
sub-model, which the macOS 26 / newer BNNS graph compiler rejects:
BNNS Graph Compile: Function main has tensor chunk_pre_encoder_embs_out as both an input and output.
That bug was a toolchain artifact (torch 2.9.x tracing folded the identity op that kept input/output
distinct). v3 is rebuilt on torch 2.7 + coremltools 9.0, producing a clean head (no alias), verified to
load and predict on computeUnits=.all (ANE/GPU) and numerically matched to the PyTorch reference
(speaker-argmax agreement 100%).
Contents
v3/
fp16/ full-precision (229–243 MB/variant) — default quality
palettized/ 6-bit kmeans-LUT weight palettization (93–99 MB/variant)
7 variants in each (Swift ModelNames.Sortformer.Variant):
| File | Config | chunk_len | latency | use |
|---|---|---|---|---|
Sortformer_v2.1 / _v2 |
fast / default | 6 | ~0.48 s | low-latency streaming |
SortformerEfficient_v2.1 |
efficient | 25 | ~2 s | higher-throughput streaming (~4× RTFx of default) |
SortformerNvidiaLow_v2.1 / _v2 |
balanced | 6 | ~0.48 s | larger FIFO for stability |
SortformerNvidiaHigh_v2.1 / _v2 |
high-context | 340 | ~27 s | offline / best throughput |
fp16 vs palettized
6-bit palettization (matches Argmax's speakerkit recipe; GPU-safe LUT, not int8 which crashes MPSGraph):
- Size: ~2.5× smaller (e.g. highContext 243 → 99 MB) — fixes RAM-driven crashes on older devices.
- Speed: unchanged (same per-call latency on GPU).
- Accuracy: +0.9 pp DER avg on full AMI-SDM (16 mtg, forced-alignment GT, collar 0.25) — most meetings +0.0–1.0 pp, a few +2–4 pp. Use fp16 if you need the last ~1 pp; palettized to fix RAM / shrink download.
Reference DER (AMI-SDM, forced-alignment GT, collar 0.25, M5 Pro)
| Variant | DER | RTFx |
|---|---|---|
| highContext offline (tuned) | ~26.5% | ~860× |
| default streaming (0.48 s) | ~29.0% | ~49× |
Absolute DER is high because AMI-SDM (single distant mic, 4 speakers) is a hard condition; same NeMo weights as other Sortformer ports, so accuracy is model-equivalent.