File size: 2,556 Bytes
26f7abe | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
license: apache-2.0
base_model: devsy0117/ultra_diar_streaming_sortformer_8spk_v1
tags:
- speaker-diarization
- coreml
- streaming
- apple-silicon
library_name: coreml
---
# Ultra-Sortformer 8-Speaker Streaming Diarization — CoreML
CoreML conversion of
[`devsy0117/ultra_diar_streaming_sortformer_8spk_v1`](https://huggingface.co/devsy0117/ultra_diar_streaming_sortformer_8spk_v1),
the [Ultra-Sortformer](https://github.com/mago-research/Ultra-Sortformer)
fine-tune that widens NVIDIA's streaming Sortformer speaker head from four to
eight speakers with SVD-initialized rows and split-learning-rate training.
Same conversion pipeline, streaming state protocol, and Arrival-Order Speaker
Cache host algorithm as
[`aufklarer/Sortformer-Diarization-CoreML`](https://huggingface.co/aufklarer/Sortformer-Diarization-CoreML)
(the 4-speaker base). Only the head width and the emitted probability tensor
change: `speaker_preds [1, 242, 8]` (188 speaker-cache + 40 FIFO + 14 chunk
frames), with the base model's cache geometry preserved.
## Contents
| File | Purpose |
|---|---|
| `Sortformer_streaming.mlmodelc` | Compiled streaming variant — 480 ms of new audio per call, cache state crosses the interface as tensors |
| `Sortformer_streaming.mlpackage` | Source package for recompilation |
| `config_streaming.json` | Loader configuration (`num_speakers: 8`, chunk/cache dims) |
## Validation
The exported step was driven by NeMo's own `streaming_feat_loader` and
`streaming_update_async` and matched the checkpoint's native
`forward_streaming` loop (streaming-parity pytest in the conversion pipeline,
passing).
## Lineage and licensing
- Base: [`nvidia/diar_streaming_sortformer_4spk-v2.1`](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1)
(NVIDIA Open Model License).
- Fine-tune: [`devsy0117/ultra_diar_streaming_sortformer_8spk_v1`](https://huggingface.co/devsy0117/ultra_diar_streaming_sortformer_8spk_v1)
([Ultra-Sortformer](https://github.com/mago-research/Ultra-Sortformer),
Apache-2.0), trained on synthetic multi-speaker sessions built from the
AI Hub multi-speaker speech synthesis corpus (Korean).
- This repository redistributes a format conversion of that fine-tune;
no additional training was performed.
Note: the upstream project publishes real-corpus rankings only for the
4-speaker base model. Evaluate this 8-speaker variant on your own data
before preferring it — the wider head targets crowded scenes, and behavior
on 2–4 speaker audio should be regression-checked per application.
|