docs: v2 files, host mean normalization, AMI benchmark and Changelog (FluidAudio #985)
Browse filesCard for b7be41fc: FluidAudio PR https://github.com/FluidInference/FluidAudio/pull/1099; removes the incorrect 'CAM++ normalizes the fbank internally'.
README.md
CHANGED
|
@@ -18,20 +18,34 @@ on-device speaker verification / diarization on Apple Silicon. Upstream:
|
|
| 18 |
|
| 19 |
| File | Precision | Compute unit | Role |
|
| 20 |
|------|-----------|--------------|------|
|
| 21 |
-
| `
|
| 22 |
-
| `
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Pipeline
|
| 25 |
|
| 26 |
```
|
| 27 |
-
waveform → [
|
| 28 |
-
→
|
|
|
|
| 29 |
```
|
| 30 |
|
| 31 |
-
|
| 32 |
-
|
|
|
|
| 33 |
|
| 34 |
-
## Benchmark —
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
| Metric | Value |
|
| 37 |
|--------|-------|
|
|
@@ -41,6 +55,12 @@ similarity for speaker verification and diarization clustering.
|
|
| 41 |
|
| 42 |
AISHELL-1 (clean read Mandarin) is easier than the official CN-Celeb (~6-7%). CoreML↔torch embedding cosine 0.9997-0.99999.
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
## License
|
| 45 |
|
| 46 |
Weights derive from FunASR's CAM++; upstream license applies. Format conversion only.
|
|
|
|
| 18 |
|
| 19 |
| File | Precision | Compute unit | Role |
|
| 20 |
|------|-----------|--------------|------|
|
| 21 |
+
| `CamPlusPreprocessor_v2.mlmodelc` | FP32 | CPU | waveform → 80-d Kaldi fbank (Povey window, log floor after the mel filterbank) |
|
| 22 |
+
| `CamPlusPlus_v2.mlmodelc` | FP32 | CPU / GPU | mean-normalized fbank → 192-d speaker embedding |
|
| 23 |
+
| `CamPlusPreprocessor.mlmodelc` | FP32 | CPU | v1, superseded by `_v2` (Hamming window); kept for older FluidAudio releases |
|
| 24 |
+
| `CamPlusPlus.mlmodelc` | FP16 | CPU / GPU | v1, superseded by `_v2` (see Changelog); kept for older FluidAudio releases |
|
| 25 |
|
| 26 |
## Pipeline
|
| 27 |
|
| 28 |
```
|
| 29 |
+
waveform → [CamPlusPreprocessor_v2 fp32/CPU] → fbank [1,T,80]
|
| 30 |
+
→ subtract the per-utterance mean (host)
|
| 31 |
+
→ [CamPlusPlus_v2 fp32/CPU+GPU] → embedding [1,192] (L2-normalize, then cosine for verification/clustering)
|
| 32 |
```
|
| 33 |
|
| 34 |
+
Neither model normalizes the fbank: the host subtracts each mel band's mean over the utterance,
|
| 35 |
+
as 3D-Speaker and FunASR do (FluidAudio's `CampPlusEmbedder`). The 192-d embedding is used with
|
| 36 |
+
cosine similarity for speaker verification and diarization clustering.
|
| 37 |
|
| 38 |
+
## Benchmark — AMI speaker verification (v2)
|
| 39 |
+
|
| 40 |
+
413 single-speaker AMI Mix-Headset segments (3–8 s, 16 participants, 3–4 meetings each), cosine scoring:
|
| 41 |
+
|
| 42 |
+
| | v1 + host mean | v2 | PyTorch 3D-Speaker CAMPPlus |
|
| 43 |
+
|---|---|---|---|
|
| 44 |
+
| EER, within meeting (1,332 target trials) | 3.07% | 2.85% | 2.85% |
|
| 45 |
+
| EER, across meetings (4,079 target trials) | 3.48% | 3.55% | 3.55% |
|
| 46 |
+
| Cosine to PyTorch, 357 lengths 0.35–60 s | as low as −0.01 | 1.0000 | 1 |
|
| 47 |
+
|
| 48 |
+
## Benchmark — AISHELL-1 speaker verification (v1)
|
| 49 |
|
| 50 |
| Metric | Value |
|
| 51 |
|--------|-------|
|
|
|
|
| 55 |
|
| 56 |
AISHELL-1 (clean read Mandarin) is easier than the official CN-Celeb (~6-7%). CoreML↔torch embedding cosine 0.9997-0.99999.
|
| 57 |
|
| 58 |
+
## Changelog
|
| 59 |
+
|
| 60 |
+
| Date | Files | Change | Issue | Recipe | FluidAudio PR | HF commit |
|
| 61 |
+
|---|---|---|---|---|---|---|
|
| 62 |
+
| 2026-11-01 | `CamPlusPreprocessor_v2.mlmodelc`, `CamPlusPlus_v2.mlmodelc` | Kaldi Povey fbank with the log floor after the mel filterbank; each segment-pooling tail averaged over its own frames (v1 divided by 100); fp32 (v1's fp16 pointwise convs gave unrelated embeddings at some lengths, e.g. 17.9–20.4 s) | [FluidAudio #985](https://github.com/FluidInference/FluidAudio/issues/985) | model-lab `models/campplus-coreml-v2` | [#1099](https://github.com/FluidInference/FluidAudio/pull/1099) | [b7be41fc](https://huggingface.co/FluidInference/campplus-coreml/commit/b7be41fcf4bf7697d879ca71c687dc532854e8a2) |
|
| 63 |
+
|
| 64 |
## License
|
| 65 |
|
| 66 |
Weights derive from FunASR's CAM++; upstream license applies. Format conversion only.
|