alexwengg commited on
Commit
f316edb
·
verified ·
1 Parent(s): b7be41f

docs: v2 files, host mean normalization, AMI benchmark and Changelog (FluidAudio #985)

Browse files

Card for b7be41fc: FluidAudio PR https://github.com/FluidInference/FluidAudio/pull/1099; removes the incorrect 'CAM++ normalizes the fbank internally'.

Files changed (1) hide show
  1. README.md +27 -7
README.md CHANGED
@@ -18,20 +18,34 @@ on-device speaker verification / diarization on Apple Silicon. Upstream:
18
 
19
  | File | Precision | Compute unit | Role |
20
  |------|-----------|--------------|------|
21
- | `CamPlusPreprocessor.mlmodelc` | FP32 | CPU | waveform → 80-d fbank features |
22
- | `CamPlusPlus.mlmodelc` | FP16 | ANE | fbank → 192-d speaker embedding |
 
 
23
 
24
  ## Pipeline
25
 
26
  ```
27
- waveform → [Preprocessor fp32/CPU] → fbank [1,T,80]
28
- → [CAM++ fp16/ANE] → embedding [1,192] (L2-normalize, then cosine for verification/clustering)
 
29
  ```
30
 
31
- CAM++ normalizes the fbank internally. The 192-d embedding is used with cosine
32
- similarity for speaker verification and diarization clustering.
 
33
 
34
- ## Benchmark — AISHELL-1 speaker verification
 
 
 
 
 
 
 
 
 
 
35
 
36
  | Metric | Value |
37
  |--------|-------|
@@ -41,6 +55,12 @@ similarity for speaker verification and diarization clustering.
41
 
42
  AISHELL-1 (clean read Mandarin) is easier than the official CN-Celeb (~6-7%). CoreML↔torch embedding cosine 0.9997-0.99999.
43
 
 
 
 
 
 
 
44
  ## License
45
 
46
  Weights derive from FunASR's CAM++; upstream license applies. Format conversion only.
 
18
 
19
  | File | Precision | Compute unit | Role |
20
  |------|-----------|--------------|------|
21
+ | `CamPlusPreprocessor_v2.mlmodelc` | FP32 | CPU | waveform → 80-d Kaldi fbank (Povey window, log floor after the mel filterbank) |
22
+ | `CamPlusPlus_v2.mlmodelc` | FP32 | CPU / GPU | mean-normalized fbank → 192-d speaker embedding |
23
+ | `CamPlusPreprocessor.mlmodelc` | FP32 | CPU | v1, superseded by `_v2` (Hamming window); kept for older FluidAudio releases |
24
+ | `CamPlusPlus.mlmodelc` | FP16 | CPU / GPU | v1, superseded by `_v2` (see Changelog); kept for older FluidAudio releases |
25
 
26
  ## Pipeline
27
 
28
  ```
29
+ waveform → [CamPlusPreprocessor_v2 fp32/CPU] → fbank [1,T,80]
30
+ → subtract the per-utterance mean (host)
31
+ → [CamPlusPlus_v2 fp32/CPU+GPU] → embedding [1,192] (L2-normalize, then cosine for verification/clustering)
32
  ```
33
 
34
+ Neither model normalizes the fbank: the host subtracts each mel band's mean over the utterance,
35
+ as 3D-Speaker and FunASR do (FluidAudio's `CampPlusEmbedder`). The 192-d embedding is used with
36
+ cosine similarity for speaker verification and diarization clustering.
37
 
38
+ ## Benchmark — AMI speaker verification (v2)
39
+
40
+ 413 single-speaker AMI Mix-Headset segments (3–8 s, 16 participants, 3–4 meetings each), cosine scoring:
41
+
42
+ | | v1 + host mean | v2 | PyTorch 3D-Speaker CAMPPlus |
43
+ |---|---|---|---|
44
+ | EER, within meeting (1,332 target trials) | 3.07% | 2.85% | 2.85% |
45
+ | EER, across meetings (4,079 target trials) | 3.48% | 3.55% | 3.55% |
46
+ | Cosine to PyTorch, 357 lengths 0.35–60 s | as low as −0.01 | 1.0000 | 1 |
47
+
48
+ ## Benchmark — AISHELL-1 speaker verification (v1)
49
 
50
  | Metric | Value |
51
  |--------|-------|
 
55
 
56
  AISHELL-1 (clean read Mandarin) is easier than the official CN-Celeb (~6-7%). CoreML↔torch embedding cosine 0.9997-0.99999.
57
 
58
+ ## Changelog
59
+
60
+ | Date | Files | Change | Issue | Recipe | FluidAudio PR | HF commit |
61
+ |---|---|---|---|---|---|---|
62
+ | 2026-11-01 | `CamPlusPreprocessor_v2.mlmodelc`, `CamPlusPlus_v2.mlmodelc` | Kaldi Povey fbank with the log floor after the mel filterbank; each segment-pooling tail averaged over its own frames (v1 divided by 100); fp32 (v1's fp16 pointwise convs gave unrelated embeddings at some lengths, e.g. 17.9–20.4 s) | [FluidAudio #985](https://github.com/FluidInference/FluidAudio/issues/985) | model-lab `models/campplus-coreml-v2` | [#1099](https://github.com/FluidInference/FluidAudio/pull/1099) | [b7be41fc](https://huggingface.co/FluidInference/campplus-coreml/commit/b7be41fcf4bf7697d879ca71c687dc532854e8a2) |
63
+
64
  ## License
65
 
66
  Weights derive from FunASR's CAM++; upstream license applies. Format conversion only.