README.md

---
license: apache-2.0
base_model: iic/speech_eres2netv2_sv_zh-cn_16k-common
tags:
  - speaker-diarization
  - speaker-recognition
  - speaker-embedding
  - onnx
  - quantized
  - int8
  - sherpa-onnx
library_name: sherpa-onnx
---

## `diarize-embedding-eres2netv2-int8.onnx`

Statically int8-quantized [ERes2NetV2](https://arxiv.org/abs/2406.02167) speaker
embedder (3D-Speaker, `zh-cn 16k-common`), for speaker diarization under
sherpa-onnx.

- **28 MB** (fp32 source: 71 MB), 192-dim embeddings, 16 kHz.
- Drop-in for `SpeakerEmbeddingExtractorConfig`: the sherpa `metadata_props`
  (`framework`, `sample_rate`, `output_dim`, `feature_normalize_type`) are
  preserved, which the extractor requires.

### Why quantize

fp32 ERes2NetV2 separates close voices well, but its 2D convolutions cost about
**10x** CAM++'s embedding time — roughly 11 minutes on a 19-minute meeting,
untenable on the no-GPU laptops this app targets. Static quantization removes
that objection:

| build | time per 6 s window (ORT CPU, 4 threads) |
|---|---|
| fp32 | 223 ms |
| **int8 static (this file)** | **77 ms — 2.9x faster** |
| int8 *dynamic* | 528 ms — 2.4x **slower** |

Dynamic quantization is a trap here: it lowers `Conv` to `ConvInteger`, which
onnxruntime's CPU provider does not optimize. Static quantization lowers to
`QLinearConv`, which it does.

### Accuracy

Against the fp32 model on real meeting windows: embedding cosine **≥ 0.9956**
(mean 0.9973), and the pairwise-similarity matrix — what clustering actually
consumes — drifts by at most **0.019**.

End-to-end on diarization bench (DER against hand-annotated references):

| fixture | CAM++ | this model |
|---|---|---|
| 2-speaker interview, 28 min | DER 12.6%, 2 voices | DER 12.7%, 2 voices |
| 2-speaker phone call, 8 min | DER 14.6%, 2 voices | DER 14.5%, 2 voices |
| multi-speaker meeting, 19 min | 2 voices, 80/20 speech split | **3 voices, 42/37/21** |

Two-speaker recordings cannot tell these models apart. The difference appears
where it matters — a meeting with several voices, where CAM++ collapses 80% of
the speech onto one speaker.

Both models still undercount a crowded room (3 of 5 real speakers on that
meeting), so lets the user pin the speaker count rather than trust
auto-detection.

### How it was made

`quantize_static` with `QuantFormat.QOperator`, per-channel int8 weights, uint8
activations, `Conv` only, calibrated on ~40 log-mel fbank windows (600 frames
≈ 6 s, per-window global-mean normalized, matching sherpa's own preprocessing)
taken from a real meeting recording. Model metadata is copied back from the
fp32 file afterwards, since the quantizer drops it.

The script lives in the app repo (`scripts/quantize-eres2netv2.py`):

```bash
python3 scripts/quantize-eres2netv2.py \
    eres2netv2-fp32.onnx diarize-embedding-eres2netv2-int8.onnx \
    some-real-meeting.mp3
```

### Verifying this file

```
sha256  be6b162137d8b08854268a97763c007e49882f221e02950242923d40d2be157e
```

## Credits and license

The weights derive from
[`iic/speech_eres2netv2_sv_zh-cn_16k-common`](https://www.modelscope.cn/models/iic/speech_eres2netv2_sv_zh-cn_16k-common)
by the [3D-Speaker](https://github.com/modelscope/3D-Speaker) team (Apache-2.0);
the fp32 ONNX export came via
[csukuangfj/speaker-embedding-models](https://huggingface.co/csukuangfj/speaker-embedding-models).
This repository redistributes a quantized derivative under the same Apache-2.0
terms. If you use it, cite the original work:

```bibtex
@inproceedings{eres2netv2,
  title     = {{ERes2NetV2}: Boosting Short-Duration Speaker Verification
               Performance with Computational Efficiency},
  author    = {Chen, Yafeng and Zheng, Siqi and Wang, Hui and Cheng, Luyao and
               Zhu, Tinglong and Huang, Rongjie and Qian, Chong and Chen, Qian
               and Zhang, Wen and Wang, Yanmin},
  booktitle = {Interspeech},
  year      = {2024}
}