Echo Dia: V4 DiariZen fine-tuned weights

README.md

@@ -0,0 +1,54 @@
+---
+license: cc-by-nc-4.0
+library_name: transformers
+pipeline_tag: voice-activity-detection
+tags:
+- speaker-diarization
+- meeting
+- wavlm
+- diarizen
+- echo
+private: true
+---
+
+# Echo Dia (V4)
+
+Fine-tuned DiariZen-v2 (`BUT-FIT/diarizen-wavlm-large-s80-md-v2`) on a multi-domain meeting compound.
+
+## Training
+
+- **Base model**: BUT-FIT/diarizen-wavlm-large-s80-md-v2
+- **Training data**: 9.1 h compound (AMI 3.5h + AliMeeting 2.6h + NOTSOFAR 3.0h)
+- **Strategy**: WavLM layer 23 unfrozen, lr_wavlm=2.5e-6, lr_head=1e-4
+- **Augmentation**: SpecAugment (time + freq mask) + audio noise injection
+- **Duration**: 60 minutes on RTX A6000 (Phase 3 winner V4)
+- **Best DER val** (ES2011a, 18 min): 17.69%
+
+## Test set DER (collar=0, with overlap)
+
+| Dataset | DER strict | DER col=0.25 | n_meetings |
+|---|---|---|---|
+| AMI test | 17.34% | 13.95% | 2 |
+| AliMeeting test | 14.14% | 8.66% | 5 |
+| NOTSOFAR test | 13.49% | 8.38% | 5 |
+
+## Usage
+
+```python
+import torch
+from diarizen.pipelines.inference import DiariZenPipeline
+
+# Load v2 base, then inject Echo Dia weights
+pipe = DiariZenPipeline.from_pretrained("BUT-FIT/diarizen-wavlm-large-s80-md-v2")
+sd = torch.load("pytorch_model.bin", map_location="cuda:0", weights_only=False)
+pipe._segmentation.model.load_state_dict(sd, strict=False)
+
+# Run
+result = pipe("audio.wav")
+for seg, _, spk in result.itertracks(yield_label=True):
+    print(f"{seg.start:.1f}-{seg.end:.1f}  {spk}")
+```
+
+## License
+
+CC BY-NC 4.0 (inherited from base model). Non-commercial use only.

config.json

@@ -0,0 +1,3 @@
+{
+  "library_name": "transformers"
+}

config.toml

@@ -0,0 +1,37 @@
+[model]
+path = "diarizen.models.eend.model_wavlm_conformer.Model"
+
+[model.args]
+wavlm_src = "wavlm_large_s80_md"
+wavlm_layer_num = 25
+wavlm_feat_dim = 1024
+attention_in = 256
+ffn_hidden = 1024
+num_head = 4
+num_layer = 4
+dropout = 0.1
+max_speakers_per_chunk = 4
+max_speakers_per_frame = 4
+chunk_size = 16
+use_posi = false
+output_activate_function = false
+selected_channel = 0
+
+[inference.args]
+seg_duration = 16
+segmentation_step = 0.1
+batch_size = 32
+apply_median_filtering = true
+
+[clustering.args]
+method = "VBxClustering"
+min_speakers = 1
+max_speakers = 20
+ahc_criterion = "distance"
+ahc_threshold = 0.6
+Fa = 0.07
+Fb = 0.8
+lda_dim = 128
+max_iters = 20
+
+

plda/plda.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b77bcd840692710dd3496f62ecfeed8d8e5f002fd991b785079b244eab7d255
+size 133852

plda/xvec_transform.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:325f1ce8e48f7e55e9c8aa47e05d2766b7c48c4b25b8de8dd751e7a4cc5fbe8f
+size 134376

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8f5c8ed69cb2ed78860587a97b3c0425284eeb793c99d4d84a46b8ff71ff692
+size 278071935