mazesmazes commited on
Commit
3adc10a
·
verified ·
1 Parent(s): 1c9abe3

Model card: document speaker diarization

Browse files
Files changed (1) hide show
  1. README.md +32 -4
README.md CHANGED
@@ -84,15 +84,43 @@ for word in result["words"]:
84
  print(word)
85
  ```
86
 
87
- ### Speaker Labels
 
 
 
 
 
 
 
 
88
 
89
  ```python
90
  result = pipe("meeting.wav", return_speakers=True, num_speakers=2)
91
- for word in result["words"]:
92
- print(word)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
93
  ```
94
 
95
- `return_speakers=True` implies `return_timestamps=True`. You can pass `num_speakers`, or `min_speakers` / `max_speakers`, when the speaker count is known.
96
 
97
  ### GPU Inference
98
 
 
84
  print(word)
85
  ```
86
 
87
+ ### Speaker Diarization
88
+
89
+ `return_speakers=True` adds speaker labels. It first transcribes and aligns words (it implies `return_timestamps=True`). It then splits the audio into speech regions with [TEN VAD](https://github.com/TEN-framework/ten-vad), embeds each region with [ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb), and clusters the embeddings into speakers. Each word is assigned to the speaker whose segment it overlaps.
90
+
91
+ Diarization needs a few extra packages:
92
+
93
+ ```bash
94
+ pip install speechbrain ten-vad scikit-learn scipy
95
+ ```
96
 
97
  ```python
98
  result = pipe("meeting.wav", return_speakers=True, num_speakers=2)
99
+
100
+ print(result["text"])
101
+
102
+ # Speaker turns
103
+ for seg in result["speaker_segments"]:
104
+ print(f"{seg['start']:6.2f}-{seg['end']:6.2f} {seg['speaker']}")
105
+ # 0.00- 2.90 SPEAKER_0
106
+ # 3.36- 6.47 SPEAKER_1
107
+ # 6.93- 9.81 SPEAKER_0
108
+
109
+ # Words with timestamps and speakers
110
+ for w in result["words"]:
111
+ print(f"{w['start']:6.2f}-{w['end']:6.2f} {w['speaker']} {w['word']}")
112
+ # 0.00- 0.19 SPEAKER_0 Hi,
113
+ # 0.22- 0.69 SPEAKER_0 Daniel.
114
+ # ...
115
+ ```
116
+
117
+ **Setting the speaker count.** Pass `num_speakers` when you know it. Otherwise the number of speakers is estimated from the audio, and on short clips that estimate tends to split one voice into several speakers. You can also bound the estimate with `min_speakers` / `max_speakers`:
118
+
119
+ ```python
120
+ result = pipe("panel.wav", return_speakers=True, min_speakers=2, max_speakers=4)
121
  ```
122
 
123
+ If diarization fails (for example, a missing package), transcription still succeeds. The error is returned under `result["diarization_error"]` instead of raising.
124
 
125
  ### GPU Inference
126