Automatic Speech Recognition
Transformers
Safetensors
English
asr_model
asr
speech-recognition
speech-to-text
audio
speech-llm
word-timestamps
speaker-diarization
qwen
granite-speech
lora
custom_code
Eval Results (legacy)
Instructions to use mazesmazes/tiny-audio with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mazesmazes/tiny-audio with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="mazesmazes/tiny-audio", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSpeechSeq2Seq model = AutoModelForSpeechSeq2Seq.from_pretrained("mazesmazes/tiny-audio", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Model card: document speaker diarization
Browse files
README.md
CHANGED
|
@@ -84,15 +84,43 @@ for word in result["words"]:
|
|
| 84 |
print(word)
|
| 85 |
```
|
| 86 |
|
| 87 |
-
### Speaker
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
```python
|
| 90 |
result = pipe("meeting.wav", return_speakers=True, num_speakers=2)
|
| 91 |
-
|
| 92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
```
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
### GPU Inference
|
| 98 |
|
|
|
|
| 84 |
print(word)
|
| 85 |
```
|
| 86 |
|
| 87 |
+
### Speaker Diarization
|
| 88 |
+
|
| 89 |
+
`return_speakers=True` adds speaker labels. It first transcribes and aligns words (it implies `return_timestamps=True`). It then splits the audio into speech regions with [TEN VAD](https://github.com/TEN-framework/ten-vad), embeds each region with [ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb), and clusters the embeddings into speakers. Each word is assigned to the speaker whose segment it overlaps.
|
| 90 |
+
|
| 91 |
+
Diarization needs a few extra packages:
|
| 92 |
+
|
| 93 |
+
```bash
|
| 94 |
+
pip install speechbrain ten-vad scikit-learn scipy
|
| 95 |
+
```
|
| 96 |
|
| 97 |
```python
|
| 98 |
result = pipe("meeting.wav", return_speakers=True, num_speakers=2)
|
| 99 |
+
|
| 100 |
+
print(result["text"])
|
| 101 |
+
|
| 102 |
+
# Speaker turns
|
| 103 |
+
for seg in result["speaker_segments"]:
|
| 104 |
+
print(f"{seg['start']:6.2f}-{seg['end']:6.2f} {seg['speaker']}")
|
| 105 |
+
# 0.00- 2.90 SPEAKER_0
|
| 106 |
+
# 3.36- 6.47 SPEAKER_1
|
| 107 |
+
# 6.93- 9.81 SPEAKER_0
|
| 108 |
+
|
| 109 |
+
# Words with timestamps and speakers
|
| 110 |
+
for w in result["words"]:
|
| 111 |
+
print(f"{w['start']:6.2f}-{w['end']:6.2f} {w['speaker']} {w['word']}")
|
| 112 |
+
# 0.00- 0.19 SPEAKER_0 Hi,
|
| 113 |
+
# 0.22- 0.69 SPEAKER_0 Daniel.
|
| 114 |
+
# ...
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
**Setting the speaker count.** Pass `num_speakers` when you know it. Otherwise the number of speakers is estimated from the audio, and on short clips that estimate tends to split one voice into several speakers. You can also bound the estimate with `min_speakers` / `max_speakers`:
|
| 118 |
+
|
| 119 |
+
```python
|
| 120 |
+
result = pipe("panel.wav", return_speakers=True, min_speakers=2, max_speakers=4)
|
| 121 |
```
|
| 122 |
|
| 123 |
+
If diarization fails (for example, a missing package), transcription still succeeds. The error is returned under `result["diarization_error"]` instead of raising.
|
| 124 |
|
| 125 |
### GPU Inference
|
| 126 |
|