Automatic Speech Recognition
NeMo
speech-recognition
fastconformer
indic
multilingual
telephony
call-center
zero-shot-language-identification
rnn-t
Eval Results (legacy)
Instructions to use TieIncred/VaaniCall with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use TieIncred/VaaniCall with NeMo:
import nemo.collections.asr as nemo_asr asr_model = nemo_asr.models.ASRModel.from_pretrained("TieIncred/VaaniCall") transcriptions = asr_model.transcribe(["file.wav"]) - Notebooks
- Google Colab
- Kaggle
Fix model card images with Hub resolve URLs
Browse files
README.md
CHANGED
|
@@ -79,7 +79,7 @@ model-index:
|
|
| 79 |
---
|
| 80 |
|
| 81 |
<div align="center">
|
| 82 |
-
<img src="assets/banner.svg" alt="VaaniCall — multilingual telephony ASR for India" width="100%">
|
| 83 |
</div>
|
| 84 |
|
| 85 |
<br>
|
|
@@ -92,7 +92,7 @@ By [Tausif Iqbal](https://huggingface.co/TieIncred). Built on NVIDIA NeMo FastCo
|
|
| 92 |
VaaniCall transcribes a phone call without a language code. It infers language and script, and it is trained through a 21-stage telephony pipeline so AMR, packet loss, and handset noise look like training data — not a domain shift.
|
| 93 |
|
| 94 |
<div align="center">
|
| 95 |
-
<img src="assets/languages.svg" alt="Supported languages: Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Gujarati, Punjabi, Odia, English" width="100%">
|
| 96 |
</div>
|
| 97 |
|
| 98 |
---
|
|
@@ -114,7 +114,7 @@ VaaniCall transcribes a phone call without a language code. It infers language a
|
|
| 114 |
All numbers are **Word Error Rate** (lower is better). Evaluations use **500 clips per language**.
|
| 115 |
|
| 116 |
<div align="center">
|
| 117 |
-
<img src="assets/wer_overall_vs_whisper.png" alt="Overall WER: VaaniCall vs Whisper large-v3 on IndicVoices and FLEURS" width="100%">
|
| 118 |
</div>
|
| 119 |
|
| 120 |
| Benchmark | Condition | VaaniCall | Whisper large-v3 | Relative WER cut |
|
|
@@ -126,7 +126,7 @@ All numbers are **Word Error Rate** (lower is better). Evaluations use **500 cli
|
|
| 126 |
Whisper large-v3 still leads on **English**. VaaniCall is the Indic-language model.
|
| 127 |
|
| 128 |
<div align="center">
|
| 129 |
-
<img src="assets/telephony_robustness.png" alt="VaaniCall WER stays flat from clean to telephonic audio" width="100%">
|
| 130 |
</div>
|
| 131 |
|
| 132 |
---
|
|
@@ -136,7 +136,7 @@ Whisper large-v3 still leads on **English**. VaaniCall is the Indic-language mod
|
|
| 136 |
### vs Whisper large-v3 — IndicVoices, telephonic
|
| 137 |
|
| 138 |
<div align="center">
|
| 139 |
-
<img src="assets/wer_indicvoices_telephonic.png" alt="Per-language telephonic WER vs Whisper large-v3" width="100%">
|
| 140 |
</div>
|
| 141 |
|
| 142 |
<details>
|
|
@@ -184,7 +184,7 @@ Whisper large-v3 still leads on **English**. VaaniCall is the Indic-language mod
|
|
| 184 |
Held-out read speech. Odia is not in FLEURS.
|
| 185 |
|
| 186 |
<div align="center">
|
| 187 |
-
<img src="assets/wer_fleurs.png" alt="FLEURS out-of-domain WER vs Whisper large-v3" width="100%">
|
| 188 |
</div>
|
| 189 |
|
| 190 |
<details>
|
|
@@ -211,7 +211,7 @@ Held-out read speech. Odia is not in FLEURS.
|
|
| 211 |
### Per-language profile
|
| 212 |
|
| 213 |
<div align="center">
|
| 214 |
-
<img src="assets/wer_by_language.png" alt="VaaniCall WER by language on telephonic IndicVoices" width="100%">
|
| 215 |
</div>
|
| 216 |
|
| 217 |
Strongest Indic languages on phone audio: **Punjabi 14.5% · Bengali 15.4% · Hindi 16.0% · Marathi 17.1%**.
|
|
@@ -221,7 +221,7 @@ Strongest Indic languages on phone audio: **Punjabi 14.5% · Bengali 15.4% · Hi
|
|
| 221 |
VaaniCall is a telephony specialist on top of VAANI. On phone-channel audio it improves **9 of 10** Indic languages. On clean audio the two models are statistically tied (24.37% vs 24.38% WER).
|
| 222 |
|
| 223 |
<div align="center">
|
| 224 |
-
<img src="assets/delta_vs_vaani_base.png" alt="WER reduction versus VAANI base under telephonic conditions" width="100%">
|
| 225 |
</div>
|
| 226 |
|
| 227 |
<details>
|
|
@@ -320,7 +320,7 @@ Largest telephonic gaps: Malayalam −5.61 pp, Telugu −5.49 pp. English exclud
|
|
| 320 |
## How it works
|
| 321 |
|
| 322 |
<div align="center">
|
| 323 |
-
<img src="assets/architecture.svg" alt="Phone audio into FastConformer encoder, RNNT decoder, transcript with no language ID" width="100%">
|
| 324 |
</div>
|
| 325 |
|
| 326 |
### Training
|
|
@@ -328,7 +328,7 @@ Largest telephonic gaps: Malayalam −5.61 pp, Telugu −5.49 pp. English exclud
|
|
| 328 |
Two-phase fine-tune from the VAANI FastConformer checkpoint.
|
| 329 |
|
| 330 |
<div align="center">
|
| 331 |
-
<img src="assets/training.svg" alt="Phase 1 language alignment, phase 2 acoustic refinement" width="100%">
|
| 332 |
</div>
|
| 333 |
|
| 334 |
1. **Language alignment** — encoder frozen; decoder learns script and language mapping.
|
|
@@ -343,7 +343,7 @@ Trained on [AI4Bharat IndicVoices](https://huggingface.co/datasets/ai4bharat/Ind
|
|
| 343 |
Every training batch can pass through a 21-stage, GPU-accelerated call-channel simulator.
|
| 344 |
|
| 345 |
<div align="center">
|
| 346 |
-
<img src="assets/pipeline.svg" alt="Talker, handset, codec, network, and line stages of the augmentation pipeline" width="100%">
|
| 347 |
</div>
|
| 348 |
|
| 349 |
<details>
|
|
|
|
| 79 |
---
|
| 80 |
|
| 81 |
<div align="center">
|
| 82 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/banner.svg" alt="VaaniCall — multilingual telephony ASR for India" width="100%">
|
| 83 |
</div>
|
| 84 |
|
| 85 |
<br>
|
|
|
|
| 92 |
VaaniCall transcribes a phone call without a language code. It infers language and script, and it is trained through a 21-stage telephony pipeline so AMR, packet loss, and handset noise look like training data — not a domain shift.
|
| 93 |
|
| 94 |
<div align="center">
|
| 95 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/languages.svg" alt="Supported languages: Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Gujarati, Punjabi, Odia, English" width="100%">
|
| 96 |
</div>
|
| 97 |
|
| 98 |
---
|
|
|
|
| 114 |
All numbers are **Word Error Rate** (lower is better). Evaluations use **500 clips per language**.
|
| 115 |
|
| 116 |
<div align="center">
|
| 117 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/wer_overall_vs_whisper.png" alt="Overall WER: VaaniCall vs Whisper large-v3 on IndicVoices and FLEURS" width="100%">
|
| 118 |
</div>
|
| 119 |
|
| 120 |
| Benchmark | Condition | VaaniCall | Whisper large-v3 | Relative WER cut |
|
|
|
|
| 126 |
Whisper large-v3 still leads on **English**. VaaniCall is the Indic-language model.
|
| 127 |
|
| 128 |
<div align="center">
|
| 129 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/telephony_robustness.png" alt="VaaniCall WER stays flat from clean to telephonic audio" width="100%">
|
| 130 |
</div>
|
| 131 |
|
| 132 |
---
|
|
|
|
| 136 |
### vs Whisper large-v3 — IndicVoices, telephonic
|
| 137 |
|
| 138 |
<div align="center">
|
| 139 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/wer_indicvoices_telephonic.png" alt="Per-language telephonic WER vs Whisper large-v3" width="100%">
|
| 140 |
</div>
|
| 141 |
|
| 142 |
<details>
|
|
|
|
| 184 |
Held-out read speech. Odia is not in FLEURS.
|
| 185 |
|
| 186 |
<div align="center">
|
| 187 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/wer_fleurs.png" alt="FLEURS out-of-domain WER vs Whisper large-v3" width="100%">
|
| 188 |
</div>
|
| 189 |
|
| 190 |
<details>
|
|
|
|
| 211 |
### Per-language profile
|
| 212 |
|
| 213 |
<div align="center">
|
| 214 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/wer_by_language.png" alt="VaaniCall WER by language on telephonic IndicVoices" width="100%">
|
| 215 |
</div>
|
| 216 |
|
| 217 |
Strongest Indic languages on phone audio: **Punjabi 14.5% · Bengali 15.4% · Hindi 16.0% · Marathi 17.1%**.
|
|
|
|
| 221 |
VaaniCall is a telephony specialist on top of VAANI. On phone-channel audio it improves **9 of 10** Indic languages. On clean audio the two models are statistically tied (24.37% vs 24.38% WER).
|
| 222 |
|
| 223 |
<div align="center">
|
| 224 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/delta_vs_vaani_base.png" alt="WER reduction versus VAANI base under telephonic conditions" width="100%">
|
| 225 |
</div>
|
| 226 |
|
| 227 |
<details>
|
|
|
|
| 320 |
## How it works
|
| 321 |
|
| 322 |
<div align="center">
|
| 323 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/architecture.svg" alt="Phone audio into FastConformer encoder, RNNT decoder, transcript with no language ID" width="100%">
|
| 324 |
</div>
|
| 325 |
|
| 326 |
### Training
|
|
|
|
| 328 |
Two-phase fine-tune from the VAANI FastConformer checkpoint.
|
| 329 |
|
| 330 |
<div align="center">
|
| 331 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/training.svg" alt="Phase 1 language alignment, phase 2 acoustic refinement" width="100%">
|
| 332 |
</div>
|
| 333 |
|
| 334 |
1. **Language alignment** — encoder frozen; decoder learns script and language mapping.
|
|
|
|
| 343 |
Every training batch can pass through a 21-stage, GPU-accelerated call-channel simulator.
|
| 344 |
|
| 345 |
<div align="center">
|
| 346 |
+
<img src="https://huggingface.co/TieIncred/VaaniCall/resolve/main/assets/pipeline.svg" alt="Talker, handset, codec, network, and line stages of the augmentation pipeline" width="100%">
|
| 347 |
</div>
|
| 348 |
|
| 349 |
<details>
|