Instructions to use PxALabs/whisper-indian-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use PxALabs/whisper-indian-lora with PEFT:
Task type is invalid.
- Transformers
How to use PxALabs/whisper-indian-lora with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="PxALabs/whisper-indian-lora")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("PxALabs/whisper-indian-lora", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string
whisper-indian-lora
A LoRA adapter for Whisper Medium that adapts it to Indian-accented English. Part of IndiVoice-DeepASR by PxA Labs.
Commercial ASR systems lose accuracy on Indian English accents. IndiVoice-DeepASR fine-tunes Whisper with low-rank adapters so the adaptation can be trained on free cloud GPUs (Kaggle and Colab) and shipped as a small add-on to the base model.
Model details
| Base model | openai/whisper-medium |
| Method | LoRA (PEFT): rank 16, alpha 32, dropout 0.05, on q_proj and v_proj |
| Training data | ai4bharat/Svarah, Indian-accented English (gated dataset) |
| Language | English (Indian accents) |
| Developed by | Purvansh Joshi and Archit Mittal, PxA Labs |
| Code | github.com/PxA-Labs/IndiVoice-DeepASR |
How to use
import torch
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor
base = WhisperForConditionalGeneration.from_pretrained("openai/whisper-medium", torch_dtype=torch.float16).to("cuda")
model = PeftModel.from_pretrained(base, "PxALabs/whisper-indian-lora")
processor = WhisperProcessor.from_pretrained("openai/whisper-medium")
# audio: a 16 kHz mono float array
inputs = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to("cuda", torch.float16)
ids = model.generate(input_features=inputs, language="en", task="transcribe")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])
The repository also contains final/ (adapter plus processor files) and last-checkpoint/ (full trainer state, for resuming training).
Training
- 2,000 steps, learning rate 1e-4 (linear schedule, 50 warm-up steps), effective batch size 32 (8 per device × 2 GPUs × 2 accumulation steps), mixed precision, seed 42.
- Training and validation data come from one Svarah manifest, split 90/10 at random (
train_test_split(test_size=0.1, seed=42)).
Evaluation
Word error rate (WER, %) on the 10% validation split, as logged during training:
| Step | 200 | 400 | 600 | 800 | 1000 | 1200 | 1400 | 1600 | 1800 | 2000 |
|---|---|---|---|---|---|---|---|---|---|---|
| Validation loss | 0.270 | 0.227 | 0.196 | 0.170 | 0.150 | 0.132 | 0.119 | 0.107 | 0.100 | 0.097 |
| WER | 22.95 | 25.87 | 27.98 | 22.57 | 19.29 | 32.99 | 25.84 | 27.21 | 26.74 | 27.28 |
The final evaluation (step 2000) gives WER 27.28. The lowest logged WER was 19.29, at step 1000.
Read these numbers with care:
- Validation loss falls steadily while WER moves up and down, so WER on this split is noisy.
- The split is random over utterances, so the same speakers can appear in training and validation. Expect WER on new speakers to be higher.
- This is not the official Svarah benchmark protocol, and no base Whisper Medium result on the same split is reported yet, so the numbers can't be compared directly with published Svarah results.
Limitations
- Trained only on Indian-accented English. It is not a model for Hindi or other Indian languages.
- Not evaluated for fairness across individual accents, genders or recording conditions.
- Evaluate on your own audio before relying on it.
Framework versions
PEFT 0.18.1 · Transformers 4.57.6 · PyTorch 2.10.0+cu128 · Datasets 4.8.3 · Tokenizers 0.22.2
Citation
@misc{indivoice2026,
author = {Purvansh Joshi and Archit Mittal},
title = {IndiVoice-DeepASR: Efficient Adaptation of Multilingual Speech Models for Indian Accents},
year = {2026},
publisher = {GitHub},
howpublished = {\url{https://github.com/PxA-Labs/IndiVoice-DeepASR}}
}
- Downloads last month
- 60
Model tree for PxALabs/whisper-indian-lora
Base model
openai/whisper-medium