Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string

whisper-indian-lora

A LoRA adapter for Whisper Medium that adapts it to Indian-accented English. Part of IndiVoice-DeepASR by PxA Labs.

Commercial ASR systems lose accuracy on Indian English accents. IndiVoice-DeepASR fine-tunes Whisper with low-rank adapters so the adaptation can be trained on free cloud GPUs (Kaggle and Colab) and shipped as a small add-on to the base model.

Model details

Base model openai/whisper-medium
Method LoRA (PEFT): rank 16, alpha 32, dropout 0.05, on q_proj and v_proj
Training data ai4bharat/Svarah, Indian-accented English (gated dataset)
Language English (Indian accents)
Developed by Purvansh Joshi and Archit Mittal, PxA Labs
Code github.com/PxA-Labs/IndiVoice-DeepASR

How to use

import torch
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor

base = WhisperForConditionalGeneration.from_pretrained("openai/whisper-medium", torch_dtype=torch.float16).to("cuda")
model = PeftModel.from_pretrained(base, "PxALabs/whisper-indian-lora")
processor = WhisperProcessor.from_pretrained("openai/whisper-medium")

# audio: a 16 kHz mono float array
inputs = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to("cuda", torch.float16)
ids = model.generate(input_features=inputs, language="en", task="transcribe")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

The repository also contains final/ (adapter plus processor files) and last-checkpoint/ (full trainer state, for resuming training).

Training

  • 2,000 steps, learning rate 1e-4 (linear schedule, 50 warm-up steps), effective batch size 32 (8 per device × 2 GPUs × 2 accumulation steps), mixed precision, seed 42.
  • Training and validation data come from one Svarah manifest, split 90/10 at random (train_test_split(test_size=0.1, seed=42)).

Evaluation

Word error rate (WER, %) on the 10% validation split, as logged during training:

Step 200 400 600 800 1000 1200 1400 1600 1800 2000
Validation loss 0.270 0.227 0.196 0.170 0.150 0.132 0.119 0.107 0.100 0.097
WER 22.95 25.87 27.98 22.57 19.29 32.99 25.84 27.21 26.74 27.28

The final evaluation (step 2000) gives WER 27.28. The lowest logged WER was 19.29, at step 1000.

Read these numbers with care:

  • Validation loss falls steadily while WER moves up and down, so WER on this split is noisy.
  • The split is random over utterances, so the same speakers can appear in training and validation. Expect WER on new speakers to be higher.
  • This is not the official Svarah benchmark protocol, and no base Whisper Medium result on the same split is reported yet, so the numbers can't be compared directly with published Svarah results.

Limitations

  • Trained only on Indian-accented English. It is not a model for Hindi or other Indian languages.
  • Not evaluated for fairness across individual accents, genders or recording conditions.
  • Evaluate on your own audio before relying on it.

Framework versions

PEFT 0.18.1 · Transformers 4.57.6 · PyTorch 2.10.0+cu128 · Datasets 4.8.3 · Tokenizers 0.22.2

Citation

@misc{indivoice2026,
  author = {Purvansh Joshi and Archit Mittal},
  title = {IndiVoice-DeepASR: Efficient Adaptation of Multilingual Speech Models for Indian Accents},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/PxA-Labs/IndiVoice-DeepASR}}
}
Downloads last month
60
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for PxALabs/whisper-indian-lora

Adapter
(141)
this model

Dataset used to train PxALabs/whisper-indian-lora