Aadhil96/whisper-small-sinhala

Model Summary

A Sinhala speech-to-text model: openai/whisper-small fine-tuned on the OpenSLR SLR52 Large Sinhala ASR corpus.

On 3,000 held-out clips from speakers not used in training, it reaches 9.13% CER and 36.47% WER. These scores are likely optimistic, because most test sentences also appear in the training data. See Evaluation Results before quoting them.

Usage

Try it in the Gradio demo notebook on Kaggle, or run it yourself:

import librosa
from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="Aadhil96/whisper-small-sinhala", chunk_length_s=30)

audio, _ = librosa.load("clip.wav", sr=16000, mono=True)
result = asr({"raw": audio, "sampling_rate": 16000},
             generate_kwargs={"language": "sinhala", "task": "transcribe"})
print(result["text"])

Model Characteristics

• Base model: openai/whisper-small
• Task: Sinhala transcription
• Training: 6,000 steps, batch size 16, learning rate 1e-5, mixed precision, on one NVIDIA T4 GPU
• Checkpoint selection: lowest CER on a 1,000-clip validation set

Data Overview

Corpus: OpenSLR SLR52, about 185K utterances of Sinhala speech, licensed under CC BY-SA 4.0.

Split: by speaker, so nobody in the validation or test sets appears in training.

Split	Speakers	Clips
Train	432	160,953 (194.7 hours)
Validation	23	1,000 sampled
Test	23	3,000 sampled

Text processing: transcripts are Unicode NFC-normalized, and anything outside the Sinhala block is removed, including digits, Latin text and punctuation. Zero-width joiners are kept in the training text because Sinhala conjuncts need them.

Filtering: clips over 20 seconds were removed. 178,378 of the corpus's 185,293 clips were kept. Most of the rest (6,865) were dropped when transcripts were matched and normalized, and the cause has not been investigated.

Evaluation Results

All models were scored on the same 3,000 clips with the same code, using plain greedy decoding. Zero-width joiners were removed before scoring.

Model	WER	CER	Median clip CER	Runaway outputs*
This model	36.47%	9.13%	6.67%	0
whisper-large-v3, unmodified	340.41%	327.51%	94.44%	938
whisper-small, unmodified	444.67%	744.62%	806.90%	2,636

*Output more than 3 times the length of the reference, usually from a repetition loop.

Treating clips as independent, a bootstrap gives a 95% interval for this model's WER of about 35.4% to 37.6%. The true uncertainty is larger, because the test set has only 23 speakers.

Please read these results with care:

• Sentence overlap: 2,125 of the 3,000 test sentences (71%) also appear in the training data, spoken by other people. Performance on new sentences is likely worse, and no score on unseen sentences alone has been measured.
• Baselines: the unmodified models were run without Whisper's usual fallback settings for escaping repetition loops. That inflates their error rates, so the table does not show how much this model improves on them, and those figures should not be read as an improvement claim.
• WER and CER: clips are short, with a median of 4 words, so one wrong letter counts as a whole wrong word. Some errors are only differences in word spacing.

Limitations

• It outputs Sinhala script only, with no digits, Latin text or punctuation.
• It was trained on clips of up to 20 seconds of recorded speech. It has not been tested on noisy audio, phone calls, heavy code-switching with English, or other real-world conditions.
• It has not been evaluated across dialects or speaker groups.

Licence and Attribution

This model is released under CC BY-SA 4.0 because it is derived from SLR52, which uses that licence. The base model, openai/whisper-small, is released by OpenAI under Apache 2.0.

Corpus citation: Kjartansson, O., Sarin, S., Pipatsrisawat, K., Jansche, M., and Ha, L. (2018). Crowd-Sourced Speech Corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali. SLTU 2018.