nmoyo45/zu_whisper

Whisper-Small Finetuned for isiZulu ASR

Model Details

Model Description

This model is a fine-tuned version of OpenAI's Whisper-small, optimized for isiZulu Automatic Speech Recognition (ASR). It has been trained on the NCHLT isiZUlu Speech Corpus to improve its performance on isiXhosa speech transcription tasks.

Base Model

Name: openai/whisper-small Type: Automatic Speech Recognition (ASR) Original language: Multilingual

Performance

• Word Error Rate (WER): 31.87%
• Character Error Rate (CER): 9.43%

Usage

To use this model for inference:

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("nmoyo45/zu_whisper")
processor = WhisperProcessor.from_pretrained("nmoyo45/zu_whisper")

# Prepare your audio file (16kHz sampling rate)
audio_input = ...  # Load your audio file here

# Process the audio
input_features = processor(audio_input, sampling_rate=16000, return_tensors="pt").input_features

# Generate token ids
predicted_ids = model.generate(input_features)

# Decode the token ids to text
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)

print(transcription)

Dataset:

NCHLT isiZulu Speech Corpus:

• Size: Approximately 56 hours of transcribed speech
• Speakers: 210 (98 female, 112 male)
• Content: Prompted speech (3-5 word utterances read from a smartphone screen)
• Source: Audio recordings smartphone-collected in non-studio environment
• License: Creative Commons Attribution 3.0 Unported License (CC BY 3.0): http://creativecommons.org/licenses/by/3.0/legalcode
• Citation: N.J. de Vries, M.H. Davel, J. Badenhorst, W.D. Basson, F. de Wet, E. Barnard and A. de Waal, "A smartphone-based ASR data collection tool for under-resourced languages", Speech Communication, Volume 56, January 2014, pp 119–131.

Lwazi isiZulu ASR Corpus:

• Speakers: 199 Speakers
• Content: ~14 elicited utterances, ~16 phonetically balanced read sentences
• License: Creative Commons Attribution 2.5 South Africa License: http://creativecommons.org/licenses/by/2.5/za/legalcode
• Citation: E. Barnard, M. Davel and C. van Heerden, "ASR Corpus Design for Resource-Scarce Languages," in Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech), Brighton, United Kingdom, September 2009, pp. 2847-2850.