---
license: apache-2.0
language:
- zu
metrics:
- cer
- wer
base_model:
- openai/whisper-small
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- audio
- automatic-speech-recognition
---
# Whisper-Small Finetuned for isiZulu ASR

## Model Details

### Model Description

This model is a fine-tuned version of OpenAI's Whisper-small, optimized for isiZulu Automatic Speech Recognition (ASR). It has been trained on the NCHLT isiZUlu Speech Corpus to improve its performance on isiXhosa speech transcription tasks.

### Base Model
Name: openai/whisper-small
Type: Automatic Speech Recognition (ASR)
Original language: Multilingual

### Performance
- Word Error Rate (WER): 31.87%
- Character Error Rate (CER): 9.43%

### Usage
To use this model for inference:
```python
from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("nmoyo45/zu_whisper")
processor = WhisperProcessor.from_pretrained("nmoyo45/zu_whisper")

# Prepare your audio file (16kHz sampling rate)
audio_input = ...  # Load your audio file here

# Process the audio
input_features = processor(audio_input, sampling_rate=16000, return_tensors="pt").input_features

# Generate token ids
predicted_ids = model.generate(input_features)

# Decode the token ids to text
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)

print(transcription)
```

### Dataset:
#### NCHLT isiZulu Speech Corpus:
- Size: Approximately 56 hours of transcribed speech
- Speakers: 210 (98 female, 112 male)
- Content: Prompted speech (3-5 word utterances read from a smartphone screen)
- Source: Audio recordings smartphone-collected in non-studio environment
- License: Creative Commons Attribution 3.0 Unported License (CC BY 3.0): http://creativecommons.org/licenses/by/3.0/legalcode	
- Citation: N.J. de Vries, M.H. Davel, J. Badenhorst, W.D. Basson, F. de Wet, E. Barnard and A. de Waal, "A smartphone-based ASR data collection tool for under-resourced languages", Speech Communication, Volume 56, January 2014, pp 119–131.	

#### Lwazi isiZulu ASR Corpus:
- Speakers: 199 Speakers
- Content: ~14 elicited utterances, ~16 phonetically balanced read sentences
- License: Creative Commons Attribution 2.5 South Africa License: http://creativecommons.org/licenses/by/2.5/za/legalcode
- Citation: E. Barnard, M. Davel and C. van Heerden, "ASR Corpus Design for Resource-Scarce Languages," in Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech), Brighton, United Kingdom, September 2009, pp. 2847-2850.