|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- zu |
|
|
metrics: |
|
|
- cer |
|
|
- wer |
|
|
base_model: |
|
|
- openai/whisper-small |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
library_name: transformers |
|
|
tags: |
|
|
- audio |
|
|
- automatic-speech-recognition |
|
|
--- |
|
|
# Whisper-Small Finetuned for isiZulu ASR |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
This model is a fine-tuned version of OpenAI's Whisper-small, optimized for isiZulu Automatic Speech Recognition (ASR). It has been trained on the NCHLT isiZUlu Speech Corpus to improve its performance on isiXhosa speech transcription tasks. |
|
|
|
|
|
### Base Model |
|
|
Name: openai/whisper-small |
|
|
Type: Automatic Speech Recognition (ASR) |
|
|
Original language: Multilingual |
|
|
|
|
|
### Performance |
|
|
- Word Error Rate (WER): 31.87% |
|
|
- Character Error Rate (CER): 9.43% |
|
|
|
|
|
### Usage |
|
|
To use this model for inference: |
|
|
```python |
|
|
from transformers import WhisperForConditionalGeneration, WhisperProcessor |
|
|
import torch |
|
|
|
|
|
# Load model and processor |
|
|
model = WhisperForConditionalGeneration.from_pretrained("nmoyo45/zu_whisper") |
|
|
processor = WhisperProcessor.from_pretrained("nmoyo45/zu_whisper") |
|
|
|
|
|
# Prepare your audio file (16kHz sampling rate) |
|
|
audio_input = ... # Load your audio file here |
|
|
|
|
|
# Process the audio |
|
|
input_features = processor(audio_input, sampling_rate=16000, return_tensors="pt").input_features |
|
|
|
|
|
# Generate token ids |
|
|
predicted_ids = model.generate(input_features) |
|
|
|
|
|
# Decode the token ids to text |
|
|
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True) |
|
|
|
|
|
print(transcription) |
|
|
``` |
|
|
|
|
|
### Dataset: |
|
|
#### NCHLT isiZulu Speech Corpus: |
|
|
- Size: Approximately 56 hours of transcribed speech |
|
|
- Speakers: 210 (98 female, 112 male) |
|
|
- Content: Prompted speech (3-5 word utterances read from a smartphone screen) |
|
|
- Source: Audio recordings smartphone-collected in non-studio environment |
|
|
- License: Creative Commons Attribution 3.0 Unported License (CC BY 3.0): http://creativecommons.org/licenses/by/3.0/legalcode |
|
|
- Citation: N.J. de Vries, M.H. Davel, J. Badenhorst, W.D. Basson, F. de Wet, E. Barnard and A. de Waal, "A smartphone-based ASR data collection tool for under-resourced languages", Speech Communication, Volume 56, January 2014, pp 119–131. |
|
|
|
|
|
#### Lwazi isiZulu ASR Corpus: |
|
|
- Speakers: 199 Speakers |
|
|
- Content: ~14 elicited utterances, ~16 phonetically balanced read sentences |
|
|
- License: Creative Commons Attribution 2.5 South Africa License: http://creativecommons.org/licenses/by/2.5/za/legalcode |
|
|
- Citation: E. Barnard, M. Davel and C. van Heerden, "ASR Corpus Design for Resource-Scarce Languages," in Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech), Brighton, United Kingdom, September 2009, pp. 2847-2850. |
