File size: 2,671 Bytes
8a53c4f 22bff1d dcd86f9 22bff1d 60e7812 22bff1d 6a46794 22bff1d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
license: apache-2.0
language:
- zu
metrics:
- cer
- wer
base_model:
- openai/whisper-small
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags:
- audio
- automatic-speech-recognition
---
# Whisper-Small Finetuned for isiZulu ASR
## Model Details
### Model Description
This model is a fine-tuned version of OpenAI's Whisper-small, optimized for isiZulu Automatic Speech Recognition (ASR). It has been trained on the NCHLT isiZUlu Speech Corpus to improve its performance on isiXhosa speech transcription tasks.
### Base Model
Name: openai/whisper-small
Type: Automatic Speech Recognition (ASR)
Original language: Multilingual
### Performance
- Word Error Rate (WER): 31.87%
- Character Error Rate (CER): 9.43%
### Usage
To use this model for inference:
```python
from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch
# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("nmoyo45/zu_whisper")
processor = WhisperProcessor.from_pretrained("nmoyo45/zu_whisper")
# Prepare your audio file (16kHz sampling rate)
audio_input = ... # Load your audio file here
# Process the audio
input_features = processor(audio_input, sampling_rate=16000, return_tensors="pt").input_features
# Generate token ids
predicted_ids = model.generate(input_features)
# Decode the token ids to text
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription)
```
### Dataset:
#### NCHLT isiZulu Speech Corpus:
- Size: Approximately 56 hours of transcribed speech
- Speakers: 210 (98 female, 112 male)
- Content: Prompted speech (3-5 word utterances read from a smartphone screen)
- Source: Audio recordings smartphone-collected in non-studio environment
- License: Creative Commons Attribution 3.0 Unported License (CC BY 3.0): http://creativecommons.org/licenses/by/3.0/legalcode
- Citation: N.J. de Vries, M.H. Davel, J. Badenhorst, W.D. Basson, F. de Wet, E. Barnard and A. de Waal, "A smartphone-based ASR data collection tool for under-resourced languages", Speech Communication, Volume 56, January 2014, pp 119–131.
#### Lwazi isiZulu ASR Corpus:
- Speakers: 199 Speakers
- Content: ~14 elicited utterances, ~16 phonetically balanced read sentences
- License: Creative Commons Attribution 2.5 South Africa License: http://creativecommons.org/licenses/by/2.5/za/legalcode
- Citation: E. Barnard, M. Davel and C. van Heerden, "ASR Corpus Design for Resource-Scarce Languages," in Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech), Brighton, United Kingdom, September 2009, pp. 2847-2850. |