README.md

---
language: en
license: cc-by-nc-sa-4.0
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
  - automatic-speech-recognition
  - whisper
  - african-accented-english
  - clinical-asr
  - afrispeech
base_model: openai/whisper-medium
datasets:
  - intronhealth/afrispeech-200
metrics:
  - wer
model-index:
  - name: whisper-medium-afrispeech
    results:
      - task:
          type: automatic-speech-recognition
          name: Automatic Speech Recognition
        dataset:
          name: AfriSpeech-200 (16 kHz)
          type: Professor/afrispeech-200-16khz
        metrics:
          - type: wer
            value: 20.13
            name: Test WER
---

# whisper-medium-afrispeech

Fine-tune of [`openai/whisper-medium`](https://huggingface.co/openai/whisper-medium) on
**AfriSpeech-200** for **African-accented English** speech recognition, spanning both
**general** and **clinical/medical** domains.

- **Base model:** openai/whisper-medium (769M)
- **Training data:** [Professor/afrispeech-200-16khz](https://huggingface.co/datasets/Professor/afrispeech-200-16khz) — a clean 16 kHz mono build of [AfriSpeech-200](https://huggingface.co/datasets/intronhealth/afrispeech-200) (Intron Health): ~200 h, 120 accents, 13 countries, clinical + general.
- **Language:** English (African accents)

## Results (test split, 6,178 clips)

### Fine-tuned vs. zero-shot baseline
Zero-shot = `openai/whisper-medium` evaluated on the *same* test set with no fine-tuning.

| Metric | Zero-shot (baseline) | **Fine-tuned (this model)** | Improvement |
|---|---|---|---|
| Overall WER | 43.22% | **20.13%** | **−23.1 pts (−53%)** |
| Clinical WER | 50.55% | **27.47%** | **−23.1 pts (−46%)** |
| General WER | 36.09% | **12.98%** | **−23.1 pts (−64%)** |

Fine-tuning **more than halves WER** across the board.

### WER by domain
| Domain | WER | n |
|---|---|---|
| General | 12.98% | 2,670 |
| **Clinical** | **27.47%** | 3,508 |
| **Overall** | **20.13%** | 6,178 |

Clinical speech remains ~2× harder than general — medical terminology (drug names,
conditions, dosages) drives most of the error, so domain matters when reporting WER.

### WER by accent (accents with n ≥ 25)
Performance varies widely across accents (**~7% to ~45%**):

| Best accents | WER | | Hardest accents | WER |
|---|---|---|---|---|
| okirika | 7.4% | | khana | 44.6% |
| afrikaans | 8.0% | | nyandang | 44.1% |
| brass | 8.6% | | | |
| ikwere | 8.9% | | | |
| twi | 10.3% | | | |

The hardest accents are mostly smaller, under-represented ones — an important
coverage/equity consideration. Full per-accent numbers: `eval_wer_breakdown.csv`.

## Usage

```python
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model = WhisperForConditionalGeneration.from_pretrained("Professor/whisper-medium-afrispeech")
processor = WhisperProcessor.from_pretrained("Professor/whisper-medium-afrispeech")

# audio: a 16 kHz mono waveform (numpy array)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
ids = model.generate(inputs.input_features, language="english", task="transcribe")
print(processor.batch_decode(ids, skip_special_tokens=True)[0])
```

## Training

- **Objective:** full fine-tune (all parameters), 3 epochs
- **Precision / hardware:** bf16 + gradient checkpointing on a single NVIDIA A40 (48 GB)
- **Batch:** 32 effective (16 × 2 grad-accum) · **LR:** 1e-5 (500 warmup, linear decay) · **best model by eval WER**
- **Data handling:** on-the-fly log-Mel feature extraction; clips > 30 s filtered out;
  labels truncated to Whisper's 448-token decoder cap.

## Limitations & biases

- **Clinical error rate is still elevated (~27%)** — verify medical terms; not for unsupervised clinical use.
- **Large accent disparity (~7–45% WER)** — weaker on under-represented accents.
- **≤ 30 s audio**; **English only**; inherits Whisper's general biases.

## Dataset, license & attribution

Trained on AfriSpeech-200 by **Intron Health**, released under **CC-BY-NC-SA-4.0**
(non-commercial, share-alike, attribution). This derivative carries the same license.

```bibtex
@article{olatunji2023afrispeech,
  title={AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR},
  author={Olatunji, Tobi and others},
  journal={Transactions of the Association for Computational Linguistics},
  year={2023}
}
```