Soloba Bambara ASR, adapted to environmental and sanitation reports (Map Action, v3)

Speech-to-text for Bambara, fine-tuned by Map Action so that citizens can report environmental and sanitation problems by voice. It is a fine-tune of RobotsMali/soloba-ctc-0.6b-v3 (FastConformer CTC, 0.6 B parameters, CC-BY-4.0). All credit for the base model goes to RobotsMali.

Results

Word error rate, lower is better. Text is lower-cased and punctuation removed. Same test clips before and after.

Test set Clips Base model This model
Map Action voice reports (held-out sentences) 16 41.1 % 33.8 %
General Bambara (RobotsMali/afvoices, held-out) 200 33.5 % 27.9 %
Domain clips from RobotsMali/bam-asr-early test 48 57.8 % 54.7 %

Character error rate on the Map Action reports: 16.8 % β†’ 14.6 %.

Training

  • 2,600 steps, batch 16, AdamW, learning rate 5e-5 with cosine decay, on one A40 GPU (32 minutes).
  • Data: 102 Map Action recordings with human-corrected transcripts (weighted Γ—8), 4,154 domain clips filtered from open RobotsMali corpora (see the dataset Map-Action/bambara-asr-environnement), and 1,300 general clips from RobotsMali/afvoices to avoid forgetting.
  • Held-out test sentences were never used for training.

Limitations, please read

  • One speaker. The Map Action recordings used for training and for the first test set come from a single male speaker. The gain on that test set partly reflects adaptation to his voice. Expect worse results on other voices.
  • Code-switching. People speak urban Bambara mixed with French. References spell French words the French way ("carburant", "forage"); the model often writes them phonetically. WER counts these as errors even when the word was understood, which is why CER is much lower than WER.
  • The 30 % WER target is met on general Bambara, not yet on voice reports.
  • Transcripts must be shown as machine-generated, next to the original audio.

Usage

import nemo.collections.asr as nemo_asr   # nemo_toolkit[asr]==2.5.0
model = nemo_asr.models.ASRModel.restore_from("soloba-ma-v3.nemo")
print(model.transcribe(["report.wav"])[0].text)   # 16 kHz mono

Earlier iterations (-v1, -v2) are kept for reproducibility. Measurement files are in this repository.

Downloads last month
27
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Map-Action/soloba-bm-map-action-v3