Amharic-ASR-v2

Benchmarked on the full SaarAI Amharic ASR leaderboard (9 test sets): 27.15% average WER with the language model, down from 28.19% for v1. See Results for the full comparison.

Amharic-ASR-v2 is SaarAI's second Amharic speech recognition model for real-world and telephone speech. It has the same two parts as Amharic-ASR-v1:

  1. Acoustic model (new): the v1 acoustic model (SaarAI/03082026-135002-01-aug-full-data, 600M-parameter w2v-bert-2.0 CTC) fine-tuned for about 5 more epochs on ~203k Amharic utterances (1,188 h) with stronger telecom-channel augmentation.
  2. Language model (same as v1): the word 4-gram KenLM model (~200M words of Amharic text), used in pyctcdecode beam search with alpha=0.5, beta=1.5, beam 100.

The 411-token character vocabulary is identical to v1, so the v1 LM works unchanged.

Developed by SaarAI
Language Amharic (am, Ge'ez script)
Architecture w2v-bert-2.0 encoder + 1 adapter layer + CTC head (411 tokens) + KenLM 4-gram beam search
Size 0.61B parameters + 613 MB language model
Input / output 16 kHz mono audio → lowercased Amharic text, no punctuation
License CC-BY-4.0

🎧 Quick start

pip install transformers torch torchaudio pyctcdecode kenlm   # kenlm needs Python <= 3.12
from transformers import pipeline

asr = pipeline("automatic-speech-recognition",
               model="SaarAI/Amharic-ASR-v2",
               device=0)
print(asr("audio.wav")["text"])                                       # uses the LM
print(asr("call.wav", chunk_length_s=20, stride_length_s=4)["text"])  # long audio

Full control (LM vs no-LM decoding):

import torch, torchaudio
from transformers import AutoModelForCTC, Wav2Vec2ProcessorWithLM

repo = "SaarAI/Amharic-ASR-v2"
processor = Wav2Vec2ProcessorWithLM.from_pretrained(repo)
model = AutoModelForCTC.from_pretrained(repo).eval().to("cuda")

wav, sr = torchaudio.load("audio.wav")
wav = torchaudio.functional.resample(wav.mean(0), sr, 16_000)
inputs = processor(wav.numpy(), sampling_rate=16_000, return_tensors="pt").to("cuda")
with torch.no_grad():
    logits = model(**inputs).logits

text  = processor.batch_decode(logits.cpu().numpy(), beam_width=100).text[0]  # + 4-gram LM
no_lm = processor.tokenizer.batch_decode(logits.argmax(-1))[0]               # fastest

📈 Results

Dev sets during training (no-LM decoding)

Scored by the training script on raw references (punctuation kept), so WER is higher than with the leaderboard normaliser.

Dev set Utterances WER ↓ CER ↓
WaxalNLP amh validation 2,912 23.74% 7.74%
FLEURS am_et validation 223 21.70% 5.06%

SaarAI ASR leaderboard (WER ↓)

Every model is run through the same open_asr_leaderboard pipeline and normaliser. v2 + LM uses the bundled decoder settings (alpha=0.5, beta=1.5, beam 100).

Test set badrex/Ethio-ASR-amharic addisai/addis-scribe-streaming snapwre/hohe-asr-amharic (+ LM) v1 (+ LM) v2 (no LM) v2 (+ LM)
waxal_amh (WAXAL read speech, test) 23.82% 25.68% 22.34% 25.10% 22.05% 22.94%
TeleASR-Amharic: clean studio 38.64% 22.93% 19.24% 21.93% 20.59% 20.85%
TeleASR-Amharic: ambient room noise 40.89% 22.94% 21.09% 21.42% 21.88% 21.31%
TeleASR-Amharic: distant / low-volume mic 38.69% 20.76% 19.25% 22.30% 20.58% 20.69%
TeleASR-Amharic: harsh environment 44.40% 26.16% 23.60% 23.79% 23.97% 22.94%
TeleASR-Amharic: wideband phone (VoLTE) 47.53% 30.36% 26.15% 24.38% 25.76% 24.05%
TeleASR-Amharic: randomized phone channel 48.97% 31.84% 27.97% 24.83% 26.43% 24.27%
TeleASR-Amharic: narrowband 2G + packet loss 77.59% 66.31% 62.93% 48.20% 50.12% 44.64%
afriswitch_amh (Amharic–English code-switching) 48.28% 37.64% 38.63% 41.78% 41.85% 42.64%
Average (9 test sets) 45.42% 31.62% 29.02% 28.19% 28.14% 27.15%
  • v2 + LM has the best average and wins on all three phone-channel sets and harsh environment. It beats v1 + LM on 8 of 9 sets, with the biggest gain on narrowband 2G (48.20% → 44.64%; hohe 62.93%).
  • snapwre/hohe-asr-amharic is better on clean and near-clean audio: clean studio (19.24%), distant mic (19.25%) and ambient noise (21.09%).
  • addisai/addis-scribe-streaming is best on code-switching (37.64%), with hohe close behind (38.63%).
  • The LM helps most on telephone and noisy audio (up to −5.5 points on 2G). On clean read speech, the v1-tuned weights make results slightly worse (WAXAL +0.9, clean studio +0.3).
  • Speed (RTFx): ~107 without the LM, ~89 with it (hohe + LM: ~55).

Caution on WAXAL scores: the v2 training data includes auto-labelled WAXAL audio from 13 speakers who also appear in the WaxalNLP validation and test sets (no utterances are shared). v2's WAXAL numbers are therefore somewhat optimistic. TeleASR-Amharic and AfriSwitch have no such overlap.

LM re-tune for v2 (dev sets)

Grid search over alpha/beta (beam 100) on cached v2 logits, with leaderboard-style normalisation. The dev sets are a 600-utterance sample of SaarAI/waxal-amharic-combined validation and FLEURS am_et validation + test (739 utterances). This is why the no-LM WER here differs from the training-time dev numbers above.

LM Dev set Best α Best β WER no LM → + LM CER no LM → + LM
v1 4-gram (bundled) WAXAL 0.1 0.5 20.06% → 19.01% 7.54% → 7.39%
v1 4-gram (bundled) FLEURS 0.1 1.0 19.03% → 18.15% 6.38% → 6.24%
v1 5-gram WAXAL 0.2 1.5 20.06% → 19.06% 7.54% → 7.37%
v1 5-gram FLEURS 0.1 1.0 19.03% → 18.01% 6.38% → 6.23%
bigger 4-gram (284M words) WAXAL 0.1 0.5 20.06% → 18.99% 7.54% → 7.38%
bigger 4-gram (284M words) FLEURS 0.1 0.5 19.03% → 18.09% 6.38% → 6.27%

Takeaways:

  • The LM gives about 1 point absolute WER for v2 (~5% relative). v1 gained more, because the v2 acoustic model is stronger.
  • v2 wants a much lower LM weight than v1: alpha≈0.1 instead of 0.5. The shipped alpha=0.5, beta=1.5 were tuned for v1 and are likely sub-optimal for v2.
  • The three LMs perform about the same (within ~0.1 WER). The bigger 4-gram (17.6M sentences from FineWeb-2, Wikipedia, HPLT-2, MADLAD-400, AfriBERTa and SaarAI text, test sets filtered out) is 924 MB versus 613 MB and does not clearly beat the bundled one.

🏗️ How it was built

1. Acoustic model

  • Start: SaarAI/03082026-135002-01-aug-full-data (the v1 acoustic model).

  • Code: Ethio-ASR codebase, CTC loss.

    • learning rate 2e-5, linear schedule, 5% warm-up, bf16, weight decay 0.01
    • effective batch 64 (16 × 4 accumulation), length-grouped batches
    • full fine-tuning, 1 adapter layer, SpecAugment time masking p=0.05, layer drop 0.05
    • ~5 epochs on one A100 80 GB (resumed once from step 9,600); best checkpoint chosen on the mean of WAXAL validation and FLEURS validation
  • Training data (train split, 6,016 duplicates removed):

    Source Utterances Hours
    WaxalNLP (amh) 38,021 189.7
    WAXAL auto-labelled v2 (3-model vote) 63,192 314.3
    BDU Amharic ASR 32,705 101.1
    Common Voice (amh) 563 0.9
    Leyu: Gonder / Wello / Gojjam / Shewa dialects 38,052 409.0
    Leyu: Addis Ababa dialect (new) 9,183 115.8
    snapwre/amharic-speech (new; up-votes > down-votes) 13,769 42.1
    ALFFA Amharic (new; morpheme splits re-joined) 7,420 14.8
    Total 202,905 1,187.7

    WAXAL auto-labels come from a vote between Amharic-ASR-v1, snapwre/hohe-asr-amharic and addisai/addis-scribe-streaming.

  • On-the-fly augmentation: Gaussian noise (p=0.2, SNR 15–40 dB), random gain (p=0.3, −15 to +5 dB), plus one pipeline per utterance sampled from: augmented, over-augmented, low audio, telecom (VoLTE), over-augmented telecom, randomized telecom channel (normal and over-augmented, double weight) and narrowband G.711 (double weight).

2. Language model and decoder

Identical to Amharic-ASR-v1: word 4-gram KenLM (4.07M unigrams) from FineWeb-2, Wikipedia and SaarAI Amharic text, filtered against all leaderboard test sets and FLEURS. Decoder alpha=0.5, beta=1.5, beam_width=100, unk_score_offset=-10, tuned for v1. The v2 dev-set re-tune (above) points to alpha≈0.1, beta=0.5–1.0. You can override these at decode time, e.g. processor.batch_decode(logits, alpha=0.1, beta=0.5, beam_width=100).


📁 Repository contents

File Purpose
model.safetensors, config.json w2v-bert-2.0 CTC acoustic model (v2)
preprocessor_config.json SeamlessM4T feature extractor (80-dim log-mel, 16 kHz)
vocab.json, tokenizer_config.json, special_tokens_map.json, added_tokens.json CTC character tokenizer (same as v1)
alphabet.json pyctcdecode alphabet
language_model/amh_4gram.bin, unigrams.txt, attrs.json KenLM 4-gram, vocabulary and default decoder settings (same as v1)
decoding_config.json settings for the SaarAI open_asr_leaderboard kenlm_ctc runner
alpha_beta.json record of the decoder settings
metrics.json final dev-set metrics from training

⚠️ Limitations

  • Output format: lowercased, no punctuation; numbers come out as Amharic words.
  • Decoder defaults are v1's: the bundled alpha=0.5, beta=1.5 were tuned for v1. The v2 dev re-tune prefers alpha≈0.1 (see Results), which may recover the small LM losses on clean speech.
  • Hard conditions: narrowband 2G + packet loss (44.6% WER) and Amharic–English code-switching (42.6%) are still the weakest sets.
  • Coverage and bias: several dialects (Gonder, Wello, Gojjam, Shewa, Addis Ababa) are covered, but not all Amharic varieties, speaker groups or recording conditions.

🙏 Acknowledgements

Built on Amharic-ASR-v1, the Ethio-ASR project (badrex/Ethio-ASR-amharic, code, arXiv:2603.23654) and TeleASR-Amharic (arXiv:2610.26448). Thanks to the creators of WAXAL, BDU Amharic, Common Voice, Leyu, snapwre/amharic-speech, ALFFA, w2v-bert-2.0, FineWeb-2, Amharic Wikipedia, KenLM and pyctcdecode, and to the authors of snapwre/hohe-asr-amharic and addisai/addis-scribe-streaming, whose models helped auto-label WAXAL.


📚 Citation

@misc{marilign2026teleasramharic,
  title         = {TeleASR-Amharic: A Benchmark for Robust Amharic ASR in Realistic Telecommunication Conditions},
  author        = {Dagnachew Mekonnen Marilign and Enrique Molero and Abrham Haile and Louis Powell and Israel Abebe Azime},
  year          = {2026},
  eprint        = {2610.26448},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  note          = {Accepted to the AACL-IJCNLP 2026 TrustAudio Workshop},
  url           = {https://arxiv.org/abs/2610.26448}
}

@misc{abdullah2026ethioasr,
  title         = {Ethio-ASR: Joint Multilingual Speech Recognition and Language Identification for Ethiopian Languages},
  author        = {Badr M. Abdullah and Israel Abebe Azime and Atnafu Lambebo Tonja and Jesujoba O. Alabi and
                   Abel Mulat Alemu and Eyob G. Hagos and Bontu Fufa Balcha and Mulubrhan A. Nerea and
                   Debela Desalegn Yadeta and Dagnachew Mekonnen Marilign and Amanuel Temesgen Fentahun and
                   Tadesse Kebede and Israel D. Gebru and Michael Melese Woldeyohannis and
                   Walelign Tewabe Sewunetie and Bernd M{\"o}bius and Dietrich Klakow},
  year          = {2026},
  eprint        = {2603.23654},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2603.23654}
}
Downloads last month
3
Safetensors
Model size
0.6B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for SaarAI/Amharic-ASR-v2

Finetuned
(1)
this model

Datasets used to train SaarAI/Amharic-ASR-v2

Papers for SaarAI/Amharic-ASR-v2