Flo976/w2v-bert-malagasy

w2v-BERT 2.0 — Malagasy speech recognition, robust on short utterances

12.7 % word error rate on short Malagasy utterances (2–8 s), against 21.5 % for a Whisper medium fine-tuned on the same data — and 5× faster on GPU.

This CTC model transcribes Malagasy speech. It is built for voice assistants and dictation, where speech arrives as short segments cut by voice-activity detection. Most Malagasy ASR models are trained and measured on 15–30 s recordings, and make about 65 % more errors on short segments; this one was trained on both.

Try it: Flo976/malagasy-asr-demo · Language model: Flo976/malagasy-kenlm-4gram-v2 · Punctuation: Flo976/malagasy-punctuation · CPU version: Flo976/w2v-bert-malagasy-onnx-int8

Résumé en français. Modèle de reconnaissance vocale malgache (w2v-BERT 2.0, CTC), entraîné pour rester fiable sur les énoncés courts d'un assistant vocal. Sur 11 625 énoncés de 2 à 8 s jamais vus à l'entraînement, il fait 12,7 % d'erreurs de mots avec le modèle de langue v2, contre 21,5 % pour un Whisper medium fine-tuné sur les mêmes données, et répond 5 fois plus vite. Il ne régresse pas sur les extraits longs (10,1 %). Sortie en minuscules, sans ponctuation : un modèle séparé remet majuscules et ponctuation.

Results

Short utterances (2–8 s) — the voice-assistant regime

11,625 utterances (16.0 h, 94 speakers), cut from the test split by forced alignment at word boundaries. No test speaker appears in training.

Model	Decoding	WER	CER
this model	KenLM 4-gram v2 + apostrophe rule	12.68 %	3.63 %
this model	KenLM 4-gram + apostrophe rule	13.20 %	3.71 %
this model	greedy	14.33 %	3.76 %
badrex/w2v-bert-2.0-malagasy-asr	KenLM 4-gram ¹	17.77 %	5.45 %
badrex/w2v-bert-2.0-malagasy-asr	greedy	19.87 %	5.79 %
Flo976/whisper-malagasy-medium	greedy	21.49 %	8.97 %

¹ Measured with the first version of this card's decoding, which passed raw logits instead of log-probabilities to pyctcdecode. With that same decoding this model scored 13.63 % (13.40 % with the apostrophe rule), so badrex comparisons with a language model hold at equal decoding. Passing log-probabilities brings 0.2 point on short utterances (95 % CI [−0.23, −0.15]); the 134 M-word language model v2 brings a further 0.5 point ([−0.65, −0.40]).

Each comparison quoted on this card is significant at 95 % (paired bootstrap on the same utterances, 2,000 resamples). Against the Whisper medium, this model gains 8.1 points (95 % CI [−8.64, −7.64]). At equal decoding this model beats its starting point badrex by 5.5 points greedy (95 % CI [−5.76, −5.34]) and by 4.1 points with the language model ([−4.34, −3.93]).

Full-length clips — no regression

The 3,101 original test clips (16.5 h, median 18 s).

Model	Decoding	WER	CER
this model	KenLM 4-gram v2 + apostrophe rule	10.13 %	2.59 %
this model	KenLM 4-gram + apostrophe rule	10.41 %	2.63 %
badrex/w2v-bert-2.0-malagasy-asr	KenLM 4-gram ²	10.86 %	2.67 %
this model	greedy	12.01 %	2.85 %
badrex/w2v-bert-2.0-malagasy-asr	greedy	12.20 %	2.88 %
Flo976/whisper-malagasy-medium	greedy	13.07 %	3.48 %

² Raw logits, no apostrophe rule. With that same decoding this model scored 10.60 %, the figure first published on this card.

Speed

One utterance at a time (2–8 s), as a voice assistant receives them:

Model	Hardware	Median latency	90th percentile
this model, greedy	RTX 5070 Ti	52 ms	65 ms
this model, with KenLM	RTX 5070 Ti	74 ms	92 ms
Whisper medium (Flo976)	RTX 5070 Ti	259 ms	648 ms
this model, greedy	2 CPU cores (Intel Core Ultra 7 265K)	1.45 s	1.53 s
int8 ONNX version, greedy	2 CPU cores (Intel Core Ultra 7 265K)	0.63 s ³

³ Measured on 100 utterances of 4.8 s on average, against 1.70 s for this model in the same run. No significant accuracy loss on 1,500 short utterances; see its card. The gain depends on the processor: on a Hugging Face cpu-basic Space it was slower than fp32 (3.0 s against 1.85 s for a 5 s phrase).

A CTC model reads the audio in one pass; Whisper writes its output token by token, so its latency grows with the sentence. On CPU the acoustic model dominates: the language model adds no measurable time. Set PyTorch's thread count to the cores actually allotted (torch.set_num_threads): in a container, the default host count oversubscribes the CPU and triples latency.

Usage

Output is lowercase, without punctuation, with numbers written as digits. Audio must be 16 kHz mono.

Greedy decoding

import librosa
import torch
from transformers import AutoModelForCTC, AutoProcessor

processor = AutoProcessor.from_pretrained("Flo976/w2v-bert-malagasy")
model = AutoModelForCTC.from_pretrained("Flo976/w2v-bert-malagasy").eval()

audio, _ = librosa.load("audio.wav", sr=16000, mono=True)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(processor.batch_decode(logits.argmax(-1))[0])

With the Malagasy language model (recommended)

pip install --no-deps pyctcdecode==0.5.0
pip install pygtrie https://github.com/kpu/kenlm/archive/master.zip

import json, re
import numpy as np
from huggingface_hub import snapshot_download
from pyctcdecode import build_ctcdecoder

lm = snapshot_download("Flo976/malagasy-kenlm-4gram-v2")
params = json.load(open(f"{lm}/params.json"))
unigrams = [w.strip() for w in open(f"{lm}/unigrams.txt", encoding="utf-8") if w.strip()]
tok = processor.tokenizer
vocab = tok.get_vocab()
labels = ["" if t == tok.pad_token else " " if t == tok.word_delimiter_token else "⁇" if t == tok.unk_token else t
          for t in sorted(vocab, key=vocab.get)]
p = params["short" if len(audio) / 16000 <= 10 else "long"]  # α 0.6 β 0.5 up to 10 s, α 0.5 β 0.5 above
decoder = build_ctcdecoder(labels, kenlm_model_path=f"{lm}/{params['lm']}", unigrams=unigrams,
                           alpha=p["alpha"], beta=p["beta"])  # build once, reuse

logp = torch.log_softmax(logits[0].float(), -1).numpy()  # log-probabilities, not raw logits
text = decoder.decode(logp, beam_width=100)
text = re.sub(r"'\s+", "'", re.sub(r"\s+'", "'", text))  # "amin' ny" -> "amin'ny"
print(text)

The parameters were tuned on the validation split only. Pass log-probabilities to pyctcdecode: the first version of this card passed raw logits, which costs 0.2 point. The language model v2 contains web text; for a model under a clean CC-BY-SA licence, use Flo976/malagasy-kenlm-4gram with α = 0.3, β = −1.0 up to 10 s and β = 0.0 above (13.20 % and 10.41 %).

A spell-checking pass on top of the language model was measured and hurts (+0.17 point); it helps only without the language model (−0.38 point).

Casing, punctuation and confidence

• Flo976/malagasy-punctuation restores capital letters and punctuation without changing any word.
• The demo also shows a calibrated confidence for each word (asr_extras.py in its files): a word at 80 % is right about 8 times out of 10. Asking the user to repeat the 10 % least confident utterances lowers the WER of the others from 12.7 % to 11.4 % on the short test set.

Recordings longer than about 30 s are best split at pauses; see app.py in the demo for a simple approach.

Training

Starting point	badrex/w2v-bert-2.0-malagasy-asr, itself fine-tuned from facebook/w2v-bert-2.0
Data	train split of badrex/malagasy-speech-full: 28,168 full clips (147.7 h) + 105,284 short segments (144.0 h) cut from those same clips
Short segments	CTC forced alignment of the reference transcript, cut at word boundaries into 2–8 s pieces
Objective	CTC, character vocabulary (a–z, 0–9, apostrophe, hyphen)
Schedule	learning rate 1e-5, cosine, 500 warm-up steps; up to 400 s of audio per optimizer step
Regularization	SpecAugment, time mask 0.05, feature mask 0.10
Precision	bfloat16, gradient checkpointing
Length	best checkpoint at step 3,200 (0.65 epoch); stopped at step 4,800 after four evaluations without improvement
Hardware	one RTX 5070 Ti (16 GB), about 3.5 hours

The gain comes from the short segments: continuing badrex's training on full-length clips alone, with the same recipe, gave no improvement on validation over 0.75 epoch.

Evaluation protocol

• Speaker-disjoint splits: 56 speakers in train, 95 in validation, 94 in test, no overlap (verified).
• Shared normalization of references and hypotheses: Unicode NFC, lowercase, punctuation removed, apostrophes and hyphens kept inside words.
• Choices on validation only: checkpoint selection, language-model parameters (tuned on half the speakers, checked on the other half) and post-processing rules. The test split was used once per frozen configuration.
• Significance: paired bootstrap on the same utterances; a difference is reported only when its 95 % confidence interval excludes zero.

Limitations

• Short utterances are a proxy. They are cut from read descriptions of images, not conversational speech captured by a microphone and a voice-activity detector. Results on real assistant traffic may differ.
• Segment boundaries come from badrex's alignment, which may slightly favour models of the same family over Whisper on the short-utterance set.
• Under 3 s remains the weakest regime: 15.9 % WER with the first language model; on 0.5–2 s segments, 24.4 % greedy.
• Noise: with added noise at 10 dB SNR, greedy WER on short utterances rises from 14.3 % to 18.7 %, and to 30.9 % at 0 dB.
• Single training run: run-to-run variance was not measured.
• Domain and dialect: the training data is read speech, mostly official Malagasy (Merina). Coastal dialects and noisy environments are not evaluated.
• No punctuation or casing in the model output (see Flo976/malagasy-punctuation), numbers as digits.

License and attribution

This model is released under CC-BY-4.0. It is a derivative of badrex/w2v-bert-2.0-malagasy-asr by Badr al-Absi (CC-BY-4.0), itself derived from facebook/w2v-bert-2.0 (MIT). Please credit both when you reuse it.

The training data, badrex/malagasy-speech-full, does not declare a license or a source on the Hub. The language models are published separately: malagasy-kenlm-4gram under CC-BY-SA-3.0 (Malagasy Wikipedia), and malagasy-kenlm-4gram-v2 with mixed sources including web text.

Citation

@misc{w2v-bert-malagasy-2026,
  author    = {Florent Didelot},
  title     = {w2v-BERT 2.0 for Malagasy speech recognition, robust on short utterances},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Flo976/w2v-bert-malagasy}
}

Developed for Milo Voice, a Malagasy voice assistant, by Flo976 — Florent Didelot / Sooatek. Questions and feedback, especially from native speakers, are welcome in the discussions.