Flo976/malagasy-punctuation

Malagasy casing and punctuation restoration for speech recognition

Speech recognition models such as Flo976/w2v-bert-malagasy output lowercase text without punctuation. This model puts back capital letters, commas, periods, question and exclamation marks and colons, without ever changing a word, and restores accented letters that ASR normalization strips (tan na → tanàna).

in:  vehivavy fohy volo iray ary miloko mainty manamboatra volo manoloana ny fitaratra
out: Vehivavy fohy volo iray ary miloko mainty, manamboatra volo manoloana ny fitaratra.

Résumé en français. Remet les majuscules et la ponctuation dans la sortie d'un modèle de reconnaissance vocale malgache, sans jamais changer les mots. Sur la vraie sortie de l'ASR, le texte final mis en forme compte 18,5 % d'erreurs contre 21,7 % pour une mise en forme minimale (majuscule initiale et point final). Les points sont bien placés (F1 79), les virgules une fois sur deux environ (F1 52).

What it does

For every word it predicts a case (lower, Capitalized, UPPER, mixed) and the mark that follows it: none, , . ? ! : or ;. Since it classifies words instead of generating text, it cannot invent, drop or rewrite words: every call asserts that normalize(output) == normalize(input), where normalize is the lowercase, punctuation-free form used to score the ASR. This assertion held on every test output and in a fuzz test.

Results

Test split of badrex/malagasy-speech-full, 3,101 clips, speakers never seen in training. Formatted WER compares the final text to the original transcript with its casing and punctuation, without any normalization: it measures what a reader actually sees.

Input	System	Formatted WER	F1 macro , . ? ! :	F1 comma	F1 period	Case accuracy
real ASR output (10.6 % WER)	capitalize first word + final period	21.73 %	13.1	0	65.6	96.0 %
real ASR output (10.6 % WER)	this model	18.47 %	45.2	51.9	79.0	97.1 %
reference words	capitalize first word + final period	13.09 %	13.3	0	66.3	96.1 %
reference words	this model	9.18 %	51.2	52.7	79.8	97.3 %

On real ASR output the gain over the trivial baseline is −3.26 points of formatted WER (paired bootstrap, 95 % CI [−3.51, −3.02]). With perfect labels, the floor would be 2.4 %: quotes, brackets and some accents are out of scope.

Usage

import os, sys
from huggingface_hub import snapshot_download

path = snapshot_download("Flo976/malagasy-punctuation")
sys.path.insert(0, path)
from punctuate import Restorer

restorer = Restorer(path)  # CPU by default; Restorer(path, device="cuda") on GPU
print(restorer.restore("tena mamy tokoa ity voankazo ity ary tsy mifidy"))

Requirements: torch, transformers, sentencepiece, numpy. Long texts are processed in sliding windows of 150 words. On two CPU threads, a 32-word utterance takes about 80 ms.

The demo also capitalizes the word after ., ? and !, as 99.6 % of sentence starts are in the validation transcripts. That rule is not included in the measurements above.

Training

Base model	Davlan/xlm-roberta-base-finetuned-malagasy, XLM-R base adapted to Malagasy
Head	one token-classification head, 28 classes (4 cases × 7 marks), read on the first sub-token of each word
Data	0.93 M words of raw train-split transcripts of badrex/malagasy-speech-full + 1.5 M words of filtered Malagasy Wikipedia sentences
Robustness	half of the training inputs receive synthetic CTC-style errors (substituted letters, merged or split words)
Accent table	built from the training text, restores letters that normalization strips; same no-word-change guarantee
Selection	on the validation split only; the test was used once
Hardware	one RTX 5070 Ti, 13 minutes

Limitations

• Commas remain hard (F1 about 52): transcribers place them inconsistently.
• Question and exclamation marks are rare in the test (20 and 35 cases): their scores are very uncertain. Semicolons are never predicted.
• Domain: trained on read image descriptions and Wikipedia. Dialogue and questions, as in a voice assistant, are less well punctuated.
• Size: 1.1 GB in fp32. Dynamic int8 quantization breaks it (comma F1 from 54 to 19): do not quantize.
• Pauses in the audio are not used. Adding them helped a separate classifier, but they are not integrated in this model.

License

CC-BY-SA-4.0, because the model is trained on Malagasy Wikipedia (CC BY-SA). The base model, Davlan/xlm-roberta-base-finetuned-malagasy, declares no license on its card; XLM-R itself is MIT. The speech dataset, badrex/malagasy-speech-full, declares no license either. Please credit Davlan (David Adelani) for the base model.

Made for Milo Voice by Flo976 — Florent Didelot / Sooatek.