README.md

metadata

title: FonBench
emoji: 🗣️
colorFrom: indigo
colorTo: yellow
sdk: static
pinned: true
license: apache-2.0
short_description: The public speech-recognition leaderboard for Fon

FonBench 🇧🇯

The public speech-recognition leaderboard for Fon, a tonal language spoken by over two million people in Benin.

This page carries the leaderboard itself: rankings, queue and submission form. It reads its data live from the same database as the evaluation service, so nothing here is a stale copy.

Evaluation runs elsewhere — a static Space cannot execute Python. Models submitted here are picked up by the evaluator at Kimyayd/FonBench, which holds the GPU and the read access to the private test set.

Metrics

Fon is written with tones (á, ɔ́, ě…) that change the meaning of words, but corpora don't follow the same convention — some mark no tone at all. A raw WER is therefore not comparable from one corpus to the next.

• WER_seg — word errors with tones stripped. Comparable everywhere.
• WER_ton — errors on tone marks alone. Not computed when the corpus doesn't annotate tones, so the figure is never misleading.
• T-WER = WER_seg + 2 × WER_ton — the headline metric.
• RTFx — seconds of audio per second of compute. Higher is faster; hardware-dependent, so only compare at equal hardware.

Test set

2,555 utterances, 4.98 hours, 45 speakers, not published — a test set that circulates stops being a test set. Its 45 speakers are strictly disjoint from the 471 training speakers, verified, zero in common. It is not secret: request access to JMLdata/fon-test-v1 and you can recompute any row yourself.

Verify any number

Scoring code, a standalone evaluator and one script per evaluated model: github.com/Izzoudine/EvalScripts. Expect agreement within ±0.0002 — CTC padding depends on batch composition, and we would rather document that than round the published figures to three decimals.