README.md

---
title: FonBench
emoji: 🗣️
colorFrom: indigo
colorTo: yellow
sdk: static
pinned: true
license: apache-2.0
short_description: The public speech-recognition leaderboard for Fon
---

# FonBench 🇧🇯

The public speech-recognition leaderboard for **Fon**, a tonal language
spoken by over two million people in Benin.

This page carries the leaderboard itself: rankings, queue and submission
form. It reads its data live from the same database as the evaluation
service, so nothing here is a stale copy.

Evaluation runs elsewhere — a static Space cannot execute Python. Models
submitted here are picked up by the evaluator at
[Kimyayd/FonBench](https://huggingface.co/spaces/Kimyayd/FonBench), which
holds the GPU and the read access to the private test set.

## Metrics

Fon is written with tones (á, ɔ́, ě…) that change the meaning of words, but
corpora don't follow the same convention — some mark no tone at all. A raw
WER is therefore not comparable from one corpus to the next.

- **WER_seg** — word errors with tones stripped. Comparable everywhere.
- **WER_ton** — errors on tone marks alone. Not computed when the corpus
  doesn't annotate tones, so the figure is never misleading.
- **T-WER** = `WER_seg + 2 × WER_ton` — the headline metric.
- **RTFx** — seconds of audio per second of compute. Higher is faster;
  hardware-dependent, so only compare at equal hardware.

## Test set

2,555 utterances, 4.98 hours, 45 speakers, **not published** — a test set
that circulates stops being a test set. Its 45 speakers are strictly
disjoint from the 471 training speakers, verified, zero in common. It is
not secret: request access to `JMLdata/fon-test-v1` and you can recompute
any row yourself.

## Verify any number

Scoring code, a standalone evaluator and one script per evaluated model:
[github.com/Izzoudine/EvalScripts](https://github.com/Izzoudine/EvalScripts).
Expect agreement within ±0.0002 — CTC padding depends on batch composition,
and we would rather document that than round the published figures to three
decimals.