diacnet-1.0

diacnet-1.0 restores diacritics/accents to text that's been typed or scraped without them, across 10 languages. Fine-tuned from google/byt5-small — character/byte-level rather than word-level, so it handles Yoruba tone marks, Vietnamese combining diacritics, and Polish/Turkish special characters through the same mechanism, no per-language vocabulary needed.

• Single joint model, all 10 languages — a language tag prefix (<yor>, <vie>, etc.) tells the model which diacritic inventory to apply, no separate models or an upstream language-ID step required.
• Median CER of ~0.02 across most languages (see Benchmarks) — near-perfect restoration on well-formed input.
• Fully self-supervised training — no manual annotation. Clean, already- diacritized text is the target; diacritics are deterministically stripped to create the training input.

🗒️ Model Details

Base model	google/byt5-small
Architecture	Byte-level seq2seq (T5)
Max sequence length	256 bytes (trained on sentence-level examples)
Languages	10 (see below)
Training data	olaverse/qg-passages-multi, split into sentences
Training	3 epochs, batch size 16 × grad-accum 2, lr 1e-4

Languages: yo vi ig ha pl tr pt es fr it (ISO 639-1; ISO 639-3: yor vie ibo hau pol tur por spa fra ita)

Scoped deliberately to languages where diacritics are lexically meaningful — not applied to the other 15 languages in the source corpus (e.g. Swahili, Zulu, Amharic, Japanese), where diacritic restoration either doesn't apply or isn't the right frame for the script.

🏃 Usage

from transformers import AutoTokenizer, T5ForConditionalGeneration

tok = AutoTokenizer.from_pretrained("olaverse/diacnet-1.0")
model = T5ForConditionalGeneration.from_pretrained("olaverse/diacnet-1.0")

text = "<yor> se eranko naa si gbo o?"
inputs = tok(text, return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(output_ids[0], skip_special_tokens=True))
# ṣé ẹranko náà sì gbọ́ ọ?

Prefix input text with the target language tag (<yor>, <vie>, <ibo>, <hau>, <pol>, <tur>, <por>, <spa>, <fra>, <ita>). Works best on single sentences or short passages — see Known Limitations for longer text.

📊 Benchmarks

External evaluation on out-of-distribution test data — not our training distribution. ~1,000 diacritic-bearing test sentences per language: Tatoeba (es/fr/it/pt/pl/tr/vi), MasakhaNEWS (ig/ha), MENYO-20k test split (yo). Sentences were diacritic-stripped to form inputs; the original text is the reference.

Systems: diacnet-1.0 (this model — one joint model, all 10 languages) vs. copy-input (the undiacritized text unchanged — the floor).

WER — word error rate (%, lower is better)

system	es	fr	it	pt	pl	tr	vi	ig	yo	ha
copy-input (floor)	17.38	18.19	14.16	19.47	40.01	42.76	86.25	34.11	87.31	7.80
diacnet-1.0	2.07	1.45	0.62	1.61	13.18	19.49	21.60	8.00	29.08	8.27 ⚠️

CER — character error rate (%, lower is better)

system	es	fr	it	pt	pl	tr	vi	ig	yo	ha
copy-input (floor)	3.13	3.61	2.49	3.65	7.51	9.57	23.01	9.28	36.37	1.58
diacnet-1.0	0.38	0.28	0.11	0.30	2.53	4.04	8.02	2.16	12.44	2.13 ⚠️

ChrF (higher is better)

system	es	fr	it	pt	pl	tr	vi	ig	yo	ha
copy-input (floor)	87.08	85.51	89.80	85.43	72.40	69.45	27.21	69.10	16.93	93.49
diacnet-1.0	98.41	98.89	99.53	98.75	91.11	85.87	77.65	92.61	68.90	93.28 ⚠️

Reading the results honestly

• diacnet-1.0 beats the do-nothing floor on 9 of 10 languages, often by a wide margin: Italian WER 14.2 → 0.6, French 18.2 → 1.5, Vietnamese 86.3 → 21.6.
• Yorùbá remains the hardest language at 29.1 WER — still well above the Romance-language scores. A dedicated Yorùbá-only specialist, Davlan/mt5_base_yoruba_adr (roughly 2× diacnet's size, trained on human-authored text), scored 16.4 WER on an earlier run of this same test set — still ahead of us. Yorùbá tone restoration is largely a semantic disambiguation problem: the same base letters can spell several different words (e.g. ogun war / ògùn medicine / ogún twenty), so closing this gap needs more human-authored Yorùbá training data, not just more parameters.
• Hausa: restoration currently does not work (⚠️). diacnet scores worse than doing nothing (8.3 vs 7.8 WER). With only ~15 Hausa examples in training, the model never learned the hooked-letter inventory (ɓ ɗ ƙ) and largely passes text through unrestored. Do not use diacnet-1.0 for Hausa; fixing this is the top priority for 1.1.
• The earlier ~0.02 median CER figure came from a held-out slice of our own training distribution; the numbers above, on external text, are the ones to trust.

Evaluation code and test sets are open — see DiacBench (olaverse/diacbench) — so these numbers are reproducible and other models can be compared on the same footing.

🔜 Toward diacnet-1.1

1. Hausa — train on adequate Hausa data (Wura, MasakhaNEWS); target: clearly beat the copy-input floor.
2. Yorùbá — add human-authored training text (e.g. MENYO-20k train split) to close the gap to the monolingual specialist.
3. Turkish/Vietnamese — remaining errors concentrate in lexically ambiguous cases; larger-capacity variants under evaluation.

Training data & licensing

Fine-tuned from google/byt5-small (Apache-2.0) on olaverse/qg-passages-multi (Apache-2.0), split into sentences and paired with a diacritic-stripped copy of each sentence as the self-supervised training input. Released under Apache-2.0.

Citation

@misc{diacnet-1.0,
  title  = {diacnet-1.0},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/diacnet-1.0}
}