Turath-OCR-Fix

Turath-OCR-Fix repairs the damage OCR does to Arabic and Persian text. It restores the hamzas an engine flattened (الاسلام → الإسلام), puts back the dots it lost from پ چ ژ گ (کفت → گفت), undoes the letter substitutions that turn a word into a non-word (حيات ه → حياته), strips tatweel and orphaned diacritics, fixes comma spacing, and re-normalizes Persian letters that drifted into Arabic forms and back.

It is a character-level model — google/byt5-base (582M parameters), byte vocabulary — so it sees the letters themselves rather than subword tokens, which is what a task like this needs. Built by Sadiqoon Technologies.

نموذج يُصلح ما يفسده المسح الضوئيّ في النصوص العربية والفارسية: يردّ الهمزات الساقطة، ويعيد النقاط المطموسة من پ چ ژ گ، ويصحّح الحروف الملتبسة، ويحذف التطويل والتشكيل اليتيم، ويضبط المسافات وعلامات الترقيم، ويعيد الحروف الفارسية إلى رسمها. نموذج على مستوى الحرف (ByT5) لأن المهمة مهمة حروف لا كلمات.

Evaluation

600 held-out passages, character error rate against the clean original:

	CER	In plain terms
OCR-damaged input	2.65%	one wrong character every ~38 — roughly 26 errors on a printed page
After Turath-OCR-Fix	0.74%	one wrong character every ~135 — roughly 7 errors on a page

72% of the character errors removed. On a 300-page book that is the difference between some 7,800 corrupted characters and some 2,200: a text a proofreader can finish, instead of one they would rather retype.

The model is conservative by design: it corrects what it recognizes and leaves the rest untouched rather than rewriting the passage. In heritage texts a residual error is far cheaper than an invented one — a wrong letter is visible to any reader, a fluent guess is not.

Training

98,000 passage pairs built from a large Arabic and Persian encyclopedic corpus with sound orthography. Each clean passage was paired with a corrupted version produced by an OCR-noise model: letter confusions drawn from real shape-similarity groups (ب/ت/ن/ي/ث, ج/ح/خ, د/ذ, ص/ض, ع/غ…), hamza flattening, Persian dot loss, Arabic↔Persian letter drift, tatweel insertion, ZWNJ loss, character drops and duplications, word merges and splits, and punctuation damage — applied at four intensity levels from 8% to 35% of words. One epoch.

Usage

from transformers import AutoTokenizer, T5ForConditionalGeneration
tok = AutoTokenizer.from_pretrained("sadiqoon/turath-ocr-fix")
model = T5ForConditionalGeneration.from_pretrained("sadiqoon/turath-ocr-fix").eval()

def fix(text):
    t = tok(text, return_tensors="pt", truncation=True, max_length=384)
    return tok.decode(model.generate(**t, max_new_tokens=384)[0], skip_special_tokens=True)

fix("الاسلام دیـن الفطرة,وقد جاء بما يوافق العقل فى كل شى")

Feed it a sentence or a short passage at a time — up to ~380 bytes works best. For a whole book, split on sentence boundaries and process in batches.

Citation

@misc{sadiqoon2026turathocrfix,
  title  = {Turath-OCR-Fix: Character-Level Post-Correction for Arabic and Persian OCR},
  author = {Sadiqoon Technologies},
  year   = {2026},
  url    = {https://huggingface.co/sadiqoon/turath-ocr-fix}
}

License & Contact

MIT. Built and maintained by Sadiqoon Technologies Ltd, London. Questions and collaboration: info@sadiqoon.uk