pandanlabs/dots-mocr-arabic-classical

Pandan Arabic OCR v0.1 (research preview)

Pandan Arabic OCR is a specialised vision-language model for faithful transcription of classical Arabic books. It is trained to keep the vowel marks (harakat) visible on the page and not to add marks the page does not print. Version 0.1 performs best on clean, fully vowelled typeset pages; results on scans, and on old low-resolution scans in particular, are more variable.

v0.1 is a fine-tune of dots.mocr by Pandan Labs.

Research preview. v0.1's weights are here to try, and feedback is welcome. Read Where it works today first: it is strongest on typeset print.

Release evaluation

Check	Result
Model version	v0.1 (training run lora-001)
Baseline	dots.mocr, revision e539fbb, same pages, same settings
Train / test separation	By book: no test or validation book is used in training
Main evaluation	Held-out books with references verified by a person: Kashf ash-Shubuhat, Sharh Ma'na at-Taghut (test); Thalathat al-Usul, Nawaqid al-Islam (validation); 160 pages
Statistical test	Paired bootstrap over pages, 95% confidence interval for each difference
Gate on readable print (typeset; scans at about 300 dpi once verified)	Passed. Vowel error rate 0.030 → 0.006 (95% CI of the change −0.028 to −0.020, 116 pages); vowel marks added that the page does not print: 14.5 → 2.0 per 1,000 letters; no pages failed to read; verse match rate 0.92 → 0.97
Old low-resolution scans (separate, softer rule: fail only if clearly much worse than base)	Passed narrowly. Vowel error rate 0.386 → 0.476 on Sharh Ma'na at-Taghut (CI +0.045 to +0.138, limit +0.05)
Letter error rate, all 267 evaluated pages (serving with the loop retry, see How to use)	0.061 → 0.040; old low-resolution scans 0.114 → 0.070
Release decision	Research preview

Where it works today

Kind of book	v0.1
Typeset classical print, fully vowelled	Works. 96% of pages at 95% accuracy or better, two thirds at 99% or better (verified references).
Scans at about 300 dpi, partly vowelled or unvowelled	Promising preliminary results; reference verification in progress.
Old low-resolution scans	Not yet for vowel marks. Letters read better than the base model (letter error 0.114 → 0.070), but vowel marks are about as good as base, sometimes worse. This is the focus of the next version.

Faithfulness to the page

For partly vowelled and unvowelled pages, accuracy of vowel marks is less telling than how many marks the model adds that the page does not print, and how many printed marks it drops. Per 1,000 letters, base model → v0.1:

Print	Vowelling	Pages	Marks added	Marks dropped	References
Typeset print	full	116	14.5 → 2.0	4.0 → 1.7	verified
Scan, about 300 dpi	full	9	75.9 → 21.8	113.4 → 74.6	provisional
Scan, about 300 dpi	partial	19	62.8 → 13.5	15.0 → 9.5	provisional
Scan, about 300 dpi	none	31	3.8 → 8.1	6.2 → 1.9	provisional
Old low-resolution scan	full	7	78.6 → 77.2	14.2 → 39.4	verified and provisional
Old low-resolution scan	partial	58	31.4 → 27.5	12.4 → 5.2	verified and provisional
Old low-resolution scan	none	27	63.3 → 41.6	12.9 → 4.3	verified and provisional

On unvowelled 300 dpi scans v0.1 adds more marks than the base model (3.8 → 8.1 per 1,000 letters), even though its overall vowel error rate is lower. Unvowelled pages still carry vowelled Qur'an quotations, so "none" describes the running text.

How accurately v0.1 reads each kind of print

Each page falls in one accuracy band. Verified means a person checked the reference text against the page; provisional means the reference is an unchecked machine transcription, so those figures are preliminary.

Print	References	Pages	99% and above	95–99%	80–95%	below 80%	Failed to read
Typeset print	verified	116	78 (67%)	33 (28%)	5 (4%)	0 (0%)	0
Scan, about 300 dpi	provisional	59	3 (5%)	36 (61%)	12 (20%)	7 (12%)	1
Old low-resolution scan	verified	44	1 (2%)	6 (14%)	31 (70%)	5 (11%)	1
Old low-resolution scan	provisional	48	2 (4%)	16 (33%)	24 (50%)	4 (8%)	2

Provisional references: Sharh Hadith Inna Kunna fi Jahiliyya, Sharh al-Jami' li-'Ibadat Allah Wahdah, Sharh as-Sunna (al-Barbahari), al-'Arabiyya bayna Yadayk.

Reading the figures

• Accuracy: 1 − character error rate over the whole page text, letters and vowel marks together, punctuation, brackets, digits and spacing included (edit distance in characters ÷ length of the reference).
• Letters: the same on bare letters only, with vowel marks, punctuation and digits set aside.
• Vowel marks: 1 − diacritic error rate, the share of letters (among those both texts share) whose vowel marks differ, counting letters that carry a mark in either text.

A page can show 100% for letters and for vowel marks and still score below 100% overall: the overall figure also counts punctuation, brackets, digits and spacing.

All figures are measured after the optional verse layer (see below), which replaces recognised Qur'an quotations with the canonical text, and with the loop retry described under How to use.

In the examples below, the left side is the page as printed and the right side is the model's text: red marks a wrong word, yellow an extra word that is not on the page, and struck-through red a word the model missed. The line at the bottom of each card gives the base model's score on the same page.

99% and above

Thalathat al-Usul, typeset print, full vowelling, verified reference: 100.0% accuracy, 100.0% letters, 100.0% vowel marks.

Kashf ash-Shubuhat, typeset print, full vowelling, verified reference: 100.0% accuracy, 100.0% letters, 100.0% vowel marks.

Nawaqid al-Islam, typeset print, full vowelling, verified reference: 99.7% accuracy, 100.0% letters, 100.0% vowel marks.

95–99%

Kashf ash-Shubuhat, typeset print, full vowelling, verified reference: 98.9% accuracy, 99.4% letters, 98.5% vowel marks.

Thalathat al-Usul, typeset print, full vowelling, verified reference: 98.8% accuracy, 100.0% letters, 100.0% vowel marks.

Sharh Ma'na at-Taghut, old low-resolution scan, partial vowelling, verified reference: 97.3% accuracy, 97.7% letters, 95.4% vowel marks.

80–95%

Sharh Ma'na at-Taghut, old low-resolution scan, partial vowelling, verified reference: 94.7% accuracy, 99.2% letters, 72.5% vowel marks.

Kashf ash-Shubuhat, typeset print, full vowelling, verified reference: 93.2% accuracy, 92.2% letters, 98.7% vowel marks.

Sharh Hadith Inna Kunna fi Jahiliyya, scan, about 300 dpi, partial vowelling, provisional reference: 94.8% accuracy, 96.9% letters, 90.6% vowel marks.

below 80%

Sharh Ma'na at-Taghut, old low-resolution scan, none vowelling, verified reference: 65.1% accuracy, 78.4% letters, 6.7% vowel marks.

al-'Arabiyya bayna Yadayk, scan, about 300 dpi, full vowelling, provisional reference: 78.2% accuracy, 85.5% letters, 77.0% vowel marks.

Sharh al-Jami' li-'Ibadat Allah Wahdah, old low-resolution scan, partial vowelling, provisional reference: 78.0% accuracy, 94.1% letters, 74.5% vowel marks.

The verse layer (optional post-processing)

The Pandan verse layer is a separate step, not part of the model weights. It finds Qur'an quotations in the model's output, matches each to an ayah, and replaces it with the King Fahd Complex (Hafs) Uthmani text of exactly the part quoted. This is canonicalisation: where a book prints a verse in a different spelling, the layer's output deliberately differs from the page. For strict diplomatic transcription, use the model's output without the layer. The figures on this page are measured after the layer.

How to use

This repository holds the full merged model (base plus fine-tune, about 6 GB), loaded like dots.mocr. v0.1 is revision 037b472. Tested with vLLM 0.30.0 on one 24 GB GPU (L4):

vllm serve pandanlabs/dots-mocr-arabic-classical --revision 037b472f2a2fd004e02ce012549844af119eea62 --trust-remote-code \
  --chat-template-content-format string --max-model-len 16384 --limit-mm-per-prompt '{"image": 1}'

--trust-remote-code runs Python code from the model repository (dots.mocr's model definition). Pin a revision you have reviewed.

Send one page per request, about 2048 px on the long side, with temperature 0 and up to 4,096 output tokens. The image tokens go before the prompt in the text part:

{"model": "pandanlabs/dots-mocr-arabic-classical", "temperature": 0, "max_tokens": 4096,
 "messages": [{"role": "user", "content": [
   {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
   {"type": "text", "text": "<|img|><|imgpad|><|endofimg|>Extract the text content from this image."}]}]}

Rarely, on title and front-matter pages, the model repeats a line until it reaches the token limit (finish_reason is length). Ask for that page once more with "repetition_penalty": 1.1; leave pages that finish normally alone, since a penalty on every page would also discourage words a page genuinely repeats. On the evaluation pages this retry fixed two of three such pages and lowered the letter error rate on old low-resolution scans from 0.126 to 0.070.

The output is Markdown: # headings, **bold**, ﴿verse﴾ [السورة: ن], and footnotes as [^n] after a --- line. Running headers and page numbers are left out.

How it was evaluated

Every comparison is against the base model on the same pages: vowel error rate, letter error rate, vowel marks added and dropped, runs of skipped or repeated words, and how often quoted verses match an ayah, each with a 95% confidence interval over pages. Results are reported for each book, kind of print and level of vowelling, and a version is promoted only if no slice of readable print gets worse.

Training data

Pages typeset from classical Arabic text (the Tashkeela corpus, hadith collections, and books whose transcriptions a person verified), rendered with a range of fonts, vowelling levels and scan effects, plus real pages of verified books. No page or paragraph of a test or validation book was used.

Limitations

• Old low-resolution scans: close to the base model and sometimes below it, especially on small verse type the model cannot make out.
• Unvowelled scans: letters read well; v0.1 still adds some vowel marks the page does not print.
• Honorific signs (such as ﷺ and ﵁) are sometimes written out in words.
• A few title and front-matter pages produce repeated text; the retry above fixes most, not all.
• Scans at about 300 dpi are measured against provisional references until they are verified.

Next

The next version focuses on generalising to real scans, partly vowelled and unvowelled print, and degraded pages, rather than on clean typeset print, where v0.1 is already close to its ceiling.

Licence

Released under the terms of dots.mocr (MIT, with the dots.mocr licence agreement, included).

Contact

Pandan Labs, hello@pandanlabs.ai