comma-lightonocr-3-4b

lightonai/LightOnOCR-3-4B fine-tuned for CATMuS-compliant, line-by-line transcription of medieval Latin manuscript pages: one line of output per physical written line, in reading order. CATMuS is a graphemic standard — the letters and signs as written, abbreviations kept, no expansion or normalisation.

LightOnOCR-3 is LightOn's document-OCR model, built on the Qwen3.5 vision-language architecture. This fine-tune keeps that architecture unchanged and retrains it on 9,906 pages of this collection.

Results

243 held-out pages, each in a different hand, none seen in training. CER is quoted after NFD normalisation, because the corpus writes ũ both precomposed and as u + combining tilde, and charging two edits for one glyph measures the encoding rather than the reading.

model	CER (NFD)	macro	WER (NFD)	line recall (NFD)	truncated
LightOnOCR-3-4B, zero-shot (same job)	0.3876	0.3486	0.8218	0.0056	2.9%
this model	0.1228	0.1036	0.3605	0.1681	0.4%
LightOnOCR-3-0.8B, full FT	0.1206	0.1018	0.3613	0.1678	0.4%
chandra-ocr-2 (5.3B), LoRA	0.1232	0.1039	0.3639	0.1668	0.8%
Qwen3.5-2B, full FT	0.1251	0.1066	0.3616	0.1647	0.8%
Qwen3.5-9B, full FT	0.1168	0.0985	0.3541	0.1751	0.4%

Greedy decoding, 3,072 max new tokens, 2,048 visual tokens per page, eval batch 4. Treat differences under 0.02 CER as ties — four seeds of one configuration in this project span 0.021.

There is no exact control for this model, and it is no better than the 0.8B. Both attempts to fine-tune stock Qwen3.5-4B on these pages hit the cluster's 48-hour limit before finishing, so the 4B can only be bracketed against neighbours. It lands at 0.1228 against the LightOnOCR-3-0.8B fine-tune's 0.1206 — within noise, and evaluated on different hardware (see below). Five times the parameters bought nothing measurable on this task; this project has seen the same plateau around 2B with stock Qwen.

Prefer comma-lightonocr-3-0.8b unless you have reason to want the larger model.

Comparing across GPUs

This model was evaluated on an H200; most comparison rows on an RTX PRO 6000. That matters more than it sounds: untrained LightOnOCR-3-4B scored 0.3674 on one GPU type and 0.3876 on the other — the same weights, pages and code, 0.0202 apart. bf16 arithmetic differs between architectures and greedy decoding amplifies it. Treat gaps under ~0.03 between this model and RTX-scored rows as ties. The zero-shot row above was scored in the same job as this model, so that one comparison is exact.

Usage

import torch
from PIL import Image
from huggingface_hub import hf_hub_download
from transformers import AutoModelForImageTextToText, AutoProcessor

REPO = "wjbmattingly/comma-lightonocr-3-4b"
model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(REPO)

# The prompt is part of the model: it was trained to answer this exact instruction.
# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.
prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read()

image = Image.open("page.jpg").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True, enable_thinking=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=3072, do_sample=False,
                     repetition_penalty=1.1, use_cache=True)
print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

• prompt.txt is required. This model was trained on a 11,629-character instruction, shipped in this repo; LightOnOCR-3 itself was trained on an empty prompt and emits block-labelled markdown with bounding boxes, which is not this task.
• enable_thinking=False. Passed explicitly so the generation prompt closes the thinking block regardless of which template variant is loaded.
• The image budget is in the repo. processor_config.json is capped at 2,048 visual tokens, the budget used in training and scoring. LightOnOCR-3 ships uncapped (~16k tokens); loading a stock processor shows the page at a resolution this model never trained on.
• repetition_penalty=1.1 is this project's measured serving setting: it removes most looping on sparse pages at a small cost in line recall. The table above is greedy (penalty 1.0); set 1.0 to reproduce it.
• generation_config.json sets use_cache: true. LightOnOCR-3-0.8B ships false (a training artefact), which makes generation recompute the whole sequence at every step; every number above was produced with the cache on.

Training

Full fine-tune of all 4,539,265,536 parameters (vision tower included), 3 epochs, 3,717 steps, learning rate 5e-05, cosine schedule, effective batch 8, bf16, 2,048 visual tokens per page, on H200. 2 of 9,906 pages exceeded the 8,192-token length budget and were dropped (the stock-Qwen control dropped 1).

Data: comma-project/deep-jsonl.

Limits

A research artifact for cultural-heritage and digital-humanities work, not a production OCR service. Trained on one collection, it should be expected to degrade on other hands, languages, genres and centuries. It reproduces what is written, including historical spelling and language; transcribing a record is not an endorsement of its contents. Verify any name, date or figure before relying on it.