DeepOCR v4

A line-level text recognition model for historical print and handwriting. v4 is DeepOCR v3 fine-tuned on more transcriptions and given a subword decoder that writes the line, while the CTC head checks the candidates against the image. On unseen handwritten letters it makes half the errors of v3. Code, training and architecture: DeepOCR.

Files

File What it is
deepocr-v4.pt The model weights (EMA), with every architecture setting, the decoder's config and its tokenizer stored inside
union_bytes_v3.bin Byte-level 6-gram KenLM, for CTC beam search
union_bytes_v3.dict The LM's token list, index-parallel to the CTC head's 261-token byte vocabulary

Use

import torch
from PIL import Image
from infer import load_model, transcribe

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model, tokenizer = load_model('deepocr-v4.pt', device)
print(transcribe(model, tokenizer, [Image.open('line.png')], device, use_decoder=True)[0])

use_decoder=True reads with the subword decoder: it writes four candidate lines, the CTC head scores each against the image, and the best overall wins. This is the most accurate way to read with v4. CTC beam search with the KenLM (transcribe_beam, set up as on the v3 card) also works.

Input is one text line as an RGB image; it is scaled to 256 px high and read up to 6144 px wide. Four flags choose the transcription convention: long_s (ſ), oblique_hyphen (⸗), typographic_punctuation and expand_abbreviations. Set them in infer.CONVENTION; all off by default.

Use the DeepOCR code from commit 163c1b2 onwards. DeepLabel reads with v4's subword decoder as well.

Model

461.6M parameters, fine-tuned from v3:

  • 156.0M for reading the image: a grouped CNN backbone, four mixture-of-experts transformer layers (64 experts, 8 active per column, plus one shared expert) and a CTC head over bytes, so it can emit any Unicode character.
  • 305.6M for the subword decoder: a 238M OLMo 3 language model distilled from HabsburgLLM, with its 48k tokenizer, extended by cross-attention to the image features.

A line reads the same whether it is processed alone or in a batch with longer lines, up to floating-point rounding.

Evaluation

Character error rate (CER), lower is better.

Material Decoding v3 v4
Held-out validation sample, 2,000 lines greedy 7.86% 7.47%
beam 10 + KenLM 7.44% 7.04%
subword decoder 6.82%
Unseen handwritten letters from around 1900, 1,972 lines beam 10 + KenLM 9.36% 5.95%
subword decoder 4.56%

The validation sample covers all training collections, including some added after v3 was trained.

Training data

Public datasets of historical print and handwriting, and line transcriptions made in DeepLabel.

Limitations

  • Reads single lines only; it needs a line detector or segmentation in front of it.
  • A line of one or two words gives the decoder little to go on, and it may write a more common word than the one on the page.
  • Upside-down lines are not detected; turn the crop the right way up first.
  • In runs of leader dots the decoder can write one dot more or fewer; beam search counts them more exactly.
  • Reads best on material like its training data: European print and handwriting from the medieval to the modern period. Languages that are rare in the training data, such as Czech, read clearly worse.

License

Apache-2.0.

Citation

@software{fleischhacker_deepocr,
  author = {Fleischhacker, David},
  title  = {DeepOCR},
  year   = {2026},
  url    = {https://github.com/m2key1/DeepOCR}
}

The subword decoder is distilled from HabsburgLLM:

@software{fleischhacker_habsburgllm,
  author = {Fleischhacker, David},
  title  = {HabsburgLLM},
  year   = {2026},
  url    = {https://github.com/m2key1/HabsburgLLM}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for M2key1/deepocr-v4

Finetuned
(1)
this model

Collection including M2key1/deepocr-v4