small-models-for-glam/page-ocr-flags

Page OCR flags

A small image model that looks at a scanned page before OCR and answers two questions:

1. Is there any text on this side of the leaf to transcribe? (no_text)
2. Is LightOnOCR-3-0.8B likely to fail on this page? (lo_blank_write, lo_trunc, lo_loop)

Use it to skip pages with nothing to read, to send risky pages to a different OCR setup, or to explain bad OCR output after the fact. It is for libraries, archives and anyone else running OCR over a large number of page scans. It was trained mostly on 19th- and 20th-century printed books from the Biodiversity Heritage Library (BHL), so check it on a sample of your own pages before you rely on it.

MobileNetV4-conv-small, 2.5M parameters, 384 × 384 input, about 50 pages per second on one CPU thread. For the stricter question "is this a page of body text?", use the companion model text-page-filter.

A score near 1 means no text. The picture page is a miss: most labelled picture-only pages score as text.

Outputs

Seven independent sigmoid outputs (multi-label, not a softmax):

Output	Meaning	Use
no_text	No text to transcribe on this side of the leaf: blank paper, show-through only, a library stamp only	Skip OCR, or treat OCR output as suspect
lo_blank_write	Page has no text, but LightOnOCR-3 writes text anyway (usually it reads show-through)	Explain invented output
lo_trunc	LightOnOCR-3 reaches the 4,096-token output limit we used (long tables, or repetition)	Route to a larger token budget or another engine
lo_loop	LightOnOCR-3 repeats itself (dot leaders, repeated lines)	Route or re-run
table, dense	Page has a table / dense text (soft teacher answers)	Routing hints
faint	Experimental, do not use (see limitations)	–

How to use

import timm, torch
from PIL import Image

model = timm.create_model("hf-hub:small-models-for-glam/page-ocr-flags", pretrained=True).eval()
cfg = timm.data.resolve_data_config({}, model=model)   # 384 x 384, squash (no crop), ImageNet mean/std
transform = timm.data.create_transform(**cfg)
labels = model.pretrained_cfg["label_names"]

page = Image.open("page.jpg").convert("L").convert("RGB")   # trained on grey pages
with torch.inference_mode():
    probs = torch.sigmoid(model(transform(page).unsqueeze(0)))[0]
print(dict(zip(labels, probs.round(decimals=3).tolist())))

Convert pages to grey first, as in the example: the model was trained on grey pages.

Suggested thresholds for no_text

Score	Decision	On the 250 hand-labelled hard pages
> 0.8	No text: skip, or flag any OCR output as invented	67/67 blank, 83/90 show-through, 1/54 text
0.2 – 0.8	Unsure: route or check	a mix: 5 faint, 5 show-through, 4 text, 3 stamp, 1 picture
< 0.2	Text: OCR normally	49/54 text

Batch inference on the GPU (NVIDIA DALI, JPEG bytes in)

import numpy as np, timm, torch
from nvidia.dali import fn, pipeline_def, types   # pip install nvidia-dali-cuda130 --extra-index-url https://pypi.nvidia.com
from torch.utils.dlpack import from_dlpack

BATCH = 64

@pipeline_def(batch_size=BATCH, num_threads=4, device_id=0)
def pages():
    enc = fn.external_source(name="enc", dtype=types.UINT8)
    img = fn.decoders.image(enc, device="mixed", output_type=types.GRAY)   # decode on the GPU
    return fn.resize(img, resize_x=384, resize_y=384, interp_type=types.INTERP_LINEAR, antialias=True)

pipe = pages(); pipe.build()
model = timm.create_model("hf-hub:small-models-for-glam/page-ocr-flags", pretrained=True).eval().cuda().half()
mean = torch.tensor([0.485, 0.456, 0.406], device="cuda").view(1, 3, 1, 1)
std = torch.tensor([0.229, 0.224, 0.225], device="cuda").view(1, 3, 1, 1)

def score(jpeg_bytes):                                  # a list of BATCH encoded JPEGs
    pipe.feed_input("enc", [np.frombuffer(b, dtype=np.uint8) for b in jpeg_bytes])
    out, = pipe.run()
    x = from_dlpack(out.as_tensor()).permute(0, 3, 1, 2).float() / 255   # (B, 1, 384, 384)
    x = ((x.expand(-1, 3, -1, -1) - mean) / std).half()
    with torch.inference_mode():
        return torch.sigmoid(model(x).float()).cpu()

Evaluation

Test set	Result
2,165 BHL pages with ground-truth transcriptions	no_text at 0.5 keeps 0.999 of text pages and removes 0.994 of empty pages
250 hand-labelled hard pages (one labeller; chosen deliberately, so not a random sample)	no_text AUROC 0.970 (round 1) and 0.935 (round 2)
2,086 pages from 133 books not used in training, scored against LightOnOCR-3-0.8B's real outputs	AUROC 0.941 (lo_blank_write), 0.936 (lo_trunc), 0.930 (lo_loop)

Full evaluation tables

Numbers are for the released checkpoint (seed 1), plus mean ± SD over three training seeds where shown.

Pages with ground-truth transcriptions: 2,165 BHL pages. Text = at least 80 characters of ground truth.

	Result
no_text AUROC, text pages vs empty pages	1.000
Text pages kept at 0.5	0.999 (1,736 / 1,737)
Empty pages removed at 0.5	0.994
Pages with 1–79 characters (mostly plates with captions)	kept as text (removed: 0.047)

Hand-labelled hard pages: 250 pages, one labeller, chosen where page models disagreed or flagged something, plus random audits.

	AUROC, no text vs text
Round 1, 78 pages (BHL, Library of Congress and London Medical Officer of Health reports)	0.970 (3 seeds: 0.976 ± 0.005)
Round 2, 172 pages (BHL and Medical Officer of Health reports; books not used in training)	0.935 (3 seeds: 0.923 ± 0.015)

Label	< 0.2	0.2 – 0.8	> 0.8
text	49	4	1
faint text	6	5	2
stamp only	4	3	14
picture, no text	4	1	0
blank	0	0	67
show-through only	2	5	83

LightOnOCR-3 failures: 2,086 pages from 133 books not used in training.

Output	Positives	AUROC	Precision @0.5	Recall @0.5	AUROC, 3 seeds
lo_blank_write	323	0.941	0.64	0.83	0.941 ± 0.003
lo_trunc	141	0.936	0.43	0.72	0.930 ± 0.007
lo_loop	143	0.930	0.44	0.72	0.922 ± 0.007

The loop reference is a simple rule (tail compression, repeated characters). It misses some loops the model flags, so some "false positives" are real failures.

Speed

Decoding the page image usually takes longer than running the model.

Setup	Pages per second
Model only, 1 CPU thread (PyTorch / ONNX Runtime, fp32)	~50 / ~85
Model only, NVIDIA L4 (fp16, batch 64)	~3,000–4,000
JPEG in, decoded on the GPU with NVIDIA DALI (L4 / RTX PRO 6000)	~300 / ~430
JPEG 2000 in, decoded on the CPU (Pillow/OpenJPEG)	~1.5–3 per CPU core

More speed figures and tips for large runs

Setup	Pages per second
Model only, NVIDIA RTX PRO 6000 (fp16, torch.compile)	~40,000
JPEG derivative, 1,024 px, decoded with DALI (A10G)	~2,000
JPEG 2000, decoded on the GPU with nvImageCodec (nvJPEG2000) (L4 / RTX PRO 6000)	~80 / ~120

• JPEG derivatives: for repeated runs over one collection, store small JPEG derivatives once. At 1,024 px (long side) they give the same decisions as full-size input. 512 px is about twice as fast but agrees slightly less.
• GPU preprocessing matches the reference: the same decision on every test page with DALI and JPEG input (largest score difference 0.015), and on 99.6% of pages with nvImageCodec, JPEG 2000 input and an area resize.
• JPEG 2000 on the GPU: nvJPEG2000 (via nvImageCodec or DALI) decoded all 60 BHL files we tested. Before batching, check that each file is complete (a complete codestream ends with FF D9), and decode on the CPU any page the GPU returns empty.
• ONNX: fp32 export gives identical scores. Static int8 quantisation changed the scores a lot in our test, so we do not recommend it.

Limitations

• Domain: mostly 19th–20th-century printed books. Manuscripts, newspapers and modern documents are untested.
• Labels: the hand labels come from one labeller in one pass. Blank vs show-through, and faint text vs blank, are fuzzy boundaries.
• Conventions: a page with only a library stamp, or an inserted "BLANK PAGE" card, counts as no text. Short captions and page numbers count as text: use text-page-filter to remove those.
• Do not use faint. It copies the teacher's "faded" answer, which fires on faded paper and show-through, so it is not a faint-text detector.
• The LightOn outputs only apply to LightOnOCR-3-0.8B (vLLM, temperature 0.2, 4,096 tokens). Other OCR models fail in other ways, so do not use these outputs for them.

Training

• Data: 17,653 training pages from about 3,000 public-domain BHL books. Books with hand-labelled pages are held out.
• Targets: soft answers from a VLM page-question teacher (openjev 4B, MIT, with a LoRA adapter) for no_text, table, dense and faint, plus the real outputs of LightOnOCR-3-0.8B (Apache-2.0) on every training page for the lo_* outputs.
• Recipe: MobileNetV4-conv-small from ImageNet, grey pages squashed to 384 × 384, BCE on soft targets, 8 epochs, 104 s on one A100.
• Teacher provenance: the openjev page-question adapter was trained on DocLayNet, BHL page types and RVL-CDIP pages. RVL-CDIP has unclear licence terms. This model was trained on BHL images and the teacher's answers only.

Other kinds of material

This model was built for printed books. If a more general version (manuscripts, newspapers, modern documents) would be useful to you, feel free to open a discussion in the Community tab. Or, even better, train one yourself! The Training section is the starting point. For a different OCR model, use its outputs on your training pages as the lo_* targets.