RESULTS.md

TrueDoc on olmOCR-bench: run 97

Overall 86.8 (macro-average of the eight sections; 95% bootstrap CI 86.0-87.7), scored on 2026-09-18 with the benchmark's official scorer (olmocr 0.4.27's benchmark.py) at dataset revision 54a96a6fb6a2bd3b297e59869491db4d3625b711, over all 1,403 pages and all 7,019 checks plus the 1,394 baseline checks.

section	score	checks passed
arxiv_math	88.6	2594 / 2927
old_scans_math	86.7	397 / 458
table_tests	89.3	913 / 1022
old_scans	58.4	307 / 526
headers_footers	96.7	735 / 760
multi_column	83.6	739 / 884
long_tiny_text	91.4	404 / 442
baseline	99.9	1392 / 1394

What produced these pages

TrueDoc is a PDF-to-markdown conversion pipeline, not a model. For this run:

• 1,122 pages with a digital text layer were read from the PDF's own text on CPU, with a layout model (docling-layout-heron) for structure. No vision model read those pages.
• 281 pages with no text layer (scans, photographs of paper) were read by Infinity-Parser2-Flash (2.2B, Apache-2.0). The 102 of those that the pipeline's own OCR could not read (its word-likeness gate) were read by Infinity-Parser2-Pro (35B, Apache-2.0) instead.
• 92 picture regions on digital pages (pictures that hold text: a scanned table, a screenshot) were read by olmOCR-2-7B-1025 (Apache-2.0).
• The model readings were recorded on a rented GPU on 17 September 2026 (Infinity-Parser2 through vLLM 0.17.1 with the settings of its authors' olmOCR-bench guide, raw markdown, none of their post-processing) and on 8 September 2026 (olmOCR 2), and replayed from disk during the run. The readings are in the code repository.
• No post-processing keyed on the benchmark's category folder names. Every page was converted the same way.

The pipeline's code is at https://github.com/islandtimer/TrueDoc, tag run-97 (commit 9f9b77b); its README says how to reproduce this run from the committed readings, and docs/BENCHMARKS.md there holds every run before this one.

What is in this repository

• outputs/<section>/<page>_pg1_repeat1.md: the 1,403 page outputs exactly as scored, in the scorer's layout. Put the folder beside the dataset's bench_data/pdfs and run the scorer to re-score in minutes.
• scoring/summary.json, scoring/score_output.txt, scoring/failed_tests.jsonl: the scorer's own output, the bootstrap interval, and every failed check.
• .eval_results/olmocrbench.yaml: the entry the dataset's leaderboard reads.

How to read the number

• It is a self-reported run of the official scorer, not an independent one. Everything needed to check it is here.
• A second scoring of the same pages moves a section by about three checks, so the tenth is not significant.
• A fifth of the benchmark's pages (bench/holdout.txt in the code repository) was never looked at while the pipeline's rules were built; that fifth scores 86.0 in this run, the rest 84.8.
• As of 2026-09-18 this overall would rank second among the entries on the leaderboard by point estimate; the leading entry's 87.6 sits inside this run's interval. A live integration of the served readers (rather than replayed readings) is pending.