# TrueDoc on olmOCR-bench: run 97

**Overall 86.8** (macro-average of the eight sections; 95% bootstrap CI 86.0-87.7), scored on
2026-09-18 with the benchmark's official scorer (`olmocr` 0.4.27's `benchmark.py`) at dataset revision
`54a96a6fb6a2bd3b297e59869491db4d3625b711`, over all 1,403 pages and all 7,019 checks plus the 1,394 baseline checks.

| section | score | checks passed |
|---|---|---|
| arxiv_math | 88.6 | 2594 / 2927 |
| old_scans_math | 86.7 | 397 / 458 |
| table_tests | 89.3 | 913 / 1022 |
| old_scans | 58.4 | 307 / 526 |
| headers_footers | 96.7 | 735 / 760 |
| multi_column | 83.6 | 739 / 884 |
| long_tiny_text | 91.4 | 404 / 442 |
| baseline | 99.9 | 1392 / 1394 |

## What produced these pages

TrueDoc is a PDF-to-markdown conversion pipeline, not a model. For this run:

- **1,122 pages with a digital text layer** were read from the PDF's own text on CPU, with a layout model
  (docling-layout-heron) for structure. No vision model read those pages.
- **281 pages with no text layer** (scans, photographs of paper) were read by **Infinity-Parser2-Flash**
  (2.2B, Apache-2.0). The **102** of those that the pipeline's own OCR could not read (its word-likeness gate)
  were read by **Infinity-Parser2-Pro** (35B, Apache-2.0) instead.
- **92 picture regions on digital pages** (pictures that hold text: a scanned table, a screenshot) were read by
  **olmOCR-2-7B-1025** (Apache-2.0).
- The model readings were recorded on a rented GPU on 17 September 2026 (Infinity-Parser2 through vLLM 0.17.1 with
  the settings of its authors' olmOCR-bench guide, raw markdown, none of their post-processing) and on 8 September
  2026 (olmOCR 2), and replayed from disk during the run. The readings are in the code repository.
- No post-processing keyed on the benchmark's category folder names. Every page was converted the same way.

The pipeline's code is at https://github.com/islandtimer/TrueDoc, tag `run-97` (commit `9f9b77b`); its README says how to reproduce this run
from the committed readings, and `docs/BENCHMARKS.md` there holds every run before this one.

## What is in this repository

- `outputs/<section>/<page>_pg1_repeat1.md`: the 1,403 page outputs exactly as scored, in the scorer's layout.
  Put the folder beside the dataset's `bench_data/pdfs` and run the scorer to re-score in minutes.
- `scoring/summary.json`, `scoring/score_output.txt`, `scoring/failed_tests.jsonl`: the scorer's own output, the
  bootstrap interval, and every failed check.
- `.eval_results/olmocrbench.yaml`: the entry the dataset's leaderboard reads.

## How to read the number

- It is a self-reported run of the official scorer, not an independent one. Everything needed to check it is
  here.
- A second scoring of the same pages moves a section by about three checks, so the tenth is not significant.
- A fifth of the benchmark's pages (`bench/holdout.txt` in the code repository) was never looked at while the
  pipeline's rules were built; that fifth scores 86.0 in this run, the rest 84.8.
- As of 2026-09-18 this overall would rank second among the entries on the leaderboard by point estimate; the leading
  entry's 87.6 sits inside this run's interval. A live integration of the served readers (rather than replayed
  readings) is pending.