Download ocr-tech-mps-integration/README.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 32.3 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/README.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech-mps-integration/README.md
-
curl -L -o README.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/README.md
ocr-tech
Hybrid Nepali + English OCR and document-structure extraction toolkit.
This repository is a runnable foundation for the document-understanding plan:
- local OCR/structure pipeline first
- optional PaddleOCR/Tesseract/remote model adapters
- stable CLI commands for data prep, training recipes, parsing, and benchmarks
- explicit output schema for Markdown, JSON, tables, and figures
It does not claim that the fine-tuned recognizer is already trained. Training commands generate checked manifests, PaddleOCR label files, dictionaries, and recipe configs so long jobs can be run on a suitable host.
The active research steering checklist is tracked in docs/ACTIVE_RESEARCH_CHECKLIST.md. It records the cdjk remote synthesis leads, Gorkhapatra validation path, and paper claim gates.
Install
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -e '.[dev]'
Optional integrations:
python -m pip install -e '.[eval]'
python -m pip install -e '.[datasets,engines]'
python -m pip install -e '.[hf-recognizer]'
The eval extra installs Pillow and PyMuPDF for image/PDF evaluation artifacts,
including Gorkhapatra epaper page rendering.
The hf-recognizer extra is the real dependency set for TrOCR-style training and inference here. It includes torch, transformers, accelerate, sentencepiece, protobuf, and pillow.
CLI
ocrtech prepare-data labels.csv --out data/manifests --image-field image_path --text-field text
ocrtech prepare-data labels.csv --out data/manifests --image-field image_path --text-field text --skip-invalid
ocrtech prepare-data gauravgiri/nepali-ocr-dataset --hf --split train --limit 100 --out data/manifests
ocrtech prepare-data prashant0919/nepali-synthetic-ocr-lines --hf --split train --limit 100 --out data/manifests
ocrtech prepare-data nvidia/OCR-Synthetic-Multilingual-v1 --hf --hf-subset en --split test --limit 1000 --out data/manifests-english-test
ocrtech merge-manifests data/manifests/gauravgiri__nepali-ocr-dataset-train.jsonl data/manifests/prashant0919__nepali-synthetic-ocr-lines-train.jsonl --out data/manifests/nepali-mixed-all.jsonl
ocrtech audit-manifest data/manifests/labels-train.jsonl --require-slice nepali --require-slice english
ocrtech split-manifest data/manifests/labels-train.jsonl --out data/splits --eval-ratio 0.15
ocrtech rebalance-manifest data/splits/train.jsonl --out data/splits/train-rebalanced.jsonl --slice english --slice nepali
ocrtech audit-recognizer-corpus --train-manifest data/splits/train-rebalanced.jsonl --eval-manifest data/splits/eval.jsonl --out outputs/recognizer-corpus-audit
ocrtech build-recognizer-corpus data/manifests/english-lines.jsonl data/manifests/nepali-lines.jsonl data/manifests/mixed-crops.jsonl --out data/splits/recognizer-train-balanced.jsonl --target-latin-only 200 --target-devanagari-only 200 --target-mixed 100 --target-real 100 --target-synthetic 100
ocrtech materialize-references data/splits/eval.jsonl --out data/references --rewritten-manifest data/splits/eval-with-refs.jsonl
ocrtech create-eval-pack --out data/eval-pack --input-format image --degradation clean --degradation scan
ocrtech create-eval-pack --out data/eval-pack-table-figure-dev-v1 --input-format image --template table-ledger --template figure-caption --count-per-template 8 --variant-offset 100 --degradation clean --degradation scan --degradation blur --degradation low_contrast --degradation skew
ocrtech audit-eval-pack data/eval-pack/manifests/ocrtech-eval-pack.jsonl --require-claim-ready
ocrtech train-recognizer --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --out runs/recognizer
ocrtech train-recognizer --backend hf-vision-encoder-decoder --base-model microsoft/trocr-small-printed --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --out runs/hf-recognizer
ocrtech bundle-recognizer-training --recipe-dir runs/recognizer --train-manifest data/splits/train-rebalanced.jsonl --eval-manifest data/splits/eval.jsonl --out runs/recognizer-training-bundle --require-corpus-audit
ocrtech export-recognizer --training-config runs/recognizer/paddle_recognizer.yml --checkpoint runs/recognizer/checkpoints/best_accuracy --out runs/recognizer/export
ocrtech package-recognizer --inference-dir runs/recognizer/export/inference --dictionary runs/recognizer/nep_eng_dict.txt --out runs/recognizer/package --model-id ocrtech-nep-eng-rec-v1 --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --training-summary runs/recognizer/recognizer-training-summary.json --admission-validation-report outputs/validation-next-recognizer-hard50/validation.json
ocrtech package-hf-recognizer --model-dir runs/hf-recognizer/final-model --out runs/hf-recognizer/package --model-id ocrtech-hf-nep-eng-rec-v1 --base-model microsoft/trocr-small-printed --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --training-summary runs/hf-recognizer/recognizer-training-summary.json --admission-validation-report outputs/validation-next-recognizer-hard50/validation.json
ocrtech train-corrector --clean-manifest data/splits/train.jsonl --out runs/corrector
ocrtech train-corrector --hf-dataset cfilt/RoundTripOCR-nepali --hf-split train --limit 1000 --out runs/corrector-cfilt
ocrtech audit-model runs/recognizer/package/model-card.json
ocrtech audit-remote-host --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --workdir '~/ocr-tech' --require-path '~/ocr-tech' --min-free-gb 20 --require-paddle-training --out outputs/remote-audit
ocrtech bootstrap-remote-host --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --workdir '~/ocr-tech' --out outputs/remote-bootstrap
ocrtech sync-remote-workspace --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --local-root . --remote-workdir '~/ocr-tech' --exclude .venv --exclude outputs --exclude runs --out outputs/remote-sync
ocrtech preflight-claim --eval-manifest data/eval-pack/manifests/ocrtech-eval-pack.jsonl --references-dir data/eval-pack/references --baselines candidate,tesseract,stock-paddle --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json --train-manifest data/splits/train.jsonl --out outputs/preflight
ocrtech parse-doc examples/sample-page.txt --out outputs/sample --engine sidecar
ocrtech parse-doc page.png --out outputs/page --engine candidate --model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
ocrtech benchmark examples/sample-page.txt --out outputs/bench --baselines ours --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
ocrtech summarize-benchmark --benchmark-report outputs/bench/report.json --out outputs/bench-summary --candidate ours --baseline tesseract --metric cer
ocrtech benchmark-error-analysis --benchmark-report outputs/bench/report.json --out outputs/bench-error-analysis --system surya --baseline tesseract --metric cer
ocrtech export-failure-manifest --source-manifest data/splits/eval.jsonl --top-failures outputs/bench-error-analysis/top-failures.json --out data/splits/eval-hard.jsonl --split hard_eval
ocrtech recognizer-admission-decision --validation-report outputs/validation-next-recognizer-hard50/validation.json --out outputs/recognizer-admission-next
ocrtech paper-benchmark-table --summary outputs/paper-claimpack-summary-tesseract/summary.json --summary outputs/paper-claimpack-summary-surya/summary.json --summary outputs/paper-claimpack-summary-stock-paddle/summary.json --out outputs/paper-claimpack-table --system candidate --system surya --system tesseract --system stock-paddle --label 'candidate=Candidate' --label 'surya=Surya' --label 'tesseract=Tesseract' --label 'stock-paddle=Stock Paddle' --source-preference 'candidate=outputs/paper-claimpack-summary-tesseract/summary.json' --source-preference 'tesseract=outputs/paper-claimpack-summary-tesseract/summary.json' --source-preference 'surya=outputs/paper-claimpack-summary-surya/summary.json' --source-preference 'stock-paddle=outputs/paper-claimpack-summary-stock-paddle/summary.json' --metric cer --metric table_cell_f1 --metric reading_order_pair_accuracy --metric latency_seconds
ocrtech calibrate-quality-router --experiment-report outputs/calibration-experiment/experiment.json --out outputs/quality-router-calibration --model-id ocrtech-tesseract-quality-router --primary-engine tesseract --secondary-engine tesseract --primary-engine-kwarg 'language=nep+eng' --primary-engine-kwarg 'psm=6' --secondary-engine-kwarg 'language=nep+eng' --secondary-engine-kwarg 'psm=13'
ocrtech calibrate-script-router --experiment-report outputs/script-calibration/experiment.json --out outputs/script-router-calibration --model-id ocrtech-tesseract-surya-script-router --primary-engine tesseract --secondary-engine surya --primary-engine-kwarg 'language=nep+eng' --primary-engine-kwarg 'psm=6' --secondary-engine-kwarg 'mode=auto'
ocrtech calibrate-tesseract-psm-ensemble --primary-experiment-report outputs/psm13-calibration/experiment.json --variant-experiment psm7=outputs/psm7-calibration/experiment.json --variant-experiment psm11=outputs/psm11-calibration/experiment.json --variant-experiment psm13=outputs/psm13-calibration/experiment.json --out outputs/psm-ensemble-calibration --model-id ocrtech-tesseract-psm-ensemble
ocrtech run-experiment --eval-manifest data/eval-pack/manifests/ocrtech-eval-pack.jsonl --out outputs/experiment --baselines candidate,tesseract,stock-paddle --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
ocrtech review-claim --experiment outputs/internal-claim/experiment.json --experiment outputs/ocrbench-v2/experiment.json --config claim-review.json --model-config runs/recognizer/package/model-card.json --out outputs/claim-review
ocrtech validate-claim --benchmark-report outputs/bench/report.json --config validation.json --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json --out outputs/validation
parse-doc writes:
document.jsondocument.mdtables/*.csvtables/*.htmlfigures/*.pngfigures/metadata.jsonquality.json
Figure exports are explicit. If an engine supplies figure.image_path,
parse-doc copies that file into figures/. If an engine supplies only a
figure bbox and the source is a regular image or page-bundle directory,
parse-doc crops the region into figures/{figure_id}.png when Pillow is
available. For bundles, it uses figure.page_index to select the page image. If
the artifact cannot be created, figures/metadata.json records
metadata.export_warnings instead of silently dropping the figure image.
When --fallback-engine or --fallback-model-config is configured,
document.json records metadata.fallback with the primary engine, fallback
engine/model card, confidence threshold, trigger reason, and success/error
outcome. Benchmarks surface this as diagnostic metrics such as
fallback_triggered, fallback_succeeded, and fallback_failed.
The same fallback flags are available on benchmark and run-experiment for
candidate/ours runs, and experiment metadata records the fallback model-card
path, SHA-256, and confidence threshold.
Engines
auto tries deterministic sidecar OCR first, then PaddleOCR, then Tesseract. Optional engines fail loudly when binaries or packages are missing.
Sidecar inputs are useful for tests and staged pipelines. For page.png, provide one of:
page.png.ocr.jsonpage.ocr.jsonpage.png.txtpage.txt
Data
Use prepare-data to verify actual fields and image paths before training. By default it fails on the first bad row. Use --skip-invalid when the upstream dataset is messy and you want a usable manifest plus an explicit reject log instead of a silent partial corpus.
Use merge-manifests to assemble multi-source corpora without ad hoc cat pipelines, then audit-manifest before training and split-manifest to create deterministic leakage-safe train/eval manifests. The split groups exact sample signatures before assignment so duplicate image+text examples cannot land on both sides.
Known Hugging Face adapters now stamp verified slice metadata automatically:
gauravgiri/nepali-ocr-dataset:nepali,realprashant0919/nepali-synthetic-ocr-lines:nepali,syntheticnvidia/OCR-Synthetic-Multilingual-v1 --hf-subset en:english,synthetic,line_crop,en
Use rebalance-manifest only on the training split after split-manifest. Rebalancing before the split would turn deliberate duplication into leakage.
Use audit-recognizer-corpus before launching a recognizer training job. It
checks the actual train/eval manifests for sample counts, Latin-script coverage,
Devanagari coverage, mixed-script examples, real/synthetic training coverage,
and train/eval leakage. This guardrail was added after the failed PaddleOCR
mixed v4 branch: a new GPU run should not start unless English preservation and
Nepali coverage are explicit in the corpus.
Use build-recognizer-corpus to assemble and upsample the training manifest by
actual text buckets, not by wishful dataset labels. It tracks latin_only,
devanagari_only, mixed_devanagari_latin, real, and synthetic coverage,
then writes a build JSON/Markdown summary next to the output manifest. It does
not invent missing categories: if a required bucket is absent from the input
manifests, the command fails so you know to add real source data before using
GPU time.
train-recognizer --backend paddleocr writes
recognizer-training-summary.json alongside the generated PaddleOCR config.
Keep that file with any remote/GPU training bundle and pass it to
package-recognizer --training-summary; it records manifest hashes, slice
counts, character coverage, label/config hashes, and launch hyperparameters.
bundle-recognizer-training records the bundled summary path and SHA-256 under
training_summary, and packaged recognizer model cards record the same
provenance when supplied. For real remote/GPU runs, use
bundle-recognizer-training --require-corpus-audit; it runs the recognizer
corpus audit and refuses to create the bundle if the corpus does not meet the
configured English, Devanagari, mixed-script, real/synthetic, and leakage gates.
The gate also checks isolated Latin-only and Devanagari-only samples; a corpus
made entirely of mixed-script rows is not enough to protect English preservation
or Nepali-only recognition.
Planned source datasets:
gauravgiri/nepali-ocr-datasetprashant0919/nepali-synthetic-ocr-linesnvidia/OCR-Synthetic-Multilingual-v1cfilt/RoundTripOCR-nepali
The verified Hugging Face adapters are intentionally dataset-specific where the upstream schema is non-standard:
gauravgiri/nepali-ocr-dataset: genericimage+textprashant0919/nepali-synthetic-ocr-lines:metadata.jsonlplus per-line image files, downloaded lazily for the requested rowscfilt/RoundTripOCR-nepali: correction pairs viaocrandcorrectnvidia/OCR-Synthetic-Multilingual-v1: HDF5 shards withannotations.line_bboxes; use--hf-subset enfor English crops
Some upstream corpora are dirty. For example, the current gauravgiri/nepali-ocr-dataset sample prefix includes null text rows, so use --skip-invalid when you want a prepared manifest plus an explicit reject log instead of a hard stop.
Baselines
The benchmark command supports:
ourstesseractstock-paddleglm-ocrpaddleocr-vl
glm-ocr and paddleocr-vl are external command hooks:
export OCRTECH_GLM_OCR_CMD='python scripts/run_glm_ocr.py --input {input} --out {out}'
export OCRTECH_PADDLEOCR_VL_CMD='python scripts/run_paddleocr_vl.py --input {input} --out {out}'
The command must include both {input} and {out} placeholders. It may print Markdown to stdout or write {out}/document.md; ocrtech now normalizes that Markdown into its own document.json schema so external VLM baselines can still be scored for text, table cells, and reading order.
The bundled wrappers are thin adapters, not magic:
scripts/run_glm_ocr.pyuses the officialtransformerspath from thezai-org/GLM-OCRmodel card.scripts/run_paddleocr_vl.pyuses the officialPaddleOCRVL(...).predict(...)path from thePaddlePaddle/PaddleOCR-VL-1.6model card.- GLM-OCR still expects a git
transformersbuild upstream. Onk2, the releasedtransformers 4.57.6package was not enough to instantiate its processor classes; a dedicated Python 3.11 env with gittransformersandtorchvisionwas required before the wrapper would run. - PaddleOCR-VL still needs the OCR extras that the official docs call out, for example
pip install "paddlex[ocr]==3.7.1".
Validation Before SOTA Claims
Use validate-claim before making any quality claim. The default validator is intentionally strict and will fail unless the benchmark report has held-out sample coverage, required slices, leakage checks, paired candidate-vs-baseline gates, and per-gate evidence such as paired wins and optional significance thresholds.
Use recognizer-admission-decision after a recognizer admission validation run.
It writes a small JSON/Markdown decision artifact: admit only for
claim_status=validated with passed=true, otherwise reject with failed
gate metrics and next actions for the next model branch.
Use create-eval-pack for internal mixed Nepali/English document tests and audit-eval-pack to check whether the pack is claim-ready. The internal pack now includes a real multi_page slice in addition to the single-page paragraph, table, form, and figure templates. Text packs are for pipeline smoke tests only; image packs are required for OCR claims.
Use preflight-claim before run-experiment on any claim-grade run. It fails early on missing references, invalid model cards, missing baseline dependencies, non-claim-ready eval packs, bad validation configs, and train/eval leakage.
For routed or thresholded candidates such as quality_select_composite and script_select_composite, claim-grade runs also require explicit calibration provenance in the model card. If the candidate was calibrated on the same eval manifest you are about to benchmark, preflight-claim now fails on purpose.
For those routed candidates, claim-grade runs also need the raw component baselines in the same benchmark. If a script_select_composite candidate is built from Tesseract and Surya, the claim run must include candidate,tesseract,surya; otherwise the repo now fails preflight because distinctness cannot be checked honestly.
Use run-experiment to benchmark every manifest input, write reproducibility metadata, and run claim validation in one output directory.
Use review-claim after the internal pack and public benchmark suites have all been run. It audits the portfolio-level evidence bundle so one green experiment does not get misrepresented as a broad SOTA claim.
Use a candidate model card for any real benchmark of ours; see docs/MODELS.md.
Use filter-manifest when a baseline is too expensive to run on the full pack and you need a reproducible exploratory subset instead of hand-editing a JSONL.
See docs/PREFLIGHT.md, docs/CLAIM_REVIEW.md, docs/EVAL_PACK.md, docs/MODELS.md, docs/EXPERIMENTS.md, and docs/VALIDATION.md.
Remote Training
Use docs/REMOTE_TRAINING.md for the current k2 host notes, audit-remote-host, and the safe SSH workflow. Do not commit SSH passwords or machine-specific secrets.
Current HF Result
The strongest verified TrOCR candidate so far is the rebalanced mixed-script run trained on k2 and evaluated on a held-out 180-sample mixed Nepali/English line set.
That run is materially better than our own earlier HF candidates:
- overall benchmark CER:
1.628157->0.586139 - English CER:
0.379729->0.300454 - Nepali CER:
7.870295->2.014560 - real-slice CER:
12.702701->1.770016
But the first named baseline comparison is still bad news for the project claim: on that same eval split, tesseract is better than the rebalanced candidate on overall, English, Nepali, real, and synthetic OCR metrics.
The next attempted rescue path also failed. On June 14, 2026, a ByT5 text-correction model trained on k2 from mined Tesseract noisy/clean pairs was packaged as a text_correction_composite candidate with raw Tesseract as the base engine. On the same held-out 180-sample eval, that composite regressed catastrophically: overall CER 6.912065 versus raw Tesseract 0.329012, with paired CER 0 wins, 170 losses, and 10 ties. That rules out the current Tesseract-plus-corrector path as a serious answer.
There is one narrower improvement worth keeping, but it is not a claim-grade win. A routed Tesseract composite now exists as quality_select_composite: it runs default Tesseract (psm=6) and a secondary variant (psm=13), scores both outputs with the local quality_score, and switches only when the secondary beats the primary by a configured margin.
The same-pack tuned run looked slightly better on the original 180-sample mixed eval, but that was not honest enough. After splitting that pack into a separate calibration half (89 samples) and untouched holdout half (91 samples), the calibrated routed candidate still improved overall CER on the holdout, just by much less:
- raw tesseract holdout overall CER:
0.342234 - routed tesseract holdout overall CER:
0.340608 - paired CER on holdout:
2wins,1loss,88ties
That makes it a plausible engineering baseline, not a frontier result. The effect size is tiny, the pack still is not claim-ready, and the real Devanagari recognizer problem remains open.
We also tested a richer calibrated Tesseract ensemble over psm=6/7/11/13 with learned quality-score biases. That looked better on the calibration half, but it failed on the untouched holdout: overall CER regressed from 0.342234 to 0.358352. That is exactly the kind of result the new calibration workflow is supposed to expose.
The more useful June 14, 2026 result is a calibrated Tesseract-plus-Surya script router on k2. Raw Surya was not competitive overall on the 91-sample holdout (2.198362 CER vs raw Tesseract 0.342234), but it was materially better on the Nepali slice. A routed script_select_composite candidate calibrated on the separate 89-sample half and evaluated on the untouched 91-sample holdout did better than raw Tesseract across every reported slice:
- raw tesseract holdout overall CER:
0.342234 - calibrated script router overall CER:
0.313084 - paired CER on holdout:
9wins,0losses,82ties
That is the strongest verified baseline improvement in the repo so far. It is still not a frontier claim, because it is one internal line-level split and not the full claim bundle, but it is a real out-of-sample gain and a better direction than the current TrOCR branch. The router now also supports routing_granularity=page for mixed multi-page documents; the calibrated k2 result above is still the original document-level calibration, not a new page-level benchmark.
We also ran an exploratory page-level script router on the same 91-sample holdout, reusing the document-level threshold 0.4. The result was identical to the document-level router (0.313084 overall CER, 9 wins, 0 losses, 82 ties vs raw Tesseract). That is expected for this pack because it is dominated by single-page line-crop samples; page routing needs a real multi-page document pack before it can prove anything new.
The internal image eval pack now has a real rendered multi_page slice with bundled page images and actual drawn table grids. On that small rendered smoke pack, the plain script router already beat Tesseract strongly on text (0.243224 vs 0.540584 overall CER) and on the dedicated table page (table_cell_f1 1.0 vs 0.0). A follow-up exploratory model with secondary_structure_backfill=true kept the same CER and backfilled Surya table structure on the multi-page report, lifting aggregate table_cell_f1 from 0.5 to 1.0 on that smoke pack. That is still not claim-grade evidence because the pack is tiny and only clean, but it is a real document-capability improvement.
The next step was to stop calling it smoke evidence and run the same backfill router on a claim-ready rendered image pack. That broader internal pack contains 25 samples across clean, scan, blur, low_contrast, and skew, and its audit passed cleanly. The calibrated backfill router also passed preflight-claim on that pack. Against raw Tesseract on the same rendered claim pack it still held up on text and structure:
- overall CER:
0.248654vs0.689214 - blur CER:
0.260039vs0.991613 - scan CER:
0.237424vs0.593631 - skew CER:
0.239317vs0.766067 - aggregate
table_cell_f1:0.955556vs0.0 - aggregate
table_exact_match:0.8vs0.0 - paired CER:
9wins,1loss,15ties
That result is good enough to keep and bad enough to describe plainly. The multi-page slice tied Tesseract on CER (0.315789 each) while still recovering table structure that Tesseract missed (table_cell_f1 0.911111 vs 0.0). The form slice regressed slightly on CER (0.231933 vs 0.230010). Aggregate reading-order pair accuracy on this rendered pack is also still worse than Tesseract (0.764 vs 0.816). So the current candidate is meaningfully better on OCR robustness and table extraction for this internal rendered pack, but it is not yet a clean document-understanding leader.
We then widened the comparison beyond Tesseract. Two more June 14, 2026 checks on k2 matter:
- raw Surya on the same
25-sample rendered claim pack is better on text CER than the calibrated backfill router:0.195260vs0.248654 - but the router is still not pointless: it ties Surya on aggregate
table_cell_f1(0.955556each) and is slightly better on aggregatereading_order_pair_accuracy(0.764000vs0.750909)
That means the current routed candidate is still not “better than what is out there” on this rendered document pack. Raw Surya is the stronger text OCR baseline here. The router is only defending a narrower position: it preserves comparable structure while staying closer to the existing Tesseract-centered stack.
We also fixed a real integration bug in the local Paddle baseline. paddleocr 3.7.0 on k2 returns v3 OCRResult objects, and the old adapter in this repo treated them like the older tuple/list API and silently dropped their text. After fixing that parser and installing paddlepaddle 3.3.1, stock-paddle became a real benchmark target on k2. On this same rendered pack it is still a weak baseline overall:
- stock-Paddle overall CER:
1.749081 - stock-Paddle aggregate
table_cell_f1:0.0 - stock-Paddle aggregate
reading_order_pair_accuracy:1.0
So the current picture is sharper now, not prettier: the routed candidate beats Tesseract and this stock-Paddle configuration on OCR robustness and table extraction, but it still loses to raw Surya on text CER.
We then used the new filter-manifest helper to run a broader but still tractable rendered-document comparison on k2: a 5-sample blur-only subset spanning paragraph, table, form, figure, and multi-page layouts. All 30 runs completed without error across candidate, tesseract, surya, stock-paddle, glm-ocr, and paddleocr-vl. The result sharpened the ranking again:
- candidate CER:
0.223273 - raw Surya CER:
0.197601 - Tesseract CER:
0.732364 - stock-Paddle CER:
2.810426 - GLM-OCR CER:
2.820269 - PaddleOCR-VL CER:
18.470663
On that subset, the current backfill candidate still held the structure line where Tesseract and the weaker VLM baselines collapsed (table_cell_f1 = 1.0 for the candidate versus 0.0 for Tesseract, stock-Paddle, GLM-OCR, and PaddleOCR-VL). But it still did not beat raw Surya on text CER. That keeps the same honest conclusion intact: on rendered document OCR text, the strongest baseline in this repo is still Surya, not the current routed candidate.
The stricter rendered-document result is now explicit too. On k2, a larger rendered claimpack-rendered-v2 split was calibrated separately and evaluated on an untouched 50-sample holdout. That calibration selected primary_script_threshold=0.0, so the routed script_select_composite collapsed to "always pick Surya" on the calibration set. On the holdout:
- candidate CER:
0.180893 - raw Surya CER:
0.180893 - candidate
table_cell_f1:0.947500 - raw Surya
table_cell_f1:0.970833
That is the ceiling of the current branch, not a win. The candidate no longer even has a text-vs-structure tradeoff story there; on this larger rendered holdout it effectively degenerates to Surya on text and still loses slightly on table structure. The claim guardrails now reject this case directly: routed candidates that collapse to one engine across all calibration samples fail candidate-calibration in both preflight-claim and review-claim.
We also tested whether a quality-based Surya/Tesseract selector could recover anything on that same rendered-v2 setup. It did not. The rendered-v2 Surya quality router calibrated to selected_threshold=0.0, then landed at:
- candidate CER:
0.181128 - raw Surya CER:
0.180893 - candidate
table_cell_f1:0.947500 - raw Surya
table_cell_f1:0.947500 - paired CER vs Surya:
0wins,1loss,49ties
That is not a distinct better engine. It is basically Surya with one extra mistake.
We then tested line_align_composite on the same rendered-v2 holdout: keep Surya text, align it to Tesseract's line scaffold, and see whether the ordering gain is worth the merge. It was not:
- candidate CER:
0.310549 - raw Surya CER:
0.180893 - candidate
table_cell_f1:0.897500 - raw Surya
table_cell_f1:0.970833 - candidate
reading_order_pair_accuracy:0.792727 - raw Surya
reading_order_pair_accuracy:0.750909 - paired CER vs Surya:
0wins,7losses,43ties
That is another useful paper result, but not a winning model branch. Better line ordering alone did not rescue the Surya/Tesseract composite family.
The upper bound is now measured too. A sample-level oracle that chooses the better of raw Surya and raw Tesseract per holdout sample using ground-truth CER landed at:
- oracle CER:
0.180529 - raw Surya CER:
0.180893 - raw Tesseract CER:
0.714207 - oracle choices: Surya
49, Tesseract1 - paired CER vs Surya:
1win,0losses,49ties
That closes the routing branch for text OCR. There is almost no remaining sample-level headroom in the Surya/Tesseract component family on rendered-v2. The next credible model-side step has to use a genuinely stronger recognizer or document model, not another selector over these same two engines.
See docs/EXPERIMENTS.md for the exact commands, artifact paths, and the current evidence gap.
If you are turning this work into a paper, start from docs/PAPER_NOTES.md. Then use docs/PAPER_RESULTS.md for the current generated comparison tables and branch chronology tied to local artifacts.