prosperious commited on
Commit
d82db76
·
verified ·
1 Parent(s): f9bc0c1

Add 1000 files (ocr-tech/tests/test_audit_limbu_book_missed_line_review_coverage.py ...)

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. ocr-tech-mps-integration/.gitignore +19 -0
  2. ocr-tech-mps-integration/.ignore +9 -0
  3. ocr-tech-mps-integration/CLAUDE.md +38 -0
  4. ocr-tech-mps-integration/README.md +371 -0
  5. ocr-tech-mps-integration/RESEARCH.md +375 -0
  6. ocr-tech-mps-integration/configs/cdjk-language-corpus-migration-v1.json +170 -0
  7. ocr-tech-mps-integration/configs/claim-review-sota-v1.json +121 -0
  8. ocr-tech-mps-integration/configs/eval-harness-recipes-v1.json +651 -0
  9. ocr-tech-mps-integration/configs/eval-target-coverage-v1.json +193 -0
  10. ocr-tech-mps-integration/configs/limbu-devanagari-ppocrv5-admission-plan-v1.json +68 -0
  11. ocr-tech-mps-integration/configs/limbu-first-ocr-pipeline-v1.json +0 -0
  12. ocr-tech-mps-integration/configs/limbu-sirijonga-augmentation-ablation-v1.json +68 -0
  13. ocr-tech-mps-integration/configs/namdhinggo-font-assets-v1.json +106 -0
  14. ocr-tech-mps-integration/configs/nepal-font-assets-v1.json +110 -0
  15. ocr-tech-mps-integration/configs/nepal-language-pack-pipeline-v1.json +1051 -0
  16. ocr-tech-mps-integration/configs/nepali-table-name-lexicon-v1.txt +10 -0
  17. ocr-tech-mps-integration/configs/recognizer-eval-pack-normalization-v1.json +165 -0
  18. ocr-tech-mps-integration/configs/validation-complex-table-surya.json +57 -0
  19. ocr-tech-mps-integration/configs/validation-hard15-surya.json +59 -0
  20. ocr-tech-mps-integration/configs/validation-hf-recognizer-next-v1-admission.json +79 -0
  21. ocr-tech-mps-integration/configs/validation-internal-sota-v1.json +120 -0
  22. ocr-tech-mps-integration/configs/validation-paddle-hard50-tesseract.json +52 -0
  23. ocr-tech-mps-integration/configs/validation-public-omnidocbench-sota-v1.json +60 -0
  24. ocr-tech-mps-integration/configs/validation-public-omnidocbench-surya.json +46 -0
  25. ocr-tech-mps-integration/configs/validation-public-real5-sota-v1.json +67 -0
  26. ocr-tech-mps-integration/configs/validation-table-ledger-surya.json +39 -0
  27. ocr-tech-mps-integration/data/dictionaries/devanagari-v1.txt +141 -0
  28. ocr-tech-mps-integration/data/dictionaries/gurungkhema-v1.txt +58 -0
  29. ocr-tech-mps-integration/data/dictionaries/kiratrai-v1.txt +58 -0
  30. ocr-tech-mps-integration/data/dictionaries/lepcha-v1.txt +74 -0
  31. ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v1.txt +76 -0
  32. ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v2.txt +91 -0
  33. ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v3.txt +98 -0
  34. ocr-tech-mps-integration/data/dictionaries/newa-clean-v1.txt +103 -0
  35. ocr-tech-mps-integration/data/dictionaries/newa-htr-v1.txt +177 -0
  36. ocr-tech-mps-integration/data/dictionaries/newa-native-v1.txt +163 -0
  37. ocr-tech-mps-integration/data/dictionaries/newa-v1.txt +97 -0
  38. ocr-tech-mps-integration/data/dictionaries/sunuwar-v1.txt +44 -0
  39. ocr-tech-mps-integration/data/dictionaries/tirhuta-v1.txt +82 -0
  40. ocr-tech-mps-integration/data/mappings/limbu-legacy/Limbu.map +146 -0
  41. ocr-tech-mps-integration/data/mappings/limbu-legacy/README.txt +1 -0
  42. ocr-tech-mps-integration/docs/ACTIVE_RESEARCH_CHECKLIST.md +386 -0
  43. ocr-tech-mps-integration/docs/CDJK_LANGUAGE_CORPUS_MIGRATION.md +310 -0
  44. ocr-tech-mps-integration/docs/CLAIM_REVIEW.md +333 -0
  45. ocr-tech-mps-integration/docs/CLAUDE_HANDOFF.md +613 -0
  46. ocr-tech-mps-integration/docs/CODEX_HANDOFF.md +0 -0
  47. ocr-tech-mps-integration/docs/DATASETS.md +381 -0
  48. ocr-tech-mps-integration/docs/EVAL_HARNESS_GOAL.md +437 -0
  49. ocr-tech-mps-integration/docs/EVAL_HARNESS_RESEARCH_GOAL.md +111 -0
  50. ocr-tech-mps-integration/docs/EVAL_HARNESS_RUNBOOK.md +803 -0
ocr-tech-mps-integration/.gitignore ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .venv/
2
+ __pycache__/
3
+ *.py[cod]
4
+ .pytest_cache/
5
+ .ruff_cache/
6
+ .mypy_cache/
7
+
8
+ data/*
9
+ !data/dictionaries/
10
+ !data/dictionaries/*.txt
11
+ outputs/
12
+ runs/
13
+ *.egg-info/
14
+
15
+ .DS_Store
16
+
17
+ spaces/limbu-ocr/models/
18
+ incoming/
19
+ logs/
ocr-tech-mps-integration/.ignore ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ .venv/
2
+ __pycache__/
3
+ *.py[cod]
4
+ .pytest_cache/
5
+
6
+ data/
7
+ outputs/
8
+ runs/
9
+ *.egg-info/
ocr-tech-mps-integration/CLAUDE.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ocr-tech — agent instructions
2
+
3
+ ## Drive documents
4
+
5
+ - `RESEARCH.md` is the research state and drive document. Read it first every
6
+ session; keep it current when results land.
7
+ - End goal: OCR covering the whole Nepal language pack, **Limbu first, Newar
8
+ (Nepal Bhasa) second** (steer 2026-07-04).
9
+
10
+ ## Model delegation policy (steer 2026-07-04)
11
+
12
+ - **Fable (lead session model)**: orchestration, the hardest reasoning, design
13
+ decisions, and final verification of all delegated work. Best model available —
14
+ do not delegate judgment calls.
15
+ - **Codex**: treat as a peer, not a subordinate. It has good independent ways of
16
+ figuring things out. Hand it substantial, well-scoped build/diagnosis/research
17
+ tasks (new tools, refactors, root-cause hunts) with verified file paths and
18
+ acceptance criteria; always verify its output independently (run its tests,
19
+ spot-check its data).
20
+ - **Opus**: near-par with the lead model but slightly weaker; usable for hard
21
+ tasks and second opinions when a peer-level check is wanted.
22
+ - **Sonnet**: tasks that require no big thought — mechanical edits, bulk file
23
+ wrangling, running known command recipes, simple searches and summaries.
24
+
25
+ ## Working rules
26
+
27
+ - Never assume file paths, schemas, or endpoint behavior — verify on disk / live
28
+ before coding against them (this repo's history has multiple wrong-assumption
29
+ scars; see RESEARCH.md "Mistakes To Avoid").
30
+ - Real CER comes from `tools/eval_metrics.py` (micro CER, codepoint + grapheme).
31
+ Never report Paddle `norm_edit_dis` as CER.
32
+ - Silver labels are machine-converted; they are training/diagnostic substrate,
33
+ not claim-grade ground truth. Claim-grade numbers need frozen, human-reviewed
34
+ eval packs.
35
+ - Long downloads/runs: launch detached (`nohup`), not under tool timeouts; watch
36
+ with polling loops. `ocrtech audit-gorkhapatra-language-pages` has no resume.
37
+ - Python runtime for Paddle work is 3.11 (`uv run --python 3.11`); silver tooling
38
+ needs `--with pymupdf --with npttf2utf`.
ocr-tech-mps-integration/README.md ADDED
@@ -0,0 +1,371 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ocr-tech
2
+
3
+ Hybrid Nepali + English OCR and document-structure extraction toolkit.
4
+
5
+ This repository is a runnable foundation for the document-understanding plan:
6
+
7
+ - local OCR/structure pipeline first
8
+ - optional PaddleOCR/Tesseract/remote model adapters
9
+ - stable CLI commands for data prep, training recipes, parsing, and benchmarks
10
+ - explicit output schema for Markdown, JSON, tables, and figures
11
+
12
+ It does not claim that the fine-tuned recognizer is already trained. Training commands generate checked manifests, PaddleOCR label files, dictionaries, and recipe configs so long jobs can be run on a suitable host.
13
+
14
+ The active research steering checklist is tracked in
15
+ [docs/ACTIVE_RESEARCH_CHECKLIST.md](docs/ACTIVE_RESEARCH_CHECKLIST.md). It
16
+ records the cdjk remote synthesis leads, Gorkhapatra validation path, and paper
17
+ claim gates.
18
+
19
+ ## Install
20
+
21
+ ```bash
22
+ python3 -m venv .venv
23
+ . .venv/bin/activate
24
+ python -m pip install -e '.[dev]'
25
+ ```
26
+
27
+ Optional integrations:
28
+
29
+ ```bash
30
+ python -m pip install -e '.[eval]'
31
+ python -m pip install -e '.[datasets,engines]'
32
+ python -m pip install -e '.[hf-recognizer]'
33
+ ```
34
+
35
+ The `eval` extra installs Pillow and PyMuPDF for image/PDF evaluation artifacts,
36
+ including Gorkhapatra epaper page rendering.
37
+
38
+ The `hf-recognizer` extra is the real dependency set for TrOCR-style training and inference here. It includes `torch`, `transformers`, `accelerate`, `sentencepiece`, `protobuf`, and `pillow`.
39
+
40
+ ## CLI
41
+
42
+ ```bash
43
+ ocrtech prepare-data labels.csv --out data/manifests --image-field image_path --text-field text
44
+ ocrtech prepare-data labels.csv --out data/manifests --image-field image_path --text-field text --skip-invalid
45
+ ocrtech prepare-data gauravgiri/nepali-ocr-dataset --hf --split train --limit 100 --out data/manifests
46
+ ocrtech prepare-data prashant0919/nepali-synthetic-ocr-lines --hf --split train --limit 100 --out data/manifests
47
+ ocrtech prepare-data nvidia/OCR-Synthetic-Multilingual-v1 --hf --hf-subset en --split test --limit 1000 --out data/manifests-english-test
48
+ ocrtech merge-manifests data/manifests/gauravgiri__nepali-ocr-dataset-train.jsonl data/manifests/prashant0919__nepali-synthetic-ocr-lines-train.jsonl --out data/manifests/nepali-mixed-all.jsonl
49
+ ocrtech audit-manifest data/manifests/labels-train.jsonl --require-slice nepali --require-slice english
50
+ ocrtech split-manifest data/manifests/labels-train.jsonl --out data/splits --eval-ratio 0.15
51
+ ocrtech rebalance-manifest data/splits/train.jsonl --out data/splits/train-rebalanced.jsonl --slice english --slice nepali
52
+ ocrtech audit-recognizer-corpus --train-manifest data/splits/train-rebalanced.jsonl --eval-manifest data/splits/eval.jsonl --out outputs/recognizer-corpus-audit
53
+ ocrtech build-recognizer-corpus data/manifests/english-lines.jsonl data/manifests/nepali-lines.jsonl data/manifests/mixed-crops.jsonl --out data/splits/recognizer-train-balanced.jsonl --target-latin-only 200 --target-devanagari-only 200 --target-mixed 100 --target-real 100 --target-synthetic 100
54
+ ocrtech materialize-references data/splits/eval.jsonl --out data/references --rewritten-manifest data/splits/eval-with-refs.jsonl
55
+ ocrtech create-eval-pack --out data/eval-pack --input-format image --degradation clean --degradation scan
56
+ ocrtech create-eval-pack --out data/eval-pack-table-figure-dev-v1 --input-format image --template table-ledger --template figure-caption --count-per-template 8 --variant-offset 100 --degradation clean --degradation scan --degradation blur --degradation low_contrast --degradation skew
57
+ ocrtech audit-eval-pack data/eval-pack/manifests/ocrtech-eval-pack.jsonl --require-claim-ready
58
+ ocrtech train-recognizer --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --out runs/recognizer
59
+ ocrtech train-recognizer --backend hf-vision-encoder-decoder --base-model microsoft/trocr-small-printed --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --out runs/hf-recognizer
60
+ ocrtech bundle-recognizer-training --recipe-dir runs/recognizer --train-manifest data/splits/train-rebalanced.jsonl --eval-manifest data/splits/eval.jsonl --out runs/recognizer-training-bundle --require-corpus-audit
61
+ ocrtech export-recognizer --training-config runs/recognizer/paddle_recognizer.yml --checkpoint runs/recognizer/checkpoints/best_accuracy --out runs/recognizer/export
62
+ ocrtech package-recognizer --inference-dir runs/recognizer/export/inference --dictionary runs/recognizer/nep_eng_dict.txt --out runs/recognizer/package --model-id ocrtech-nep-eng-rec-v1 --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --training-summary runs/recognizer/recognizer-training-summary.json --admission-validation-report outputs/validation-next-recognizer-hard50/validation.json
63
+ ocrtech package-hf-recognizer --model-dir runs/hf-recognizer/final-model --out runs/hf-recognizer/package --model-id ocrtech-hf-nep-eng-rec-v1 --base-model microsoft/trocr-small-printed --train-manifest data/splits/train.jsonl --eval-manifest data/splits/eval.jsonl --training-summary runs/hf-recognizer/recognizer-training-summary.json --admission-validation-report outputs/validation-next-recognizer-hard50/validation.json
64
+ ocrtech train-corrector --clean-manifest data/splits/train.jsonl --out runs/corrector
65
+ ocrtech train-corrector --hf-dataset cfilt/RoundTripOCR-nepali --hf-split train --limit 1000 --out runs/corrector-cfilt
66
+ ocrtech audit-model runs/recognizer/package/model-card.json
67
+ ocrtech audit-remote-host --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --workdir '~/ocr-tech' --require-path '~/ocr-tech' --min-free-gb 20 --require-paddle-training --out outputs/remote-audit
68
+ ocrtech bootstrap-remote-host --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --workdir '~/ocr-tech' --out outputs/remote-bootstrap
69
+ ocrtech sync-remote-workspace --host 100.97.238.103 --user k2 --password-env OCRTECH_REMOTE_PASSWORD --local-root . --remote-workdir '~/ocr-tech' --exclude .venv --exclude outputs --exclude runs --out outputs/remote-sync
70
+ ocrtech preflight-claim --eval-manifest data/eval-pack/manifests/ocrtech-eval-pack.jsonl --references-dir data/eval-pack/references --baselines candidate,tesseract,stock-paddle --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json --train-manifest data/splits/train.jsonl --out outputs/preflight
71
+ ocrtech parse-doc examples/sample-page.txt --out outputs/sample --engine sidecar
72
+ ocrtech parse-doc page.png --out outputs/page --engine candidate --model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
73
+ ocrtech benchmark examples/sample-page.txt --out outputs/bench --baselines ours --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
74
+ ocrtech summarize-benchmark --benchmark-report outputs/bench/report.json --out outputs/bench-summary --candidate ours --baseline tesseract --metric cer
75
+ ocrtech benchmark-error-analysis --benchmark-report outputs/bench/report.json --out outputs/bench-error-analysis --system surya --baseline tesseract --metric cer
76
+ ocrtech export-failure-manifest --source-manifest data/splits/eval.jsonl --top-failures outputs/bench-error-analysis/top-failures.json --out data/splits/eval-hard.jsonl --split hard_eval
77
+ ocrtech recognizer-admission-decision --validation-report outputs/validation-next-recognizer-hard50/validation.json --out outputs/recognizer-admission-next
78
+ ocrtech paper-benchmark-table --summary outputs/paper-claimpack-summary-tesseract/summary.json --summary outputs/paper-claimpack-summary-surya/summary.json --summary outputs/paper-claimpack-summary-stock-paddle/summary.json --out outputs/paper-claimpack-table --system candidate --system surya --system tesseract --system stock-paddle --label 'candidate=Candidate' --label 'surya=Surya' --label 'tesseract=Tesseract' --label 'stock-paddle=Stock Paddle' --source-preference 'candidate=outputs/paper-claimpack-summary-tesseract/summary.json' --source-preference 'tesseract=outputs/paper-claimpack-summary-tesseract/summary.json' --source-preference 'surya=outputs/paper-claimpack-summary-surya/summary.json' --source-preference 'stock-paddle=outputs/paper-claimpack-summary-stock-paddle/summary.json' --metric cer --metric table_cell_f1 --metric reading_order_pair_accuracy --metric latency_seconds
79
+ ocrtech calibrate-quality-router --experiment-report outputs/calibration-experiment/experiment.json --out outputs/quality-router-calibration --model-id ocrtech-tesseract-quality-router --primary-engine tesseract --secondary-engine tesseract --primary-engine-kwarg 'language=nep+eng' --primary-engine-kwarg 'psm=6' --secondary-engine-kwarg 'language=nep+eng' --secondary-engine-kwarg 'psm=13'
80
+ ocrtech calibrate-script-router --experiment-report outputs/script-calibration/experiment.json --out outputs/script-router-calibration --model-id ocrtech-tesseract-surya-script-router --primary-engine tesseract --secondary-engine surya --primary-engine-kwarg 'language=nep+eng' --primary-engine-kwarg 'psm=6' --secondary-engine-kwarg 'mode=auto'
81
+ ocrtech calibrate-tesseract-psm-ensemble --primary-experiment-report outputs/psm13-calibration/experiment.json --variant-experiment psm7=outputs/psm7-calibration/experiment.json --variant-experiment psm11=outputs/psm11-calibration/experiment.json --variant-experiment psm13=outputs/psm13-calibration/experiment.json --out outputs/psm-ensemble-calibration --model-id ocrtech-tesseract-psm-ensemble
82
+ ocrtech run-experiment --eval-manifest data/eval-pack/manifests/ocrtech-eval-pack.jsonl --out outputs/experiment --baselines candidate,tesseract,stock-paddle --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json
83
+ ocrtech review-claim --experiment outputs/internal-claim/experiment.json --experiment outputs/ocrbench-v2/experiment.json --config claim-review.json --model-config runs/recognizer/package/model-card.json --out outputs/claim-review
84
+ ocrtech validate-claim --benchmark-report outputs/bench/report.json --config validation.json --candidate-model-config runs/recognizer/package/model-card.json --fallback-model-config runs/vlm-fallback/package/model-card.json --out outputs/validation
85
+ ```
86
+
87
+ `parse-doc` writes:
88
+
89
+ - `document.json`
90
+ - `document.md`
91
+ - `tables/*.csv`
92
+ - `tables/*.html`
93
+ - `figures/*.png`
94
+ - `figures/metadata.json`
95
+ - `quality.json`
96
+
97
+ Figure exports are explicit. If an engine supplies `figure.image_path`,
98
+ `parse-doc` copies that file into `figures/`. If an engine supplies only a
99
+ figure bbox and the source is a regular image or page-bundle directory,
100
+ `parse-doc` crops the region into `figures/{figure_id}.png` when Pillow is
101
+ available. For bundles, it uses `figure.page_index` to select the page image. If
102
+ the artifact cannot be created, `figures/metadata.json` records
103
+ `metadata.export_warnings` instead of silently dropping the figure image.
104
+
105
+ When `--fallback-engine` or `--fallback-model-config` is configured,
106
+ `document.json` records `metadata.fallback` with the primary engine, fallback
107
+ engine/model card, confidence threshold, trigger reason, and success/error
108
+ outcome. Benchmarks surface this as diagnostic metrics such as
109
+ `fallback_triggered`, `fallback_succeeded`, and `fallback_failed`.
110
+ The same fallback flags are available on `benchmark` and `run-experiment` for
111
+ candidate/ours runs, and experiment metadata records the fallback model-card
112
+ path, SHA-256, and confidence threshold.
113
+
114
+ ## Engines
115
+
116
+ `auto` tries deterministic sidecar OCR first, then PaddleOCR, then Tesseract. Optional engines fail loudly when binaries or packages are missing.
117
+
118
+ Sidecar inputs are useful for tests and staged pipelines. For `page.png`, provide one of:
119
+
120
+ - `page.png.ocr.json`
121
+ - `page.ocr.json`
122
+ - `page.png.txt`
123
+ - `page.txt`
124
+
125
+ ## Data
126
+
127
+ Use `prepare-data` to verify actual fields and image paths before training. By default it fails on the first bad row. Use `--skip-invalid` when the upstream dataset is messy and you want a usable manifest plus an explicit reject log instead of a silent partial corpus.
128
+
129
+ Use `merge-manifests` to assemble multi-source corpora without ad hoc `cat` pipelines, then `audit-manifest` before training and `split-manifest` to create deterministic leakage-safe train/eval manifests. The split groups exact sample signatures before assignment so duplicate image+text examples cannot land on both sides.
130
+
131
+ Known Hugging Face adapters now stamp verified slice metadata automatically:
132
+
133
+ - `gauravgiri/nepali-ocr-dataset`: `nepali`, `real`
134
+ - `prashant0919/nepali-synthetic-ocr-lines`: `nepali`, `synthetic`
135
+ - `nvidia/OCR-Synthetic-Multilingual-v1 --hf-subset en`: `english`, `synthetic`, `line_crop`, `en`
136
+
137
+ Use `rebalance-manifest` only on the training split after `split-manifest`. Rebalancing before the split would turn deliberate duplication into leakage.
138
+
139
+ Use `audit-recognizer-corpus` before launching a recognizer training job. It
140
+ checks the actual train/eval manifests for sample counts, Latin-script coverage,
141
+ Devanagari coverage, mixed-script examples, real/synthetic training coverage,
142
+ and train/eval leakage. This guardrail was added after the failed PaddleOCR
143
+ mixed v4 branch: a new GPU run should not start unless English preservation and
144
+ Nepali coverage are explicit in the corpus.
145
+
146
+ Use `build-recognizer-corpus` to assemble and upsample the training manifest by
147
+ actual text buckets, not by wishful dataset labels. It tracks `latin_only`,
148
+ `devanagari_only`, `mixed_devanagari_latin`, `real`, and `synthetic` coverage,
149
+ then writes a build JSON/Markdown summary next to the output manifest. It does
150
+ not invent missing categories: if a required bucket is absent from the input
151
+ manifests, the command fails so you know to add real source data before using
152
+ GPU time.
153
+
154
+ `train-recognizer --backend paddleocr` writes
155
+ `recognizer-training-summary.json` alongside the generated PaddleOCR config.
156
+ Keep that file with any remote/GPU training bundle and pass it to
157
+ `package-recognizer --training-summary`; it records manifest hashes, slice
158
+ counts, character coverage, label/config hashes, and launch hyperparameters.
159
+ `bundle-recognizer-training` records the bundled summary path and SHA-256 under
160
+ `training_summary`, and packaged recognizer model cards record the same
161
+ provenance when supplied. For real remote/GPU runs, use
162
+ `bundle-recognizer-training --require-corpus-audit`; it runs the recognizer
163
+ corpus audit and refuses to create the bundle if the corpus does not meet the
164
+ configured English, Devanagari, mixed-script, real/synthetic, and leakage gates.
165
+ The gate also checks isolated Latin-only and Devanagari-only samples; a corpus
166
+ made entirely of mixed-script rows is not enough to protect English preservation
167
+ or Nepali-only recognition.
168
+
169
+ Planned source datasets:
170
+
171
+ - `gauravgiri/nepali-ocr-dataset`
172
+ - `prashant0919/nepali-synthetic-ocr-lines`
173
+ - `nvidia/OCR-Synthetic-Multilingual-v1`
174
+ - `cfilt/RoundTripOCR-nepali`
175
+
176
+ The verified Hugging Face adapters are intentionally dataset-specific where the upstream schema is non-standard:
177
+
178
+ - `gauravgiri/nepali-ocr-dataset`: generic `image` + `text`
179
+ - `prashant0919/nepali-synthetic-ocr-lines`: `metadata.jsonl` plus per-line image files, downloaded lazily for the requested rows
180
+ - `cfilt/RoundTripOCR-nepali`: correction pairs via `ocr` and `correct`
181
+ - `nvidia/OCR-Synthetic-Multilingual-v1`: HDF5 shards with `annotations.line_bboxes`; use `--hf-subset en` for English crops
182
+
183
+ Some upstream corpora are dirty. For example, the current `gauravgiri/nepali-ocr-dataset` sample prefix includes null `text` rows, so use `--skip-invalid` when you want a prepared manifest plus an explicit reject log instead of a hard stop.
184
+
185
+ ## Baselines
186
+
187
+ The benchmark command supports:
188
+
189
+ - `ours`
190
+ - `tesseract`
191
+ - `stock-paddle`
192
+ - `glm-ocr`
193
+ - `paddleocr-vl`
194
+
195
+ `glm-ocr` and `paddleocr-vl` are external command hooks:
196
+
197
+ ```bash
198
+ export OCRTECH_GLM_OCR_CMD='python scripts/run_glm_ocr.py --input {input} --out {out}'
199
+ export OCRTECH_PADDLEOCR_VL_CMD='python scripts/run_paddleocr_vl.py --input {input} --out {out}'
200
+ ```
201
+
202
+ The command must include both `{input}` and `{out}` placeholders. It may print Markdown to stdout or write `{out}/document.md`; `ocrtech` now normalizes that Markdown into its own `document.json` schema so external VLM baselines can still be scored for text, table cells, and reading order.
203
+
204
+ The bundled wrappers are thin adapters, not magic:
205
+
206
+ - `scripts/run_glm_ocr.py` uses the official `transformers` path from the `zai-org/GLM-OCR` model card.
207
+ - `scripts/run_paddleocr_vl.py` uses the official `PaddleOCRVL(...).predict(...)` path from the `PaddlePaddle/PaddleOCR-VL-1.6` model card.
208
+ - GLM-OCR still expects a git `transformers` build upstream. On `k2`, the released `transformers 4.57.6` package was not enough to instantiate its processor classes; a dedicated Python 3.11 env with git `transformers` and `torchvision` was required before the wrapper would run.
209
+ - PaddleOCR-VL still needs the OCR extras that the official docs call out, for example `pip install "paddlex[ocr]==3.7.1"`.
210
+
211
+ ## Validation Before SOTA Claims
212
+
213
+ Use `validate-claim` before making any quality claim. The default validator is intentionally strict and will fail unless the benchmark report has held-out sample coverage, required slices, leakage checks, paired candidate-vs-baseline gates, and per-gate evidence such as paired wins and optional significance thresholds.
214
+
215
+ Use `recognizer-admission-decision` after a recognizer admission validation run.
216
+ It writes a small JSON/Markdown decision artifact: `admit` only for
217
+ `claim_status=validated` with `passed=true`, otherwise `reject` with failed
218
+ gate metrics and next actions for the next model branch.
219
+
220
+ Use `create-eval-pack` for internal mixed Nepali/English document tests and `audit-eval-pack` to check whether the pack is claim-ready. The internal pack now includes a real `multi_page` slice in addition to the single-page paragraph, table, form, and figure templates. Text packs are for pipeline smoke tests only; image packs are required for OCR claims.
221
+
222
+ Use `preflight-claim` before `run-experiment` on any claim-grade run. It fails early on missing references, invalid model cards, missing baseline dependencies, non-claim-ready eval packs, bad validation configs, and train/eval leakage.
223
+
224
+ For routed or thresholded candidates such as `quality_select_composite` and `script_select_composite`, claim-grade runs also require explicit calibration provenance in the model card. If the candidate was calibrated on the same eval manifest you are about to benchmark, `preflight-claim` now fails on purpose.
225
+
226
+ For those routed candidates, claim-grade runs also need the raw component baselines in the same benchmark. If a `script_select_composite` candidate is built from Tesseract and Surya, the claim run must include `candidate,tesseract,surya`; otherwise the repo now fails preflight because distinctness cannot be checked honestly.
227
+
228
+ Use `run-experiment` to benchmark every manifest input, write reproducibility metadata, and run claim validation in one output directory.
229
+
230
+ Use `review-claim` after the internal pack and public benchmark suites have all been run. It audits the portfolio-level evidence bundle so one green experiment does not get misrepresented as a broad SOTA claim.
231
+
232
+ Use a candidate model card for any real benchmark of `ours`; see [docs/MODELS.md](docs/MODELS.md).
233
+ Use `filter-manifest` when a baseline is too expensive to run on the full pack and you need a reproducible exploratory subset instead of hand-editing a JSONL.
234
+
235
+ See [docs/PREFLIGHT.md](docs/PREFLIGHT.md), [docs/CLAIM_REVIEW.md](docs/CLAIM_REVIEW.md), [docs/EVAL_PACK.md](docs/EVAL_PACK.md), [docs/MODELS.md](docs/MODELS.md), [docs/EXPERIMENTS.md](docs/EXPERIMENTS.md), and [docs/VALIDATION.md](docs/VALIDATION.md).
236
+
237
+ ## Remote Training
238
+
239
+ Use `docs/REMOTE_TRAINING.md` for the current `k2` host notes, `audit-remote-host`, and the safe SSH workflow. Do not commit SSH passwords or machine-specific secrets.
240
+
241
+ ## Current HF Result
242
+
243
+ The strongest verified TrOCR candidate so far is the rebalanced mixed-script run trained on `k2` and evaluated on a held-out `180`-sample mixed Nepali/English line set.
244
+
245
+ That run is materially better than our own earlier HF candidates:
246
+
247
+ - overall benchmark CER: `1.628157` -> `0.586139`
248
+ - English CER: `0.379729` -> `0.300454`
249
+ - Nepali CER: `7.870295` -> `2.014560`
250
+ - real-slice CER: `12.702701` -> `1.770016`
251
+
252
+ But the first named baseline comparison is still bad news for the project claim: on that same eval split, `tesseract` is better than the rebalanced candidate on overall, English, Nepali, real, and synthetic OCR metrics.
253
+
254
+ The next attempted rescue path also failed. On June 14, 2026, a ByT5 text-correction model trained on `k2` from mined Tesseract noisy/clean pairs was packaged as a `text_correction_composite` candidate with raw Tesseract as the base engine. On the same held-out `180`-sample eval, that composite regressed catastrophically: overall CER `6.912065` versus raw Tesseract `0.329012`, with paired CER `0` wins, `170` losses, and `10` ties. That rules out the current Tesseract-plus-corrector path as a serious answer.
255
+
256
+ There is one narrower improvement worth keeping, but it is not a claim-grade win. A routed Tesseract composite now exists as `quality_select_composite`: it runs default Tesseract (`psm=6`) and a secondary variant (`psm=13`), scores both outputs with the local `quality_score`, and switches only when the secondary beats the primary by a configured margin.
257
+
258
+ The same-pack tuned run looked slightly better on the original `180`-sample mixed eval, but that was not honest enough. After splitting that pack into a separate calibration half (`89` samples) and untouched holdout half (`91` samples), the calibrated routed candidate still improved overall CER on the holdout, just by much less:
259
+
260
+ - raw tesseract holdout overall CER: `0.342234`
261
+ - routed tesseract holdout overall CER: `0.340608`
262
+ - paired CER on holdout: `2` wins, `1` loss, `88` ties
263
+
264
+ That makes it a plausible engineering baseline, not a frontier result. The effect size is tiny, the pack still is not claim-ready, and the real Devanagari recognizer problem remains open.
265
+
266
+ We also tested a richer calibrated Tesseract ensemble over `psm=6/7/11/13` with learned quality-score biases. That looked better on the calibration half, but it failed on the untouched holdout: overall CER regressed from `0.342234` to `0.358352`. That is exactly the kind of result the new calibration workflow is supposed to expose.
267
+
268
+ The more useful June 14, 2026 result is a calibrated Tesseract-plus-Surya script router on `k2`. Raw Surya was not competitive overall on the `91`-sample holdout (`2.198362` CER vs raw Tesseract `0.342234`), but it was materially better on the Nepali slice. A routed `script_select_composite` candidate calibrated on the separate `89`-sample half and evaluated on the untouched `91`-sample holdout did better than raw Tesseract across every reported slice:
269
+
270
+ - raw tesseract holdout overall CER: `0.342234`
271
+ - calibrated script router overall CER: `0.313084`
272
+ - paired CER on holdout: `9` wins, `0` losses, `82` ties
273
+
274
+ That is the strongest verified baseline improvement in the repo so far. It is still not a frontier claim, because it is one internal line-level split and not the full claim bundle, but it is a real out-of-sample gain and a better direction than the current TrOCR branch. The router now also supports `routing_granularity=page` for mixed multi-page documents; the calibrated `k2` result above is still the original document-level calibration, not a new page-level benchmark.
275
+
276
+ We also ran an exploratory page-level script router on the same `91`-sample holdout, reusing the document-level threshold `0.4`. The result was identical to the document-level router (`0.313084` overall CER, `9` wins, `0` losses, `82` ties vs raw Tesseract). That is expected for this pack because it is dominated by single-page line-crop samples; page routing needs a real multi-page document pack before it can prove anything new.
277
+
278
+ The internal image eval pack now has a real rendered `multi_page` slice with bundled page images and actual drawn table grids. On that small rendered smoke pack, the plain script router already beat Tesseract strongly on text (`0.243224` vs `0.540584` overall CER) and on the dedicated table page (`table_cell_f1` `1.0` vs `0.0`). A follow-up exploratory model with `secondary_structure_backfill=true` kept the same CER and backfilled Surya table structure on the multi-page report, lifting aggregate `table_cell_f1` from `0.5` to `1.0` on that smoke pack. That is still not claim-grade evidence because the pack is tiny and only `clean`, but it is a real document-capability improvement.
279
+
280
+ The next step was to stop calling it smoke evidence and run the same backfill router on a claim-ready rendered image pack. That broader internal pack contains `25` samples across `clean`, `scan`, `blur`, `low_contrast`, and `skew`, and its audit passed cleanly. The calibrated backfill router also passed `preflight-claim` on that pack. Against raw Tesseract on the same rendered claim pack it still held up on text and structure:
281
+
282
+ - overall CER: `0.248654` vs `0.689214`
283
+ - blur CER: `0.260039` vs `0.991613`
284
+ - scan CER: `0.237424` vs `0.593631`
285
+ - skew CER: `0.239317` vs `0.766067`
286
+ - aggregate `table_cell_f1`: `0.955556` vs `0.0`
287
+ - aggregate `table_exact_match`: `0.8` vs `0.0`
288
+ - paired CER: `9` wins, `1` loss, `15` ties
289
+
290
+ That result is good enough to keep and bad enough to describe plainly. The multi-page slice tied Tesseract on CER (`0.315789` each) while still recovering table structure that Tesseract missed (`table_cell_f1` `0.911111` vs `0.0`). The form slice regressed slightly on CER (`0.231933` vs `0.230010`). Aggregate reading-order pair accuracy on this rendered pack is also still worse than Tesseract (`0.764` vs `0.816`). So the current candidate is meaningfully better on OCR robustness and table extraction for this internal rendered pack, but it is not yet a clean document-understanding leader.
291
+
292
+ We then widened the comparison beyond Tesseract. Two more June 14, 2026 checks on `k2` matter:
293
+
294
+ - raw Surya on the same `25`-sample rendered claim pack is better on text CER than the calibrated backfill router: `0.195260` vs `0.248654`
295
+ - but the router is still not pointless: it ties Surya on aggregate `table_cell_f1` (`0.955556` each) and is slightly better on aggregate `reading_order_pair_accuracy` (`0.764000` vs `0.750909`)
296
+
297
+ That means the current routed candidate is still not “better than what is out there” on this rendered document pack. Raw Surya is the stronger text OCR baseline here. The router is only defending a narrower position: it preserves comparable structure while staying closer to the existing Tesseract-centered stack.
298
+
299
+ We also fixed a real integration bug in the local Paddle baseline. `paddleocr 3.7.0` on `k2` returns v3 `OCRResult` objects, and the old adapter in this repo treated them like the older tuple/list API and silently dropped their text. After fixing that parser and installing `paddlepaddle 3.3.1`, `stock-paddle` became a real benchmark target on `k2`. On this same rendered pack it is still a weak baseline overall:
300
+
301
+ - stock-Paddle overall CER: `1.749081`
302
+ - stock-Paddle aggregate `table_cell_f1`: `0.0`
303
+ - stock-Paddle aggregate `reading_order_pair_accuracy`: `1.0`
304
+
305
+ So the current picture is sharper now, not prettier: the routed candidate beats Tesseract and this stock-Paddle configuration on OCR robustness and table extraction, but it still loses to raw Surya on text CER.
306
+
307
+ We then used the new `filter-manifest` helper to run a broader but still tractable rendered-document comparison on `k2`: a `5`-sample blur-only subset spanning paragraph, table, form, figure, and multi-page layouts. All `30` runs completed without error across `candidate`, `tesseract`, `surya`, `stock-paddle`, `glm-ocr`, and `paddleocr-vl`. The result sharpened the ranking again:
308
+
309
+ - candidate CER: `0.223273`
310
+ - raw Surya CER: `0.197601`
311
+ - Tesseract CER: `0.732364`
312
+ - stock-Paddle CER: `2.810426`
313
+ - GLM-OCR CER: `2.820269`
314
+ - PaddleOCR-VL CER: `18.470663`
315
+
316
+ On that subset, the current backfill candidate still held the structure line where Tesseract and the weaker VLM baselines collapsed (`table_cell_f1 = 1.0` for the candidate versus `0.0` for Tesseract, stock-Paddle, GLM-OCR, and PaddleOCR-VL). But it still did not beat raw Surya on text CER. That keeps the same honest conclusion intact: on rendered document OCR text, the strongest baseline in this repo is still Surya, not the current routed candidate.
317
+
318
+ The stricter rendered-document result is now explicit too. On `k2`, a larger rendered `claimpack-rendered-v2` split was calibrated separately and evaluated on an untouched `50`-sample holdout. That calibration selected `primary_script_threshold=0.0`, so the routed `script_select_composite` collapsed to "always pick Surya" on the calibration set. On the holdout:
319
+
320
+ - candidate CER: `0.180893`
321
+ - raw Surya CER: `0.180893`
322
+ - candidate `table_cell_f1`: `0.947500`
323
+ - raw Surya `table_cell_f1`: `0.970833`
324
+
325
+ That is the ceiling of the current branch, not a win. The candidate no longer even has a text-vs-structure tradeoff story there; on this larger rendered holdout it effectively degenerates to Surya on text and still loses slightly on table structure. The claim guardrails now reject this case directly: routed candidates that collapse to one engine across all calibration samples fail `candidate-calibration` in both `preflight-claim` and `review-claim`.
326
+
327
+ We also tested whether a quality-based Surya/Tesseract selector could recover anything on that same rendered-v2 setup. It did not. The rendered-v2 Surya quality router calibrated to `selected_threshold=0.0`, then landed at:
328
+
329
+ - candidate CER: `0.181128`
330
+ - raw Surya CER: `0.180893`
331
+ - candidate `table_cell_f1`: `0.947500`
332
+ - raw Surya `table_cell_f1`: `0.947500`
333
+ - paired CER vs Surya: `0` wins, `1` loss, `49` ties
334
+
335
+ That is not a distinct better engine. It is basically Surya with one extra mistake.
336
+
337
+ We then tested `line_align_composite` on the same rendered-v2 holdout: keep Surya text, align it to Tesseract's line scaffold, and see whether the ordering gain is worth the merge. It was not:
338
+
339
+ - candidate CER: `0.310549`
340
+ - raw Surya CER: `0.180893`
341
+ - candidate `table_cell_f1`: `0.897500`
342
+ - raw Surya `table_cell_f1`: `0.970833`
343
+ - candidate `reading_order_pair_accuracy`: `0.792727`
344
+ - raw Surya `reading_order_pair_accuracy`: `0.750909`
345
+ - paired CER vs Surya: `0` wins, `7` losses, `43` ties
346
+
347
+ That is another useful paper result, but not a winning model branch. Better line ordering alone did not rescue the Surya/Tesseract composite family.
348
+
349
+ The upper bound is now measured too. A sample-level oracle that chooses the better of raw Surya and raw Tesseract per holdout sample using ground-truth CER landed at:
350
+
351
+ - oracle CER: `0.180529`
352
+ - raw Surya CER: `0.180893`
353
+ - raw Tesseract CER: `0.714207`
354
+ - oracle choices: Surya `49`, Tesseract `1`
355
+ - paired CER vs Surya: `1` win, `0` losses, `49` ties
356
+
357
+ That closes the routing branch for text OCR. There is almost no remaining sample-level headroom in the Surya/Tesseract component family on rendered-v2. The next credible model-side step has to use a genuinely stronger recognizer or document model, not another selector over these same two engines.
358
+
359
+ See [docs/EXPERIMENTS.md](docs/EXPERIMENTS.md) for the exact commands, artifact paths, and the current evidence gap.
360
+
361
+ If you are turning this work into a paper, start from [docs/PAPER_NOTES.md](docs/PAPER_NOTES.md). Then use [docs/PAPER_RESULTS.md](docs/PAPER_RESULTS.md) for the current generated comparison tables and branch chronology tied to local artifacts.
362
+
363
+ ## References
364
+
365
+ - [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
366
+ - [PP-StructureV3 docs](https://paddlepaddle.github.io/PaddleOCR/latest/version3.x/pipeline_usage/PP-StructureV3.html)
367
+ - [PaddleX table structure recognition docs](https://paddlepaddle.github.io/PaddleX/3.4/en/module_usage/tutorials/ocr_modules/table_structure_recognition.html)
368
+ - [PaddleOCR-VL-1.6](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6)
369
+ - [Tesseract docs](https://tesseract-ocr.github.io/docs/)
370
+ - [GLM-OCR](https://huggingface.co/zai-org/GLM-OCR)
371
+ - [Surya OCR](https://github.com/datalab-to/surya)
ocr-tech-mps-integration/RESEARCH.md ADDED
@@ -0,0 +1,375 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Research State: Nepal Language Pack OCR
2
+
3
+ Updated: 2026-08-09
4
+
5
+ ## Current Goal
6
+
7
+ The end goal is an OCR system covering the multiple languages of the Nepal pack,
8
+ built out of Paddle, **with Limbu and Newar (Nepal Bhasa) as the first two focus
9
+ languages** (user steer, 2026-07-04). The system should accept real book/page images
10
+ or PDFs for any language in the pack and output editable Unicode text with page,
11
+ line, crop, confidence, and provenance evidence preserved. Limbu stays the proven
12
+ template — it has the strongest recognizer and the deepest harvest. Newar is the
13
+ deliberate second: it has its own native script (Prachalit), real cultural demand,
14
+ and is currently the weakest trained recognizer, so it exercises every part of the
15
+ method that Limbu already proved. Every stage that was built Limbu-specific must
16
+ generalize: harvest, silver-label extraction, recognizer training, script routing,
17
+ page/book OCR, and eval. The product is the full pipeline: page
18
+ detection, script-aware routing to per-script recognizers, target-region and
19
+ furniture filtering, reading order, assembly, Unicode/JSON/Markdown export, and
20
+ review artifacts for hard lines. YOLO line detection stays a diagnostic/probe;
21
+ Paddle-first is the production story.
22
+
23
+ ## The Language Pack
24
+
25
+ Two inventories exist and they are not the same thing:
26
+
27
+ - The registry `languages/nepal-v1.json` tracks 18 target languages (17 Nepal + English)
28
+ with scripts, fonts, dictionaries, and corpus status.
29
+ - The Gorkhapatra Naya Nepal archive publishes far more: the 2026-07-04 harvest found
30
+ **50 normalized languages (53 raw labels) in 400 articles over ~7 months** (see the
31
+ harvest section). Labels include all registry languages plus Chamling Rai, Bahing,
32
+ Jirel, Chepang, Baram, Tajpuriya, Kumal, Doteli, Bajhangi, Dandeldhureli, Hyolmo,
33
+ Mugal, Marwadi, Urdu, Sanskrit, Awadhi, Kisan, Achhami, Bote, Hayu, Meche, Lhomi,
34
+ Uraon, Bhote, Danuwar, Majhi, Magahi, Bajjika, Khash Jumli, and more.
35
+ Label variants must be normalized before counting (थारु/थारू, याक्ख/याक्खा,
36
+ ह्योलमो/ह्योल्मो, लिम्बु/लिम्बू, चाम्लिङ vs चाम्लिङ राई) —
37
+ `tools/build_gorkhapatra_language_pack_inventory.py` owns that normalization and the
38
+ label→registry mapping.
39
+
40
+ Per-language recognizer state (grapheme CER, honest micro metric, from the re-grounded
41
+ 2026-06-18 scorecard in `docs/RECOGNIZER_GOAL_PROGRESS.md`):
42
+
43
+ - **Limbu/Sirijonga 0.041 real-print** (champion:
44
+ `runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23`,
45
+ cp CER 0.011 on the frozen 255-line Gorkhapatra silver eval).
46
+ - **8 Devanagari languages 0.075–0.129 real-print** (Majhi, Magar, Sunuwar-Deva, Magahi,
47
+ Bhojpuri, Thami, Wambule, Darai) from one shared Devanagari recognizer.
48
+ - **Sunuwar (Jenticha) 0.121 real-crop** after the same-font retrain (0.342 synth-only).
49
+ - **Tirhuta 0.343 and Newa 0.613 synthetic-only** — weak, transliteration-fed, honest
50
+ fails; Newa needs real printed-register Nepal Lipi, not more pixels.
51
+ - **Kirat Rai (70 real crops) and Lepcha (132 real crops)** blocked on legacy-font
52
+ converter coverage outside their eval editions; **Gurung Khema** has no real print
53
+ anywhere (revival script) — synthetic-only by nature, claim-scoped accordingly.
54
+
55
+ The near-term pack claim should therefore be script-family shaped: one Sirijonga
56
+ recognizer, one shared Devanagari recognizer covering the many Devanagari-written
57
+ languages, plus per-script experts (Jenticha, Kirat Rai, Lepcha, Prachalit/Newa,
58
+ Tirhuta, Gurung Khema, Ol Chiki, Tibetan-script) admitted one by one as real eval
59
+ crops exist.
60
+
61
+ **Focus order (2026-07-04 steer): Limbu first, Newar second.** Newar state: the
62
+ Prachalit recognizer is the weakest in the pack (grapheme CER 0.613 on its own
63
+ synthetic eval; transliteration-fed training failed; real printed-register Nepal
64
+ Lipi is the known lever, and only 2 Unicode Newa fonts exist). The Devanagari-Newar
65
+ route moved fast on 2026-07-04: 6,800 real silver lines harvested from 20
66
+ Gorkhapatra pages; `newar` added to the registry (Devanagari primary + Prachalit
67
+ secondary; audit passes); a frozen by-date pack built at
68
+ `outputs/newar-devanagari-silver-pack-v1/` (train 4,096 / dev 1,528 / test 1,139
69
+ lines; test dates 2082-07-25 and 2083-01-05; zero date/text overlap independently
70
+ verified; pack sha256
71
+ `1fa7b23b6c18701857aea686fd55f3075e104d224730896f6ee35ba25e7ccb85`); and the
72
+ **first real-print Devanagari-Newar baseline** was scored locally (CPU export
73
+ route, `tools/score_recognizer_pack.py`, per `docs/LOCAL_RECOGNIZER_SCORING.md`):
74
+ the shared deva-v2 recognizer, never trained on Newar, reads the frozen test at
75
+ **codepoint CER 0.0622, grapheme CER 0.1312, exact-line 0.158** (n=1,139 silver
76
+ lines — machine-converted labels, diagnostic, not claim-grade). The same-day
77
+ warm-start fine-tune (**newar-ft-v1**: deva-v2 base, LR 2e-5 cosine, 60 epochs,
78
+ batch 64, selection on the all-batch2 dev split, best_epoch 46; G6 L4 instance
79
+ `ashish-experiments`, ~68 min; artifacts + train log at
80
+ `outputs/aws-newar-deva-ft-v1/`) scores the same frozen test at **codepoint CER
81
+ 0.0465, grapheme CER 0.0955, exact-line 0.245** — a 25/27% relative CER drop and
82
+ +55% exact-line over the deva-v2 baseline, scored through the identical local
83
+ CPU path. **Newar's grapheme CER now sits inside deva-v2's own-language range
84
+ (0.075–0.129): the second focus language reached Devanagari-pack parity from
85
+ zero in one day.** Same claim boundary: silver machine-converted labels,
86
+ diagnostic, not claim-grade until native review. Next Newar lever: native
87
+ Prachalit (real printed-register data), and page-level Newar eval for the
88
+ region-filtering track.
89
+
90
+ ## Source Rule
91
+
92
+ For Gorkhapatra/Naya Nepal language pages, the Naya Nepal page is discovery metadata
93
+ only. Use `https://gorkhapatraonline.com/publications/naya-nepal` to find the language,
94
+ date, and article, but do not use the blog/card image for OCR, training, or evaluation
95
+ when the full paper exists. The authoritative source is the matching Gorkhapatra epaper
96
+ PDF from the date-range endpoint. The correct pair is the rendered PDF page image plus
97
+ the text layer extracted from that same PDF page, with PDF URL/hash, page number,
98
+ rendered image hash, extraction command/version, font evidence, and converter/map hash
99
+ recorded. For Limbu/Sirijonga the text layer is Namdhinggo-style legacy converted with
100
+ `src/ocrtech/limbu_legacy.py` + `data/mappings/limbu-legacy/Limbu.map`. The same
101
+ pattern — classify the text layer, then build an explicit converter only when the font
102
+ requires it — replicates to every other language. Manual transcription is the fallback
103
+ for no usable text layer, not the default.
104
+
105
+ ## Getting More Gorkhapatra Pages (the specified harvest recipe)
106
+
107
+ All numbers below were verified live on 2026-07-04.
108
+
109
+ **Archive size.** The Naya Nepal publication archive paginates at
110
+ `https://gorkhapatraonline.com/publications/naya-nepal?page=N` with 5 articles per
111
+ page and runs to page ~352 before drying up: **~1,750 language-page articles total**
112
+ (about 2 per day going back years). The 60-page crawl already yielded 300 items across
113
+ 53 labels spanning BS 2082-10-18 → 2083-03-19; deeper pages reach correspondingly
114
+ older dates.
115
+
116
+ **Epaper endpoint behavior (the critical constraint).** The date-range endpoint
117
+ `https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=A&pdfdaterange2=B&slug=gorkhapatra`
118
+ **caps results at 30 PDFs per query and ignores `page=N` pagination**. A wide window
119
+ silently drops most dates — the 2026-07-04 six-window discovery pass aligned only
120
+ 111/300 items for exactly this reason, with unmatched dates interleaved among matched
121
+ ones. **Seven-day windows return complete daily coverage (7/7 verified).** Always
122
+ harvest with weekly windows.
123
+
124
+ **The recipe** (one command; add one `--epaper-url` per week to cover the publication
125
+ date span being harvested):
126
+
127
+ ```bash
128
+ env PYTHONPATH=src uv run --python 3.11 --with pymupdf \
129
+ ocrtech audit-gorkhapatra-language-pages \
130
+ --out outputs/gorkhapatra-language-pages-full-<date> \
131
+ --publication-url https://gorkhapatraonline.com/publications/naya-nepal \
132
+ --publication-pages 80 \
133
+ --epaper-url 'https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-28&pdfdaterange2=2026-07-04&slug=gorkhapatra' \
134
+ --epaper-url '...one per week, going back...' \
135
+ --download-pdfs \
136
+ --render-pages \
137
+ --timeout-seconds 180
138
+ ```
139
+
140
+ The command aligns every publication card (all languages at once — alignment is
141
+ language-agnostic) to its same-date PDF, downloads each PDF once (cached by URL),
142
+ searches pages for the language-page markers (`भाषा पृष्ठ` and its legacy-encoding
143
+ form `efiff k[i7`), and renders hit pages at 2×. The 2026-07-04 run with 23 weekly
144
+ windows (2026-01-25 → 2026-07-04, publication-pages 80) completed with
145
+ **400 publication items, 195 aligned dates, 394 rendered language-page hits, zero
146
+ warnings** at `outputs/gorkhapatra-language-pages-full-2026-07-04/`. Batch 2
147
+ (2025-08-03 → 2026-01-24, 25 weekly windows, publication-pages 160) added 398 more
148
+ items: **798 items, 294 aligned dates, 589 further page hits**, at
149
+ `outputs/gorkhapatra-language-pages-full-batch2-2026-07-04/`. The merged normalized
150
+ inventory (`tools/build_gorkhapatra_language_pack_inventory.py`) shows **56
151
+ languages, 983 rendered language-page hits, most languages with ~20–27 pages
152
+ spanning 14 months**, at `outputs/language-pack-inventory-merged-2026-07-04/`. For
153
+ comparison, the entire previous state of the project had 8 usable pages for one
154
+ language. Batches cost ~2–2.5 GB each; the remaining archive depth (back to at least
155
+ March 2024 — verified the epaper endpoint serves complete weeks there) is another
156
+ ~2× and stays month-batchable with the same recipe.
157
+
158
+ **Scaling to the full archive.** Full depth is ~700+ unique dates ≈ 35 GB of PDFs.
159
+ Do not do it in one run: batch by month (4–5 weekly windows + the matching
160
+ publication-page depth), run `extract-gorkhapatra-language-page-pdf-text` per batch,
161
+ then keep rendered pages, extracted text, and hashes; PDFs can be pruned after
162
+ extraction because URL+SHA-256 make them re-fetchable. The remote 2 TB volume
163
+ (`cdjk@100.110.148.43:/Volumes/TRANSCEND`) is the right home for a full-depth mirror.
164
+
165
+ **Per-language downstream.** After harvest, run per language:
166
+
167
+ ```bash
168
+ env PYTHONPATH=src uv run --python 3.11 --with pymupdf \
169
+ ocrtech extract-gorkhapatra-language-page-pdf-text \
170
+ <pack-manifest.jsonl> \
171
+ --candidate-language <label> \
172
+ --target-font <font> \
173
+ --out outputs/<language>-pdf-text-<date>
174
+ ```
175
+
176
+ then classify each page's text layer (native Unicode / legacy font / Devanagari
177
+ transcription / Latin / none) and only then decide converter vs manual transcription.
178
+
179
+ **Font/text-layer facts already established** (30-page, 15-language extraction at
180
+ `outputs/gorkhapatra-multilingual-language-pages-current-2026-06-24-pdf-text/`):
181
+
182
+ - The body font of non-Limbu language pages is **`nayanepal` — a legacy 8-bit
183
+ Preeti-family encoding, not Unicode** (8,845 of 11,791 spans; the text layer reads
184
+ `k[i7 ;+of]hs` where the page shows पृष्ठ संयोजक). **The converter already
185
+ exists**: `nepal_ttf2utf.convert_devanagari(font="nayanepal")` in the sibling repo
186
+ `/Users/cdjk/github/llm/nepal-ttf2utf` (Preeti base map + nayanepal extension
187
+ glyphs + NFC, ~97–99% clean on real Gorkhapatra pages), already used by
188
+ `tools/build_gorkhapatra_devanagari_real.py` for the 79k-row real Devanagari
189
+ corpus. The remaining leverage is wiring it **per language**: run it over each
190
+ harvested language page and emit per-language silver line packs with the same
191
+ faithfulness gate (clean-Unicode allow-set, `conversion_gap` accounting, leakage
192
+ checks) that the Devanagari pack builder already implements. The same sibling repo
193
+ also hosts the limbu, kiratrai, lepcha, and sunuwar legacy converters.
194
+ - Tibetan-script fonts (`TibetanMachine`, `MonlamUniOuChan5`, `Qomolangma-*`) appear on
195
+ Mugal and related pages, but **every such span has empty extracted text (verified
196
+ 2026-07-04 across all 2026-06-24 samples)** — Tibetan-script pages are visual-only,
197
+ like the known Limbu `2082-11-12` date. They need OCR bootstrap or manual
198
+ transcription; no converter is possible.
199
+ - `Gorkhapatra-*`, `Nagarik-*`, `TimesNewRoman*`, `MyriadPro-*` are masthead/furniture
200
+ fonts — useful as non-target filters for target-region selection.
201
+ - `NamdhinggoSILL` appears only on Limbu dates (the existing Limbu path).
202
+
203
+ ## What We Built (Limbu template, still true)
204
+
205
+ The Gorkhapatra alignment path: publication-page discovery, epaper-date matching, PDF
206
+ download/rendering, page-marker search, PDF-native span/font extraction, review
207
+ sheets, reference templates, and dashboard visualization. The expanded Limbu harvest
208
+ has 9 discovered dates, 8 usable PDF-text-layer pages, 2,441 line rows, 2,430 usable
209
+ silver labels (`outputs/limbu-gorkhapatra-pdf-backed-silver-2026-07-03-v1/`).
210
+
211
+ The Limbu recognizer training/eval path: 48,507-row combined pack; real-heavy
212
+ fine-tune champion at codepoint CER 0.011336 / grapheme 0.024736 / exact-line 0.729 on
213
+ the frozen 255-line silver eval. Synthetic hardening lessons: targeted+Unicode packs
214
+ regressed; v3 dictionary warm-start catastrophically failed (~0.98 CER); keep the
215
+ real-heavy v2 champion; future gains come from real/book-domain harvesting or a
216
+ deliberate rehead, not blind synth rows.
217
+
218
+ The page/book OCR loop: local + HF Space (`spaces/limbu-ocr/`, deployed private at
219
+ `https://huggingface.co/spaces/ampixa/limbu-ocr`) with line OCR, Paddle Page OCR,
220
+ Paddle Book OCR, image-line fallback, runtime checks, annotated pages, and crops.
221
+
222
+ Page-level diagnostics on the 8-page silver pack: Rows LTR + y/conf70 filter scores
223
+ page cp CER 0.414330, line F1 0.963009, matched-line cp CER 0.026039, order accuracy
224
+ 0.993176. Recognition on matched lines is strong; **full-page CER is dominated by page
225
+ reconstruction, target-region selection, and furniture filtering** — that is the
226
+ pipeline problem to solve, and it is language-independent.
227
+
228
+ Supervised line-detector infrastructure from CDC PDF geometry (287 pages, 26,160
229
+ annotations, mAP50 ≈ 0.921 on PDF-rendered val) — evidence the detector path works,
230
+ not a solved product.
231
+
232
+ A claim-grade eval harness (versioned packs, hashes, provenance, normalization
233
+ profiles, cp/grapheme CER, CIs, paired comparisons, leakage manifests) — parked as a
234
+ guardrail; do not let it eat work blocks that should go to the pipeline.
235
+
236
+ ## Current Working State
237
+
238
+ The strongest component is the Sirijonga line recognizer. The weakest component is
239
+ end-to-end page/book reconstruction: target-region selection, furniture filtering,
240
+ line preservation, and ordering. Of the three Limbu-locked code spots, two were
241
+ generalized on 2026-07-04 (Codex worker, tests green): `_script_ratio` in
242
+ `src/ocrtech/engines.py` is now driven by the shared `SCRIPT_CODEPOINT_RANGES` table
243
+ covering all pack scripts (devanagari, limbu/sirijonga, latin, sunuwar/jenticha,
244
+ lepcha, kirat_rai, newa/prachalit, tirhuta, gurung_khema, ol_chiki, tibetan, with
245
+ aliases), and the image-line filter in `src/ocrtech/image_lines.py` accepts any
246
+ script key from that same table. The third spot fell the same day: the
247
+ `src/ocrtech/limbu_runtime.py` preflight now takes `required_script` (default
248
+ `limbu`, validated against the shared table; CLI `--required-script`), so a router
249
+ card for any pack script can be preflighted. The engine classes themselves (`CandidateEngine`, `PaddleOcrEngine`,
250
+ `ScriptSelectCompositeEngine`, `build_image_line_document`, `structure.py` reading
251
+ order) are already parameterized — the rest of the generalization is registry-driven
252
+ wiring, not a rewrite.
253
+
254
+ ## Apple MPS Recognizer Training Evidence (2026-08-09)
255
+
256
+ Stock PaddlePaddle is not the immediate Apple-GPU route. A direct PaddlePaddle
257
+ 3.3.1 probe on `k2` exposed no custom devices and rejected `mps`, `metal`, and
258
+ `custom_device:metal`. Paddle's official `PaddleCustomDevice` MPS backend exists,
259
+ but is currently a version-0.0.1 prototype with too small an operator surface to
260
+ establish PP-OCRv5 training.
261
+
262
+ The practical path is an Apache-2.0 vendored subset of
263
+ `frotms/PaddleOCR2Pytorch` pinned at
264
+ `c05307fbd61e575d55f0fef0334026c7536aec76`: PPLCNetV3 scale 0.95 plus the
265
+ SVTR/CTC branch of the PP-OCRv5 mobile recognizer. Dynamic pooling preserves the
266
+ recognition width (`T = input_width / 4`), and a localized contiguity fix before
267
+ the SVTR encoder's `conv3` makes the model safe for the tested MPS runtimes. The
268
+ model and log-softmax run on MPS; `CTCLoss` runs on CPU and autograd returns
269
+ finite gradients to MPS for the optimizer update. The NRTR auxiliary branch is
270
+ frozen and not part of this CTC loss, so this is not a complete reproduction of
271
+ PaddleOCR's multi-loss training recipe.
272
+
273
+ One-step evidence is green on both Macs. The end-to-end `mbp2` run used Python
274
+ 3.11.15 and Torch 2.13.0: `[4,3,48,160] -> [4,40,7]`, loss 29.883989, 582 finite
275
+ MPS gradient tensors, and a non-zero optimizer update across synthetic Latin,
276
+ Devanagari, and Limbu samples. Direct width-320 forward/backward/update checks
277
+ also passed on `mbp2` and on `k2` with Torch 2.12. These cold one-step timings
278
+ are diagnostics, not benchmarks.
279
+
280
+ The checkpoint boundary is strict: this backend writes PyTorch checkpoints,
281
+ not Paddle `.pdparams`; conversion requires explicit tensor mapping and frozen
282
+ inference-parity tests. `mbp2` and `k2` should run independent experiments with
283
+ distinct seeds and run directories, not multi-host MPS DDP. The remaining proof
284
+ is a multi-epoch real-manifest run with resume/reload, memory observation, and
285
+ canonical held-out CER. See `docs/PADDLEOCR_MPS.md` for exact smoke artifacts,
286
+ hashes, compatibility limits, and host guidance.
287
+
288
+ ## The Paddle Pipeline (whole pipeline, not just recognizers)
289
+
290
+ Target architecture, all stages Paddle-first and language-pack-parameterized by the
291
+ per-language pipeline config **`configs/nepal-language-pack-pipeline-v1.json`**
292
+ (50 languages: label, registry id, scripts, recognizer artifact, dictionary,
293
+ converter, text-layer class, harvest counts — generated 2026-07-04 from the
294
+ inventory):
295
+
296
+ 1. **Capture/prep** — existing `prepare-limbu-capture` path, already
297
+ language-agnostic in behavior; rename/rescope.
298
+ 2. **Detection** — Paddle text detection as production default; YOLO line detector and
299
+ morphology image-line boxes stay as diagnostics/rescue passes.
300
+ 3. **Script identification & routing** — generalize `_script_ratio` to a
301
+ Unicode-block-driven scorer over the pack's script list (Devanagari, Limbu,
302
+ Jenticha/Sunuwar, Lepcha, Kirat Rai, Newa, Tirhuta, Gurung Khema, Ol Chiki,
303
+ Tibetan, Latin), then route each line to the script's recognizer from the registry.
304
+ 4. **Recognition** — per-script PP-OCRv5 recognizers (existing runs) with per-script
305
+ dictionaries and input-width profiles (Sirijonga width-480 lesson).
306
+ 5. **Target-region & furniture filtering** — non-oracle region selection using layout
307
+ plus font/language signals; furniture fonts and masthead geometry as negative
308
+ evidence. This is the main open research item; score against the Rows LTR +
309
+ y/conf70 baseline.
310
+ 6. **Reading order & assembly** — `auto_layout` selection (wide-column RTL vs rows)
311
+ per page, conservative assembly stays diagnostic.
312
+ 7. **Export** — Unicode text, JSON with provenance, Markdown; review queues +
313
+ dashboards per page (existing Limbu artifact shapes, renamed pack-wide).
314
+ 8. **Eval guardrail** — frozen real-page packs per language: page CER/WER, line
315
+ P/R/F1, reading order, blank/extra/missed lines, review burden.
316
+
317
+ PP-StructureV3 and PaddleOCR-VL remain the documented fallback/teacher direction for
318
+ hard layouts, per `docs/NEPAL_MULTILINGUAL_OCR_GOAL.md`.
319
+
320
+ ## Next Work
321
+
322
+ 1. ~~Finish the 2026-07-04 full harvest and inventory~~ **Done 2026-07-04**: 394
323
+ language-page hits, 50 languages, inventory manifests at
324
+ `outputs/language-pack-inventory-full-2026-07-04/`.
325
+ 2. ~~Per-language silver packs via the existing `nayanepal` converter~~ **Done
326
+ 2026-07-04** (`tools/build_language_pack_silver_line_packs.py`, tests green): the
327
+ batch-1 run produced **116,287 kept silver lines across 50 languages, zero
328
+ errors**, with per-line provenance (PDF hash, page, bbox, raw legacy bytes,
329
+ converted text, image hash) at `outputs/language-pack-silver-lines-2026-07-04/`;
330
+ spot-checked conversions read as clean, language-plausible Devanagari. The
331
+ batch-2 run over the older 589 page hits kept **183,324 more lines (49 languages,
332
+ zero errors)** at `outputs/language-pack-silver-lines-batch2-2026-07-04/` —
333
+ **299,611 per-language-attributed silver lines total** across the two batches
334
+ (~2.4 GB of crops). Run the tool with
335
+ `uv run --python 3.11 --with pymupdf --with npttf2utf` — `nepal_ttf2utf` needs the
336
+ `npttf2utf` base-map package at runtime, and without it every span fails
337
+ (cleanly, with per-span error records, but kept=0). **Honest caveat**: lines
338
+ are page-attributed, not language-verified — a language page also carries Nepali
339
+ furniture/boxes, and pages whose body is not nayanepal Devanagari (Urdu, the
340
+ Tibetan-script pages) yield mostly furniture lines. A language-ID/lexicon filter
341
+ before per-language recognizer training is the follow-up gate.
342
+ 3. **Newar track (second focus language)**: add the Newar registry entry
343
+ (Devanagari + Prachalit rows); build the Devanagari-written Newar recognizer
344
+ eval/train split from the 6,800 harvested silver lines (by-date split, frozen
345
+ test); keep native Prachalit as the separate longer arc — real printed-register
346
+ Nepal Lipi harvesting, not more transliteration, is the lever.
347
+ 4. **Classify Tibetan-script pages** (Hyolmo/Mugal/Bhote) and decide converter vs
348
+ native Unicode.
349
+ 5. ~~Generalize the pipeline wiring~~ **Done 2026-07-04**: all three Limbu-locked
350
+ spots generalized (shared script table, filter keys, `--required-script`
351
+ preflight), tests green; `configs/nepal-language-pack-pipeline-v1.json` holds the
352
+ per-language wiring.
353
+ 6. **Target-region/furniture filtering stage** for Paddle page/book OCR, scored
354
+ against the current baseline on the 8-page Limbu pack and at least one
355
+ Devanagari-language pack (Newar is the natural candidate given the focus order).
356
+ 7. Only after page reconstruction improves, export high-quality real/silver line
357
+ crops back into recognizer training, per language.
358
+ 8. Month-batch the archive back-harvest toward the full ~1,750-article depth, storing
359
+ the mirror on the remote 2 TB volume.
360
+
361
+ ## Mistakes To Avoid
362
+
363
+ Do not use Naya Nepal blog images as training/eval evidence when the PDF exists. Do
364
+ not use wide epaper date windows — the endpoint silently caps at 30 PDFs per query;
365
+ weekly windows only. Do not treat unnormalized Gorkhapatra labels as distinct
366
+ languages (थारु vs थारू). Do not treat the `nayanepal` text layer as Unicode — it is
367
+ legacy Preeti-family and needs the converter before any silver label is real. Do not
368
+ report Paddle `norm_edit_dis` as CER; real CER comes from `tools/eval_metrics.py`
369
+ micro CER over canonical predictions. Do not treat synthetic data as claim evidence,
370
+ and label Gurung Khema results as synthetic-by-nature. Do not promote the
371
+ targeted+Unicode checkpoint or v3 dictionary warm-starts. Do not spend long blocks on
372
+ generic eval-harness work unless it directly improves the book/page OCR loop. Do not
373
+ frame video-to-page work as OCR scope; reduce video to stills first. Do not present
374
+ page OCR as solved: matched-line recognition is strong, but full-page CER is still
375
+ dominated by reconstruction and region selection.
ocr-tech-mps-integration/configs/cdjk-language-corpus-migration-v1.json ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "id": "cdjk-language-corpus-migration-v1",
3
+ "created_date": "2026-06-16",
4
+ "source_host": "cdjk@100.110.148.43",
5
+ "source_machine": "cdjks-MacBook-Pro.local",
6
+ "source_storage": {
7
+ "primary_external_volume": "/Volumes/TRANSCEND",
8
+ "primary_external_volume_size": "3.6Ti",
9
+ "primary_external_volume_available_at_inventory": "2.0Ti"
10
+ },
11
+ "destination_root": "data/external-language-resources/cdjk-100.110.148.43",
12
+ "copy_policy": {
13
+ "purpose": "migrate text corpora, dictionaries, lexicons, grammar notes, and manifests for OCR synthesis planning",
14
+ "include_suffixes": [".txt", ".tsv", ".csv", ".json", ".jsonl", ".md", ".yaml", ".yml"],
15
+ "excluded_trees": [".git", ".pytest_cache", ".playwright-mcp", ".ruff_cache", "__pycache__", "node_modules", ".venv", ".venvs"],
16
+ "excluded_suffixes": [
17
+ ".wav",
18
+ ".mp3",
19
+ ".flac",
20
+ ".m4a",
21
+ ".mp4",
22
+ ".mov",
23
+ ".png",
24
+ ".jpg",
25
+ ".jpeg",
26
+ ".pdf",
27
+ ".zip",
28
+ ".pt",
29
+ ".pth",
30
+ ".onnx",
31
+ ".safetensors",
32
+ ".bin",
33
+ ".ckpt"
34
+ ],
35
+ "notes": [
36
+ "The migration is intentionally selective because the local machine had about 33GiB free at copy time.",
37
+ "Generated data under data/ is ignored by git; this config records the reproducible migration policy."
38
+ ]
39
+ },
40
+ "source_roots": [
41
+ {
42
+ "label": "dhimal-toolkit",
43
+ "remote_path": "/Users/cdjk/github/llm/dhimal-toolkit",
44
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/dhimal-toolkit"
45
+ },
46
+ {
47
+ "label": "gurung-language-toolkit",
48
+ "remote_path": "/Users/cdjk/github/llm/gurung-language-toolkit",
49
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/gurung-language-toolkit"
50
+ },
51
+ {
52
+ "label": "limbu-speech-toolkit",
53
+ "remote_path": "/Users/cdjk/github/llm/limbu-speech-toolkit",
54
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit"
55
+ },
56
+ {
57
+ "label": "ltk",
58
+ "remote_path": "/Users/cdjk/github/llm/ltk",
59
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/ltk"
60
+ },
61
+ {
62
+ "label": "magar-toolkit",
63
+ "remote_path": "/Users/cdjk/github/llm/magar-toolkit",
64
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/magar-toolkit"
65
+ },
66
+ {
67
+ "label": "rai-speech-toolkit",
68
+ "remote_path": "/Users/cdjk/github/llm/rai-speech-toolkit",
69
+ "local_path": "data/external-language-resources/cdjk-100.110.148.43/rai-speech-toolkit"
70
+ }
71
+ ],
72
+ "migration_result": {
73
+ "transferred_files": 2307,
74
+ "total_files_after_copy": 2308,
75
+ "total_size_bytes": 1689175338,
76
+ "local_size_human": "1.6G",
77
+ "suffix_counts": {
78
+ "json": 1071,
79
+ "txt": 686,
80
+ "md": 209,
81
+ "jsonl": 179,
82
+ "tsv": 155,
83
+ "csv": 4,
84
+ "yml": 3
85
+ }
86
+ },
87
+ "audits": {
88
+ "resource_audit": "outputs/cdjk-language-resource-audit-2026-06-16/synthesis-resource-audit.json",
89
+ "limbu_text_audit": "outputs/cdjk-limbu-limdic-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
90
+ "limbu_only_text_audit": "outputs/cdjk-limbu-only-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
91
+ "tamang_text_audit": "outputs/cdjk-tamang-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
92
+ "magar_text_audit": "outputs/cdjk-magar-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
93
+ "gurung_text_audit": "outputs/cdjk-gurung-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
94
+ "dhimal_parallel_text_audit": "outputs/cdjk-dhimal-parallel-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
95
+ "dhimal_mono_text_audit": "outputs/cdjk-dhimal-mono-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
96
+ "magar_parallel_extra_text_audit": "outputs/cdjk-magar-parallel-extra-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
97
+ "magar_mono_extra_text_audit": "outputs/cdjk-magar-mono-extra-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
98
+ "rai_bantawa_text_audit": "outputs/cdjk-rai-bantawa-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
99
+ "rai_chamling_text_audit": "outputs/cdjk-rai-chamling-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
100
+ "rai_kulung_text_audit": "outputs/cdjk-rai-kulung-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
101
+ "rai_athpariya_text_audit": "outputs/cdjk-rai-athpariya-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
102
+ "rai_khaling_text_audit": "outputs/cdjk-rai-khaling-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
103
+ "rai_yakkha_text_audit": "outputs/cdjk-rai-yakkha-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
104
+ "rai_dumi_text_audit": "outputs/cdjk-rai-dumi-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
105
+ "rai_thulung_text_audit": "outputs/cdjk-rai-thulung-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
106
+ "rai_koyu_text_audit": "outputs/cdjk-rai-koyu-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
107
+ "rai_lohorung_text_audit": "outputs/cdjk-rai-lohorung-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
108
+ "rai_sunwar_text_audit": "outputs/cdjk-rai-sunwar-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
109
+ "rai_limbu_text_audit": "outputs/cdjk-rai-limbu-text-audit-2026-06-16/synthesis-text-manifest-audit.json",
110
+ "limbu_rendered_smoke_audit": "outputs/cdjk-limbu-rendered-smoke-audit-2026-06-16-limbu-only/rendered-line-manifest-audit.json"
111
+ },
112
+ "prepared_outputs": {
113
+ "limbu_limbu_only_text": "data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl",
114
+ "limbu_all_limdic_text": "data/synthesis-text/cdjk-limbu-limdic-2026-06-16/limbu-limdic-text.jsonl",
115
+ "tamang_text": "data/synthesis-text/cdjk-tamang-2026-06-16/tamang-text.jsonl",
116
+ "magar_text": "data/synthesis-text/cdjk-magar-2026-06-16/magar-text.jsonl",
117
+ "gurung_text": "data/synthesis-text/cdjk-gurung-2026-06-16/toolkit-parallel-text.jsonl",
118
+ "dhimal_parallel_text": "data/synthesis-text/cdjk-dhimal-parallel-2026-06-16/toolkit-parallel-text.jsonl",
119
+ "dhimal_mono_text": "data/synthesis-text/cdjk-dhimal-mono-2026-06-16/toolkit-parallel-text.jsonl",
120
+ "magar_parallel_extra_text": "data/synthesis-text/cdjk-magar-parallel-extra-2026-06-16/toolkit-parallel-text.jsonl",
121
+ "magar_mono_extra_text": "data/synthesis-text/cdjk-magar-mono-extra-2026-06-16/toolkit-parallel-text.jsonl",
122
+ "rai_bantawa_text": "data/synthesis-text/cdjk-rai-bantawa-2026-06-16/toolkit-parallel-text.jsonl",
123
+ "rai_chamling_text": "data/synthesis-text/cdjk-rai-chamling-2026-06-16/toolkit-parallel-text.jsonl",
124
+ "rai_kulung_text": "data/synthesis-text/cdjk-rai-kulung-2026-06-16/toolkit-parallel-text.jsonl",
125
+ "rai_athpariya_text": "data/synthesis-text/cdjk-rai-athpariya-2026-06-16/toolkit-parallel-text.jsonl",
126
+ "rai_khaling_text": "data/synthesis-text/cdjk-rai-khaling-2026-06-16/toolkit-parallel-text.jsonl",
127
+ "rai_yakkha_text": "data/synthesis-text/cdjk-rai-yakkha-2026-06-16/toolkit-parallel-text.jsonl",
128
+ "rai_dumi_text": "data/synthesis-text/cdjk-rai-dumi-2026-06-16/toolkit-parallel-text.jsonl",
129
+ "rai_thulung_text": "data/synthesis-text/cdjk-rai-thulung-2026-06-16/toolkit-parallel-text.jsonl",
130
+ "rai_koyu_text": "data/synthesis-text/cdjk-rai-koyu-2026-06-16/toolkit-parallel-text.jsonl",
131
+ "rai_lohorung_text": "data/synthesis-text/cdjk-rai-lohorung-2026-06-16/toolkit-parallel-text.jsonl",
132
+ "rai_sunwar_text": "data/synthesis-text/cdjk-rai-sunwar-2026-06-16/toolkit-parallel-text.jsonl",
133
+ "rai_limbu_text": "data/synthesis-text/cdjk-rai-limbu-2026-06-16/toolkit-parallel-text.jsonl",
134
+ "limbu_rendered_smoke_manifest": "data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/manifests/rendered-line-manifest.jsonl",
135
+ "limbu_rendered_smoke_labels": "data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/labels/train.txt"
136
+ },
137
+ "prepared_manifest_summary": {
138
+ "manifest_count": 20,
139
+ "total_samples": 228212,
140
+ "total_duplicates": 45258,
141
+ "script_counts": {
142
+ "Devanagari": 202560,
143
+ "Mixed_Devanagari_Latin": 10337,
144
+ "Latin": 6521,
145
+ "Limbu": 8772,
146
+ "Mixed_Limbu_Devanagari": 14,
147
+ "Mixed_Limbu_Latin": 3,
148
+ "Other": 5
149
+ }
150
+ },
151
+ "language_coverage_leads": [
152
+ "limbu",
153
+ "tamang",
154
+ "magar",
155
+ "gurung",
156
+ "dhimal",
157
+ "bantawa",
158
+ "chamling",
159
+ "athpariya",
160
+ "khaling",
161
+ "kulung",
162
+ "sampang",
163
+ "sunwar",
164
+ "thulung",
165
+ "lohorung",
166
+ "yakkha",
167
+ "dumi",
168
+ "koyu"
169
+ ]
170
+ }
ocr-tech-mps-integration/configs/claim-review-sota-v1.json ADDED
@@ -0,0 +1,121 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "require_shared_candidate_model": true,
3
+ "require_fallback_model_present": true,
4
+ "require_candidate_fallback_provenance": true,
5
+ "require_shared_fallback_model": true,
6
+ "require_model_audit_passed": true,
7
+ "require_fallback_model_audit_passed": true,
8
+ "require_trained_recognizer": true,
9
+ "require_model_admission": true,
10
+ "required_experiments": [
11
+ {
12
+ "name": "internal-nepali-english-document-pack",
13
+ "datasets": ["ocrtech-eval-pack"],
14
+ "require_validation_status": "validated",
15
+ "required_validation_config": "configs/validation-internal-sota-v1.json",
16
+ "require_baselines": ["candidate", "surya", "tesseract", "stock-paddle", "glm-ocr", "paddleocr-vl"],
17
+ "required_metrics": [
18
+ "cer",
19
+ "wer",
20
+ "document_reconstruction_similarity",
21
+ "table_cell_f1",
22
+ "reading_order_pair_accuracy",
23
+ "figure_detection_f1",
24
+ "figure_caption_cer"
25
+ ],
26
+ "required_metric_slices": {
27
+ "cer": ["nepali", "english"],
28
+ "wer": ["nepali", "english"],
29
+ "document_reconstruction_similarity": ["table", "reading_order", "figure"],
30
+ "table_cell_f1": ["table"],
31
+ "reading_order_pair_accuracy": ["reading_order"],
32
+ "figure_detection_f1": ["figure"],
33
+ "figure_caption_cer": ["figure"]
34
+ },
35
+ "required_output_artifacts": ["document_json", "document_md", "quality_json"],
36
+ "required_output_artifact_slices": {
37
+ "table_csv_html": ["table"],
38
+ "figure_metadata": ["figure"],
39
+ "figure_images": ["figure"]
40
+ },
41
+ "required_validation_gates": [
42
+ {"baseline": "tesseract", "metric": "cer", "slices": ["nepali"]},
43
+ {"baseline": "tesseract", "metric": "wer", "slices": ["nepali"]},
44
+ {"baseline": "stock-paddle", "metric": "cer", "slices": ["nepali"]},
45
+ {"baseline": "stock-paddle", "metric": "cer", "slices": ["english"]},
46
+ {"baseline": "stock-paddle", "metric": "wer", "slices": ["english"]},
47
+ {"baseline": "glm-ocr", "metric": "cer", "slices": ["nepali"]},
48
+ {"baseline": "surya", "metric": "table_cell_f1", "slices": ["table"]},
49
+ {"baseline": "surya", "metric": "figure_detection_f1", "slices": ["figure"]},
50
+ {"baseline": "surya", "metric": "figure_caption_cer", "slices": ["figure"]},
51
+ {"baseline": "paddleocr-vl", "metric": "reading_order_pair_accuracy", "slices": ["reading_order"]},
52
+ {"baseline": "paddleocr-vl", "metric": "document_reconstruction_similarity", "slices": ["figure"]}
53
+ ],
54
+ "require_candidate_fallback_metrics": true,
55
+ "max_candidate_fallback_trigger_rate": 0.35,
56
+ "max_candidate_fallback_trigger_rate_by_slice": {
57
+ "english": 0.10,
58
+ "nepali": 0.20
59
+ },
60
+ "max_candidate_fallback_failed_rate": 0.0,
61
+ "required_slices": ["nepali", "english", "table", "reading_order", "figure"]
62
+ },
63
+ {
64
+ "name": "public-omnidocbench",
65
+ "datasets": ["opendatalab/OmniDocBench"],
66
+ "require_validation_status": "validated",
67
+ "required_validation_config": "configs/validation-public-omnidocbench-sota-v1.json",
68
+ "require_baselines": ["candidate", "surya", "paddleocr-vl"],
69
+ "required_metrics": ["cer", "document_reconstruction_similarity", "table_cell_f1", "table_structure_similarity", "reading_order_pair_accuracy"],
70
+ "required_metric_slices": {
71
+ "table_cell_f1": ["document_parsing"],
72
+ "table_structure_similarity": ["document_parsing"],
73
+ "reading_order_pair_accuracy": ["document_parsing"],
74
+ "document_reconstruction_similarity": ["document_parsing"]
75
+ },
76
+ "required_output_artifacts": ["document_json", "document_md", "quality_json"],
77
+ "required_output_artifact_slices": {
78
+ "table_csv_html": ["table"]
79
+ },
80
+ "required_validation_gates": [
81
+ {"baseline": "surya", "metric": "cer", "slices": ["public", "document_parsing"]},
82
+ {"baseline": "surya", "metric": "table_cell_f1", "slices": ["public", "table"]},
83
+ {"baseline": "paddleocr-vl", "metric": "document_reconstruction_similarity", "slices": ["public", "document_parsing"]}
84
+ ],
85
+ "require_candidate_fallback_metrics": true,
86
+ "max_candidate_fallback_trigger_rate": 0.35,
87
+ "max_candidate_fallback_trigger_rate_by_slice": {
88
+ "public": 0.35
89
+ },
90
+ "max_candidate_fallback_failed_rate": 0.0,
91
+ "required_slices": ["public", "document_parsing"]
92
+ },
93
+ {
94
+ "name": "public-real5-omnidocbench-or-olmocr",
95
+ "eval_manifest_patterns": ["*Real5-OmniDocBench*", "*olmOCR-bench*"],
96
+ "require_validation_status": "validated",
97
+ "required_validation_config": "configs/validation-public-real5-sota-v1.json",
98
+ "require_baselines": ["candidate", "surya", "glm-ocr", "paddleocr-vl"],
99
+ "required_metrics": ["cer", "wer", "document_reconstruction_similarity", "reading_order_pair_accuracy"],
100
+ "required_metric_slices": {
101
+ "document_reconstruction_similarity": ["public"],
102
+ "reading_order_pair_accuracy": ["public"]
103
+ },
104
+ "required_output_artifacts": ["document_json", "document_md", "quality_json"],
105
+ "required_validation_gates": [
106
+ {"baseline": "surya", "metric": "cer", "slices": ["public"]},
107
+ {"baseline": "surya", "metric": "wer", "slices": ["public"]},
108
+ {"baseline": "glm-ocr", "metric": "cer", "slices": ["public"]},
109
+ {"baseline": "glm-ocr", "metric": "wer", "slices": ["public"]},
110
+ {"baseline": "paddleocr-vl", "metric": "document_reconstruction_similarity", "slices": ["public"]}
111
+ ],
112
+ "require_candidate_fallback_metrics": true,
113
+ "max_candidate_fallback_trigger_rate": 0.35,
114
+ "max_candidate_fallback_trigger_rate_by_slice": {
115
+ "public": 0.35
116
+ },
117
+ "max_candidate_fallback_failed_rate": 0.0,
118
+ "required_slices": ["public"]
119
+ }
120
+ ]
121
+ }
ocr-tech-mps-integration/configs/eval-harness-recipes-v1.json ADDED
@@ -0,0 +1,651 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "eval-harness-recipes-v1",
3
+ "updated": "2026-06-19",
4
+ "recipes": [
5
+ {
6
+ "id": "newa-zenodo-native-retrain-oracle-crop",
7
+ "task": "recognizer_oracle_crop",
8
+ "language": "newa",
9
+ "script": "Newa",
10
+ "model_family": "newa-native-paddleocr-remote",
11
+ "runner": {
12
+ "type": "external_command",
13
+ "command_template": "REPLACE_WITH_REMOTE_NEWA_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
14
+ "requires_operator_resolution": true,
15
+ "notes": "GPU checkpoint path is not checked into this repo; resolve from the remote native-retrain handoff before scoring."
16
+ },
17
+ "eval_pack": {
18
+ "label_file": "outputs/eval-packs/newa-zenodo-6967421-v1/eval.txt",
19
+ "data_dir": "outputs/eval-packs/newa-zenodo-6967421-v1",
20
+ "normalization_profile": "NFC_preserve_whitespace",
21
+ "manifest_file": "outputs/eval-packs/newa-zenodo-6967421-v1/manifests/recognizer-eval.jsonl",
22
+ "summary_file": "outputs/eval-packs/newa-zenodo-6967421-v1/summary.json",
23
+ "dataset": "newa-zenodo-6967421-prachalit-real",
24
+ "domain": "real_handwritten",
25
+ "gt_status": "machine_converted",
26
+ "split": "eval",
27
+ "eval_pack_version": "newa-zenodo-6967421-v1",
28
+ "frozen_eval_pack": true,
29
+ "unseen_for_selection": false,
30
+ "claim_eligible": false
31
+ },
32
+ "leakage_sources": [
33
+ {
34
+ "label_file": "data/newa-native-render/train.txt",
35
+ "data_dir": "data/newa-native-render",
36
+ "split": "train"
37
+ },
38
+ {
39
+ "label_file": "data/newa-native-render/eval.txt",
40
+ "data_dir": "data/newa-native-render",
41
+ "split": "dev"
42
+ }
43
+ ],
44
+ "claim_blockers": [
45
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/newa-zenodo-6967421-v1/audit/recognizer-eval-pack-audit.json",
46
+ "eval pack not declared unseen for model/checkpoint selection",
47
+ "GT is machine-converted and lacks native-reader calibration",
48
+ "row-level claim eligibility is false",
49
+ "remote checkpoint path must be resolved before reproducible scoring"
50
+ ]
51
+ },
52
+ {
53
+ "id": "sunuwar-jenticha-samefont-oracle-crop",
54
+ "task": "recognizer_oracle_crop",
55
+ "language": "sunuwar",
56
+ "script": "Sunuwar",
57
+ "model_family": "sunuwar-samefont-paddleocr-remote",
58
+ "runner": {
59
+ "type": "external_command",
60
+ "command_template": "REPLACE_WITH_REMOTE_SUNUWAR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
61
+ "requires_operator_resolution": true,
62
+ "notes": "Remote same-font checkpoint is referenced by handoff but not packaged locally."
63
+ },
64
+ "eval_pack": {
65
+ "label_file": "outputs/eval-packs/sunuwar-herald-real-v1/eval.txt",
66
+ "data_dir": "outputs/eval-packs/sunuwar-herald-real-v1",
67
+ "normalization_profile": "NFC_preserve_whitespace",
68
+ "manifest_file": "outputs/eval-packs/sunuwar-herald-real-v1/manifests/recognizer-eval.jsonl",
69
+ "summary_file": "outputs/eval-packs/sunuwar-herald-real-v1/summary.json",
70
+ "dataset": "sikkim-herald-sunuwar-real",
71
+ "domain": "real_print",
72
+ "gt_status": "machine_converted",
73
+ "split": "eval",
74
+ "eval_pack_version": "sunuwar-herald-real-v1",
75
+ "frozen_eval_pack": true,
76
+ "unseen_for_selection": true,
77
+ "claim_eligible": false
78
+ },
79
+ "leakage_sources": [
80
+ {
81
+ "label_file": "data/sunuwar-train-pack/train.txt",
82
+ "data_dir": "data/sunuwar-train-pack",
83
+ "split": "train"
84
+ },
85
+ {
86
+ "label_file": "data/sunuwar-train-pack/eval.txt",
87
+ "data_dir": "data/sunuwar-train-pack",
88
+ "split": "dev"
89
+ }
90
+ ],
91
+ "claim_blockers": [
92
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/sunuwar-herald-real-v1/audit/recognizer-eval-pack-audit.json",
93
+ "GT is machine-converted and lacks native-reader calibration",
94
+ "row-level claim eligibility is false",
95
+ "remote checkpoint path must be resolved before reproducible scoring"
96
+ ]
97
+ },
98
+ {
99
+ "id": "lepcha-herald-real-oracle-crop",
100
+ "task": "recognizer_oracle_crop",
101
+ "language": "lepcha",
102
+ "script": "Lepcha",
103
+ "model_family": "lepcha-paddleocr-unresolved",
104
+ "runner": {
105
+ "type": "external_command",
106
+ "command_template": "REPLACE_WITH_LEPCHA_PADDLEOCR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
107
+ "requires_operator_resolution": true,
108
+ "notes": "No packaged local Lepcha checkpoint is registered yet; resolve the exact recognizer and prediction command before scoring."
109
+ },
110
+ "eval_pack": {
111
+ "label_file": "outputs/eval-packs/lepcha-herald-real-v1/eval.txt",
112
+ "data_dir": "outputs/eval-packs/lepcha-herald-real-v1",
113
+ "normalization_profile": "NFC_preserve_whitespace",
114
+ "manifest_file": "outputs/eval-packs/lepcha-herald-real-v1/manifests/recognizer-eval.jsonl",
115
+ "summary_file": "outputs/eval-packs/lepcha-herald-real-v1/summary.json",
116
+ "dataset": "sikkim-herald-lepcha-real",
117
+ "domain": "real_print",
118
+ "gt_status": "machine_converted",
119
+ "split": "eval",
120
+ "eval_pack_version": "lepcha-herald-real-v1",
121
+ "frozen_eval_pack": true,
122
+ "unseen_for_selection": true,
123
+ "claim_eligible": false
124
+ },
125
+ "leakage_sources": [
126
+ {
127
+ "label_file": "data/synth-corpora/lepcha/train.txt",
128
+ "data_dir": "data/synth-corpora/lepcha",
129
+ "split": "train"
130
+ },
131
+ {
132
+ "label_file": "data/synth-corpora/lepcha/eval.txt",
133
+ "data_dir": "data/synth-corpora/lepcha",
134
+ "split": "dev"
135
+ }
136
+ ],
137
+ "claim_blockers": [
138
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/lepcha-herald-real-v1/audit/recognizer-eval-pack-audit.json",
139
+ "GT is machine-converted and lacks native-reader calibration",
140
+ "row-level claim eligibility is false",
141
+ "packaged local model card is absent",
142
+ "external prediction command must be resolved before reproducible scoring"
143
+ ]
144
+ },
145
+ {
146
+ "id": "kiratrai-herald-real-oracle-crop",
147
+ "task": "recognizer_oracle_crop",
148
+ "language": "kiratrai",
149
+ "script": "KiratRai",
150
+ "model_family": "kiratrai-paddleocr-unresolved",
151
+ "runner": {
152
+ "type": "external_command",
153
+ "command_template": "REPLACE_WITH_KIRATRAI_PADDLEOCR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
154
+ "requires_operator_resolution": true,
155
+ "notes": "No packaged local Kirat Rai checkpoint is registered yet; resolve the exact recognizer and prediction command before scoring."
156
+ },
157
+ "eval_pack": {
158
+ "label_file": "outputs/eval-packs/kiratrai-herald-real-v1/eval.txt",
159
+ "data_dir": "outputs/eval-packs/kiratrai-herald-real-v1",
160
+ "normalization_profile": "NFC_preserve_whitespace",
161
+ "summary_file": "outputs/eval-packs/kiratrai-herald-real-v1/summary.json",
162
+ "dataset": "sikkim-herald-kiratrai-real",
163
+ "domain": "real_print",
164
+ "gt_status": "machine_converted",
165
+ "split": "eval",
166
+ "eval_pack_version": "kiratrai-herald-real-v1",
167
+ "frozen_eval_pack": true,
168
+ "unseen_for_selection": true,
169
+ "claim_eligible": false,
170
+ "manifest_file": "outputs/eval-packs/kiratrai-herald-real-v1/manifests/recognizer-eval.jsonl"
171
+ },
172
+ "leakage_sources": [
173
+ {
174
+ "label_file": "data/kiratrai-train/real-renders/labels.txt",
175
+ "data_dir": "data/kiratrai-train/real-renders",
176
+ "split": "train"
177
+ },
178
+ {
179
+ "label_file": "data/synth-corpora/kiratrai/train.txt",
180
+ "data_dir": "data/synth-corpora/kiratrai",
181
+ "split": "train"
182
+ },
183
+ {
184
+ "label_file": "data/synth-corpora/kiratrai/eval.txt",
185
+ "data_dir": "data/synth-corpora/kiratrai",
186
+ "split": "dev"
187
+ }
188
+ ],
189
+ "claim_blockers": [
190
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/kiratrai-herald-real-v1/audit/recognizer-eval-pack-audit.json",
191
+ "GT is machine-converted and lacks native-reader calibration",
192
+ "row-level claim eligibility is false",
193
+ "packaged local model card is absent",
194
+ "external prediction command must be resolved before reproducible scoring"
195
+ ]
196
+ },
197
+ {
198
+ "id": "limbu-sirijonga-gorkhapatra-oracle-crop",
199
+ "task": "recognizer_oracle_crop",
200
+ "language": "limbu",
201
+ "script": "Limbu",
202
+ "model_family": "limbu-sirijonga-ppocrv5-mobile",
203
+ "runner": {
204
+ "type": "external_command",
205
+ "command_template": "REPLACE_WITH_LIMBU_PADDLEOCR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
206
+ "requires_operator_resolution": true,
207
+ "training_summary": "runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/recognizer-training-summary.json"
208
+ },
209
+ "eval_pack": {
210
+ "label_file": "outputs/eval-packs/limbu-gorkhapatra-real-v1/eval.txt",
211
+ "data_dir": "outputs/eval-packs/limbu-gorkhapatra-real-v1",
212
+ "normalization_profile": "NFC_preserve_whitespace",
213
+ "dataset": "gorkhapatra-limbu-real-crops",
214
+ "domain": "real_print",
215
+ "gt_status": "machine_converted",
216
+ "split": "eval",
217
+ "eval_pack_version": "limbu-gorkhapatra-real-v1",
218
+ "frozen_eval_pack": true,
219
+ "unseen_for_selection": false,
220
+ "claim_eligible": false,
221
+ "manifest_file": "outputs/eval-packs/limbu-gorkhapatra-real-v1/manifests/recognizer-eval.jsonl",
222
+ "summary_file": "outputs/eval-packs/limbu-gorkhapatra-real-v1/summary.json"
223
+ },
224
+ "leakage_sources": [
225
+ {
226
+ "label_file": "runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/labels/train.txt",
227
+ "data_dir": ".",
228
+ "split": "train"
229
+ },
230
+ {
231
+ "label_file": "runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/labels/eval.txt",
232
+ "data_dir": ".",
233
+ "split": "dev"
234
+ }
235
+ ],
236
+ "claim_blockers": [
237
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/limbu-gorkhapatra-real-v1/audit/recognizer-eval-pack-audit.json",
238
+ "eval pack not declared unseen for model/checkpoint selection",
239
+ "GT is machine-converted and lacks native-reader calibration",
240
+ "row-level claim eligibility is false",
241
+ "packaged local model card is absent"
242
+ ]
243
+ },
244
+ {
245
+ "id": "limbu-devanagari-cdc-ppocrv5-devclean-heldout",
246
+ "task": "recognizer_oracle_crop",
247
+ "language": "limbu",
248
+ "script": "Devanagari",
249
+ "model_family": "limbu-devanagari-cdc-ppocrv5-mobile",
250
+ "runner": {
251
+ "type": "external_command",
252
+ "command_template": "REPLACE_WITH_LIMBU_DEVANAGARI_PADDLEOCR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
253
+ "requires_operator_resolution": true,
254
+ "training_summary": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/training-recipes/ppocrv5-mobile-devclean/recognizer-training-summary.json",
255
+ "config": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/training-recipes/ppocrv5-mobile-devclean/paddle_recognizer.yml",
256
+ "dictionary": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/training-recipes/ppocrv5-mobile-devclean/nep_eng_dict.txt",
257
+ "training_bundle": "runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21/limbu-devanagari-cdc-ppocrv5-mobile-devclean-training-bundle.tar.gz",
258
+ "selected_checkpoint_on_test": false,
259
+ "notes": "Train from the dev-clean bundle, select/tune only on the 258-row dev split, then fill in checkpoint and prediction command for this sealed 31-row eval split."
260
+ },
261
+ "eval_pack": {
262
+ "label_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/eval.txt",
263
+ "data_dir": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21",
264
+ "normalization_profile": "NFC_preserve_whitespace",
265
+ "manifest_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/manifests/recognizer-eval.jsonl",
266
+ "summary_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/summaries/eval.summary.json",
267
+ "script_inventory_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/script-inventory.json",
268
+ "dataset": "limbu-devanagari-cdc-line-crop-corpus-draft",
269
+ "domain": "real_print",
270
+ "gt_status": "curriculum_text_unreviewed",
271
+ "split": "eval",
272
+ "eval_pack_version": "limbu-devanagari-cdc-line-crop-corpus-draft-v1",
273
+ "frozen_eval_pack": false,
274
+ "unseen_for_selection": true,
275
+ "claim_eligible": false
276
+ },
277
+ "leakage_sources": [
278
+ {
279
+ "manifest_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/train-manifest.jsonl",
280
+ "split": "train_dev"
281
+ }
282
+ ],
283
+ "claim_blockers": [
284
+ "trained checkpoint path and reproducible prediction command are unresolved until the PP-OCRv5 bundle is run",
285
+ "eval pack is not frozen",
286
+ "GT is unreviewed curriculum text and lacks human/native-reader calibration",
287
+ "row-level claim eligibility is false",
288
+ "packaged local model card is absent"
289
+ ]
290
+ },
291
+ {
292
+ "id": "devanagari-shared-gorkhapatra-oracle-crop",
293
+ "task": "recognizer_oracle_crop",
294
+ "language": "devanagari_shared",
295
+ "script": "Devanagari",
296
+ "model_family": "devanagari-ppocrv5-mobile",
297
+ "runner": {
298
+ "type": "external_command",
299
+ "command_template": "REPLACE_WITH_DEVANAGARI_PADDLEOCR_PREDICT_COMMAND --labels {label_file} --data {data_dir} --out {prediction_file}",
300
+ "requires_operator_resolution": true,
301
+ "training_summary": "runs/devanagari-ppocrv5-mobile-2026-06-17/recognizer-training-summary.json"
302
+ },
303
+ "eval_pack": {
304
+ "label_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/eval.txt",
305
+ "data_dir": "outputs/eval-packs/gorkhapatra-devanagari-real-v1",
306
+ "normalization_profile": "NFC_preserve_whitespace",
307
+ "summary_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/summary.json",
308
+ "dataset": "gorkhapatra-devanagari-real",
309
+ "domain": "real_print",
310
+ "gt_status": "machine_converted",
311
+ "split": "eval",
312
+ "eval_pack_version": "gorkhapatra-devanagari-real-v1",
313
+ "frozen_eval_pack": true,
314
+ "unseen_for_selection": false,
315
+ "claim_eligible": false,
316
+ "manifest_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/manifests/recognizer-eval.jsonl"
317
+ },
318
+ "leakage_sources": [
319
+ {
320
+ "label_file": "runs/devanagari-ppocrv5-mobile-2026-06-17/labels/train.txt",
321
+ "data_dir": ".",
322
+ "split": "train"
323
+ },
324
+ {
325
+ "label_file": "runs/devanagari-ppocrv5-mobile-2026-06-17/labels/eval.txt",
326
+ "data_dir": ".",
327
+ "split": "dev"
328
+ }
329
+ ],
330
+ "claim_blockers": [
331
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/gorkhapatra-devanagari-real-v1/audit/recognizer-eval-pack-audit.json",
332
+ "eval pack not declared unseen for model/checkpoint selection",
333
+ "GT is machine-converted and lacks native-reader calibration",
334
+ "row-level claim eligibility is false",
335
+ "shared recognizer still needs per-language slice reporting",
336
+ "packaged local model card is absent"
337
+ ]
338
+ },
339
+ {
340
+ "id": "hf-recognizer-next-v1-model-card-smoke",
341
+ "task": "recognizer_oracle_crop",
342
+ "language": "nepali_mixed",
343
+ "script": "Devanagari",
344
+ "model_family": "hf-recognizer-next-v1",
345
+ "runner": {
346
+ "type": "model_config",
347
+ "model_config": "outputs/aws-g6-hf-recognizer-next-v1/package/model-card-cpu.json",
348
+ "training_summary": "outputs/aws-g6-hf-recognizer-next-v1/runs-hf-recognizer-next-v1/recognizer-training-summary.json"
349
+ },
350
+ "eval_pack": {
351
+ "label_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/eval.txt",
352
+ "data_dir": "outputs/eval-packs/gorkhapatra-devanagari-real-v1",
353
+ "normalization_profile": "NFC_preserve_whitespace",
354
+ "summary_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/summary.json",
355
+ "dataset": "gorkhapatra-devanagari-real",
356
+ "domain": "real_print",
357
+ "gt_status": "machine_converted",
358
+ "split": "eval",
359
+ "eval_pack_version": "gorkhapatra-devanagari-real-v1",
360
+ "frozen_eval_pack": true,
361
+ "unseen_for_selection": false,
362
+ "claim_eligible": false,
363
+ "manifest_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/manifests/recognizer-eval.jsonl"
364
+ },
365
+ "leakage_sources": [
366
+ {
367
+ "manifest_file": "data/splits-recognizer-next-v1/train.jsonl",
368
+ "split": "train"
369
+ },
370
+ {
371
+ "manifest_file": "data/splits-recognizer-next-v1/eval.jsonl",
372
+ "split": "dev"
373
+ }
374
+ ],
375
+ "claim_blockers": [
376
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/gorkhapatra-devanagari-real-v1/audit/recognizer-eval-pack-audit.json",
377
+ "model is generic mixed recognizer, not a per-language claim model",
378
+ "eval pack not declared unseen for model/checkpoint selection",
379
+ "GT is machine-converted and lacks native-reader calibration"
380
+ ]
381
+ },
382
+ {
383
+ "id": "paddle-recognizer-next-v1-model-card-smoke",
384
+ "task": "recognizer_oracle_crop",
385
+ "language": "nepali_mixed",
386
+ "script": "Devanagari",
387
+ "model_family": "paddle-recognizer-next-v1",
388
+ "runner": {
389
+ "type": "model_config",
390
+ "model_config": "outputs/aws-g6-paddle-recognizer-next-v1/slim/runs/paddle-recognizer-next-v1/package/model-card.json",
391
+ "training_summary": "outputs/aws-g6-paddle-recognizer-next-v1/slim/runs/paddle-recognizer-next-v1/recognizer-training-summary.json"
392
+ },
393
+ "eval_pack": {
394
+ "label_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/eval.txt",
395
+ "data_dir": "outputs/eval-packs/gorkhapatra-devanagari-real-v1",
396
+ "normalization_profile": "NFC_preserve_whitespace",
397
+ "summary_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/summary.json",
398
+ "dataset": "gorkhapatra-devanagari-real",
399
+ "domain": "real_print",
400
+ "gt_status": "machine_converted",
401
+ "split": "eval",
402
+ "eval_pack_version": "gorkhapatra-devanagari-real-v1",
403
+ "frozen_eval_pack": true,
404
+ "unseen_for_selection": false,
405
+ "claim_eligible": false,
406
+ "manifest_file": "outputs/eval-packs/gorkhapatra-devanagari-real-v1/manifests/recognizer-eval.jsonl"
407
+ },
408
+ "leakage_sources": [
409
+ {
410
+ "manifest_file": "data/splits-recognizer-next-v1/train.jsonl",
411
+ "split": "train"
412
+ },
413
+ {
414
+ "manifest_file": "data/splits-recognizer-next-v1/eval.jsonl",
415
+ "split": "dev"
416
+ }
417
+ ],
418
+ "claim_blockers": [
419
+ "canonical eval pack audit is not claim-ready; see outputs/eval-packs/gorkhapatra-devanagari-real-v1/audit/recognizer-eval-pack-audit.json",
420
+ "model is generic mixed recognizer, not a per-language claim model",
421
+ "eval pack not declared unseen for model/checkpoint selection",
422
+ "GT is machine-converted and lacks native-reader calibration"
423
+ ]
424
+ },
425
+ {
426
+ "id": "limbu-sirijonga-gorkhapatra-page-current-router",
427
+ "task": "page_level_ocr",
428
+ "language": "limbu",
429
+ "script": "Limbu",
430
+ "model_family": "limbu-first-router-v1",
431
+ "runner": {
432
+ "type": "external_command",
433
+ "command_template": "uv run python tools/copy_page_ocr_document.py --source-documents-dir outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/operational-image-line-width480-minwidth005-conf070-documents --sample-id {sample_id} --input-path {input_path} --out {output_dir} --document-json {document_json} --force",
434
+ "requires_operator_resolution": false,
435
+ "model_name": "operational-image-line-width480-minwidth005-conf070",
436
+ "checkpoint": "outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/artifacts/inference.pdiparams",
437
+ "config": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/recognizer-width-ablation-v1/model-cards/input-width-480/model-card.json",
438
+ "dictionary": "outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/artifacts/limbu-sirijonga-v2.txt",
439
+ "selected_checkpoint_on_test": false,
440
+ "notes": "Replays the current operational Sirijonga page OCR documents into the recipe runner. This makes the page gate reproducible from local artifacts, but it is not fresh OCR inference."
441
+ },
442
+ "eval_pack": {
443
+ "eval_manifest": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/manifests/eval-current-schema-gorkhapatra-sirijonga.jsonl",
444
+ "ocr_output_kind": "sidecar_fixture",
445
+ "normalization_profile": "NFC_preserve_whitespace",
446
+ "summary_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/summary-current-schema-input.json",
447
+ "script_inventory_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/script-inventory.json",
448
+ "dataset": "gorkhapatra-limbu-sirijonga-page-draft",
449
+ "domain": "real_print",
450
+ "gt_status": "machine_converted_unreviewed",
451
+ "split": "eval",
452
+ "eval_pack_version": "gorkhapatra-limbu-sirijonga-page-draft-v1-current-schema-input",
453
+ "frozen_eval_pack": false,
454
+ "unseen_for_selection": true,
455
+ "claim_eligible": false
456
+ },
457
+ "leakage_sources": [
458
+ {
459
+ "manifest_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/train-dev-empty.jsonl",
460
+ "split": "train_dev_placeholder"
461
+ }
462
+ ],
463
+ "claim_blockers": [
464
+ "page reference is machine-converted and unreviewed",
465
+ "page eval pack is not frozen",
466
+ "row-level claim eligibility is false",
467
+ "current full-page OCR has unresolved review rows and weak detector/layout recall",
468
+ "recipe replays pre-materialized operational OCR documents rather than running fresh page OCR"
469
+ ]
470
+ },
471
+ {
472
+ "id": "limbu-sirijonga-gorkhapatra-book-conf070-coverage-gated",
473
+ "task": "page_level_ocr",
474
+ "language": "limbu",
475
+ "script": "Limbu",
476
+ "model_family": "limbu-sirijonga-gorkhapatra-conf070-book-replay",
477
+ "runner": {
478
+ "type": "external_command",
479
+ "command_template": "uv run python tools/score_limbu_book_ocr_candidate.py --book-output outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-replay --book-audit outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-replay/limbu-book-output-audit.json --documents-dir outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-replay/score-documents --eval-manifest outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-sidecar-replay-inputs/eval-replay.jsonl --ocr-output-kind sidecar_fixture --summary-file outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/summary-current-schema-input.json --script-inventory-file outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/script-inventory.json --train-manifest outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/train-dev-empty.jsonl --system-name limbu-sirijonga-gorkhapatra-conf070-book-replay --model-name operational-image-line-width480-minwidth005-conf070 --checkpoint outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/artifacts/inference.pdiparams --config outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/recognizer-width-ablation-v1/model-cards/input-width-480/model-card.json --dictionary outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/artifacts/limbu-sirijonga-v2.txt --language limbu --script Limbu --domain real_print --gt-status machine_converted_unreviewed --eval-pack-version gorkhapatra-limbu-sirijonga-page-draft-v1-current-schema-input --no-frozen-eval-pack --unseen-for-selection --no-selected-checkpoint-on-test --no-claim-eligible --line-iou-threshold 0.5 --review-sample-size 50 --champion-metrics outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/score-gorkhapatra-operational-conf070-review-application-pending/metrics.json --review-coverage-report outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-missed-line-review-coverage-with-missed-refs/missed-line-review-coverage.json --baseline-codepoint-cer 0.7393608074011775 --baseline-grapheme-cer 0.7682623819778035 --baseline-wer 0.9189686924493554 --min-codepoint-improvement 0 --max-grapheme-regression 0 --max-wer-regression 0 --max-champion-codepoint-regression 0 --max-champion-grapheme-regression 0 --max-champion-wer-regression 0 --max-review-uncovered-or-weak-count 0 --max-review-extra-uncovered-or-weak-count 0 --min-line-f1 0.87 --min-reading-order-accuracy 0.77 --max-blank-line-rate 0 --max-extra-line-rate 0.02 --max-missed-line-rate 0.23 --min-scored-pages 1 --min-claim-pages 30 --out {output_dir} # input {input_path}",
480
+ "requires_operator_resolution": false,
481
+ "notes": "Sirijonga book-page replay of the current operational conf070 OCR document. The command scores the materialized book projection and gates it against the selected Sirijonga diagnostic plus unresolved-line coverage."
482
+ },
483
+ "eval_pack": {
484
+ "eval_manifest": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-sidecar-replay-inputs/eval-replay.jsonl",
485
+ "ocr_output_kind": "sidecar_fixture",
486
+ "normalization_profile": "NFC_preserve_whitespace",
487
+ "summary_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/summary-current-schema-input.json",
488
+ "script_inventory_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/script-inventory.json",
489
+ "dataset": "gorkhapatra-limbu-sirijonga-page-draft",
490
+ "domain": "real_print",
491
+ "gt_status": "machine_converted_unreviewed",
492
+ "split": "eval",
493
+ "eval_pack_version": "gorkhapatra-limbu-sirijonga-page-draft-v1-current-schema-input",
494
+ "frozen_eval_pack": false,
495
+ "unseen_for_selection": true,
496
+ "claim_eligible": false
497
+ },
498
+ "evidence": {
499
+ "book_output": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-replay",
500
+ "score_admission": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/score-admit-gorkhapatra-operational-conf070-book-replay-coverage-gated/limbu-book-ocr-score-admission.json",
501
+ "admission_report": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/score-admit-gorkhapatra-operational-conf070-book-replay-coverage-gated/admission/limbu-book-ocr-candidate-admission.json",
502
+ "metrics": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/score-admit-gorkhapatra-operational-conf070-book-replay-coverage-gated/score/metrics.json",
503
+ "review_coverage_report": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-missed-line-review-coverage-with-missed-refs/missed-line-review-coverage.json",
504
+ "unresolved_review_packet": "outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-operational-conf070-book-unresolved-review-packet-with-missed-refs/book-unresolved-review-packet.tsv"
505
+ },
506
+ "leakage_sources": [
507
+ {
508
+ "manifest_file": "outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/train-dev-empty.jsonl",
509
+ "split": "train_dev_placeholder"
510
+ }
511
+ ],
512
+ "claim_blockers": [
513
+ "page reference is machine-converted and unreviewed",
514
+ "page eval pack is not frozen",
515
+ "row-level claim eligibility is false",
516
+ "claim_safe=false because the score has claim_eligible_rows=0 and only 1 scored page",
517
+ "79 unresolved review packet rows still require reviewed correction or safe automatic recovery",
518
+ "Sirijonga candidate is operationally admitted only for the current 1-page Gorkhapatra draft set"
519
+ ]
520
+ },
521
+ {
522
+ "id": "limbu-devanagari-cdc-page-live-tesseract",
523
+ "task": "page_level_ocr",
524
+ "language": "limbu",
525
+ "script": "Devanagari",
526
+ "model_family": "limbu-devanagari-cdc-live-page-baseline",
527
+ "runner": {
528
+ "type": "external_command",
529
+ "command_template": "REPLACE_WITH_LIMBU_DEVANAGARI_PAGE_OCR_COMMAND --input {input_path} --out {output_dir}",
530
+ "requires_operator_resolution": true,
531
+ "notes": "Tracks the CDC Devanagari-written Limbu full-page draft gate; Tesseract is a failed baseline until the trained recognizer is admitted.",
532
+ "output_audit": {
533
+ "command_template": "uv run ocrtech audit-limbu-output {output_dir} --require-capture-prep --min-capture-prepared-width 1000 --min-capture-prepared-height 1000 --min-capture-prepared-entropy 0.5 --min-capture-prepared-luminance-stddev 1.0 --min-capture-prepared-edge-stddev 1.0 --min-line-count 30 --min-average-line-confidence 0.80 --min-quality-score 0.85 --require-no-pending-review --require-no-dropped-image-lines --require-script devanagari_limbu",
534
+ "required_flags": [
535
+ "--require-capture-prep",
536
+ "--min-capture-prepared-width",
537
+ "--min-capture-prepared-height",
538
+ "--min-capture-prepared-entropy",
539
+ "--min-capture-prepared-luminance-stddev",
540
+ "--min-capture-prepared-edge-stddev",
541
+ "--min-line-count",
542
+ "--min-average-line-confidence",
543
+ "--min-quality-score",
544
+ "--require-no-pending-review",
545
+ "--require-no-dropped-image-lines"
546
+ ],
547
+ "required_scripts": [
548
+ "devanagari_limbu"
549
+ ]
550
+ }
551
+ },
552
+ "eval_pack": {
553
+ "eval_manifest": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/manifests/eval.jsonl",
554
+ "ocr_output_kind": "live_detector",
555
+ "normalization_profile": "NFC_preserve_whitespace",
556
+ "summary_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/summary.json",
557
+ "script_inventory_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/script-inventory.json",
558
+ "dataset": "limbu-devanagari-cdc-curriculum-page-draft",
559
+ "domain": "real_print",
560
+ "gt_status": "curriculum_text_unreviewed",
561
+ "split": "eval",
562
+ "eval_pack_version": "limbu-devanagari-cdc-curriculum-linebox-draft-v1",
563
+ "frozen_eval_pack": false,
564
+ "unseen_for_selection": true,
565
+ "claim_eligible": false
566
+ },
567
+ "leakage_sources": [
568
+ {
569
+ "manifest_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/train-dev-empty.jsonl",
570
+ "split": "train_dev_placeholder"
571
+ }
572
+ ],
573
+ "claim_blockers": [
574
+ "CDC page references are unreviewed curriculum/PDF-aligned draft evidence",
575
+ "page eval pack is not frozen",
576
+ "row-level claim eligibility is false",
577
+ "Tesseract baseline is not an adequate Devanagari-written Limbu primary",
578
+ "reproducible page OCR command must be resolved after recognizer admission"
579
+ ]
580
+ },
581
+ {
582
+ "id": "limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated",
583
+ "task": "page_level_ocr",
584
+ "language": "limbu",
585
+ "script": "Devanagari",
586
+ "model_family": "limbu-devanagari-cdc-ppocrv5-page-marker-v2",
587
+ "runner": {
588
+ "type": "external_command",
589
+ "command_template": "uv run python tools/score_limbu_book_ocr_candidate.py --book-output outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application --book-audit outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application/candidate-line-application-audit.json --champion-metrics outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2/score/metrics.json --review-coverage-report outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/missed-line-review-coverage.json --out {output_dir} --documents-dir {documents_dir} # input {input_path}",
590
+ "requires_operator_resolution": false,
591
+ "notes": "Current Devanagari-written Limbu book-page candidate. The command scores the materialized book projection and applies old-baseline, champion non-regression, and unresolved-line coverage gates.",
592
+ "output_audit": {
593
+ "command_template": "uv run python tools/audit_candidate_line_review_application.py --application-dir {output_dir}",
594
+ "required_flags": [
595
+ "--require-capture-prep",
596
+ "--min-capture-prepared-width",
597
+ "--min-capture-prepared-height",
598
+ "--min-capture-prepared-entropy",
599
+ "--min-capture-prepared-luminance-stddev",
600
+ "--min-capture-prepared-edge-stddev",
601
+ "--min-line-count",
602
+ "--min-average-line-confidence",
603
+ "--min-quality-score",
604
+ "--require-no-pending-review",
605
+ "--require-no-dropped-image-lines"
606
+ ],
607
+ "required_scripts": [
608
+ "devanagari_limbu"
609
+ ]
610
+ }
611
+ },
612
+ "eval_pack": {
613
+ "eval_manifest": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-replay-inputs/eval-replay.jsonl",
614
+ "ocr_output_kind": "sidecar_fixture",
615
+ "normalization_profile": "NFC_preserve_whitespace",
616
+ "summary_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/summary.json",
617
+ "script_inventory_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/script-inventory.json",
618
+ "dataset": "limbu-devanagari-cdc-curriculum-page-draft",
619
+ "domain": "real_print",
620
+ "gt_status": "curriculum_text_unreviewed",
621
+ "split": "eval",
622
+ "eval_pack_version": "limbu-devanagari-cdc-curriculum-linebox-draft-v1",
623
+ "frozen_eval_pack": false,
624
+ "unseen_for_selection": true,
625
+ "claim_eligible": false
626
+ },
627
+ "evidence": {
628
+ "book_output": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application",
629
+ "score_admission": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/limbu-book-ocr-score-admission.json",
630
+ "admission_report": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/admission/limbu-book-ocr-candidate-admission.json",
631
+ "metrics": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/score/metrics.json",
632
+ "review_coverage_report": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/missed-line-review-coverage.json",
633
+ "unresolved_review_packet": "outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-extra-predictions/book-unresolved-review-packet.tsv"
634
+ },
635
+ "leakage_sources": [
636
+ {
637
+ "manifest_file": "outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/train-dev-empty.jsonl",
638
+ "split": "train_dev_placeholder"
639
+ }
640
+ ],
641
+ "claim_blockers": [
642
+ "CDC page references are unreviewed curriculum/PDF-aligned draft evidence",
643
+ "page eval pack is not frozen",
644
+ "row-level claim eligibility is false",
645
+ "claim_safe=false because the score has claim_eligible_rows=0 and only 3 scored pages",
646
+ "18 unresolved review rows still require reviewed correction or safe automatic recovery",
647
+ "Devanagari candidate is operationally admitted only for the current 3-page CDC draft set"
648
+ ]
649
+ }
650
+ ]
651
+ }
ocr-tech-mps-integration/configs/eval-target-coverage-v1.json ADDED
@@ -0,0 +1,193 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "eval-target-coverage-v1",
3
+ "updated": "2026-06-22",
4
+ "scope_note": "Tracks intended Nepal/diaspora OCR evaluation coverage. Claim readiness is determined by linked eval-pack normalization recipes, eval run recipes, and their blockers; target presence alone is not evidence.",
5
+ "target_sets": [
6
+ {
7
+ "id": "limbu-first-two-script-ocr",
8
+ "description": "Required coverage for the Limbu-first OCR pipeline: Sirijonga-script Limbu and Devanagari-written Limbu must both be represented by claim-ready eval harness entries before any two-script Limbu OCR claim.",
9
+ "required_for_claim": true,
10
+ "requires_page_eval_recipe": true,
11
+ "requires_book_candidate_evidence": true,
12
+ "targets": [
13
+ {
14
+ "language": "limbu",
15
+ "display_name": "Limbu/Yakthung",
16
+ "script": "Limbu",
17
+ "first_script_target": "Limbu/Sirijonga",
18
+ "eval_status": "registered_real_pack",
19
+ "normalization_pack_id": "limbu-gorkhapatra-real-v1",
20
+ "eval_recipe_id": "limbu-sirijonga-gorkhapatra-oracle-crop",
21
+ "page_eval_recipe_id": "limbu-sirijonga-gorkhapatra-book-conf070-coverage-gated",
22
+ "notes": "Tracks the current real Sirijonga recognizer pack and the operationally admitted one-page book replay. Still blocked by machine-converted GT, row claim eligibility, one-page sample size, unseen-selection status, and missing packaged model-card evidence."
23
+ },
24
+ {
25
+ "language": "limbu",
26
+ "display_name": "Limbu/Yakthung",
27
+ "script": "Devanagari",
28
+ "first_script_target": "Devanagari-written Limbu",
29
+ "eval_status": "registered_real_pack",
30
+ "normalization_pack_id": "limbu-devanagari-cdc-line-crop-corpus-draft-v1",
31
+ "eval_recipe_id": "limbu-devanagari-cdc-ppocrv5-devclean-heldout",
32
+ "page_eval_recipe_id": "limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated",
33
+ "notes": "Tracks the current Devanagari-written Limbu CDC book-page candidate: page-marker-v2 with champion non-regression and unresolved-line coverage admission. It is still not claim-ready until the pack is frozen and text/line references are reviewed."
34
+ }
35
+ ]
36
+ },
37
+ {
38
+ "id": "first-ten-document-ocr",
39
+ "description": "First ten language targets from docs/NEPAL_MULTILINGUAL_OCR_GOAL.md.",
40
+ "required_for_claim": false,
41
+ "targets": [
42
+ {
43
+ "language": "limbu",
44
+ "display_name": "Limbu/Yakthung",
45
+ "script": "Limbu",
46
+ "first_script_target": "Limbu/Sirijonga",
47
+ "eval_status": "partial_real_recognizer_pack",
48
+ "normalization_pack_id": "limbu-gorkhapatra-real-v1",
49
+ "eval_recipe_id": "limbu-sirijonga-gorkhapatra-oracle-crop",
50
+ "notes": "First non-Devanagari target; current pack lacks row provenance and native-reader calibration."
51
+ },
52
+ {
53
+ "language": "nepali",
54
+ "display_name": "Nepali",
55
+ "script": "Devanagari",
56
+ "first_script_target": "Devanagari",
57
+ "eval_status": "shared_devanagari_pack",
58
+ "normalization_pack_id": "gorkhapatra-devanagari-real-v1",
59
+ "eval_recipe_id": "devanagari-shared-gorkhapatra-oracle-crop",
60
+ "notes": "Core baseline language; current shared pack still needs per-language row slices for a Nepali-only claim."
61
+ },
62
+ {
63
+ "language": "english",
64
+ "display_name": "English",
65
+ "script": "Latin",
66
+ "first_script_target": "Latin",
67
+ "eval_status": "missing_real_eval_pack",
68
+ "notes": "Must not regress for mixed documents, but no claim-grade recognizer eval pack is registered here."
69
+ },
70
+ {
71
+ "language": "tamang",
72
+ "display_name": "Tamang",
73
+ "script": "Devanagari",
74
+ "first_script_target": "Devanagari first; Tamyig audit later",
75
+ "eval_status": "missing_real_eval_pack",
76
+ "notes": "Local corpus exists per project docs; no registered real OCR eval pack yet."
77
+ },
78
+ {
79
+ "language": "magar",
80
+ "display_name": "Magar",
81
+ "script": "Devanagari",
82
+ "first_script_target": "Devanagari first",
83
+ "eval_status": "missing_real_eval_pack",
84
+ "notes": "Gorkhapatra item confirmed in project docs; no registered real OCR eval pack yet."
85
+ },
86
+ {
87
+ "language": "wambule",
88
+ "display_name": "Wambule",
89
+ "script": "Devanagari",
90
+ "first_script_target": "Devanagari first",
91
+ "eval_status": "missing_real_eval_pack",
92
+ "notes": "Gorkhapatra item confirmed in project docs; no registered real OCR eval pack yet."
93
+ },
94
+ {
95
+ "language": "bantawa_rai",
96
+ "display_name": "Bantawa/Rai",
97
+ "script": "Devanagari",
98
+ "first_script_target": "Devanagari or Kirat Rai depending source",
99
+ "eval_status": "missing_real_eval_pack",
100
+ "notes": "Rai-family target; Kirat Rai native-script pack is tracked separately as current real-pack coverage."
101
+ },
102
+ {
103
+ "language": "kulung",
104
+ "display_name": "Kulung",
105
+ "script": "Devanagari",
106
+ "first_script_target": "Devanagari first",
107
+ "eval_status": "missing_real_eval_pack",
108
+ "notes": "Gorkhapatra item confirmed in project docs; no registered real OCR eval pack yet."
109
+ },
110
+ {
111
+ "language": "yakkha",
112
+ "display_name": "Yakkha",
113
+ "script": "Devanagari",
114
+ "first_script_target": "Devanagari first",
115
+ "eval_status": "missing_real_eval_pack",
116
+ "notes": "Gorkhapatra item confirmed in project docs; no registered real OCR eval pack yet."
117
+ },
118
+ {
119
+ "language": "tharu_rana",
120
+ "display_name": "Tharu/Rana Tharu",
121
+ "script": "Devanagari",
122
+ "first_script_target": "Devanagari first",
123
+ "eval_status": "missing_real_eval_pack",
124
+ "notes": "Multiple varieties; labels must remain explicit before any claim."
125
+ }
126
+ ]
127
+ },
128
+ {
129
+ "id": "current-real-recognizer-packs",
130
+ "description": "Current real recognizer eval packs registered for the harness.",
131
+ "required_for_claim": true,
132
+ "targets": [
133
+ {
134
+ "language": "newa",
135
+ "display_name": "Newa / Nepal Bhasa",
136
+ "script": "Newa",
137
+ "first_script_target": "Prachalit/Newa",
138
+ "eval_status": "registered_real_pack",
139
+ "normalization_pack_id": "newa-zenodo-6967421-v1",
140
+ "eval_recipe_id": "newa-zenodo-native-retrain-oracle-crop"
141
+ },
142
+ {
143
+ "language": "sunuwar",
144
+ "display_name": "Sunuwar / Jenticha",
145
+ "script": "Sunuwar",
146
+ "first_script_target": "Jenticha/Sunuwar",
147
+ "eval_status": "registered_real_pack",
148
+ "normalization_pack_id": "sunuwar-herald-real-v1",
149
+ "eval_recipe_id": "sunuwar-jenticha-samefont-oracle-crop"
150
+ },
151
+ {
152
+ "language": "lepcha",
153
+ "display_name": "Lepcha",
154
+ "script": "Lepcha",
155
+ "first_script_target": "Lepcha",
156
+ "eval_status": "registered_real_pack",
157
+ "normalization_pack_id": "lepcha-herald-real-v1",
158
+ "eval_recipe_id": "lepcha-herald-real-oracle-crop",
159
+ "notes": "Eval recipe exists, but model command, GT trust, and row claim eligibility blockers remain."
160
+ },
161
+ {
162
+ "language": "limbu",
163
+ "display_name": "Limbu/Yakthung",
164
+ "script": "Limbu",
165
+ "first_script_target": "Limbu/Sirijonga",
166
+ "eval_status": "registered_real_pack",
167
+ "normalization_pack_id": "limbu-gorkhapatra-real-v1",
168
+ "eval_recipe_id": "limbu-sirijonga-gorkhapatra-oracle-crop"
169
+ },
170
+ {
171
+ "language": "kiratrai",
172
+ "display_name": "Kirat Rai",
173
+ "script": "KiratRai",
174
+ "first_script_target": "Kirat Rai",
175
+ "eval_status": "registered_real_pack",
176
+ "normalization_pack_id": "kiratrai-herald-real-v1",
177
+ "eval_recipe_id": "kiratrai-herald-real-oracle-crop",
178
+ "notes": "Eval recipe exists, but model command, GT trust, provenance, and row claim eligibility blockers remain."
179
+ },
180
+ {
181
+ "language": "devanagari_shared",
182
+ "display_name": "Shared Devanagari Gorkhapatra",
183
+ "script": "Devanagari",
184
+ "first_script_target": "Shared Devanagari",
185
+ "eval_status": "registered_real_pack",
186
+ "normalization_pack_id": "gorkhapatra-devanagari-real-v1",
187
+ "eval_recipe_id": "devanagari-shared-gorkhapatra-oracle-crop",
188
+ "notes": "Shared pack must still be sliced into explicit language rows before supporting individual language claims."
189
+ }
190
+ ]
191
+ }
192
+ ]
193
+ }
ocr-tech-mps-integration/configs/limbu-devanagari-ppocrv5-admission-plan-v1.json ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "limbu-devanagari-ppocrv5-admission-plan-v1",
3
+ "updated": "2026-06-21",
4
+ "plans": [
5
+ {
6
+ "id": "limbu-devanagari-cdc-ppocrv5-devclean-admission",
7
+ "language": "limbu",
8
+ "script": "Devanagari",
9
+ "model_family": "limbu-devanagari-cdc-ppocrv5-mobile",
10
+ "purpose": "Train a same-register Devanagari-written Limbu recognizer from real CDC line crops, select checkpoints only on the dev split, and reserve the 31-row eval split for a single held-out admission score after checkpoint selection.",
11
+ "training_bundle": {
12
+ "archive": "runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21/limbu-devanagari-cdc-ppocrv5-mobile-devclean-training-bundle.tar.gz",
13
+ "archive_sha256": "01b3689603309c13cb31ef31ff7dc0def65b556a92269d33483ee11da6e8c14d",
14
+ "manifest": "runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21/recognizer-training-bundle.json",
15
+ "manifest_sha256": "95322aa852b14662e22c53ddbc1ae8346f37a51a7d1ae8348ec4f7a5fbf8da9f",
16
+ "recipe_backend": "paddleocr",
17
+ "run_command_after_extract": "python PaddleOCR/tools/train.py -c training-recipes/ppocrv5-mobile-devclean/paddle_recognizer.yml",
18
+ "bundle_train_manifest": "manifests/train.jsonl",
19
+ "bundle_dev_manifest": "manifests/dev.jsonl"
20
+ },
21
+ "selection_policy": {
22
+ "train_manifest": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/manifests/train.jsonl",
23
+ "dev_manifest": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/manifests/dev.jsonl",
24
+ "train_plus_dev_manifest": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/train-manifest.jsonl",
25
+ "sealed_eval_manifest": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/manifests/recognizer-eval.jsonl",
26
+ "sealed_eval_label_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/eval.txt",
27
+ "allowed_checkpoint_selection_splits": [
28
+ "train",
29
+ "dev"
30
+ ],
31
+ "forbidden_checkpoint_selection_splits": [
32
+ "eval"
33
+ ],
34
+ "min_train_samples": 5000,
35
+ "min_dev_samples": 200,
36
+ "min_sealed_eval_samples": 31,
37
+ "expected_split_counts": {
38
+ "train": 5551,
39
+ "dev": 258,
40
+ "eval": 31,
41
+ "train_plus_dev": 5809
42
+ },
43
+ "required_no_overlap_fields": [
44
+ "sample_id",
45
+ "image_path",
46
+ "image_sha256",
47
+ "text_sha256",
48
+ "sample_sha256"
49
+ ]
50
+ },
51
+ "admission_recipe": {
52
+ "registry": "configs/eval-harness-recipes-v1.json",
53
+ "recipe_id": "limbu-devanagari-cdc-ppocrv5-devclean-heldout",
54
+ "requires_unseen_for_selection": true,
55
+ "requires_selected_checkpoint_on_test": false,
56
+ "requires_eval_split": "eval",
57
+ "requires_leakage_source_manifest": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/train-manifest.jsonl",
58
+ "claim_blockers_until_resolved": [
59
+ "trained checkpoint path and reproducible prediction command are unresolved until the PP-OCRv5 bundle is run",
60
+ "eval pack is not frozen",
61
+ "GT is unreviewed curriculum text and lacks human/native-reader calibration",
62
+ "row-level claim eligibility is false",
63
+ "packaged local model card is absent"
64
+ ]
65
+ }
66
+ }
67
+ ]
68
+ }
ocr-tech-mps-integration/configs/limbu-first-ocr-pipeline-v1.json ADDED
The diff for this file is too large to render. See raw diff
 
ocr-tech-mps-integration/configs/limbu-sirijonga-augmentation-ablation-v1.json ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "ablation_id": "limbu-sirijonga-augmentation-ablation-v1",
3
+ "language": "limbu",
4
+ "script": "Limbu / Sirijonga",
5
+ "purpose": "Compare clean-only, lighting-augmented, and camera-angle/phone-photo synthetic OCR line training for Limbu/Sirijonga as training ablations only.",
6
+ "claim_policy": {
7
+ "claim_evidence_eligible": false,
8
+ "reason": "All rows are rendered synthetic training or ablation data. Accuracy claims require held-out real Gorkhapatra/Naya Nepal references."
9
+ },
10
+ "source_bundle": {
11
+ "rendered_split": "outputs/sirijonga-limbu-rendered-split-namdhinggo-augmented-2026-06-16",
12
+ "font_readiness_report": "outputs/font-asset-readiness-namdhinggo-2026-06-16/font-asset-readiness.json",
13
+ "font_sha256": "1b46b16277e4b6784bd19306a9474281888b656e9c57233b4e7dc63bd1229d6c",
14
+ "degradation_seed": 29,
15
+ "full_split_audit": "outputs/sirijonga-limbu-rendered-split-namdhinggo-augmented-audit-2026-06-16/rendered-degradation-split-audit.json"
16
+ },
17
+ "dictionary": "data/dictionaries/limbu-sirijonga-v1.txt",
18
+ "backend": {
19
+ "name": "paddleocr",
20
+ "base_model": "https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams",
21
+ "status": "narrow_limbu_ablation_only",
22
+ "guardrail": "The broader Devanagari PP-OCRv3 mixed recognizer family is stopped for engine promotion by prior admission evidence. These runs are allowed only as a Limbu/Sirijonga synthetic augmentation ablation and must not be promoted without real-document validation."
23
+ },
24
+ "training_defaults": {
25
+ "train_batch_size": 8,
26
+ "eval_batch_size": 8,
27
+ "learning_rate": 0.0001,
28
+ "epochs": 1
29
+ },
30
+ "arms": [
31
+ {
32
+ "name": "clean",
33
+ "profiles": ["clean"],
34
+ "train_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/train-clean.jsonl",
35
+ "eval_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/eval-clean.jsonl",
36
+ "audit": "outputs/ablation-limbu-namdhinggo-augmented-v1/audit-clean/rendered-degradation-split-audit.json",
37
+ "recipe_dir": "runs/sirijonga-limbu-paddle-ablation-clean-v1",
38
+ "train_count": 150,
39
+ "eval_count": 27
40
+ },
41
+ {
42
+ "name": "lighting",
43
+ "profiles": ["clean", "low_light", "uneven_light"],
44
+ "train_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/train-lighting.jsonl",
45
+ "eval_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/eval-lighting.jsonl",
46
+ "audit": "outputs/ablation-limbu-namdhinggo-augmented-v1/audit-lighting/rendered-degradation-split-audit.json",
47
+ "recipe_dir": "runs/sirijonga-limbu-paddle-ablation-lighting-v1",
48
+ "train_count": 450,
49
+ "eval_count": 81
50
+ },
51
+ {
52
+ "name": "camera",
53
+ "profiles": ["clean", "camera_left", "camera_right", "phone_photo"],
54
+ "train_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/train-camera.jsonl",
55
+ "eval_manifest": "data/ablation-limbu-namdhinggo-augmented-v1/eval-camera.jsonl",
56
+ "audit": "outputs/ablation-limbu-namdhinggo-augmented-v1/audit-camera/rendered-degradation-split-audit.json",
57
+ "recipe_dir": "runs/sirijonga-limbu-paddle-ablation-camera-v1",
58
+ "train_count": 600,
59
+ "eval_count": 108
60
+ }
61
+ ],
62
+ "promotion_gates": [
63
+ "Every arm must pass rendered-degradation split audit with zero issues and warnings.",
64
+ "Every trained arm must be evaluated on the same held-out synthetic eval arm and on manually reviewed real Gorkhapatra/Naya Nepal Limbu references.",
65
+ "No synthetic-only result may be used as a paper accuracy claim.",
66
+ "A model can move toward engine integration only if it improves Limbu/Sirijonga real-reference OCR without regressing English, Nepali, table extraction, reading order, or figure/caption behavior in the broader document pipeline."
67
+ ]
68
+ }
ocr-tech-mps-integration/configs/namdhinggo-font-assets-v1.json ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "manifest_id": "namdhinggo-font-assets-v1",
3
+ "description": "SIL Namdhinggo 3.100 font assets for Limbu/Sirijonga OCR synthesis. Assets are extracted from the upstream release ZIP and verified by package and member SHA-256.",
4
+ "assets": [
5
+ {
6
+ "id": "namdhinggo-regular",
7
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
8
+ "archive_member": "Namdhinggo-3.100/Namdhinggo-Regular.ttf",
9
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
10
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Regular.ttf",
11
+ "sha256": "1b46b16277e4b6784bd19306a9474281888b656e9c57233b4e7dc63bd1229d6c",
12
+ "license": "OFL-1.1",
13
+ "source": "SIL font-namdhinggo release v3.100"
14
+ },
15
+ {
16
+ "id": "namdhinggo-medium",
17
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
18
+ "archive_member": "Namdhinggo-3.100/Namdhinggo-Medium.ttf",
19
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
20
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Medium.ttf",
21
+ "sha256": "d29b8f9481dcf84567d2525a90561c8173375fbd1c761880b593c0956f3fdcb6",
22
+ "license": "OFL-1.1",
23
+ "source": "SIL font-namdhinggo release v3.100"
24
+ },
25
+ {
26
+ "id": "namdhinggo-semibold",
27
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
28
+ "archive_member": "Namdhinggo-3.100/Namdhinggo-SemiBold.ttf",
29
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
30
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-SemiBold.ttf",
31
+ "sha256": "f15fe69ee3c93e7f92609940ef17112e6d25807ca3df19cf81798928028c3397",
32
+ "license": "OFL-1.1",
33
+ "source": "SIL font-namdhinggo release v3.100"
34
+ },
35
+ {
36
+ "id": "namdhinggo-bold",
37
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
38
+ "archive_member": "Namdhinggo-3.100/Namdhinggo-Bold.ttf",
39
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
40
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Bold.ttf",
41
+ "sha256": "76da18746fdbe1c6db4c1d0222a8f06dbc9d206e362ae06c928c6810123f57da",
42
+ "license": "OFL-1.1",
43
+ "source": "SIL font-namdhinggo release v3.100"
44
+ },
45
+ {
46
+ "id": "namdhinggo-extrabold",
47
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
48
+ "archive_member": "Namdhinggo-3.100/Namdhinggo-ExtraBold.ttf",
49
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
50
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-ExtraBold.ttf",
51
+ "sha256": "ce3d01b4e99bcfe6a02941ac4c6f96f550b31a2af10e53454e1069e823bd9b2c",
52
+ "license": "OFL-1.1",
53
+ "source": "SIL font-namdhinggo release v3.100"
54
+ },
55
+ {
56
+ "id": "namdhinggo-ofl",
57
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
58
+ "archive_member": "Namdhinggo-3.100/OFL.txt",
59
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
60
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/OFL.txt",
61
+ "sha256": "64df695392d8facd1ed4b35e7158f9957073741c7f76c252c7423d5aae699cee",
62
+ "license": "OFL-1.1",
63
+ "source": "SIL font-namdhinggo release v3.100"
64
+ },
65
+ {
66
+ "id": "namdhinggo-ofl-faq",
67
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
68
+ "archive_member": "Namdhinggo-3.100/OFL-FAQ.txt",
69
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
70
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/OFL-FAQ.txt",
71
+ "sha256": "d6a2891c2470d349f5b3bb4d41c7901527c9a8016a29bfdd1cbec98b9420104e",
72
+ "license": "OFL-1.1",
73
+ "source": "SIL font-namdhinggo release v3.100"
74
+ },
75
+ {
76
+ "id": "namdhinggo-readme",
77
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
78
+ "archive_member": "Namdhinggo-3.100/README.txt",
79
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
80
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/README.txt",
81
+ "sha256": "99c8a1904df3f148628ce633a438011440114b912ddfa95b99c0132084466b64",
82
+ "license": "OFL-1.1",
83
+ "source": "SIL font-namdhinggo release v3.100"
84
+ },
85
+ {
86
+ "id": "namdhinggo-fontlog",
87
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
88
+ "archive_member": "Namdhinggo-3.100/FONTLOG.txt",
89
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
90
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/FONTLOG.txt",
91
+ "sha256": "505040639d06948d1a7ee154be618b55846e7561622c118af3fc62371508efa9",
92
+ "license": "OFL-1.1",
93
+ "source": "SIL font-namdhinggo release v3.100"
94
+ },
95
+ {
96
+ "id": "namdhinggo-fontmanifest",
97
+ "url": "https://github.com/silnrsi/font-namdhinggo/releases/download/v3.100/Namdhinggo-3.100.zip",
98
+ "archive_member": "Namdhinggo-3.100/fontmanifest.json",
99
+ "archive_sha256": "073369fbaf064f9075bfbc65e1a76776f449a280c4c367f3cb560bebf70b3915",
100
+ "target_path": "data/fonts/namdhinggo/Namdhinggo-3.100/fontmanifest.json",
101
+ "sha256": "4113c4fc112949eb5c72de21d3a98a77f13b334fa2281e3b78a85d1abdbf3721",
102
+ "license": "OFL-1.1",
103
+ "source": "SIL font-namdhinggo release v3.100"
104
+ }
105
+ ]
106
+ }
ocr-tech-mps-integration/configs/nepal-font-assets-v1.json ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "manifest_id": "nepal-font-assets-v1",
3
+ "description": "License-clear Unicode font assets used for Nepal multilingual OCR synthesis and renderability audits. Font binaries are provisioned into ignored data/fonts runtime data; this manifest records reproducible source URLs and SHA-256 checks.",
4
+ "assets": [
5
+ {
6
+ "id": "kanchenjunga-regular",
7
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/kanchenjunga/Kanchenjunga-Regular.ttf",
8
+ "target_path": "data/fonts/google-fonts-2026-06-16/kanchenjunga/Kanchenjunga-Regular.ttf",
9
+ "sha256": "45609f8cc90d4733d3d1665346d359afc1a659def340d0f369121af034322ef9",
10
+ "license": "OFL-1.1",
11
+ "source": "Google Fonts ofl/kanchenjunga"
12
+ },
13
+ {
14
+ "id": "kanchenjunga-bold",
15
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/kanchenjunga/Kanchenjunga-Bold.ttf",
16
+ "target_path": "data/fonts/google-fonts-2026-06-16/kanchenjunga/Kanchenjunga-Bold.ttf",
17
+ "sha256": "67a9a29cd49a17832a5eee8d2747fb9a66ee32e49dc52f5dd43887ff24eb0935",
18
+ "license": "OFL-1.1",
19
+ "source": "Google Fonts ofl/kanchenjunga"
20
+ },
21
+ {
22
+ "id": "kanchenjunga-ofl",
23
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/kanchenjunga/OFL.txt",
24
+ "target_path": "data/fonts/google-fonts-2026-06-16/kanchenjunga/OFL.txt",
25
+ "sha256": "b6d9eb7d23db4cb33585dc14ae4cd4c4d4f90f0d20e1673197d92cb9af76283f",
26
+ "license": "OFL-1.1",
27
+ "source": "Google Fonts ofl/kanchenjunga"
28
+ },
29
+ {
30
+ "id": "kanchenjunga-upstream-info",
31
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/kanchenjunga/upstream_info.md",
32
+ "target_path": "data/fonts/google-fonts-2026-06-16/kanchenjunga/upstream_info.md",
33
+ "sha256": "fc6cc71cf5168d56715f60fd77c79d5837e8777a416afd7f29382749f664c90d",
34
+ "license": "OFL-1.1",
35
+ "source": "Google Fonts ofl/kanchenjunga"
36
+ },
37
+ {
38
+ "id": "noto-sans-tirhuta-regular",
39
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosanstirhuta/NotoSansTirhuta-Regular.ttf",
40
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosanstirhuta/NotoSansTirhuta-Regular.ttf",
41
+ "sha256": "ad7123ee63118b83ed2f723591e5e861baad8dd157508b8339362850c6036efe",
42
+ "license": "OFL-1.1",
43
+ "source": "Google Fonts ofl/notosanstirhuta"
44
+ },
45
+ {
46
+ "id": "noto-sans-tirhuta-ofl",
47
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosanstirhuta/OFL.txt",
48
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosanstirhuta/OFL.txt",
49
+ "sha256": "9bf9c1ab9b7837bb65fb40dd7f6b706a28199cd633b80dcebe532bddc56594cb",
50
+ "license": "OFL-1.1",
51
+ "source": "Google Fonts ofl/notosanstirhuta"
52
+ },
53
+ {
54
+ "id": "noto-sans-tirhuta-upstream-info",
55
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosanstirhuta/upstream_info.md",
56
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosanstirhuta/upstream_info.md",
57
+ "sha256": "5bedd6efb51804b87c4dc169e5aa912bb432027f2a5cd3b5a872f192e2464b59",
58
+ "license": "OFL-1.1",
59
+ "source": "Google Fonts ofl/notosanstirhuta"
60
+ },
61
+ {
62
+ "id": "noto-serif-devanagari-variable",
63
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notoserifdevanagari/NotoSerifDevanagari%5Bwdth%2Cwght%5D.ttf",
64
+ "target_path": "data/fonts/google-fonts-2026-06-16/notoserifdevanagari/NotoSerifDevanagari-VariableFont_wdth,wght.ttf",
65
+ "sha256": "1191e07bfeb062d80e252eb85b0eafdfbda1e350707a2a60628668e8f677dbbb",
66
+ "license": "OFL-1.1",
67
+ "source": "Google Fonts ofl/notoserifdevanagari"
68
+ },
69
+ {
70
+ "id": "noto-serif-devanagari-ofl",
71
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notoserifdevanagari/OFL.txt",
72
+ "target_path": "data/fonts/google-fonts-2026-06-16/notoserifdevanagari/OFL.txt",
73
+ "sha256": "a216f6f8d85c7228093e0ee5e258d9d377e6671f68acb4db1930b29583d0f331",
74
+ "license": "OFL-1.1",
75
+ "source": "Google Fonts ofl/notoserifdevanagari"
76
+ },
77
+ {
78
+ "id": "noto-serif-devanagari-upstream-info",
79
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notoserifdevanagari/upstream_info.md",
80
+ "target_path": "data/fonts/google-fonts-2026-06-16/notoserifdevanagari/upstream_info.md",
81
+ "sha256": "44dfc44c289514753ec0b3dd499bfbca110c907fd4dfc9c21cfb7716af3d94e3",
82
+ "license": "OFL-1.1",
83
+ "source": "Google Fonts ofl/notoserifdevanagari"
84
+ },
85
+ {
86
+ "id": "noto-sans-ol-chiki-variable",
87
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosansolchiki/NotoSansOlChiki%5Bwght%5D.ttf",
88
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosansolchiki/NotoSansOlChiki-VariableFont_wght.ttf",
89
+ "sha256": "c9c31988656f49eccec9588825ab3b5045099c2f850ef98f356f976e8a596b4d",
90
+ "license": "OFL-1.1",
91
+ "source": "Google Fonts ofl/notosansolchiki"
92
+ },
93
+ {
94
+ "id": "noto-sans-ol-chiki-ofl",
95
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosansolchiki/OFL.txt",
96
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosansolchiki/OFL.txt",
97
+ "sha256": "b30754bbe6bea30b666a60d513ae80435a197979c9500e8380f83acd99dc763b",
98
+ "license": "OFL-1.1",
99
+ "source": "Google Fonts ofl/notosansolchiki"
100
+ },
101
+ {
102
+ "id": "noto-sans-ol-chiki-upstream-info",
103
+ "url": "https://raw.githubusercontent.com/google/fonts/main/ofl/notosansolchiki/upstream_info.md",
104
+ "target_path": "data/fonts/google-fonts-2026-06-16/notosansolchiki/upstream_info.md",
105
+ "sha256": "16e693164bea4aa66cdf0cf4f64fa113f8b1722f001775c2b5a8a2853f50da6e",
106
+ "license": "OFL-1.1",
107
+ "source": "Google Fonts ofl/notosansolchiki"
108
+ }
109
+ ]
110
+ }
ocr-tech-mps-integration/configs/nepal-language-pack-pipeline-v1.json ADDED
@@ -0,0 +1,1051 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "config_id": "nepal-language-pack-pipeline-v1",
3
+ "updated": "2026-07-04",
4
+ "description": "Language-pack-parameterized Paddle OCR pipeline registry: per-language script routing, converter, recognizer, and harvest state. Companion to RESEARCH.md sections 'The Paddle Pipeline' and 'Getting More Gorkhapatra Pages'.",
5
+ "pipeline_stages": [
6
+ "capture_prep",
7
+ "detection",
8
+ "script_routing",
9
+ "recognition",
10
+ "target_region_filtering",
11
+ "reading_order_assembly",
12
+ "export",
13
+ "eval_guardrail"
14
+ ],
15
+ "harvest_sources": {
16
+ "audit": "outputs/gorkhapatra-language-pages-full-2026-07-04/gorkhapatra-language-pages.json",
17
+ "inventory": "outputs/language-pack-inventory-full-2026-07-04/language-pack-inventory.json"
18
+ },
19
+ "script_recognizers": {
20
+ "limbu_sirijonga": {
21
+ "status": "champion",
22
+ "artifact": "runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23",
23
+ "dictionary": "data/dictionaries/limbu-sirijonga-v2.txt",
24
+ "input_width": 480,
25
+ "eval": "grapheme CER 0.041 real Gorkhapatra (frozen silver eval cp 0.011)"
26
+ },
27
+ "devanagari": {
28
+ "status": "trained",
29
+ "artifact": "outputs/devanagari-recognizer-v2-2026-06-18",
30
+ "dictionary": "data/dictionaries/devanagari-v1.txt",
31
+ "eval": "grapheme CER 0.075-0.129 across 8 languages, real Gorkhapatra"
32
+ },
33
+ "sunuwar_jenticha": {
34
+ "status": "trained_remote_needs_export",
35
+ "artifact": "aws-ebs (same-font retrain 2026-06-19; not exported locally)",
36
+ "dictionary": "data/dictionaries/sunuwar-v1.txt",
37
+ "eval": "cp=graph CER 0.121 real Sikkim Herald crops"
38
+ },
39
+ "kiratrai": {
40
+ "status": "trained_weak_eval",
41
+ "artifact": "runs/kiratrai-ppocrv5-mobile",
42
+ "dictionary": "data/dictionaries/kiratrai-v1.txt",
43
+ "eval": "70 real crops only; converter coverage blocks expansion"
44
+ },
45
+ "newa_prachalit": {
46
+ "status": "trained_weak",
47
+ "artifact": "outputs/newa-recognizer-v3-2026-06-18",
48
+ "dictionary": "data/dictionaries/newa-native-v1.txt",
49
+ "eval": "grapheme CER 0.613 synthetic-only (honest fail)",
50
+ "focus_note": "Newar is focus language 2 (steer 2026-07-04); Devanagari route first, native Prachalit needs real printed-register data"
51
+ },
52
+ "tirhuta": {
53
+ "status": "trained_weak",
54
+ "artifact": "outputs/tirhuta-recognizer-v1-2026-06-18",
55
+ "dictionary": "data/dictionaries/tirhuta-v1.txt",
56
+ "eval": "grapheme CER 0.343 synthetic-only"
57
+ },
58
+ "lepcha": {
59
+ "status": "data_only",
60
+ "artifact": null,
61
+ "dictionary": "data/dictionaries/lepcha-v1.txt",
62
+ "eval": "132 real crops; converter blocked outside eval editions"
63
+ },
64
+ "gurung_khema": {
65
+ "status": "data_only_synthetic_by_nature",
66
+ "artifact": null,
67
+ "dictionary": "data/dictionaries/gurungkhema-v1.txt",
68
+ "eval": "no real print exists (revival script)"
69
+ },
70
+ "tibetan": {
71
+ "status": "unstarted",
72
+ "artifact": null,
73
+ "dictionary": null,
74
+ "eval": "Gorkhapatra Tibetan-script pages are visual-only (verified 2026-07-04)"
75
+ },
76
+ "ol_chiki": {
77
+ "status": "unstarted",
78
+ "artifact": null,
79
+ "dictionary": null,
80
+ "eval": null
81
+ },
82
+ "devanagari_limbu": {
83
+ "status": "trained",
84
+ "artifact": "runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21",
85
+ "dictionary": "data/dictionaries/devanagari-v1.txt",
86
+ "eval": "CDC Devanagari-Limbu book path"
87
+ },
88
+ "perso_arabic": {
89
+ "status": "unstarted",
90
+ "artifact": null,
91
+ "dictionary": null,
92
+ "eval": "Urdu pages; script/text-layer class unverified"
93
+ }
94
+ },
95
+ "languages": [
96
+ {
97
+ "label": "खश जुम्ली",
98
+ "aliases_seen": [
99
+ "खश जुम्ली"
100
+ ],
101
+ "registry_id": null,
102
+ "primary_script": "devanagari",
103
+ "secondary_script": null,
104
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
105
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
106
+ "recognizer_ref": "devanagari",
107
+ "harvest_2026_07_04": {
108
+ "items": 9,
109
+ "aligned": 6,
110
+ "page_hits": 13,
111
+ "unique_dates": 9
112
+ },
113
+ "status": "harvested_pending_silver_pack"
114
+ },
115
+ {
116
+ "label": "अवधी",
117
+ "aliases_seen": [
118
+ "अवधी"
119
+ ],
120
+ "registry_id": null,
121
+ "primary_script": "devanagari",
122
+ "secondary_script": null,
123
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
124
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
125
+ "recognizer_ref": "devanagari",
126
+ "harvest_2026_07_04": {
127
+ "items": 9,
128
+ "aligned": 5,
129
+ "page_hits": 12,
130
+ "unique_dates": 9
131
+ },
132
+ "status": "harvested_pending_silver_pack"
133
+ },
134
+ {
135
+ "label": "थामी",
136
+ "aliases_seen": [
137
+ "थामी"
138
+ ],
139
+ "registry_id": null,
140
+ "primary_script": "devanagari",
141
+ "secondary_script": null,
142
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
143
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
144
+ "recognizer_ref": "devanagari",
145
+ "harvest_2026_07_04": {
146
+ "items": 9,
147
+ "aligned": 6,
148
+ "page_hits": 12,
149
+ "unique_dates": 9
150
+ },
151
+ "status": "harvested_pending_silver_pack"
152
+ },
153
+ {
154
+ "label": "राजवंशी",
155
+ "aliases_seen": [
156
+ "राजवंशी"
157
+ ],
158
+ "registry_id": "rajbanshi",
159
+ "primary_script": "devanagari",
160
+ "secondary_script": null,
161
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
162
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
163
+ "recognizer_ref": "devanagari",
164
+ "harvest_2026_07_04": {
165
+ "items": 9,
166
+ "aligned": 6,
167
+ "page_hits": 12,
168
+ "unique_dates": 9
169
+ },
170
+ "status": "harvested_pending_silver_pack"
171
+ },
172
+ {
173
+ "label": "ल्होमी",
174
+ "aliases_seen": [
175
+ "ल्होमी"
176
+ ],
177
+ "registry_id": null,
178
+ "primary_script": "devanagari",
179
+ "secondary_script": null,
180
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
181
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
182
+ "recognizer_ref": "devanagari",
183
+ "harvest_2026_07_04": {
184
+ "items": 9,
185
+ "aligned": 6,
186
+ "page_hits": 12,
187
+ "unique_dates": 9
188
+ },
189
+ "status": "harvested_pending_silver_pack"
190
+ },
191
+ {
192
+ "label": "ताजपुरिया",
193
+ "aliases_seen": [
194
+ "ताजपुरिया"
195
+ ],
196
+ "registry_id": null,
197
+ "primary_script": "devanagari",
198
+ "secondary_script": null,
199
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
200
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
201
+ "recognizer_ref": "devanagari",
202
+ "harvest_2026_07_04": {
203
+ "items": 9,
204
+ "aligned": 5,
205
+ "page_hits": 11,
206
+ "unique_dates": 9
207
+ },
208
+ "status": "harvested_pending_silver_pack"
209
+ },
210
+ {
211
+ "label": "किसान",
212
+ "aliases_seen": [
213
+ "किसान"
214
+ ],
215
+ "registry_id": null,
216
+ "primary_script": "devanagari",
217
+ "secondary_script": null,
218
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
219
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
220
+ "recognizer_ref": "devanagari",
221
+ "harvest_2026_07_04": {
222
+ "items": 10,
223
+ "aligned": 5,
224
+ "page_hits": 10,
225
+ "unique_dates": 10
226
+ },
227
+ "status": "harvested_pending_silver_pack"
228
+ },
229
+ {
230
+ "label": "माझी",
231
+ "aliases_seen": [
232
+ "माझी"
233
+ ],
234
+ "registry_id": null,
235
+ "primary_script": "devanagari",
236
+ "secondary_script": null,
237
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
238
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
239
+ "recognizer_ref": "devanagari",
240
+ "harvest_2026_07_04": {
241
+ "items": 10,
242
+ "aligned": 5,
243
+ "page_hits": 10,
244
+ "unique_dates": 10
245
+ },
246
+ "status": "harvested_pending_silver_pack"
247
+ },
248
+ {
249
+ "label": "सुनुवार",
250
+ "aliases_seen": [
251
+ "सुनुवार"
252
+ ],
253
+ "registry_id": "sunuwar",
254
+ "primary_script": "devanagari",
255
+ "secondary_script": "sunuwar_jenticha",
256
+ "text_layer_class": "legacy_font_nayanepal_devanagari (verified render 2082-10-16; native Jenticha chart box present)",
257
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal')",
258
+ "recognizer_ref": "devanagari",
259
+ "harvest_2026_07_04": {
260
+ "items": 10,
261
+ "aligned": 5,
262
+ "page_hits": 10,
263
+ "unique_dates": 10
264
+ },
265
+ "status": "harvested_pending_silver_pack"
266
+ },
267
+ {
268
+ "label": "कुमाल",
269
+ "aliases_seen": [
270
+ "कुमाल"
271
+ ],
272
+ "registry_id": null,
273
+ "primary_script": "devanagari",
274
+ "secondary_script": null,
275
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
276
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
277
+ "recognizer_ref": "devanagari",
278
+ "harvest_2026_07_04": {
279
+ "items": 9,
280
+ "aligned": 5,
281
+ "page_hits": 10,
282
+ "unique_dates": 9
283
+ },
284
+ "status": "harvested_pending_silver_pack"
285
+ },
286
+ {
287
+ "label": "थारु",
288
+ "aliases_seen": [
289
+ "थारु",
290
+ "थारू"
291
+ ],
292
+ "registry_id": "rana-tharu",
293
+ "primary_script": "devanagari",
294
+ "secondary_script": null,
295
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
296
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
297
+ "recognizer_ref": "devanagari",
298
+ "harvest_2026_07_04": {
299
+ "items": 9,
300
+ "aligned": 5,
301
+ "page_hits": 10,
302
+ "unique_dates": 9
303
+ },
304
+ "status": "harvested_pending_silver_pack"
305
+ },
306
+ {
307
+ "label": "बज्जिका",
308
+ "aliases_seen": [
309
+ "बज्जिका"
310
+ ],
311
+ "registry_id": null,
312
+ "primary_script": "devanagari",
313
+ "secondary_script": null,
314
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
315
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
316
+ "recognizer_ref": "devanagari",
317
+ "harvest_2026_07_04": {
318
+ "items": 9,
319
+ "aligned": 5,
320
+ "page_hits": 10,
321
+ "unique_dates": 9
322
+ },
323
+ "status": "harvested_pending_silver_pack"
324
+ },
325
+ {
326
+ "label": "बान्तावा",
327
+ "aliases_seen": [
328
+ "बान्तावा"
329
+ ],
330
+ "registry_id": "bantawa",
331
+ "primary_script": "devanagari",
332
+ "secondary_script": null,
333
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
334
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
335
+ "recognizer_ref": "devanagari",
336
+ "harvest_2026_07_04": {
337
+ "items": 9,
338
+ "aligned": 5,
339
+ "page_hits": 10,
340
+ "unique_dates": 9
341
+ },
342
+ "status": "harvested_pending_silver_pack"
343
+ },
344
+ {
345
+ "label": "बाहिङ",
346
+ "aliases_seen": [
347
+ "बाहिङ"
348
+ ],
349
+ "registry_id": null,
350
+ "primary_script": "devanagari",
351
+ "secondary_script": null,
352
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
353
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
354
+ "recognizer_ref": "devanagari",
355
+ "harvest_2026_07_04": {
356
+ "items": 9,
357
+ "aligned": 5,
358
+ "page_hits": 10,
359
+ "unique_dates": 9
360
+ },
361
+ "status": "harvested_pending_silver_pack"
362
+ },
363
+ {
364
+ "label": "मुगाल",
365
+ "aliases_seen": [
366
+ "मुगाल"
367
+ ],
368
+ "registry_id": null,
369
+ "primary_script": "tibetan",
370
+ "secondary_script": null,
371
+ "text_layer_class": "tibetan_script_visual_only (verified sample 2026-06-24/0012)",
372
+ "converter": null,
373
+ "recognizer_ref": "tibetan",
374
+ "harvest_2026_07_04": {
375
+ "items": 9,
376
+ "aligned": 5,
377
+ "page_hits": 10,
378
+ "unique_dates": 9
379
+ },
380
+ "status": "harvested_pending_silver_pack"
381
+ },
382
+ {
383
+ "label": "ह्योल्मो",
384
+ "aliases_seen": [
385
+ "ह्योलमो",
386
+ "ह्योल्मो"
387
+ ],
388
+ "registry_id": null,
389
+ "primary_script": "devanagari",
390
+ "secondary_script": null,
391
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
392
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
393
+ "recognizer_ref": "devanagari",
394
+ "harvest_2026_07_04": {
395
+ "items": 9,
396
+ "aligned": 5,
397
+ "page_hits": 10,
398
+ "unique_dates": 9
399
+ },
400
+ "status": "harvested_pending_silver_pack"
401
+ },
402
+ {
403
+ "label": "मारवाडी",
404
+ "aliases_seen": [
405
+ "मारवाडी"
406
+ ],
407
+ "registry_id": null,
408
+ "primary_script": "devanagari",
409
+ "secondary_script": null,
410
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
411
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
412
+ "recognizer_ref": "devanagari",
413
+ "harvest_2026_07_04": {
414
+ "items": 10,
415
+ "aligned": 5,
416
+ "page_hits": 9,
417
+ "unique_dates": 10
418
+ },
419
+ "status": "harvested_pending_silver_pack"
420
+ },
421
+ {
422
+ "label": "दनुवार",
423
+ "aliases_seen": [
424
+ "दनुवार"
425
+ ],
426
+ "registry_id": null,
427
+ "primary_script": "devanagari",
428
+ "secondary_script": null,
429
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
430
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
431
+ "recognizer_ref": "devanagari",
432
+ "harvest_2026_07_04": {
433
+ "items": 9,
434
+ "aligned": 5,
435
+ "page_hits": 9,
436
+ "unique_dates": 9
437
+ },
438
+ "status": "harvested_pending_silver_pack"
439
+ },
440
+ {
441
+ "label": "धिमाल",
442
+ "aliases_seen": [
443
+ "धिमाल"
444
+ ],
445
+ "registry_id": "dhimal",
446
+ "primary_script": "devanagari",
447
+ "secondary_script": null,
448
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
449
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
450
+ "recognizer_ref": "devanagari",
451
+ "harvest_2026_07_04": {
452
+ "items": 9,
453
+ "aligned": 4,
454
+ "page_hits": 9,
455
+ "unique_dates": 9
456
+ },
457
+ "status": "harvested_pending_silver_pack"
458
+ },
459
+ {
460
+ "label": "भोजपुरी",
461
+ "aliases_seen": [
462
+ "भोजपुरी"
463
+ ],
464
+ "registry_id": "bhojpuri",
465
+ "primary_script": "devanagari",
466
+ "secondary_script": null,
467
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
468
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
469
+ "recognizer_ref": "devanagari",
470
+ "harvest_2026_07_04": {
471
+ "items": 10,
472
+ "aligned": 4,
473
+ "page_hits": 8,
474
+ "unique_dates": 10
475
+ },
476
+ "status": "harvested_pending_silver_pack"
477
+ },
478
+ {
479
+ "label": "उराँव",
480
+ "aliases_seen": [
481
+ "उराँव"
482
+ ],
483
+ "registry_id": null,
484
+ "primary_script": "devanagari",
485
+ "secondary_script": null,
486
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
487
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
488
+ "recognizer_ref": "devanagari",
489
+ "harvest_2026_07_04": {
490
+ "items": 9,
491
+ "aligned": 4,
492
+ "page_hits": 8,
493
+ "unique_dates": 9
494
+ },
495
+ "status": "harvested_pending_silver_pack"
496
+ },
497
+ {
498
+ "label": "लिम्बु",
499
+ "aliases_seen": [
500
+ "लिम्बू"
501
+ ],
502
+ "registry_id": "limbu",
503
+ "primary_script": "limbu_sirijonga",
504
+ "secondary_script": "devanagari_limbu",
505
+ "text_layer_class": "legacy_font_namdhinggo (verified)",
506
+ "converter": "nepal_ttf2utf limbu / src/ocrtech/limbu_legacy.py + data/mappings/limbu-legacy/Limbu.map",
507
+ "recognizer_ref": "limbu_sirijonga",
508
+ "harvest_2026_07_04": {
509
+ "items": 9,
510
+ "aligned": 4,
511
+ "page_hits": 8,
512
+ "unique_dates": 9
513
+ },
514
+ "status": "harvested_pending_silver_pack"
515
+ },
516
+ {
517
+ "label": "गुरुङ",
518
+ "aliases_seen": [
519
+ "गुरुङ"
520
+ ],
521
+ "registry_id": "gurung",
522
+ "primary_script": "devanagari",
523
+ "secondary_script": null,
524
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
525
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
526
+ "recognizer_ref": "devanagari",
527
+ "harvest_2026_07_04": {
528
+ "items": 8,
529
+ "aligned": 4,
530
+ "page_hits": 8,
531
+ "unique_dates": 8
532
+ },
533
+ "status": "harvested_pending_silver_pack"
534
+ },
535
+ {
536
+ "label": "तामाङ",
537
+ "aliases_seen": [
538
+ "तामाङ"
539
+ ],
540
+ "registry_id": "tamang",
541
+ "primary_script": "devanagari",
542
+ "secondary_script": null,
543
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
544
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
545
+ "recognizer_ref": "devanagari",
546
+ "harvest_2026_07_04": {
547
+ "items": 8,
548
+ "aligned": 4,
549
+ "page_hits": 8,
550
+ "unique_dates": 8
551
+ },
552
+ "status": "harvested_pending_silver_pack"
553
+ },
554
+ {
555
+ "label": "मगही",
556
+ "aliases_seen": [
557
+ "मगही"
558
+ ],
559
+ "registry_id": null,
560
+ "primary_script": "devanagari",
561
+ "secondary_script": null,
562
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
563
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
564
+ "recognizer_ref": "devanagari",
565
+ "harvest_2026_07_04": {
566
+ "items": 8,
567
+ "aligned": 4,
568
+ "page_hits": 8,
569
+ "unique_dates": 8
570
+ },
571
+ "status": "harvested_pending_silver_pack"
572
+ },
573
+ {
574
+ "label": "वाम्बुले",
575
+ "aliases_seen": [
576
+ "वाम्बुले"
577
+ ],
578
+ "registry_id": "wambule",
579
+ "primary_script": "devanagari",
580
+ "secondary_script": null,
581
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
582
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
583
+ "recognizer_ref": "devanagari",
584
+ "harvest_2026_07_04": {
585
+ "items": 8,
586
+ "aligned": 4,
587
+ "page_hits": 8,
588
+ "unique_dates": 8
589
+ },
590
+ "status": "harvested_pending_silver_pack"
591
+ },
592
+ {
593
+ "label": "कुलुङ",
594
+ "aliases_seen": [
595
+ "कुलुङ"
596
+ ],
597
+ "registry_id": "kulung",
598
+ "primary_script": "devanagari",
599
+ "secondary_script": null,
600
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
601
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
602
+ "recognizer_ref": "devanagari",
603
+ "harvest_2026_07_04": {
604
+ "items": 7,
605
+ "aligned": 4,
606
+ "page_hits": 8,
607
+ "unique_dates": 7
608
+ },
609
+ "status": "harvested_pending_silver_pack"
610
+ },
611
+ {
612
+ "label": "राना थारु",
613
+ "aliases_seen": [
614
+ "राना थारु"
615
+ ],
616
+ "registry_id": "rana-tharu",
617
+ "primary_script": "devanagari",
618
+ "secondary_script": null,
619
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
620
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
621
+ "recognizer_ref": "devanagari",
622
+ "harvest_2026_07_04": {
623
+ "items": 7,
624
+ "aligned": 4,
625
+ "page_hits": 8,
626
+ "unique_dates": 7
627
+ },
628
+ "status": "harvested_pending_silver_pack"
629
+ },
630
+ {
631
+ "label": "चेपाङ",
632
+ "aliases_seen": [
633
+ "चेपाङ"
634
+ ],
635
+ "registry_id": null,
636
+ "primary_script": "devanagari",
637
+ "secondary_script": null,
638
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
639
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
640
+ "recognizer_ref": "devanagari",
641
+ "harvest_2026_07_04": {
642
+ "items": 8,
643
+ "aligned": 3,
644
+ "page_hits": 7,
645
+ "unique_dates": 8
646
+ },
647
+ "status": "harvested_pending_silver_pack"
648
+ },
649
+ {
650
+ "label": "बराम",
651
+ "aliases_seen": [
652
+ "बराम"
653
+ ],
654
+ "registry_id": null,
655
+ "primary_script": "devanagari",
656
+ "secondary_script": null,
657
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
658
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
659
+ "recognizer_ref": "devanagari",
660
+ "harvest_2026_07_04": {
661
+ "items": 10,
662
+ "aligned": 3,
663
+ "page_hits": 6,
664
+ "unique_dates": 10
665
+ },
666
+ "status": "harvested_pending_silver_pack"
667
+ },
668
+ {
669
+ "label": "चाम्लिङ राई",
670
+ "aliases_seen": [
671
+ "चाम्लिङ",
672
+ "चाम्लिङ राई"
673
+ ],
674
+ "registry_id": null,
675
+ "primary_script": "devanagari",
676
+ "secondary_script": null,
677
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
678
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
679
+ "recognizer_ref": "devanagari",
680
+ "harvest_2026_07_04": {
681
+ "items": 9,
682
+ "aligned": 3,
683
+ "page_hits": 6,
684
+ "unique_dates": 9
685
+ },
686
+ "status": "harvested_pending_silver_pack"
687
+ },
688
+ {
689
+ "label": "डोट्याली",
690
+ "aliases_seen": [
691
+ "डोट्याली"
692
+ ],
693
+ "registry_id": null,
694
+ "primary_script": "devanagari",
695
+ "secondary_script": null,
696
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
697
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
698
+ "recognizer_ref": "devanagari",
699
+ "harvest_2026_07_04": {
700
+ "items": 9,
701
+ "aligned": 3,
702
+ "page_hits": 6,
703
+ "unique_dates": 9
704
+ },
705
+ "status": "harvested_pending_silver_pack"
706
+ },
707
+ {
708
+ "label": "बझाङी",
709
+ "aliases_seen": [
710
+ "बझाङी"
711
+ ],
712
+ "registry_id": null,
713
+ "primary_script": "devanagari",
714
+ "secondary_script": null,
715
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
716
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
717
+ "recognizer_ref": "devanagari",
718
+ "harvest_2026_07_04": {
719
+ "items": 9,
720
+ "aligned": 3,
721
+ "page_hits": 6,
722
+ "unique_dates": 9
723
+ },
724
+ "status": "harvested_pending_silver_pack"
725
+ },
726
+ {
727
+ "label": "मगर",
728
+ "aliases_seen": [
729
+ "मगर"
730
+ ],
731
+ "registry_id": "magar",
732
+ "primary_script": "devanagari",
733
+ "secondary_script": null,
734
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
735
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
736
+ "recognizer_ref": "devanagari",
737
+ "harvest_2026_07_04": {
738
+ "items": 9,
739
+ "aligned": 3,
740
+ "page_hits": 6,
741
+ "unique_dates": 9
742
+ },
743
+ "status": "harvested_pending_silver_pack"
744
+ },
745
+ {
746
+ "label": "अछामि",
747
+ "aliases_seen": [
748
+ "अछामि"
749
+ ],
750
+ "registry_id": null,
751
+ "primary_script": "devanagari",
752
+ "secondary_script": null,
753
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
754
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
755
+ "recognizer_ref": "devanagari",
756
+ "harvest_2026_07_04": {
757
+ "items": 8,
758
+ "aligned": 4,
759
+ "page_hits": 6,
760
+ "unique_dates": 8
761
+ },
762
+ "status": "harvested_pending_silver_pack"
763
+ },
764
+ {
765
+ "label": "जिरेल",
766
+ "aliases_seen": [
767
+ "जिरेल"
768
+ ],
769
+ "registry_id": null,
770
+ "primary_script": "devanagari",
771
+ "secondary_script": null,
772
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
773
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
774
+ "recognizer_ref": "devanagari",
775
+ "harvest_2026_07_04": {
776
+ "items": 8,
777
+ "aligned": 3,
778
+ "page_hits": 6,
779
+ "unique_dates": 8
780
+ },
781
+ "status": "harvested_pending_silver_pack"
782
+ },
783
+ {
784
+ "label": "दराई",
785
+ "aliases_seen": [
786
+ "दराई"
787
+ ],
788
+ "registry_id": null,
789
+ "primary_script": "devanagari",
790
+ "secondary_script": null,
791
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
792
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
793
+ "recognizer_ref": "devanagari",
794
+ "harvest_2026_07_04": {
795
+ "items": 8,
796
+ "aligned": 3,
797
+ "page_hits": 6,
798
+ "unique_dates": 8
799
+ },
800
+ "status": "harvested_pending_silver_pack"
801
+ },
802
+ {
803
+ "label": "नेपाल",
804
+ "aliases_seen": [
805
+ "नेपाल"
806
+ ],
807
+ "registry_id": "newar",
808
+ "primary_script": "devanagari",
809
+ "secondary_script": "newa_prachalit",
810
+ "text_layer_class": "legacy_font_nayanepal_devanagari (verified via silver run 2026-07-04: 6,800 kept lines)",
811
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
812
+ "recognizer_ref": "devanagari",
813
+ "harvest_2026_07_04": {
814
+ "items": 8,
815
+ "aligned": 3,
816
+ "page_hits": 6,
817
+ "unique_dates": 8
818
+ },
819
+ "status": "focus_language_2_pending_frozen_split"
820
+ },
821
+ {
822
+ "label": "भोटे",
823
+ "aliases_seen": [
824
+ "भोटे"
825
+ ],
826
+ "registry_id": null,
827
+ "primary_script": "devanagari",
828
+ "secondary_script": null,
829
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
830
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
831
+ "recognizer_ref": "devanagari",
832
+ "harvest_2026_07_04": {
833
+ "items": 8,
834
+ "aligned": 3,
835
+ "page_hits": 6,
836
+ "unique_dates": 8
837
+ },
838
+ "status": "harvested_pending_silver_pack"
839
+ },
840
+ {
841
+ "label": "मैथिली",
842
+ "aliases_seen": [
843
+ "मैथिली"
844
+ ],
845
+ "registry_id": "maithili",
846
+ "primary_script": "devanagari",
847
+ "secondary_script": null,
848
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
849
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
850
+ "recognizer_ref": "devanagari",
851
+ "harvest_2026_07_04": {
852
+ "items": 8,
853
+ "aligned": 3,
854
+ "page_hits": 6,
855
+ "unique_dates": 8
856
+ },
857
+ "status": "harvested_pending_silver_pack"
858
+ },
859
+ {
860
+ "label": "संस्कृत",
861
+ "aliases_seen": [
862
+ "संस्कृत"
863
+ ],
864
+ "registry_id": null,
865
+ "primary_script": "devanagari",
866
+ "secondary_script": null,
867
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
868
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
869
+ "recognizer_ref": "devanagari",
870
+ "harvest_2026_07_04": {
871
+ "items": 8,
872
+ "aligned": 3,
873
+ "page_hits": 6,
874
+ "unique_dates": 8
875
+ },
876
+ "status": "harvested_pending_silver_pack"
877
+ },
878
+ {
879
+ "label": "डडेल्धुरेली",
880
+ "aliases_seen": [
881
+ "डडेल्धुरेली"
882
+ ],
883
+ "registry_id": null,
884
+ "primary_script": "devanagari",
885
+ "secondary_script": null,
886
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
887
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
888
+ "recognizer_ref": "devanagari",
889
+ "harvest_2026_07_04": {
890
+ "items": 7,
891
+ "aligned": 3,
892
+ "page_hits": 6,
893
+ "unique_dates": 7
894
+ },
895
+ "status": "harvested_pending_silver_pack"
896
+ },
897
+ {
898
+ "label": "मेचे",
899
+ "aliases_seen": [
900
+ "मेचे"
901
+ ],
902
+ "registry_id": null,
903
+ "primary_script": "devanagari",
904
+ "secondary_script": null,
905
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
906
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
907
+ "recognizer_ref": "devanagari",
908
+ "harvest_2026_07_04": {
909
+ "items": 7,
910
+ "aligned": 3,
911
+ "page_hits": 6,
912
+ "unique_dates": 7
913
+ },
914
+ "status": "harvested_pending_silver_pack"
915
+ },
916
+ {
917
+ "label": "याक्खा",
918
+ "aliases_seen": [
919
+ "याक्ख",
920
+ "याक्खा"
921
+ ],
922
+ "registry_id": "yakkha",
923
+ "primary_script": "devanagari",
924
+ "secondary_script": null,
925
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
926
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
927
+ "recognizer_ref": "devanagari",
928
+ "harvest_2026_07_04": {
929
+ "items": 7,
930
+ "aligned": 3,
931
+ "page_hits": 6,
932
+ "unique_dates": 7
933
+ },
934
+ "status": "harvested_pending_silver_pack"
935
+ },
936
+ {
937
+ "label": "शेर्पा",
938
+ "aliases_seen": [
939
+ "शेर्पा"
940
+ ],
941
+ "registry_id": "sherpa",
942
+ "primary_script": "devanagari",
943
+ "secondary_script": null,
944
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
945
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
946
+ "recognizer_ref": "devanagari",
947
+ "harvest_2026_07_04": {
948
+ "items": 5,
949
+ "aligned": 3,
950
+ "page_hits": 5,
951
+ "unique_dates": 5
952
+ },
953
+ "status": "harvested_pending_silver_pack"
954
+ },
955
+ {
956
+ "label": "बोटे",
957
+ "aliases_seen": [
958
+ "बोटे"
959
+ ],
960
+ "registry_id": null,
961
+ "primary_script": "devanagari",
962
+ "secondary_script": null,
963
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
964
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
965
+ "recognizer_ref": "devanagari",
966
+ "harvest_2026_07_04": {
967
+ "items": 2,
968
+ "aligned": 2,
969
+ "page_hits": 5,
970
+ "unique_dates": 2
971
+ },
972
+ "status": "harvested_pending_silver_pack"
973
+ },
974
+ {
975
+ "label": "सन्थाली",
976
+ "aliases_seen": [
977
+ "सन्थाली"
978
+ ],
979
+ "registry_id": "santhali",
980
+ "primary_script": "ol_chiki",
981
+ "secondary_script": null,
982
+ "text_layer_class": "unclassified_pending_extraction",
983
+ "converter": null,
984
+ "recognizer_ref": "ol_chiki",
985
+ "harvest_2026_07_04": {
986
+ "items": 2,
987
+ "aligned": 2,
988
+ "page_hits": 5,
989
+ "unique_dates": 2
990
+ },
991
+ "status": "harvested_pending_silver_pack"
992
+ },
993
+ {
994
+ "label": "हायु",
995
+ "aliases_seen": [
996
+ "हायु"
997
+ ],
998
+ "registry_id": null,
999
+ "primary_script": "devanagari",
1000
+ "secondary_script": null,
1001
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
1002
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
1003
+ "recognizer_ref": "devanagari",
1004
+ "harvest_2026_07_04": {
1005
+ "items": 2,
1006
+ "aligned": 2,
1007
+ "page_hits": 5,
1008
+ "unique_dates": 2
1009
+ },
1010
+ "status": "harvested_pending_silver_pack"
1011
+ },
1012
+ {
1013
+ "label": "उर्दू",
1014
+ "aliases_seen": [
1015
+ "उर्दू"
1016
+ ],
1017
+ "registry_id": null,
1018
+ "primary_script": "perso_arabic",
1019
+ "secondary_script": null,
1020
+ "text_layer_class": "unclassified_pending_extraction (Perso-Arabic expected, nayanepal impossible)",
1021
+ "converter": null,
1022
+ "recognizer_ref": "perso_arabic",
1023
+ "harvest_2026_07_04": {
1024
+ "items": 8,
1025
+ "aligned": 2,
1026
+ "page_hits": 4,
1027
+ "unique_dates": 8
1028
+ },
1029
+ "status": "harvested_pending_silver_pack"
1030
+ },
1031
+ {
1032
+ "label": null,
1033
+ "aliases_seen": [
1034
+ null
1035
+ ],
1036
+ "registry_id": null,
1037
+ "primary_script": "devanagari",
1038
+ "secondary_script": null,
1039
+ "text_layer_class": "expected_legacy_font_nayanepal (pending per-page extraction)",
1040
+ "converter": "nepal_ttf2utf convert_devanagari(font='nayanepal') (pending validation on this language's pages)",
1041
+ "recognizer_ref": "devanagari",
1042
+ "harvest_2026_07_04": {
1043
+ "items": 1,
1044
+ "aligned": 1,
1045
+ "page_hits": 2,
1046
+ "unique_dates": 1
1047
+ },
1048
+ "status": "harvested_pending_silver_pack"
1049
+ }
1050
+ ]
1051
+ }
ocr-tech-mps-integration/configs/nepali-table-name-lexicon-v1.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ # Small Nepali table-name lexicon for guarded OCR correction.
2
+ # Use only with provenance in the model card; this is not a general language model.
3
+ किरण
4
+ माया
5
+ राम
6
+ सीता
7
+ हरी
8
+ गीता
9
+ सोम
10
+ रीता
ocr-tech-mps-integration/configs/recognizer-eval-pack-normalization-v1.json ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "recognizer-eval-pack-normalization-v1",
3
+ "updated": "2026-06-19",
4
+ "packs": [
5
+ {
6
+ "id": "newa-zenodo-6967421-v1",
7
+ "label_file": "data/newa-real-eval-pack/eval.txt",
8
+ "data_dir": "data/newa-real-eval-pack",
9
+ "manifest_file": "data/newa-real-eval-pack/manifest.jsonl",
10
+ "summary_file": "data/newa-real-eval-pack/summary.json",
11
+ "out": "outputs/eval-packs/newa-zenodo-6967421-v1",
12
+ "dataset": "newa-zenodo-6967421-prachalit-real",
13
+ "language": "newa",
14
+ "script": "Newa",
15
+ "domain": "real_handwritten",
16
+ "gt_status": "machine_converted",
17
+ "split": "eval",
18
+ "eval_pack_version": "newa-zenodo-6967421-v1",
19
+ "frozen_eval_pack": true,
20
+ "unseen_for_selection": false,
21
+ "claim_eligible": false,
22
+ "claim_blockers": [
23
+ "eval pack is not declared unseen for model/checkpoint selection",
24
+ "GT is machine-converted and lacks native-reader calibration",
25
+ "row-level claim eligibility is false"
26
+ ]
27
+ },
28
+ {
29
+ "id": "sunuwar-herald-real-v1",
30
+ "label_file": "data/sunuwar-real-eval-pack/real_eval.txt",
31
+ "data_dir": "data/sunuwar-real-eval-pack",
32
+ "manifest_file": "data/sunuwar-real-eval-pack/manifests/recognizer-eval.jsonl",
33
+ "summary_file": "data/sunuwar-real-eval-pack/README.json",
34
+ "out": "outputs/eval-packs/sunuwar-herald-real-v1",
35
+ "dataset": "sikkim-herald-sunuwar-real",
36
+ "language": "sunuwar",
37
+ "script": "Sunuwar",
38
+ "domain": "real_print",
39
+ "gt_status": "machine_converted",
40
+ "split": "eval",
41
+ "eval_pack_version": "sunuwar-herald-real-v1",
42
+ "frozen_eval_pack": true,
43
+ "unseen_for_selection": true,
44
+ "claim_eligible": false,
45
+ "claim_blockers": [
46
+ "GT is machine-converted and lacks native-reader calibration",
47
+ "row-level claim eligibility is false"
48
+ ]
49
+ },
50
+ {
51
+ "id": "lepcha-herald-real-v1",
52
+ "label_file": "data/lepcha-real-eval-pack/real_eval.txt",
53
+ "data_dir": "data/lepcha-real-eval-pack",
54
+ "manifest_file": "data/lepcha-real-eval-pack/manifests/recognizer-eval.jsonl",
55
+ "summary_file": "data/lepcha-real-eval-pack/README.json",
56
+ "out": "outputs/eval-packs/lepcha-herald-real-v1",
57
+ "dataset": "sikkim-herald-lepcha-real",
58
+ "language": "lepcha",
59
+ "script": "Lepcha",
60
+ "domain": "real_print",
61
+ "gt_status": "machine_converted",
62
+ "split": "eval",
63
+ "eval_pack_version": "lepcha-herald-real-v1",
64
+ "frozen_eval_pack": true,
65
+ "unseen_for_selection": true,
66
+ "claim_eligible": false,
67
+ "claim_blockers": [
68
+ "GT is machine-converted and lacks native-reader calibration",
69
+ "row-level claim eligibility is false"
70
+ ]
71
+ },
72
+ {
73
+ "id": "limbu-gorkhapatra-real-v1",
74
+ "label_file": "data/limbu-real-crops/labels.txt",
75
+ "data_dir": "data/limbu-real-crops",
76
+ "out": "outputs/eval-packs/limbu-gorkhapatra-real-v1",
77
+ "dataset": "gorkhapatra-limbu-real-crops",
78
+ "language": "limbu",
79
+ "script": "Limbu",
80
+ "domain": "real_print",
81
+ "gt_status": "machine_converted",
82
+ "split": "eval",
83
+ "eval_pack_version": "limbu-gorkhapatra-real-v1",
84
+ "frozen_eval_pack": true,
85
+ "unseen_for_selection": false,
86
+ "claim_eligible": false,
87
+ "claim_blockers": [
88
+ "source pack lacks canonical manifest and summary",
89
+ "eval pack is not declared unseen for model/checkpoint selection",
90
+ "GT is machine-converted and lacks native-reader calibration",
91
+ "row-level claim eligibility is false"
92
+ ]
93
+ },
94
+ {
95
+ "id": "limbu-devanagari-cdc-line-crop-corpus-draft-v1",
96
+ "label_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/eval.txt",
97
+ "data_dir": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21",
98
+ "manifest_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/manifests/recognizer-eval.jsonl",
99
+ "summary_file": "outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21/summaries/eval.summary.json",
100
+ "out": "outputs/eval-packs/limbu-devanagari-cdc-line-crop-corpus-draft-v1",
101
+ "dataset": "limbu-devanagari-cdc-line-crop-corpus-draft",
102
+ "language": "limbu",
103
+ "script": "Devanagari",
104
+ "domain": "real_print",
105
+ "gt_status": "curriculum_text_unreviewed",
106
+ "split": "eval",
107
+ "eval_pack_version": "limbu-devanagari-cdc-line-crop-corpus-draft-v1",
108
+ "frozen_eval_pack": false,
109
+ "unseen_for_selection": true,
110
+ "claim_eligible": false,
111
+ "claim_blockers": [
112
+ "eval pack is not frozen",
113
+ "GT is unreviewed curriculum text and lacks human/native-reader calibration",
114
+ "row-level claim eligibility is false",
115
+ "trained Devanagari recognizer checkpoint has not completed held-out admission"
116
+ ]
117
+ },
118
+ {
119
+ "id": "kiratrai-herald-real-v1",
120
+ "label_file": "data/kiratrai-real-eval-pack/eval.txt",
121
+ "data_dir": "data/kiratrai-real-eval-pack",
122
+ "summary_file": "data/kiratrai-real-eval-pack/summary.json",
123
+ "out": "outputs/eval-packs/kiratrai-herald-real-v1",
124
+ "dataset": "sikkim-herald-kiratrai-real",
125
+ "language": "kiratrai",
126
+ "script": "KiratRai",
127
+ "domain": "real_print",
128
+ "gt_status": "machine_converted",
129
+ "split": "eval",
130
+ "eval_pack_version": "kiratrai-herald-real-v1",
131
+ "frozen_eval_pack": true,
132
+ "unseen_for_selection": true,
133
+ "claim_eligible": false,
134
+ "claim_blockers": [
135
+ "source pack lacks canonical manifest",
136
+ "GT is machine-converted and lacks native-reader calibration",
137
+ "row-level claim eligibility is false"
138
+ ]
139
+ },
140
+ {
141
+ "id": "gorkhapatra-devanagari-real-v1",
142
+ "label_file": "data/gorkhapatra-devanagari-real/labels.txt",
143
+ "data_dir": "data/gorkhapatra-devanagari-real",
144
+ "summary_file": "data/gorkhapatra-devanagari-real/summary.json",
145
+ "out": "outputs/eval-packs/gorkhapatra-devanagari-real-v1",
146
+ "dataset": "gorkhapatra-devanagari-real",
147
+ "language": "devanagari_shared",
148
+ "script": "Devanagari",
149
+ "domain": "real_print",
150
+ "gt_status": "machine_converted",
151
+ "split": "eval",
152
+ "eval_pack_version": "gorkhapatra-devanagari-real-v1",
153
+ "frozen_eval_pack": true,
154
+ "unseen_for_selection": false,
155
+ "claim_eligible": false,
156
+ "claim_blockers": [
157
+ "source pack lacks canonical manifest",
158
+ "eval pack is not declared unseen for model/checkpoint selection",
159
+ "GT is machine-converted and lacks native-reader calibration",
160
+ "row-level claim eligibility is false",
161
+ "shared Devanagari pack still needs per-language source slices"
162
+ ]
163
+ }
164
+ ]
165
+ }
ocr-tech-mps-integration/configs/validation-complex-table-surya.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 40,
4
+ "required_slices": {
5
+ "complex_table": 40,
6
+ "table": 40,
7
+ "nepali": 40,
8
+ "english": 40
9
+ },
10
+ "bootstrap_iterations": 1000,
11
+ "bootstrap_confidence": 0.95,
12
+ "random_seed": 13,
13
+ "gates": [
14
+ {
15
+ "baseline": "surya",
16
+ "metric": "table_cell_f1",
17
+ "direction": "higher",
18
+ "slices": ["complex_table", "table"],
19
+ "min_pairs": 40,
20
+ "min_absolute_margin": 0.005
21
+ },
22
+ {
23
+ "baseline": "surya",
24
+ "metric": "table_exact_match",
25
+ "direction": "higher",
26
+ "slices": ["complex_table", "table"],
27
+ "min_pairs": 40,
28
+ "min_absolute_margin": 0.0,
29
+ "require_ci_above_zero": false
30
+ },
31
+ {
32
+ "baseline": "surya",
33
+ "metric": "table_structure_similarity",
34
+ "direction": "higher",
35
+ "slices": ["complex_table", "table"],
36
+ "min_pairs": 40,
37
+ "min_absolute_margin": 0.0,
38
+ "require_ci_above_zero": false
39
+ },
40
+ {
41
+ "baseline": "surya",
42
+ "metric": "table_teds_structure",
43
+ "direction": "higher",
44
+ "slices": ["complex_table", "table"],
45
+ "min_pairs": 40,
46
+ "min_absolute_margin": 0.0
47
+ },
48
+ {
49
+ "baseline": "surya",
50
+ "metric": "table_teds_text",
51
+ "direction": "higher",
52
+ "slices": ["complex_table", "table"],
53
+ "min_pairs": 40,
54
+ "min_absolute_margin": 0.005
55
+ }
56
+ ]
57
+ }
ocr-tech-mps-integration/configs/validation-hard15-surya.json ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 15,
4
+ "required_slices": {
5
+ "hard_eval": 15,
6
+ "table": 8,
7
+ "figure": 7,
8
+ "nepali": 15,
9
+ "english": 15
10
+ },
11
+ "bootstrap_iterations": 1000,
12
+ "bootstrap_confidence": 0.95,
13
+ "random_seed": 13,
14
+ "gates": [
15
+ {
16
+ "baseline": "surya",
17
+ "metric": "cer",
18
+ "direction": "lower",
19
+ "slices": ["hard_eval"],
20
+ "min_pairs": 15,
21
+ "min_absolute_margin": 0.02,
22
+ "max_p_value": 0.05
23
+ },
24
+ {
25
+ "baseline": "surya",
26
+ "metric": "cer",
27
+ "direction": "lower",
28
+ "slices": ["hard_eval", "table"],
29
+ "min_pairs": 8,
30
+ "min_absolute_margin": 0.02,
31
+ "max_p_value": 0.05
32
+ },
33
+ {
34
+ "baseline": "surya",
35
+ "metric": "cer",
36
+ "direction": "lower",
37
+ "slices": ["hard_eval", "figure"],
38
+ "min_pairs": 7,
39
+ "min_absolute_margin": 0.02,
40
+ "max_p_value": 0.05
41
+ },
42
+ {
43
+ "baseline": "surya",
44
+ "metric": "table_cell_f1",
45
+ "direction": "higher",
46
+ "slices": ["hard_eval", "table"],
47
+ "min_pairs": 8,
48
+ "min_absolute_margin": 0.01
49
+ },
50
+ {
51
+ "baseline": "surya",
52
+ "metric": "figure_caption_cer",
53
+ "direction": "lower",
54
+ "slices": ["hard_eval", "figure"],
55
+ "min_pairs": 7,
56
+ "min_absolute_margin": 0.01
57
+ }
58
+ ]
59
+ }
ocr-tech-mps-integration/configs/validation-hf-recognizer-next-v1-admission.json ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 244,
4
+ "required_slices": {
5
+ "english": 214,
6
+ "line_crop": 150,
7
+ "nepali": 94,
8
+ "real": 15,
9
+ "synthetic": 165
10
+ },
11
+ "bootstrap_iterations": 1000,
12
+ "bootstrap_confidence": 0.95,
13
+ "random_seed": 13,
14
+ "gates": [
15
+ {
16
+ "baseline": "tesseract",
17
+ "metric": "cer",
18
+ "direction": "lower",
19
+ "min_pairs": 180,
20
+ "min_absolute_margin": 0.05,
21
+ "require_ci_above_zero": false
22
+ },
23
+ {
24
+ "baseline": "tesseract",
25
+ "metric": "cer",
26
+ "direction": "lower",
27
+ "slices": ["nepali"],
28
+ "min_pairs": 30,
29
+ "min_absolute_margin": 0.05,
30
+ "require_ci_above_zero": false
31
+ },
32
+ {
33
+ "baseline": "stock-paddle",
34
+ "metric": "cer",
35
+ "direction": "lower",
36
+ "min_pairs": 180,
37
+ "min_absolute_margin": 0.10,
38
+ "min_pair_win_rate": 0.80,
39
+ "require_ci_above_zero": false
40
+ },
41
+ {
42
+ "baseline": "stock-paddle",
43
+ "metric": "cer",
44
+ "direction": "lower",
45
+ "slices": ["english", "line_crop"],
46
+ "min_pairs": 150,
47
+ "min_absolute_margin": 0.10,
48
+ "min_pair_win_rate": 0.80,
49
+ "require_ci_above_zero": false
50
+ },
51
+ {
52
+ "baseline": "stock-paddle",
53
+ "metric": "cer",
54
+ "direction": "lower",
55
+ "slices": ["nepali"],
56
+ "min_pairs": 30,
57
+ "min_absolute_margin": 0.10,
58
+ "min_pair_win_rate": 0.75,
59
+ "require_ci_above_zero": false
60
+ },
61
+ {
62
+ "baseline": "surya",
63
+ "metric": "cer",
64
+ "direction": "lower",
65
+ "min_pairs": 179,
66
+ "min_absolute_margin": 0.10,
67
+ "require_ci_above_zero": false
68
+ },
69
+ {
70
+ "baseline": "surya",
71
+ "metric": "cer",
72
+ "direction": "lower",
73
+ "slices": ["nepali"],
74
+ "min_pairs": 30,
75
+ "min_absolute_margin": 0.05,
76
+ "require_ci_above_zero": false
77
+ }
78
+ ]
79
+ }
ocr-tech-mps-integration/configs/validation-internal-sota-v1.json ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 100,
4
+ "required_slices": {
5
+ "nepali": 50,
6
+ "english": 50,
7
+ "table": 20,
8
+ "reading_order": 20,
9
+ "figure": 20
10
+ },
11
+ "bootstrap_iterations": 1000,
12
+ "bootstrap_confidence": 0.95,
13
+ "random_seed": 13,
14
+ "require_full_eval_manifest_coverage": true,
15
+ "gates": [
16
+ {
17
+ "baseline": "tesseract",
18
+ "metric": "cer",
19
+ "direction": "lower",
20
+ "slices": ["nepali"],
21
+ "min_pairs": 50,
22
+ "min_absolute_margin": 0.02,
23
+ "min_pair_win_rate": 0.55,
24
+ "max_p_value": 0.05
25
+ },
26
+ {
27
+ "baseline": "tesseract",
28
+ "metric": "wer",
29
+ "direction": "lower",
30
+ "slices": ["nepali"],
31
+ "min_pairs": 50,
32
+ "min_absolute_margin": 0.01,
33
+ "min_pair_win_rate": 0.55,
34
+ "max_p_value": 0.05
35
+ },
36
+ {
37
+ "baseline": "stock-paddle",
38
+ "metric": "cer",
39
+ "direction": "lower",
40
+ "slices": ["nepali"],
41
+ "min_pairs": 50,
42
+ "min_absolute_margin": 0.01,
43
+ "min_pair_win_rate": 0.55,
44
+ "max_p_value": 0.05
45
+ },
46
+ {
47
+ "baseline": "stock-paddle",
48
+ "metric": "cer",
49
+ "direction": "lower",
50
+ "slices": ["english"],
51
+ "min_pairs": 50,
52
+ "min_absolute_margin": 0.0,
53
+ "require_ci_above_zero": false
54
+ },
55
+ {
56
+ "baseline": "stock-paddle",
57
+ "metric": "wer",
58
+ "direction": "lower",
59
+ "slices": ["english"],
60
+ "min_pairs": 50,
61
+ "min_absolute_margin": 0.0,
62
+ "require_ci_above_zero": false
63
+ },
64
+ {
65
+ "baseline": "glm-ocr",
66
+ "metric": "cer",
67
+ "direction": "lower",
68
+ "slices": ["nepali"],
69
+ "min_pairs": 50,
70
+ "min_absolute_margin": 0.01,
71
+ "min_pair_win_rate": 0.55,
72
+ "max_p_value": 0.05
73
+ },
74
+ {
75
+ "baseline": "surya",
76
+ "metric": "table_cell_f1",
77
+ "direction": "higher",
78
+ "slices": ["table"],
79
+ "min_pairs": 20,
80
+ "min_absolute_margin": 0.01,
81
+ "min_pair_win_rate": 0.55
82
+ },
83
+ {
84
+ "baseline": "paddleocr-vl",
85
+ "metric": "reading_order_pair_accuracy",
86
+ "direction": "higher",
87
+ "slices": ["reading_order"],
88
+ "min_pairs": 20,
89
+ "min_absolute_margin": 0.01,
90
+ "min_pair_win_rate": 0.55
91
+ },
92
+ {
93
+ "baseline": "paddleocr-vl",
94
+ "metric": "document_reconstruction_similarity",
95
+ "direction": "higher",
96
+ "slices": ["figure"],
97
+ "min_pairs": 20,
98
+ "min_absolute_margin": 0.01,
99
+ "min_pair_win_rate": 0.55
100
+ },
101
+ {
102
+ "baseline": "surya",
103
+ "metric": "figure_detection_f1",
104
+ "direction": "higher",
105
+ "slices": ["figure"],
106
+ "min_pairs": 20,
107
+ "min_absolute_margin": 0.0,
108
+ "require_ci_above_zero": false
109
+ },
110
+ {
111
+ "baseline": "surya",
112
+ "metric": "figure_caption_cer",
113
+ "direction": "lower",
114
+ "slices": ["figure"],
115
+ "min_pairs": 20,
116
+ "min_absolute_margin": 0.01,
117
+ "min_pair_win_rate": 0.55
118
+ }
119
+ ]
120
+ }
ocr-tech-mps-integration/configs/validation-paddle-hard50-tesseract.json ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 50,
4
+ "required_slices": {
5
+ "paddle_hard_eval": 50,
6
+ "synthetic": 50,
7
+ "english": 38,
8
+ "line_crop": 38,
9
+ "nepali": 12
10
+ },
11
+ "bootstrap_iterations": 1000,
12
+ "bootstrap_confidence": 0.95,
13
+ "random_seed": 13,
14
+ "gates": [
15
+ {
16
+ "baseline": "tesseract",
17
+ "metric": "cer",
18
+ "direction": "lower",
19
+ "slices": ["paddle_hard_eval"],
20
+ "min_pairs": 50,
21
+ "min_absolute_margin": 0.0,
22
+ "require_ci_above_zero": false
23
+ },
24
+ {
25
+ "baseline": "tesseract",
26
+ "metric": "cer",
27
+ "direction": "lower",
28
+ "slices": ["paddle_hard_eval", "english", "line_crop"],
29
+ "min_pairs": 38,
30
+ "min_absolute_margin": 0.0,
31
+ "require_ci_above_zero": false
32
+ },
33
+ {
34
+ "baseline": "tesseract",
35
+ "metric": "wer",
36
+ "direction": "lower",
37
+ "slices": ["paddle_hard_eval", "english", "line_crop"],
38
+ "min_pairs": 38,
39
+ "min_absolute_margin": 0.0,
40
+ "require_ci_above_zero": false
41
+ },
42
+ {
43
+ "baseline": "tesseract",
44
+ "metric": "wer",
45
+ "direction": "lower",
46
+ "slices": ["paddle_hard_eval", "nepali"],
47
+ "min_pairs": 12,
48
+ "min_absolute_margin": 0.05,
49
+ "require_ci_above_zero": false
50
+ }
51
+ ]
52
+ }
ocr-tech-mps-integration/configs/validation-public-omnidocbench-sota-v1.json ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 100,
4
+ "required_slices": {
5
+ "public": 100,
6
+ "document_parsing": 100,
7
+ "table": 40
8
+ },
9
+ "bootstrap_iterations": 1000,
10
+ "bootstrap_confidence": 0.95,
11
+ "random_seed": 13,
12
+ "require_full_eval_manifest_coverage": true,
13
+ "gates": [
14
+ {
15
+ "baseline": "surya",
16
+ "metric": "cer",
17
+ "direction": "lower",
18
+ "slices": ["public", "document_parsing"],
19
+ "min_pairs": 100,
20
+ "min_absolute_margin": 0.005,
21
+ "min_pair_win_rate": 0.55
22
+ },
23
+ {
24
+ "baseline": "surya",
25
+ "metric": "table_cell_f1",
26
+ "direction": "higher",
27
+ "slices": ["public", "table"],
28
+ "min_pairs": 40,
29
+ "min_absolute_margin": 0.005,
30
+ "min_pair_win_rate": 0.55
31
+ },
32
+ {
33
+ "baseline": "surya",
34
+ "metric": "table_structure_similarity",
35
+ "direction": "higher",
36
+ "slices": ["public", "document_parsing"],
37
+ "min_pairs": 40,
38
+ "min_absolute_margin": 0.0,
39
+ "require_ci_above_zero": false
40
+ },
41
+ {
42
+ "baseline": "paddleocr-vl",
43
+ "metric": "document_reconstruction_similarity",
44
+ "direction": "higher",
45
+ "slices": ["public", "document_parsing"],
46
+ "min_pairs": 100,
47
+ "min_absolute_margin": 0.005,
48
+ "min_pair_win_rate": 0.55
49
+ },
50
+ {
51
+ "baseline": "paddleocr-vl",
52
+ "metric": "reading_order_pair_accuracy",
53
+ "direction": "higher",
54
+ "slices": ["public", "document_parsing"],
55
+ "min_pairs": 100,
56
+ "min_absolute_margin": 0.005,
57
+ "min_pair_win_rate": 0.55
58
+ }
59
+ ]
60
+ }
ocr-tech-mps-integration/configs/validation-public-omnidocbench-surya.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 100,
4
+ "required_slices": {
5
+ "public": 100,
6
+ "document_parsing": 100,
7
+ "table": 40
8
+ },
9
+ "bootstrap_iterations": 1000,
10
+ "bootstrap_confidence": 0.95,
11
+ "random_seed": 13,
12
+ "gates": [
13
+ {
14
+ "baseline": "surya",
15
+ "metric": "cer",
16
+ "direction": "lower",
17
+ "slices": ["public", "document_parsing"],
18
+ "min_pairs": 100,
19
+ "min_absolute_margin": 0.005
20
+ },
21
+ {
22
+ "baseline": "surya",
23
+ "metric": "table_cell_f1",
24
+ "direction": "higher",
25
+ "slices": ["public", "table"],
26
+ "min_pairs": 40,
27
+ "min_absolute_margin": 0.005
28
+ },
29
+ {
30
+ "baseline": "surya",
31
+ "metric": "table_teds_structure",
32
+ "direction": "higher",
33
+ "slices": ["public", "table"],
34
+ "min_pairs": 40,
35
+ "min_absolute_margin": 0.005
36
+ },
37
+ {
38
+ "baseline": "surya",
39
+ "metric": "table_teds_text",
40
+ "direction": "higher",
41
+ "slices": ["public", "table"],
42
+ "min_pairs": 40,
43
+ "min_absolute_margin": 0.005
44
+ }
45
+ ]
46
+ }
ocr-tech-mps-integration/configs/validation-public-real5-sota-v1.json ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 100,
4
+ "required_slices": {
5
+ "public": 100
6
+ },
7
+ "bootstrap_iterations": 1000,
8
+ "bootstrap_confidence": 0.95,
9
+ "random_seed": 13,
10
+ "require_full_eval_manifest_coverage": true,
11
+ "gates": [
12
+ {
13
+ "baseline": "surya",
14
+ "metric": "cer",
15
+ "direction": "lower",
16
+ "slices": ["public"],
17
+ "min_pairs": 100,
18
+ "min_absolute_margin": 0.005,
19
+ "min_pair_win_rate": 0.55
20
+ },
21
+ {
22
+ "baseline": "surya",
23
+ "metric": "wer",
24
+ "direction": "lower",
25
+ "slices": ["public"],
26
+ "min_pairs": 100,
27
+ "min_absolute_margin": 0.005,
28
+ "min_pair_win_rate": 0.55
29
+ },
30
+ {
31
+ "baseline": "glm-ocr",
32
+ "metric": "cer",
33
+ "direction": "lower",
34
+ "slices": ["public"],
35
+ "min_pairs": 100,
36
+ "min_absolute_margin": 0.005,
37
+ "min_pair_win_rate": 0.55
38
+ },
39
+ {
40
+ "baseline": "glm-ocr",
41
+ "metric": "wer",
42
+ "direction": "lower",
43
+ "slices": ["public"],
44
+ "min_pairs": 100,
45
+ "min_absolute_margin": 0.005,
46
+ "min_pair_win_rate": 0.55
47
+ },
48
+ {
49
+ "baseline": "paddleocr-vl",
50
+ "metric": "document_reconstruction_similarity",
51
+ "direction": "higher",
52
+ "slices": ["public"],
53
+ "min_pairs": 100,
54
+ "min_absolute_margin": 0.005,
55
+ "min_pair_win_rate": 0.55
56
+ },
57
+ {
58
+ "baseline": "paddleocr-vl",
59
+ "metric": "reading_order_pair_accuracy",
60
+ "direction": "higher",
61
+ "slices": ["public"],
62
+ "min_pairs": 100,
63
+ "min_absolute_margin": 0.005,
64
+ "min_pair_win_rate": 0.55
65
+ }
66
+ ]
67
+ }
ocr-tech-mps-integration/configs/validation-table-ledger-surya.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "candidate": "candidate",
3
+ "min_samples": 40,
4
+ "required_slices": {
5
+ "table": 40,
6
+ "nepali": 40,
7
+ "english": 40
8
+ },
9
+ "bootstrap_iterations": 1000,
10
+ "bootstrap_confidence": 0.95,
11
+ "random_seed": 13,
12
+ "gates": [
13
+ {
14
+ "baseline": "surya",
15
+ "metric": "table_cell_f1",
16
+ "direction": "higher",
17
+ "slices": ["table"],
18
+ "min_pairs": 40,
19
+ "min_absolute_margin": 0.01
20
+ },
21
+ {
22
+ "baseline": "surya",
23
+ "metric": "table_exact_match",
24
+ "direction": "higher",
25
+ "slices": ["table"],
26
+ "min_pairs": 40,
27
+ "min_absolute_margin": 0.01
28
+ },
29
+ {
30
+ "baseline": "surya",
31
+ "metric": "table_structure_similarity",
32
+ "direction": "higher",
33
+ "slices": ["table"],
34
+ "min_pairs": 40,
35
+ "min_absolute_margin": 0.0,
36
+ "require_ci_above_zero": false
37
+ }
38
+ ]
39
+ }
ocr-tech-mps-integration/data/dictionaries/devanagari-v1.txt ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !
2
+ "
3
+ %
4
+ '
5
+ (
6
+ )
7
+ +
8
+ ,
9
+ -
10
+ .
11
+ /
12
+ 0
13
+ 1
14
+ 2
15
+ 3
16
+ 7
17
+ :
18
+ ;
19
+ =
20
+ ?
21
+ A
22
+ C
23
+ D
24
+ N
25
+ O
26
+ [
27
+ ]
28
+ a
29
+ b
30
+ c
31
+ d
32
+ e
33
+ f
34
+ g
35
+ h
36
+ i
37
+ j
38
+ k
39
+ l
40
+ m
41
+ n
42
+ o
43
+ p
44
+ r
45
+ s
46
+ t
47
+ u
48
+ v
49
+ w
50
+ y
51
+ |
52
+ ¥
53
+ ÷
54
+ ʻ
55
+ ँ
56
+ ं
57
+ ः
58
+ अ
59
+ आ
60
+ इ
61
+ ई
62
+ उ
63
+ ऊ
64
+ ऋ
65
+ ऎ
66
+ ए
67
+ ऐ
68
+ ओ
69
+ औ
70
+ क
71
+ ख
72
+ ग
73
+ घ
74
+ ङ
75
+ च
76
+ छ
77
+ ज
78
+ झ
79
+ ञ
80
+ ट
81
+ ठ
82
+ ड
83
+ ढ
84
+ ण
85
+ त
86
+ थ
87
+ द
88
+ ध
89
+ न
90
+ ऩ
91
+ प
92
+ फ
93
+ ब
94
+ भ
95
+ म
96
+ य
97
+ र
98
+ ल
99
+ व
100
+ श
101
+ ष
102
+ स
103
+ ह
104
+ ़
105
+ ऽ
106
+ ा
107
+ ि
108
+ ी
109
+ ु
110
+ ू
111
+ ृ
112
+ ॆ
113
+ े
114
+ ै
115
+ ो
116
+ ौ
117
+ ्
118
+ ।
119
+ ०
120
+ १
121
+ २
122
+ ३
123
+ ४
124
+ ५
125
+ ६
126
+ ७
127
+ ८
128
+ ९
129
+ ॽ
130
+ ‌
131
+ ‍
132
+ –
133
+ —
134
+ ‘
135
+ ’
136
+ ‚
137
+ “
138
+ ”
139
+ …
140
+ √
141
+ ∶
ocr-tech-mps-integration/data/dictionaries/gurungkhema-v1.txt ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 𖄀
2
+ 𖄁
3
+ 𖄂
4
+ 𖄃
5
+ 𖄄
6
+ 𖄅
7
+ 𖄆
8
+ 𖄇
9
+ 𖄈
10
+ 𖄉
11
+ 𖄊
12
+ 𖄋
13
+ 𖄌
14
+ 𖄍
15
+ 𖄎
16
+ 𖄏
17
+ 𖄐
18
+ 𖄑
19
+ 𖄒
20
+ 𖄓
21
+ 𖄔
22
+ 𖄕
23
+ 𖄖
24
+ 𖄗
25
+ 𖄘
26
+ 𖄙
27
+ 𖄚
28
+ 𖄛
29
+ 𖄜
30
+ 𖄝
31
+ 𖄞
32
+ 𖄟
33
+ 𖄠
34
+ 𖄡
35
+ 𖄢
36
+ 𖄣
37
+ 𖄤
38
+ 𖄥
39
+ 𖄦
40
+ 𖄧
41
+ 𖄨
42
+ 𖄩
43
+ 𖄪
44
+ 𖄫
45
+ 𖄬
46
+ 𖄭
47
+ 𖄮
48
+ 𖄯
49
+ 𖄰
50
+ 𖄱
51
+ 𖄲
52
+ 𖄳
53
+ 𖄴
54
+ 𖄵
55
+ 𖄶
56
+ 𖄷
57
+ 𖄸
58
+ 𖄹
ocr-tech-mps-integration/data/dictionaries/kiratrai-v1.txt ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 𖵀
2
+ 𖵁
3
+ 𖵂
4
+ 𖵃
5
+ 𖵄
6
+ 𖵅
7
+ 𖵆
8
+ 𖵇
9
+ 𖵈
10
+ 𖵉
11
+ 𖵊
12
+ 𖵋
13
+ 𖵌
14
+ 𖵍
15
+ 𖵎
16
+ 𖵏
17
+ 𖵐
18
+ 𖵑
19
+ 𖵒
20
+ 𖵓
21
+ 𖵔
22
+ 𖵕
23
+ 𖵖
24
+ 𖵗
25
+ 𖵘
26
+ 𖵙
27
+ 𖵚
28
+ 𖵛
29
+ 𖵜
30
+ 𖵝
31
+ 𖵞
32
+ 𖵟
33
+ 𖵠
34
+ 𖵡
35
+ 𖵢
36
+ 𖵣
37
+ 𖵤
38
+ 𖵥
39
+ 𖵦
40
+ 𖵧
41
+ 𖵨
42
+ 𖵩
43
+ 𖵪
44
+ 𖵫
45
+ 𖵬
46
+ 𖵭
47
+ 𖵮
48
+ 𖵯
49
+ 𖵰
50
+ 𖵱
51
+ 𖵲
52
+ 𖵳
53
+ 𖵴
54
+ 𖵵
55
+ 𖵶
56
+ 𖵷
57
+ 𖵸
58
+ 𖵹
ocr-tech-mps-integration/data/dictionaries/lepcha-v1.txt ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ᰀ
2
+ ᰁ
3
+ ᰂ
4
+ ᰃ
5
+ ᰄ
6
+ ᰅ
7
+ ᰆ
8
+ ᰇ
9
+ ᰈ
10
+ ᰉ
11
+ ᰊ
12
+ ᰋ
13
+ ᰌ
14
+ ᰍ
15
+ ᰎ
16
+ ᰏ
17
+ ᰐ
18
+ ᰑ
19
+ ᰒ
20
+ ᰓ
21
+ ᰔ
22
+ ᰕ
23
+ ᰖ
24
+ ᰗ
25
+ ᰘ
26
+ ᰙ
27
+ ᰚ
28
+ ᰛ
29
+ ᰜ
30
+ ᰝ
31
+ ᰞ
32
+ ᰟ
33
+ ᰠ
34
+ ᰡ
35
+ ᰢ
36
+ ᰣ
37
+ ᰤ
38
+ ᰥ
39
+ ᰦ
40
+ ᰧ
41
+ ᰨ
42
+ ᰩ
43
+ ᰪ
44
+ ᰫ
45
+ ᰬ
46
+ ᰭ
47
+ ᰮ
48
+ ᰯ
49
+ ᰰ
50
+ ᰱ
51
+ ᰲ
52
+ ᰳ
53
+ ᰴ
54
+ ᰵ
55
+ ᰶ
56
+ ᰷
57
+ ᰻
58
+ ᰼
59
+ ᰽
60
+ ᰾
61
+ ᰿
62
+ ᱀
63
+ ᱁
64
+ ᱂
65
+ ᱃
66
+ ᱄
67
+ ᱅
68
+ ᱆
69
+ ᱇
70
+ ᱈
71
+ ᱉
72
+ ᱍ
73
+ ᱎ
74
+ ᱏ
ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v1.txt ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ )
2
+ ,
3
+ -
4
+ .
5
+ :
6
+ |
7
+ ।
8
+ ॥
9
+ ᤀ
10
+ ᤁ
11
+ ᤂ
12
+ ᤃ
13
+ ᤄ
14
+ ᤅ
15
+ ᤆ
16
+ ᤇ
17
+ ᤈ
18
+ ᤉ
19
+ ᤊ
20
+ ᤋ
21
+ ᤌ
22
+ ᤍ
23
+ ᤎ
24
+ ᤏ
25
+ ᤐ
26
+ ᤑ
27
+ ᤒ
28
+ ᤓ
29
+ ᤔ
30
+ ᤕ
31
+ ᤖ
32
+ ᤗ
33
+ ᤘ
34
+ ᤙ
35
+ ᤚ
36
+ ᤛ
37
+ ᤜ
38
+ ᤝ
39
+ ᤞ
40
+ ᤠ
41
+ ᤡ
42
+ ᤢ
43
+ ᤣ
44
+ ᤤ
45
+ ᤥ
46
+ ᤦ
47
+ ᤧ
48
+ ᤨ
49
+ ᤩ
50
+ ᤪ
51
+ ᤫ
52
+ ᤰ
53
+ ᤱ
54
+ ᤲ
55
+ ᤳ
56
+ ᤴ
57
+ ᤵ
58
+ ᤶ
59
+ ᤷ
60
+ ᤸ
61
+ ᤹
62
+ ᤺
63
+ ᤻
64
+ ᥀
65
+ ᥄
66
+ ᥅
67
+ ᥆
68
+ ᥇
69
+ ᥈
70
+ ᥉
71
+ ᥊
72
+ ᥋
73
+ ᥌
74
+ ᥍
75
+ ᥎
76
+ ᥏
ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v2.txt ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !
2
+ "
3
+ '
4
+ (
5
+ )
6
+ *
7
+ +
8
+ ,
9
+ -
10
+ .
11
+ /
12
+ :
13
+ =
14
+ ?
15
+ S
16
+ X
17
+ [
18
+ ]
19
+ a
20
+ c
21
+ i
22
+ s
23
+ t
24
+ |
25
+ ।
26
+ ॥
27
+ ᤀ
28
+ ᤁ
29
+ ᤂ
30
+ ᤃ
31
+ ᤄ
32
+ ᤅ
33
+ ᤆ
34
+ ᤇ
35
+ ᤈ
36
+ ᤋ
37
+ ᤌ
38
+ ᤍ
39
+ ᤎ
40
+ ᤏ
41
+ ᤐ
42
+ ᤑ
43
+ ᤒ
44
+ ᤓ
45
+ ᤔ
46
+ ᤕ
47
+ ᤖ
48
+ ᤗ
49
+ ᤘ
50
+ ᤙ
51
+ ᤛ
52
+ ᤜ
53
+ ᤠ
54
+ ᤡ
55
+ ᤢ
56
+ ᤣ
57
+ ᤤ
58
+ ᤥ
59
+ ᤦ
60
+ ᤧ
61
+ ᤨ
62
+ ᤩ
63
+ ᤪ
64
+ ᤫ
65
+ ᤰ
66
+ ᤱ
67
+ ᤳ
68
+ ᤴ
69
+ ᤵ
70
+ ᤶ
71
+ ᤷ
72
+ ᤸ
73
+ ᤹
74
+ ᤺
75
+ ᤻
76
+ ᥄
77
+ ᥅
78
+ ᥆
79
+ ᥇
80
+ ᥈
81
+ ᥉
82
+ ᥊
83
+ ᥋
84
+ ᥌
85
+ ᥍
86
+ ᥎
87
+ ᥏
88
+ ‌
89
+ ‘
90
+ ’
91
+ …
ocr-tech-mps-integration/data/dictionaries/limbu-sirijonga-v3.txt ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !
2
+ "
3
+ '
4
+ (
5
+ )
6
+ *
7
+ +
8
+ ,
9
+ -
10
+ .
11
+ /
12
+ :
13
+ =
14
+ ?
15
+ S
16
+ X
17
+ [
18
+ ]
19
+ a
20
+ c
21
+ i
22
+ s
23
+ t
24
+ |
25
+ ।
26
+ ॥
27
+ ᤀ
28
+ ᤁ
29
+ ᤂ
30
+ ᤃ
31
+ ᤄ
32
+ ᤅ
33
+ ᤆ
34
+ ᤇ
35
+ ᤈ
36
+ ᤋ
37
+ ᤌ
38
+ ᤍ
39
+ ᤎ
40
+ ᤏ
41
+ ᤐ
42
+ ᤑ
43
+ ᤒ
44
+ ᤓ
45
+ ᤔ
46
+ ᤕ
47
+ ᤖ
48
+ ᤗ
49
+ ᤘ
50
+ ᤙ
51
+ ᤛ
52
+ ᤜ
53
+ ᤠ
54
+ ᤡ
55
+ ᤢ
56
+ ᤣ
57
+ ᤤ
58
+ ᤥ
59
+ ᤦ
60
+ ᤧ
61
+ ᤨ
62
+ ᤩ
63
+ ᤪ
64
+ ᤫ
65
+ ᤰ
66
+ ᤱ
67
+ ᤳ
68
+ ᤴ
69
+ ᤵ
70
+ ᤶ
71
+ ᤷ
72
+ ᤸ
73
+ ᤹
74
+ ᤺
75
+ ᤻
76
+ ᥄
77
+ ᥅
78
+ ᥆
79
+ ᥇
80
+ ᥈
81
+ ᥉
82
+ ᥊
83
+ ᥋
84
+ ᥌
85
+ ᥍
86
+ ᥎
87
+ ᥏
88
+ ‌
89
+ ‘
90
+ ’
91
+ …
92
+ ᤉ
93
+ ᤊ
94
+ ᤚ
95
+ ᤝ
96
+ ᤞ
97
+ ᤲ
98
+ ᥀
ocr-tech-mps-integration/data/dictionaries/newa-clean-v1.txt ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !
2
+ %
3
+ (
4
+ )
5
+ +
6
+ ,
7
+ -
8
+ .
9
+ /
10
+ 0
11
+ 1
12
+ 2
13
+ 6
14
+ 8
15
+ 9
16
+ :
17
+ ;
18
+ ?
19
+ ~
20
+  
21
+ ¯
22
+ ‍
23
+ –
24
+ ‘
25
+ ’
26
+ “
27
+ ”
28
+ •
29
+ 𑐀
30
+ 𑐁
31
+ 𑐂
32
+ 𑐃
33
+ 𑐄
34
+ 𑐅
35
+ 𑐆
36
+ 𑐈
37
+ 𑐊
38
+ 𑐌
39
+ 𑐍
40
+ 𑐎
41
+ 𑐏
42
+ 𑐐
43
+ 𑐑
44
+ 𑐒
45
+ 𑐔
46
+ 𑐕
47
+ 𑐖
48
+ 𑐗
49
+ 𑐘
50
+ 𑐚
51
+ 𑐛
52
+ 𑐜
53
+ 𑐝
54
+ 𑐞
55
+ 𑐟
56
+ 𑐠
57
+ 𑐡
58
+ 𑐢
59
+ 𑐣
60
+ 𑐤
61
+ 𑐥
62
+ 𑐦
63
+ 𑐧
64
+ 𑐨
65
+ 𑐩
66
+ 𑐫
67
+ 𑐬
68
+ 𑐮
69
+ 𑐰
70
+ 𑐱
71
+ 𑐲
72
+ 𑐳
73
+ 𑐴
74
+ 𑐵
75
+ 𑐶
76
+ 𑐷
77
+ 𑐸
78
+ 𑐹
79
+ 𑐺
80
+ 𑐻
81
+ 𑐾
82
+ 𑐿
83
+ 𑑀
84
+ 𑑁
85
+ 𑑂
86
+ 𑑃
87
+ 𑑄
88
+ 𑑅
89
+ 𑑆
90
+ 𑑇
91
+ 𑑋
92
+ 𑑌
93
+ 𑑍
94
+ 𑑐
95
+ 𑑑
96
+ 𑑒
97
+ 𑑓
98
+ 𑑔
99
+ 𑑕
100
+ 𑑖
101
+ 𑑗
102
+ 𑑘
103
+ 𑑙
ocr-tech-mps-integration/data/dictionaries/newa-htr-v1.txt ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+
3
+ !
4
+ %
5
+ (
6
+ )
7
+ +
8
+ ,
9
+ -
10
+ .
11
+ /
12
+ 0
13
+ 1
14
+ 2
15
+ 3
16
+ 4
17
+ 6
18
+ 7
19
+ 8
20
+ 9
21
+ :
22
+ ;
23
+ ?
24
+ A
25
+ B
26
+ C
27
+ D
28
+ E
29
+ F
30
+ G
31
+ H
32
+ I
33
+ J
34
+ L
35
+ M
36
+ N
37
+ O
38
+ P
39
+ R
40
+ S
41
+ U
42
+ W
43
+ Y
44
+ [
45
+ ]
46
+ a
47
+ b
48
+ c
49
+ d
50
+ e
51
+ f
52
+ g
53
+ h
54
+ i
55
+ j
56
+ k
57
+ l
58
+ m
59
+ n
60
+ o
61
+ p
62
+ r
63
+ s
64
+ t
65
+ u
66
+ v
67
+ w
68
+ y
69
+ z
70
+ ~
71
+  
72
+ ¯
73
+ »
74
+ ‍
75
+ –
76
+ ‘
77
+ ’
78
+ “
79
+ ”
80
+ •
81
+ 𑐀
82
+ 𑐁
83
+ 𑐂
84
+ 𑐃
85
+ 𑐄
86
+ 𑐅
87
+ 𑐆
88
+ 𑐇
89
+ 𑐈
90
+ 𑐉
91
+ 𑐊
92
+ 𑐋
93
+ 𑐌
94
+ 𑐍
95
+ 𑐎
96
+ 𑐏
97
+ 𑐐
98
+ 𑐑
99
+ 𑐒
100
+ 𑐓
101
+ 𑐔
102
+ 𑐕
103
+ 𑐖
104
+ 𑐗
105
+ 𑐘
106
+ 𑐙
107
+ 𑐚
108
+ 𑐛
109
+ 𑐜
110
+ 𑐝
111
+ 𑐞
112
+ 𑐟
113
+ 𑐠
114
+ 𑐡
115
+ 𑐢
116
+ 𑐣
117
+ 𑐤
118
+ 𑐥
119
+ 𑐦
120
+ 𑐧
121
+ 𑐨
122
+ 𑐩
123
+ 𑐪
124
+ 𑐫
125
+ 𑐬
126
+ 𑐭
127
+ 𑐮
128
+ 𑐯
129
+ 𑐰
130
+ 𑐱
131
+ 𑐲
132
+ 𑐳
133
+ 𑐴
134
+ 𑐵
135
+ 𑐶
136
+ 𑐷
137
+ 𑐸
138
+ 𑐹
139
+ 𑐺
140
+ 𑐻
141
+ 𑐼
142
+ 𑐽
143
+ 𑐾
144
+ 𑐿
145
+ 𑑀
146
+ 𑑁
147
+ 𑑂
148
+ 𑑃
149
+ 𑑄
150
+ 𑑅
151
+ 𑑆
152
+ 𑑇
153
+ 𑑈
154
+ 𑑉
155
+ 𑑊
156
+ 𑑋
157
+ 𑑌
158
+ 𑑍
159
+ 𑑎
160
+ 𑑏
161
+ 𑑐
162
+ 𑑑
163
+ 𑑒
164
+ 𑑓
165
+ 𑑔
166
+ 𑑕
167
+ 𑑖
168
+ 𑑗
169
+ 𑑘
170
+ 𑑙
171
+ 𑑚
172
+ 𑑛
173
+ 𑑝
174
+ 𑑞
175
+ 𑑟
176
+ 𑑠
177
+ 𑑡
ocr-tech-mps-integration/data/dictionaries/newa-native-v1.txt ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !
2
+ %
3
+ (
4
+ )
5
+ *
6
+ +
7
+ ,
8
+ -
9
+ .
10
+ /
11
+ 0
12
+ 1
13
+ 2
14
+ 3
15
+ 4
16
+ 5
17
+ 6
18
+ 7
19
+ 8
20
+ 9
21
+ :
22
+ ;
23
+ ?
24
+ A
25
+ B
26
+ C
27
+ D
28
+ E
29
+ F
30
+ H
31
+ I
32
+ J
33
+ L
34
+ M
35
+ N
36
+ O
37
+ P
38
+ R
39
+ S
40
+ T
41
+ U
42
+ W
43
+ Y
44
+ [
45
+ ]
46
+ _
47
+ a
48
+ b
49
+ c
50
+ d
51
+ e
52
+ f
53
+ g
54
+ h
55
+ i
56
+ k
57
+ l
58
+ m
59
+ n
60
+ o
61
+ p
62
+ r
63
+ s
64
+ t
65
+ u
66
+ v
67
+ w
68
+ y
69
+ z
70
+ ~
71
+ ¯
72
+ »
73
+ ə
74
+ ˈ
75
+ ्
76
+ ।
77
+ ॥
78
+ ॰
79
+ ‍
80
+ –
81
+ ‘
82
+ ’
83
+ “
84
+ ”
85
+ •
86
+ …
87
+ ・
88
+ 𑐀
89
+ 𑐁
90
+ 𑐂
91
+ 𑐃
92
+ 𑐄
93
+ 𑐅
94
+ 𑐆
95
+ 𑐈
96
+ 𑐊
97
+ 𑐌
98
+ 𑐍
99
+ 𑐎
100
+ 𑐏
101
+ 𑐐
102
+ 𑐑
103
+ 𑐒
104
+ 𑐔
105
+ 𑐕
106
+ 𑐖
107
+ 𑐗
108
+ 𑐘
109
+ 𑐚
110
+ 𑐛
111
+ 𑐜
112
+ 𑐝
113
+ 𑐞
114
+ 𑐟
115
+ 𑐠
116
+ 𑐡
117
+ 𑐢
118
+ 𑐣
119
+ 𑐤
120
+ 𑐥
121
+ 𑐦
122
+ 𑐧
123
+ 𑐨
124
+ 𑐩
125
+ 𑐫
126
+ 𑐬
127
+ 𑐮
128
+ 𑐰
129
+ 𑐱
130
+ 𑐲
131
+ 𑐳
132
+ 𑐴
133
+ 𑐵
134
+ 𑐶
135
+ 𑐷
136
+ 𑐸
137
+ 𑐹
138
+ 𑐺
139
+ 𑐻
140
+ 𑐾
141
+ 𑐿
142
+ 𑑀
143
+ 𑑁
144
+ 𑑂
145
+ 𑑃
146
+ 𑑄
147
+ 𑑅
148
+ 𑑆
149
+ 𑑇
150
+ 𑑉
151
+ 𑑋
152
+ 𑑌
153
+ 𑑍
154
+ 𑑐
155
+ 𑑑
156
+ 𑑒
157
+ 𑑓
158
+ 𑑔
159
+ 𑑕
160
+ 𑑖
161
+ 𑑗
162
+ 𑑘
163
+ 𑑙
ocr-tech-mps-integration/data/dictionaries/newa-v1.txt ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 𑐀
2
+ 𑐁
3
+ 𑐂
4
+ 𑐃
5
+ 𑐄
6
+ 𑐅
7
+ 𑐆
8
+ 𑐇
9
+ 𑐈
10
+ 𑐉
11
+ 𑐊
12
+ 𑐋
13
+ 𑐌
14
+ 𑐍
15
+ 𑐎
16
+ 𑐏
17
+ 𑐐
18
+ 𑐑
19
+ 𑐒
20
+ 𑐓
21
+ 𑐔
22
+ 𑐕
23
+ 𑐖
24
+ 𑐗
25
+ 𑐘
26
+ 𑐙
27
+ 𑐚
28
+ 𑐛
29
+ 𑐜
30
+ 𑐝
31
+ 𑐞
32
+ 𑐟
33
+ 𑐠
34
+ 𑐡
35
+ 𑐢
36
+ 𑐣
37
+ 𑐤
38
+ 𑐥
39
+ 𑐦
40
+ 𑐧
41
+ 𑐨
42
+ 𑐩
43
+ 𑐪
44
+ 𑐫
45
+ 𑐬
46
+ 𑐭
47
+ 𑐮
48
+ 𑐯
49
+ 𑐰
50
+ 𑐱
51
+ 𑐲
52
+ 𑐳
53
+ 𑐴
54
+ 𑐵
55
+ 𑐶
56
+ 𑐷
57
+ 𑐸
58
+ 𑐹
59
+ 𑐺
60
+ 𑐻
61
+ 𑐼
62
+ 𑐽
63
+ 𑐾
64
+ 𑐿
65
+ 𑑀
66
+ 𑑁
67
+ 𑑂
68
+ 𑑃
69
+ 𑑄
70
+ 𑑅
71
+ 𑑆
72
+ 𑑇
73
+ 𑑈
74
+ 𑑉
75
+ 𑑊
76
+ 𑑋
77
+ 𑑌
78
+ 𑑍
79
+ 𑑎
80
+ 𑑏
81
+ 𑑐
82
+ 𑑑
83
+ 𑑒
84
+ 𑑓
85
+ 𑑔
86
+ 𑑕
87
+ 𑑖
88
+ 𑑗
89
+ 𑑘
90
+ 𑑙
91
+ 𑑚
92
+ 𑑛
93
+ 𑑝
94
+ 𑑞
95
+ 𑑟
96
+ 𑑠
97
+ 𑑡
ocr-tech-mps-integration/data/dictionaries/sunuwar-v1.txt ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 𑯀
2
+ 𑯁
3
+ 𑯂
4
+ 𑯃
5
+ 𑯄
6
+ 𑯅
7
+ 𑯆
8
+ 𑯇
9
+ 𑯈
10
+ 𑯉
11
+ 𑯊
12
+ 𑯋
13
+ 𑯌
14
+ 𑯍
15
+ 𑯎
16
+ 𑯏
17
+ 𑯐
18
+ 𑯑
19
+ 𑯒
20
+ 𑯓
21
+ 𑯔
22
+ 𑯕
23
+ 𑯖
24
+ 𑯗
25
+ 𑯘
26
+ 𑯙
27
+ 𑯚
28
+ 𑯛
29
+ 𑯜
30
+ 𑯝
31
+ 𑯞
32
+ 𑯟
33
+ 𑯠
34
+ 𑯡
35
+ 𑯰
36
+ 𑯱
37
+ 𑯲
38
+ 𑯳
39
+ 𑯴
40
+ 𑯵
41
+ 𑯶
42
+ 𑯷
43
+ 𑯸
44
+ 𑯹
ocr-tech-mps-integration/data/dictionaries/tirhuta-v1.txt ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 𑒀
2
+ 𑒁
3
+ 𑒂
4
+ 𑒃
5
+ 𑒄
6
+ 𑒅
7
+ 𑒆
8
+ 𑒇
9
+ 𑒈
10
+ 𑒉
11
+ 𑒊
12
+ 𑒋
13
+ 𑒌
14
+ 𑒍
15
+ 𑒎
16
+ 𑒏
17
+ 𑒐
18
+ 𑒑
19
+ 𑒒
20
+ 𑒓
21
+ 𑒔
22
+ 𑒕
23
+ 𑒖
24
+ 𑒗
25
+ 𑒘
26
+ 𑒙
27
+ 𑒚
28
+ 𑒛
29
+ 𑒜
30
+ 𑒝
31
+ 𑒞
32
+ 𑒟
33
+ 𑒠
34
+ 𑒡
35
+ 𑒢
36
+ 𑒣
37
+ 𑒤
38
+ 𑒥
39
+ 𑒦
40
+ 𑒧
41
+ 𑒨
42
+ 𑒩
43
+ 𑒪
44
+ 𑒫
45
+ 𑒬
46
+ 𑒭
47
+ 𑒮
48
+ 𑒯
49
+ 𑒰
50
+ 𑒱
51
+ 𑒲
52
+ 𑒳
53
+ 𑒴
54
+ 𑒵
55
+ 𑒶
56
+ 𑒷
57
+ 𑒸
58
+ 𑒹
59
+ 𑒺
60
+ 𑒻
61
+ 𑒼
62
+ 𑒽
63
+ 𑒾
64
+ 𑒿
65
+ 𑓀
66
+ 𑓁
67
+ 𑓂
68
+ 𑓃
69
+ 𑓄
70
+ 𑓅
71
+ 𑓆
72
+ 𑓇
73
+ 𑓐
74
+ 𑓑
75
+ 𑓒
76
+ 𑓓
77
+ 𑓔
78
+ 𑓕
79
+ 𑓖
80
+ 𑓗
81
+ 𑓘
82
+ 𑓙
ocr-tech-mps-integration/data/mappings/limbu-legacy/Limbu.map ADDED
@@ -0,0 +1,146 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ; Limbu.map
2
+ ; Copyright (c) 2007 SIL International.
3
+
4
+ ; TECkit mapping to convert from Limbu legacy encoding (as used by Sirijonga)
5
+ ; to Unicode and back
6
+
7
+ EncodingName "Limbu-Sirijonga"
8
+ DescriptiveName "Limbu"
9
+ Version "1.0"
10
+ Contact "mailto:victor_gaultney@sil.org"
11
+ RegistrationAuthority "SIL International"
12
+ RegistrationName "Limbu-001"
13
+ Copyright "(c)2007 SIL International"
14
+
15
+
16
+ Pass(Byte_Unicode)
17
+
18
+ ByteDefault 0x5E ; number sign
19
+ UniDefault replacement_character
20
+
21
+ ByteClass [CTL] = ( 0x00 .. 0x1F )
22
+ UniClass [CTL] = ( U+0000 .. U+001F )
23
+ [CTL] <> [CTL]
24
+
25
+ ; Unicode chars with no glyphs in legacy font:
26
+ ; U+1909 (LIMBU LETTER JHA)
27
+ ; U+190A (LIMBU LETTER YAN)
28
+ ; U+191A (LIMBU LETTER SSA)
29
+ ; U+1932 (LIMBU SMALL LETTER ANUSVARA)
30
+ ; U+1940 (LIMBU SIGN LOO)
31
+ ; U+1944 (LIMBU EXCLAMATION MARK)
32
+
33
+ ;mappings - * means direct from ASCII, ** means to another Unicode block
34
+ 0x20 <> U+0020 ; space*
35
+ 0x21 <> U+0021 ; exclam*
36
+ 0x22 <> U+0022 ; quotedbl*
37
+ ;0x23 <> ?
38
+ 0x24 <> U+0024 ; dollar* (NOT IN FONT)
39
+ 0x25 <> U+0025 ; percent*
40
+ 0x26 <> U+0026 ; ampersand* (NOT IN FONT)
41
+ 0x27 <> U+1922 ; limbu_vowel_sign_u
42
+ 0x28 <> U+0028 ; parenleft*
43
+ 0x29 <> U+0029 ; parenright*
44
+ 0x2A <> U+002A ; asterisk*
45
+ 0x2B <> U+002B ; plus*
46
+ 0x2C <> U+002C ; comma*
47
+ 0x2D <> U+193B ; limbu_sign_sa-i
48
+ 0x2E <> U+0965 ; devanagari_double_danda**
49
+ 0x2F <> U+002F ; solidus*
50
+ 0x30 <> U+1946 ; limbu_digit_zero
51
+ 0x31 <> U+1947 ; limbu_digit_one
52
+ 0x32 <> U+1948 ; limbu_digit_two
53
+ 0x33 <> U+1949 ; limbu_digit_three
54
+ 0x34 <> U+194A ; limbu_digit_four
55
+ 0x35 <> U+194B ; limbu_digit_five
56
+ 0x36 <> U+194C ; limbu_digit_six
57
+ 0x37 <> U+194D ; limbu_digit_seven
58
+ 0x38 <> U+194E ; limbu_digit_eight
59
+ 0x39 <> U+194F ; limbu_digit_nine
60
+ 0x3A > U+003A ; colon (0x60 is appropriate reverse mapping)
61
+ 0x3B <> U+191B ; limbu_letter_sa
62
+ 0x3C <> U+005B ; bracketleft
63
+ 0x3D <> U+003D ; equal*
64
+ 0x3E <> U+005D ; bracketright*
65
+ 0x3F <> U+003F ; question*
66
+ 0x40 <> U+0040 ; at*
67
+ 0x41 <> U+0041 ; latin_capital_letter_A (NOT IN FONT)
68
+ 0x42 <> U+003B ; semicolon
69
+ 0x43 <> U+1939 ; limbu_sign_mukphreng
70
+ 0x44 <> U+1936 ; limbu_small_letter_ma
71
+ 0x45 <> U+007B ; braceleft
72
+ 0x46 <> U+1916 ; limbu_letter_ra
73
+ 0x47 <> U+1934 ; limbu_small_letter_na
74
+ ;0x48 <> U+0048 ; latin_capital_letter_H (ORIGINAL)
75
+ 0x48 <> U+1922 U+192A ; limbu_vowel_sign_u + limbu_subjoined_letter_ra (NEW)
76
+ 0x49 <> U+1931 ; limbu_small_letter_nga
77
+ 0x4A <> U+192B ; limbu_subjoined_letter_wa
78
+ 0x4B <> U+1935 ; limbu_small_letter_pa
79
+ 0x4C <> U+1921 U+193A ; limbu_vowel_sign_i + limbu_sign_kemphreng
80
+ 0x4D <> U+193A ; limbu_sign_kemphreng
81
+ 0x4E <> U+1938 ; limbu_small_letter_la
82
+ 0x4F <> U+1929 ; limbu_subjoined_letter_ya
83
+ 0x50 <> U+1904 ; limbu_letter_gha
84
+ 0x51 <> U+003C ; less
85
+ 0x52 <> U+007D ; braceright
86
+ 0x53 <> U+1930 ; limbu_small_letter_ka
87
+ 0x54 <> U+1933 ; limbu_small_letter_ta
88
+ 0x55 <> U+1937 ; limbu_small_letter_ra
89
+ 0x56 <> U+002E ; period
90
+ 0x57 <> U+003E ; greater
91
+ ;0x58 <> ? ; is this the same as the limbu exclamation mark?
92
+ 0x59 <> U+0027 ; quotesingle
93
+ 0x5A <> U+1945 ; limbu_question_mark
94
+ 0x5B <> U+1927 ; limbu_vowel_sign_e
95
+ 0x5C <> U+005C ; reverse_solidus*
96
+ 0x5D <> U+1923 ; limbu_vowel_sign_ee (alone)
97
+ 0x5E <> U+0023 ; numbersign
98
+ 0x5F <> U+002D ; hyphen
99
+ 0x60 <> U+003A ; colon
100
+ 0x61 <> U+1912 ; limbu_letter_ba
101
+ 0x62 <> U+190D ; limbu_letter_da
102
+ 0x63 <> U+1900 ; limbu_vowel-carrier_letter
103
+ 0x64 <> U+1914 ; limbu_letter_ma
104
+ 0x65 <> U+1913 ; limbu_letter_bha
105
+ 0x66 0x5D <> U+1925 ; limbu_vowel_sign_oo
106
+ 0x66 0x7D <> U+1926 ; limbu_vowel_sign_au
107
+ 0x66 <> U+1920 ; limbu_vowel_sign_a (alone)
108
+ 0x67 <> U+190F ; limbu_letter_na
109
+ 0x68 <> U+1908 ; limbu_letter_ja
110
+ 0x69 <> U+1905 ; limbu_letter_nga
111
+ 0x6A <> U+1918 ; limbu_letter_wa
112
+ 0x6B <> U+1910 ; limbu_letter_pa
113
+ 0x6C <> U+1921 ; limbu_vowel_sign_i
114
+ 0x6D <> U+1911 ; limbu_letter_pha
115
+ 0x6E <> U+1917 ; limbu_letter_la
116
+ 0x6F <> U+1915 ; limbu_letter_ya
117
+ 0x70 <> U+1907 ; limbu_letter_cha
118
+ 0x71 <> U+192A ; limbu_subjoined_letter_ra
119
+ 0x72 <> U+1906 ; limbu_letter_ca
120
+ 0x73 <> U+1901 ; limbu_letter_ka
121
+ 0x74 <> U+190B ; limbu_letter_ta
122
+ 0x75 <> U+1903 ; limbu_letter_ga
123
+ 0x76 <> U+1902 ; limbu_letter_kha
124
+ 0x77 <> U+190E ; limbu_letter_dha
125
+ 0x78 <> U+191C ; limbu_letter_ha
126
+ 0x79 <> U+190C ; limbu_letter_tha
127
+ 0x7A <> U+1919 ; limbu_letter_sha
128
+ 0x7B <> U+1928 ; limbu_vowel_sign_o
129
+ 0x7C <> U+007C ; bar*
130
+ 0x7D <> U+1924 ; limbu_vowel_sign_ai (alone)
131
+ 0x7E <> U+007E ; asciitilde*
132
+ ;0x7F <> ;
133
+
134
+ 0xD2 <> U+201C ; quotedblleft
135
+ 0xD3 <> U+201D ; quotedblright
136
+ 0xD4 <> U+2018 ; quotesingleleft
137
+ 0xD5 <> U+2019 ; quotesingleright
138
+
139
+ Pass(Unicode)
140
+ ; to reorder the subjoined cons
141
+
142
+ UniClass [VOWEL] = ( U+1920 .. U+1928 )
143
+ UniClass [SUBJ] = ( U+1929 .. U+192B )
144
+
145
+ [VOWEL]=V [SUBJ]=S <> @S @V
146
+ [VOWEL]=V U+193A [SUBJ]=S <> @S @V U+193A
ocr-tech-mps-integration/data/mappings/limbu-legacy/README.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ Compiled and uncompiled TECkit conversion tables are included here. The Limbu one maps data using the Namdhinggo SIL L custom-encoded font to Unicode.
ocr-tech-mps-integration/docs/ACTIVE_RESEARCH_CHECKLIST.md ADDED
@@ -0,0 +1,386 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Active Research Checklist
2
+
3
+ This checklist records the current steering decision for the Nepal multilingual
4
+ OCR paper track. Keep it updated when remote data, synthesis converters,
5
+ validation packs, or claim gates change.
6
+
7
+ ## Current Goal
8
+
9
+ Build a PaddleOCR-centered document OCR engine for at least ten languages
10
+ spoken in Nepal. Limbu/Sirijonga is the first non-Devanagari target. The
11
+ engine must support OCR, reading order, tables, figures, captions, Markdown,
12
+ and structured JSON. Synthetic data can train and ablate the system, but paper
13
+ claims require real held-out validation, especially Gorkhapatra/Naya Nepal
14
+ page evidence.
15
+
16
+ ## Non-Negotiable Gates
17
+
18
+ - [ ] Keep PaddleOCR/PP-OCR as the primary engine family.
19
+ - [ ] Use PP-Structure-style document extraction for layout, reading order,
20
+ table reconstruction, figures, captions, Markdown, and JSON.
21
+ - [ ] Use PaddleOCR-VL or another VLM only as fallback, teacher, or hard-page
22
+ comparator; report fallback rate separately.
23
+ - [ ] Do not call a language synthesis-ready until a text manifest passes
24
+ `ocrtech audit-synthesis-text-manifest`.
25
+ - [ ] Do not call rendered OCR data training-ready until rendered outputs pass
26
+ `ocrtech audit-rendered-synthesis-lines`.
27
+ - [ ] For split synthesis text, render with `render-synthesis-text-split` so
28
+ train/eval labels and rendered audits are created together.
29
+ - [ ] Do not use synthetic samples as claim evidence.
30
+ - [ ] Do not promote Gorkhapatra/Naya Nepal pages to claim evidence until the
31
+ review audit and reference audit pass.
32
+ - [ ] Keep Nepali and English non-regression gates in every claim-grade run.
33
+
34
+ ## cdjk Remote Synthesis Path
35
+
36
+ Remote host verified read-only on 2026-06-16:
37
+
38
+ ```text
39
+ host: cdjk@100.110.148.43
40
+ machine: cdjks-MacBook-Pro.local
41
+ disk: /Volumes/TRANSCEND, 3.6Ti total, 2.0Ti available
42
+ ```
43
+
44
+ Confirmed `~/github/llm` leads:
45
+
46
+ - `/Users/cdjk/github/llm/limbu-speech-toolkit`
47
+ - `/Users/cdjk/github/llm/rai-speech-toolkit`
48
+ - `/Users/cdjk/github/llm/dhimal-toolkit`
49
+ - `/Users/cdjk/github/llm/gurung-language-toolkit`
50
+ - `/Users/cdjk/github/llm/magar-toolkit`
51
+ - `/Users/cdjk/github/llm/ltk`
52
+ - `/Users/cdjk/github/llm/ltk/limbu`
53
+ - `/Users/cdjk/github/llm/ltk/tamang`
54
+ - `/Users/cdjk/github/llm/ltk/magar`
55
+ - `/Users/cdjk/github/llm/nepa-newa-text-frontend`
56
+ - `/Users/cdjk/github/llm/nepali_text_frontend`
57
+ - `/Users/cdjk/github/llm/devanagari-OCR`
58
+ - `/Users/cdjk/github/llm/nllb-better`
59
+
60
+ Confirmed `/Volumes/TRANSCEND` leads:
61
+
62
+ - `/Volumes/TRANSCEND/bible_brain_nepal/lif/LIFWBT_Limbu`
63
+ - `/Volumes/TRANSCEND/bible_brain_nepal/dhi/DHIDPI_Dhimal`
64
+ - `/Volumes/TRANSCEND/bible_brain_nepal/gvr/GVRWBT_Gurung`
65
+ - `/Volumes/TRANSCEND/bible_brain_nepal/new/NEWNCL_Newari`
66
+ - `/Volumes/TRANSCEND/bible_brain_nepal/taj/TAJWBT_Tamang__Eastern`
67
+ - `/Volumes/TRANSCEND/bible_brain_nepal/npi/NPIBIB_Nepali`
68
+ - `/Volumes/TRANSCEND/bible_brain_nepal/npi/NPIDPI_Nepali`
69
+ - `/Volumes/TRANSCEND/nepali_dataset_prep`
70
+ - `/Volumes/TRANSCEND/noiseremover_nepal18_timed_pipeline/bible_brain`
71
+ - `/Volumes/TRANSCEND/noiseremover_nepal18_timed_pipeline/speaker_cluster_limbu`
72
+
73
+ Next remote steps:
74
+
75
+ - [x] Selectively migrate text corpus/dictionary resources from
76
+ `cdjk@100.110.148.43` into ignored local `data/` without writing generated
77
+ corpus data back to the source machine. Tracked policy:
78
+ `configs/cdjk-language-corpus-migration-v1.json`.
79
+ - [x] Run `ocrtech audit-synthesis-resources` on the migrated roots:
80
+ `passed=True roots=6/6 files=2308 warnings=0`.
81
+ - [x] Prepare and audit first migrated synthesis-text manifests:
82
+ Limbu Limdic `26152` rows, Limbu-only subset `8772` rows, Tamang `46314`
83
+ rows, and Magar Western dictionary `1182` rows; all audits passed with
84
+ `claim_eligible=0`.
85
+ - [x] Render a corrected Limbu-only Namdhinggo smoke set with six degradation
86
+ profiles: `120` images, rendered audit `passed=True`, `issues=0`,
87
+ `warnings=0`.
88
+ - [ ] Convert the same local migration into full split train/eval synthesis
89
+ manifests with exclusion sets and license policy before large-scale
90
+ rendering.
91
+ - [ ] Inspect actual schemas for each candidate file before writing converters.
92
+ Do not infer fields from filenames.
93
+ - [x] Run explicit field-mapped conversion recipes for Gurung, Dhimal, and Rai
94
+ dialect corpora discovered in the migration.
95
+ - [x] Run explicit field-mapped conversion for extra Magar
96
+ `mgp/source/mscore` monolingual rows and `mgp/npi` lexicon/parallel rows.
97
+ - [x] Audit 20 migrated synthesis-text manifests after the second sweep:
98
+ `228212` total rows, `202560` Devanagari rows, `8772` true Unicode Limbu
99
+ rows, and all manifest audits passed with `claim_eligible=0`.
100
+ - [ ] Add a combined index/report command for migrated synthesis-text manifests
101
+ so the aggregate counts do not depend on ad hoc `jq` commands.
102
+ - [x] Add an explicit-field converter for remote toolkit JSONL/TSV/CSV parallel
103
+ corpora.
104
+ - [x] Add `audit-synthesis-text-promotion` so remote synthesis text must carry
105
+ split policy and must not overlap excluded eval/reference text before
106
+ rendering.
107
+ - [x] Add `split-synthesis-text` so promoted train/eval synthesis text is
108
+ grouped by `text_sha256`, stamped with split policy, and promotion-audited.
109
+ - [x] Smoke-run the converter on real remote Magar toolkit JSONL and pass
110
+ `audit-synthesis-text-manifest` with Magar/Nepali Devanagari gates.
111
+ - [x] Smoke-run that converter on selected Limbu, Rai, Dhimal, Gurung, and
112
+ Magar remote toolkit files, then audit the emitted synthesis-text manifests.
113
+ - [ ] Promote remote toolkit smoke commands into full split synthesis-text
114
+ manifests that pass `split-synthesis-text` with exclusion sets and license
115
+ policy appropriate to the run.
116
+ - [ ] Prioritize remaining converters in this order: Limbu Pangloss XML/TSV,
117
+ Bible Brain text manifests, Nepali prep TSV/transcript roots.
118
+ - [ ] Record license state per source. Bible Brain and derived text/audio roots
119
+ stay `pending_review` until cleared.
120
+ - [ ] Emit source path, source SHA-256, text SHA-256, language, script, Unicode
121
+ block counts, split policy, and `claim_evidence_eligible=false` for every
122
+ synthesis text row.
123
+ - [ ] Deduplicate synthesis text against all real validation references before
124
+ rendering.
125
+ - [ ] Render promoted train/eval synthesis text with verified fonts and
126
+ `--require-font` before using labels for PaddleOCR training.
127
+ - [ ] Verify fonts and Unicode renderability before rendering new scripts.
128
+ - [x] Add Namdhinggo 3.100 as the primary Limbu/Sirijonga font candidate and
129
+ keep Noto Sans Limbu as a fallback runtime font.
130
+ - [x] Add `configs/namdhinggo-font-assets-v1.json` so Namdhinggo 3.100 is
131
+ reproducibly provisioned from the upstream SIL release ZIP with package and
132
+ extracted-member SHA-256 checks.
133
+ - [x] Add `ocrtech inventory-fonts` for directory/ZIP font collection audits
134
+ with SHA-256, cmap readability, script coverage, duplicate detection, and
135
+ JSON/CSV/Markdown outputs.
136
+ - [x] Inventory current runtime font corpus:
137
+ `outputs/font-inventory-current-2026-06-16` has `17` font files, `7`
138
+ cmap-readable fonts, and `0` duplicate SHA groups.
139
+ - [x] Inventory `/Users/cdjk/github/llm/ltk/fonts.zip`: found `janaki.ttf` and
140
+ `madan.ttf`; both are cmap-readable but do not match registry Unicode ranges,
141
+ so they stay legacy/conversion leads rather than OCR rendering fonts.
142
+ - [x] Probed `cdjk@100.110.148.43` under `~/github/llm` for font assets and
143
+ copied direct font files into ignored `data/fonts/remote-cdjk-2026-06-16`.
144
+ - [x] Inventory remote-copied fonts:
145
+ `outputs/font-inventory-remote-cdjk-2026-06-16` has `4` readable fonts. Only
146
+ Noto Sans Devanagari covers registered Devanagari targets; the 8-bit
147
+ Devanagari font stays a legacy conversion lead.
148
+ - [x] Download and inventory Google Fonts OFL batch:
149
+ Kanchenjunga Regular/Bold for Kirat Rai, Noto Sans Tirhuta for Tirhuta,
150
+ Noto Serif Devanagari for print-style Devanagari, and Noto Sans Ol Chiki
151
+ Variable for Santhali style diversity.
152
+ - [x] Add `configs/nepal-font-assets-v1.json` plus
153
+ `ocrtech prepare-font-assets` so the Google Fonts batch is reproducibly
154
+ provisioned with SHA-256 checks instead of manual curl commands.
155
+ - [x] Add `ocrtech audit-font-asset-readiness` to require passing asset
156
+ preparation, font inventory, and all-script renderability evidence before
157
+ treating a runtime font corpus as synthesis-ready.
158
+ - [x] Connect rendered synthesis audits to font readiness evidence via
159
+ `--font-readiness-report`; rendered rows now fail training-bound audits when
160
+ their `font_sha256` is not in the passed readiness manifest.
161
+ - [x] Reaudit the Namdhinggo-rendered Limbu/Sirijonga train/eval split against
162
+ Namdhinggo font readiness evidence:
163
+ `train=150`, `eval=27`, `issues=0`, `warnings=0`.
164
+ - [x] Add deterministic rendered-line degradation profiles for PaddleOCR
165
+ recognizer training: `clean`, `scan`, `low_light`, `uneven_light`,
166
+ `glare_light`, `camera_left`, `camera_right`, `camera_top`,
167
+ `camera_bottom`, and `phone_photo`. Every generated image now records the
168
+ selected profile, seed, and parameters in manifest metadata.
169
+ - [x] Add `ocrtech audit-rendered-degradation-split` to verify expected
170
+ degradation profiles per text hash and reject train/eval `text_sha256`
171
+ overlap in augmented rendered bundles.
172
+ - [x] Generate and audit the full Namdhinggo augmented Limbu/Sirijonga rendered
173
+ split: `900` train rows, `162` eval rows, six profiles per text hash,
174
+ `issues=0`, `warnings=0`.
175
+ - [x] Materialize the Limbu/Sirijonga augmentation ablation spec:
176
+ `configs/limbu-sirijonga-augmentation-ablation-v1.json`, with clean,
177
+ lighting, and camera arms plus training recipe directories for each arm.
178
+ - [x] Audit all ablation arms for profile balance and train/eval text-hash
179
+ isolation: clean `150/27`, lighting `450/81`, camera `600/108`,
180
+ all `issues=0`, `warnings=0`.
181
+ - [x] Include illumination and capture-angle robustness in the first
182
+ Sirijonga augmentation plan: `low_light`, `uneven_light`, `glare_light`,
183
+ `camera_left`, `camera_right`, `camera_top`, `camera_bottom`, and
184
+ `phone_photo` are now deterministic profiles with recorded seeds and
185
+ parameters.
186
+ - [x] Upgrade camera/phone-photo profiles from affine shear to deterministic
187
+ perspective transforms, and require `source_quad`, `output_quad`, and
188
+ `perspective_coefficients` in rendered-manifest audits.
189
+ - [x] Tighten rendered-manifest audits so lighting profiles must record
190
+ numeric brightness/contrast, gradient/blend, or glare metadata instead of
191
+ relying on profile names alone.
192
+ - [x] Add `ocrtech audit-augmentation-ablation` so Limbu/Sirijonga synthetic
193
+ ablation arms cannot be treated as promotion-ready without a passed real
194
+ Gorkhapatra verified-reference review audit.
195
+ - [x] Run the current ablation promotion audit:
196
+ `passed=False arms=3 real_evidence=0 verified_references=0 issues=1`.
197
+ All synthetic arms pass internally, but the intended blocker is missing real
198
+ verified Gorkhapatra evidence.
199
+ - [ ] Run a controlled Limbu/Sirijonga augmentation ablation: clean-only vs
200
+ lighting-only vs camera-angle/phone-photo mix, evaluated on real
201
+ Gorkhapatra/Naya Nepal pages and held-out synthetic rows without text-hash
202
+ leakage.
203
+ - [x] Promote coverage-proven Google Fonts candidates into
204
+ `languages/nepal-v1.json` and rerun all-script renderability:
205
+ `passed=True scripts=22 candidates=58 rendered=0`.
206
+ - [ ] Collect and inventory additional print-style Unicode Devanagari, Ol
207
+ Chiki, Kirat Rai, Tibetan/Tamyig, and other Nepal-script fonts with license
208
+ notes before using them for synthesis diversity.
209
+
210
+ ## Gorkhapatra Validation Path
211
+
212
+ - [ ] Start from `https://gorkhapatraonline.com/publications/naya-nepal`.
213
+ - [x] Support paginated Naya Nepal publication discovery with
214
+ `audit-gorkhapatra-language-pages --publication-pages N`; Limbu appears on
215
+ page 9 in the 2026-06-16 live scan.
216
+ - [x] Capture article publication dates and language labels.
217
+ - [x] Query the epaper endpoint for the matching publication date:
218
+ `https://epaper.gorkhapatraonline.com/epaper-list?...&slug=gorkhapatra`.
219
+ - [x] Download the direct PDF from the viewer `file=` path, not the PDF.js
220
+ shell.
221
+ - [x] Search/render the language page, including `bhasa pristha` /
222
+ `भाषा पृष्ठ` candidates.
223
+ - [x] Extract PDF-native text spans, font names, bboxes, codepoints, and
224
+ embedded fonts before OCR/manual transcription. This is now implemented as
225
+ `ocrtech extract-gorkhapatra-language-page-pdf-text`.
226
+ - [x] Run the PDF-native extractor on the current Limbu pack:
227
+ `passed=True samples=2/2 spans=776 target_spans=304 fonts=10 warnings=0`.
228
+ Page 6 (`gorkhapatra-language-page-0001-01`) has `304`
229
+ `NamdhinggoSILL` spans and extracted embedded font
230
+ `UFNWLJ+NamdhinggoSILL`; page 7 has `0` target-font spans.
231
+ - [x] Compare the blog image, PDF page, and PDF font metadata to choose the
232
+ exact page candidate. Current conclusion: page 6 is the Limbu/Sirijonga
233
+ page; page 7 is the same-date non-Limbu page.
234
+ - [ ] Build or locate a Namdhinggo/Gorkhapatra legacy encoding converter to
235
+ Unicode Limbu (`U+1900..U+194F`) so extracted PDF spans can become reference
236
+ text instead of manual transcription burden.
237
+ - [ ] Finalize only after manual review accepts exactly one candidate for the
238
+ publication/language/date group.
239
+ - [x] Add a candidate-language filter to the Gorkhapatra language-page review
240
+ audit/finalize commands so Limbu can be gated independently from other
241
+ accepted Naya Nepal languages on the same PDF.
242
+ - [x] Run the filtered Limbu verified-reference audit:
243
+ `passed=False samples=2 groups=1 accepted=1 rejected=1 issues=1 warnings=0`.
244
+ The only blocker is empty verified reference text for
245
+ `gorkhapatra-language-page-0001-01`.
246
+ - [x] Add a transcription work-order generator for accepted Gorkhapatra
247
+ language-page rows so manual labeling is explicit, auditable, and scoped by
248
+ target language.
249
+ - [x] Generate the Limbu work order:
250
+ `passed=True samples=1 blocked=1 verified=0 warnings=0`. The selected work
251
+ item is `gorkhapatra-language-page-0001-01`, page 6 of the 2083-02-12
252
+ Gorkhapatra PDF.
253
+ - [x] Correct the work-order priority: for Gorkhapatra epapers, PDF-native
254
+ extraction is upstream of OCR and manual transcription. The transcription
255
+ dashboard is fallback/verification only, especially for Limbu/Sirijonga where
256
+ OCR candidates are currently invalid.
257
+ - [x] Add `prepare-gorkhapatra-language-page-assisted-references` for
258
+ non-claim-ready machine OCR draft references. Drafts preserve
259
+ `reference_status=draft`, `claim_evidence_eligible=false`,
260
+ `machine_generated=true`, and `requires_human_review=true`.
261
+ - [x] Add `prepare-gorkhapatra-language-page-ocr-sidecars` so accepted
262
+ Gorkhapatra rows can produce reproducible OCR sidecars before assisted
263
+ reference drafting.
264
+ - [x] Run PaddleOCR sidecar generation for the current Limbu page on k2:
265
+ `pages=1`, `lines=299`, `nonzero_bbox=299`,
266
+ `average_confidence=0.5347613170992171`.
267
+ - [x] Rerun the assisted-reference command on the current Limbu page:
268
+ `passed=True samples=1 assisted=1 failed=0 skipped=3 warnings=0`. The draft
269
+ now contains `298` non-empty OCR text lines and remains non-claim-ready.
270
+ - [x] Add `prepare-gorkhapatra-language-page-verification-bundle` for
271
+ line-level human correction sheets from assisted references.
272
+ - [x] Add `apply-gorkhapatra-language-page-verification-bundle` to convert a
273
+ completed line-review CSV into verified `.ref.json` files and an updated
274
+ page-review CSV, while blocking `pending`, `needs_context`, unknown, or empty
275
+ reviewed rows.
276
+ - [x] Tighten verified-reference application so claim-eligible references also
277
+ require explicit table, figure, and caption review declarations. Checked
278
+ absent is acceptable; silently inheriting empty draft arrays is not.
279
+ - [x] Generate the current Limbu verification bundle:
280
+ `passed=True samples=1 lines=298 crops=298 missing_bboxes=0 warnings=0`.
281
+ The CSV review sheet and line crops are ready for human correction.
282
+ - [x] Add an interactive HTML review dashboard to every verification bundle so
283
+ reviewers can scan line crops, ids, order, confidence, and OCR text, set
284
+ `review_status`, enter corrected text, add notes, and export a reviewed CSV
285
+ that still must pass the audit/apply gates.
286
+ - [x] Add `audit-gorkhapatra-language-page-verification-csv` for dry-run
287
+ status counts and blocker reports before writing verified references.
288
+ - [x] Add `split-gorkhapatra-language-page-verification-csv` to produce
289
+ reviewer-sized CSV/HTML batches while preserving line ids and order.
290
+ - [x] Add `merge-gorkhapatra-language-page-verification-batches` so reviewed
291
+ batch CSVs can be recombined only when every original `(sample_id, line_id)`
292
+ appears exactly once.
293
+ - [x] Add `assign-gorkhapatra-language-page-verification-batches` so split
294
+ batches have an explicit reviewer/due-date ledger for paper-grade provenance.
295
+ - [x] Run the real Limbu split/merge smoke:
296
+ `split passed=True samples=1 lines=298 batches=6 batch_size=50 warnings=0`;
297
+ untouched-batch merge `passed=True source_lines=298 merged_lines=298 batch_files=6 issues=0 warnings=0`;
298
+ merged audit still blocks with `pending=298`.
299
+ - [x] Run the current Limbu verification CSV audit:
300
+ `passed=False samples=1 lines=298 ready=0 dropped=0 blocked=298 issues=298 warnings=0`;
301
+ all rows remain `pending`.
302
+ - [ ] Write structured references with text, reading order, tables, figures,
303
+ captions, and Markdown where present.
304
+ - [ ] Run `audit-gorkhapatra-language-page-review` and `audit-references`
305
+ before using the pack in claim-grade validation.
306
+
307
+ ## Paper Evidence Checklist
308
+
309
+ - [ ] Every benchmark report records input hashes, output artifact hashes,
310
+ model-card hashes, latency, memory, and fallback usage.
311
+ - [ ] Every public or real eval row has a parsed and audited reference.
312
+ - [ ] Claim review rejects stale artifacts, missing references, malformed
313
+ structured references, and synthetic-only evidence.
314
+ - [ ] Error analysis includes failures by language, script, layout type,
315
+ degradation, table presence, and fallback trigger reason.
316
+ - [ ] Paper tables separate OCR metrics, document-structure metrics, resource
317
+ metrics, and fallback metrics.
318
+
319
+ ## Limbu/Sirijonga Current Evidence
320
+
321
+ - [x] Built split-safe Limbu Unicode synthesis text:
322
+ `outputs/sirijonga-limbu-split-2026-06-16/train.jsonl` and
323
+ `outputs/sirijonga-limbu-split-2026-06-16/eval.jsonl`.
324
+ - [x] Split policy is
325
+ `split_synthesis_text:text_sha256:seed=17:eval_ratio=0.15`, with `170`
326
+ train rows, `30` eval rows, and `200` text-hash groups.
327
+ - [x] Rendered the pure Limbu/Sirijonga subset with font provenance:
328
+ `150` train lines and `27` eval lines passed rendered-line audits using
329
+ Noto Sans Limbu.
330
+ - [x] Rerendered the same split with Namdhinggo Regular; this is now the
331
+ preferred Sirijonga training bundle.
332
+ - [x] Reaudited the Namdhinggo-rendered split with
333
+ `--font-readiness-report outputs/font-asset-readiness-namdhinggo-2026-06-16/font-asset-readiness.json`;
334
+ the training bundle is now tied to reproducible Namdhinggo asset provenance.
335
+ - [x] Regenerated the preferred Namdhinggo split with explicit degradation
336
+ profiles after confirming the clean split remained reproducible. The
337
+ degradation split audit confirms clean/degraded variants stay on the same
338
+ split side.
339
+ - [x] Add a Limbu/Sirijonga PaddleOCR dictionary covering `U+1900..U+194F`
340
+ plus accepted punctuation.
341
+ - [x] Audit that dictionary against the Namdhinggo-rendered train/eval label
342
+ files: `76` dictionary characters, `68` required Limbu-range characters,
343
+ `64` observed label characters, `0` missing required, `0` missing labels.
344
+ Label spaces are covered by PaddleOCR `use_space_char=true`, not by a literal
345
+ whitespace dictionary row.
346
+ - [ ] Find or label real Gorkhapatra/Naya Nepal Sirijonga pages before any
347
+ claim-grade evaluation.
348
+ - [x] Found the real Limbu Naya Nepal item:
349
+ `https://gorkhapatraonline.com/news/207555`, date `2083-02-12`, article image
350
+ `https://gorkhapatraonline.com/storage/media/360954/conversions/6-original.jpg`.
351
+ - [x] Matched that date to Gorkhapatra epaper PDF
352
+ `2026-05-26-03-43-57-2083-02-12.pdf`, SHA-256
353
+ `afe1d3d2e64248745d94d511e5ca456abddd3ad828480baff87263029b987cd8`.
354
+ - [x] Rendered same-date page candidates and visually disambiguated:
355
+ page 6 is Limbu/Sirijonga, page 7 is Majhi/Devanagari.
356
+ - [x] Created an accepted-row reviewer bundle for manual reference labeling:
357
+ `data/limbu-gorkhapatra-language-page-reviewer-bundle-2026-06-16`, with
358
+ `2` images and `2` draft references.
359
+ - [x] Created a Limbu transcription work order:
360
+ `data/limbu-gorkhapatra-language-page-transcription-work-order-2026-06-16`.
361
+ It confirms one accepted Limbu page is blocked only by the empty reference
362
+ JSON, not by missing image/PDF provenance.
363
+ - [x] Created a Limbu assisted-reference draft run:
364
+ `data/limbu-gorkhapatra-language-page-assisted-references-2026-06-16`.
365
+ It now contains PaddleOCR-assisted text suggestions from k2 and keeps the
366
+ draft non-claim-ready.
367
+ - [x] Created a Limbu line-level verification bundle:
368
+ `data/limbu-gorkhapatra-language-page-verification-bundle-2026-06-16`, with
369
+ `298` line crops, a pending review CSV, and an HTML review contact sheet.
370
+ - [x] Confirmed the correct PDF-native browser review surface:
371
+ `file:///Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html`.
372
+ Use this full-page clickable PDF-span dashboard for verification. Do not use
373
+ the PDF-native verification-bundle HTML as the review UI; that was identified
374
+ as the wrong/broken interface.
375
+ - [ ] Fill the verification CSV with explicit `accept`, `corrected`, or `drop`
376
+ line statuses, either directly or via
377
+ `split-gorkhapatra-language-page-verification-csv` batches. If batches are
378
+ used, merge them with `merge-gorkhapatra-language-page-verification-batches`.
379
+ Then run `audit-gorkhapatra-language-page-verification-csv`, then run
380
+ `apply-gorkhapatra-language-page-verification-bundle` to produce verified
381
+ references and an updated review CSV with `--tables-reviewed`,
382
+ `--figures-reviewed`, and `--captions-reviewed` after those structures are checked.
383
+ - [ ] Fill verified text/layout references for the accepted Limbu page; strict
384
+ review audit currently fails as intended until that reference is complete.
385
+ - [ ] Re-run the Limbu-only gate after transcription:
386
+ `audit-gorkhapatra-language-page-review ... --candidate-language लिम्बू --require-verified-references`.
ocr-tech-mps-integration/docs/CDJK_LANGUAGE_CORPUS_MIGRATION.md ADDED
@@ -0,0 +1,310 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # cdjk Language Corpus Migration
2
+
3
+ Run date: 2026-06-16.
4
+
5
+ This document records the first local migration of language corpora and
6
+ dictionaries from `cdjk@100.110.148.43` for multilingual OCR synthesis. The raw
7
+ copy lives under ignored `data/`; the reproducible policy is tracked in
8
+ `configs/cdjk-language-corpus-migration-v1.json`.
9
+
10
+ ## Source Host
11
+
12
+ ```text
13
+ host: cdjk@100.110.148.43
14
+ machine: cdjks-MacBook-Pro.local
15
+ home: /Users/cdjk
16
+ external volume: /Volumes/TRANSCEND
17
+ external volume size: 3.6Ti
18
+ external volume free at inventory: 2.0Ti
19
+ ```
20
+
21
+ Local free space at copy time was about `33GiB`, so the migration was
22
+ selective. We copied text-like corpus resources and skipped audio, video,
23
+ models, checkpoints, PDFs, images, git metadata, caches, and virtualenvs.
24
+
25
+ ## Copied Roots
26
+
27
+ Destination:
28
+
29
+ ```text
30
+ data/external-language-resources/cdjk-100.110.148.43/
31
+ ```
32
+
33
+ Copied roots:
34
+
35
+ ```text
36
+ dhimal-toolkit
37
+ gurung-language-toolkit
38
+ limbu-speech-toolkit
39
+ ltk
40
+ magar-toolkit
41
+ rai-speech-toolkit
42
+ ```
43
+
44
+ Copy result:
45
+
46
+ ```text
47
+ files transferred: 2307
48
+ files after copy: 2308
49
+ local size: 1.6G
50
+ ```
51
+
52
+ Suffix counts:
53
+
54
+ ```text
55
+ json 1071
56
+ txt 686
57
+ md 209
58
+ jsonl 179
59
+ tsv 155
60
+ csv 4
61
+ yml 3
62
+ ```
63
+
64
+ Per-root local size:
65
+
66
+ ```text
67
+ gurung-language-toolkit 41M
68
+ magar-toolkit 43M
69
+ ltk 67M
70
+ dhimal-toolkit 94M
71
+ limbu-speech-toolkit 555M
72
+ rai-speech-toolkit 841M
73
+ total 1.6G
74
+ ```
75
+
76
+ ## Resource Audit
77
+
78
+ Command:
79
+
80
+ ```bash
81
+ uv run ocrtech audit-synthesis-resources \
82
+ --root data/external-language-resources/cdjk-100.110.148.43/ltk --label ltk \
83
+ --root data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit --label limbu-speech-toolkit \
84
+ --root data/external-language-resources/cdjk-100.110.148.43/rai-speech-toolkit --label rai-speech-toolkit \
85
+ --root data/external-language-resources/cdjk-100.110.148.43/gurung-language-toolkit --label gurung-language-toolkit \
86
+ --root data/external-language-resources/cdjk-100.110.148.43/magar-toolkit --label magar-toolkit \
87
+ --root data/external-language-resources/cdjk-100.110.148.43/dhimal-toolkit --label dhimal-toolkit \
88
+ --out outputs/cdjk-language-resource-audit-2026-06-16 \
89
+ --max-text-samples-per-root 8
90
+ ```
91
+
92
+ Result:
93
+
94
+ ```text
95
+ passed=True roots=6/6 files=2308 warnings=0
96
+ ```
97
+
98
+ ## Prepared Text Manifests
99
+
100
+ Limbu Limdic:
101
+
102
+ ```bash
103
+ uv run ocrtech prepare-limbu-limdic-text \
104
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/limbu/data/limdic_dictionary.json \
105
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/limbu/data/limdic_dictionary.tsv \
106
+ --out data/synthesis-text/cdjk-limbu-limdic-2026-06-16 \
107
+ --dataset cdjk-limbu-limdic
108
+ ```
109
+
110
+ Result:
111
+
112
+ ```text
113
+ passed=True samples=26152 duplicates=26628 warnings=0
114
+ script_counts:
115
+ Limbu: 8772
116
+ Devanagari: 8747
117
+ Mixed_Devanagari_Latin: 8616
118
+ Mixed_Limbu_Devanagari: 14
119
+ Mixed_Limbu_Latin: 2
120
+ Latin: 1
121
+ ```
122
+
123
+ The Limbu-only subset was created for Namdhinggo rendering:
124
+
125
+ ```bash
126
+ mkdir -p data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered
127
+ jq -c 'select(.script == "Limbu")' \
128
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/limbu-limdic-text.jsonl \
129
+ > data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl
130
+
131
+ uv run ocrtech audit-synthesis-text-manifest \
132
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl \
133
+ --out outputs/cdjk-limbu-only-text-audit-2026-06-16 \
134
+ --require-language limbu \
135
+ --require-script Limbu \
136
+ --min-samples 100
137
+ ```
138
+
139
+ Result:
140
+
141
+ ```text
142
+ passed=True samples=8772 claim_eligible=0 issues=0 warnings=0
143
+ ```
144
+
145
+ Tamang:
146
+
147
+ ```bash
148
+ uv run ocrtech prepare-tamang-text \
149
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/tamang/data/neptam_train.csv \
150
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/tamang/data/neptam_test.csv \
151
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/tamang/data/western_tamang_dictionary.tsv \
152
+ --out data/synthesis-text/cdjk-tamang-2026-06-16 \
153
+ --dataset cdjk-tamang-real-text
154
+ ```
155
+
156
+ Result:
157
+
158
+ ```text
159
+ passed=True samples=46314 duplicates=892 warnings=0
160
+ audit: passed=True samples=46314 claim_eligible=0 issues=0 warnings=0
161
+ ```
162
+
163
+ Magar:
164
+
165
+ ```bash
166
+ uv run ocrtech prepare-magar-text \
167
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/magar/data/western_magar_dictionary.tsv \
168
+ --source data/external-language-resources/cdjk-100.110.148.43/ltk/magar/data/western_magar_dictionary.json \
169
+ --out data/synthesis-text/cdjk-magar-2026-06-16 \
170
+ --dataset cdjk-magar-western-dictionary
171
+ ```
172
+
173
+ Result:
174
+
175
+ ```text
176
+ passed=True samples=1182 duplicates=1246 warnings=0
177
+ audit: passed=True samples=1182 claim_eligible=0 issues=0 warnings=0
178
+ ```
179
+
180
+ Important schema note: `magar-toolkit/data/lexicon/mgp-npi.tsv` and
181
+ `magar-toolkit/data/mono/magar-dhut-clean.tsv` are useful, but they do not
182
+ match the Western Magar dictionary converter schema. They need an explicit
183
+ field-mapped converter path before use.
184
+
185
+ ## Field-Mapped Toolkit Sweep
186
+
187
+ After the first Limbu/Tamang/Magar pass, the explicit-field converter was run
188
+ on Gurung, Dhimal, extra Magar, and Rai dialect sources. Mixed schemas were
189
+ split instead of being forced into one command. Empty split files were skipped.
190
+
191
+ Results:
192
+
193
+ | Manifest | Samples | Duplicates | Notes |
194
+ | --- | ---: | ---: | --- |
195
+ | Gurung toolkit | 17,061 | 79 | `gvr` target text plus Nepali source text |
196
+ | Dhimal parallel | 37,504 | 11,936 | Bible/parallel rows |
197
+ | Dhimal mono | 740 | 2 | Epustakalaya monolingual rows |
198
+ | Magar parallel extra | 20,127 | 1,391 | `mgp/npi` lexicon and MT rows |
199
+ | Magar mono extra | 3,441 | 0 | `mgp/source/mscore` monolingual rows |
200
+ | Rai Bantawa | 747 | 0 | mostly Latin phonetic text |
201
+ | Rai Chamling | 23 | 0 | Devanagari rows |
202
+ | Rai Kulung | 7,956 | 1 | Devanagari rows |
203
+ | Rai Athpariya | 7,861 | 5 | Devanagari rows |
204
+ | Rai Khaling | 1,166 | 84 | mixed Devanagari/Latin source pool |
205
+ | Rai Yakkha | 1,357 | 1,438 | Devanagari rows |
206
+ | Rai Dumi | 1,453 | 42 | Devanagari rows |
207
+ | Rai Thulung | 2,825 | 0 | mostly Latin phonetic text |
208
+ | Rai Koyu | 819 | 0 | mostly Latin phonetic text |
209
+ | Rai Lohorung | 10,202 | 42 | Devanagari rows |
210
+ | Rai Sunwar | 32,165 | 1,445 | Devanagari rows |
211
+ | Rai Limbu | 9,117 | 27 | Devanagari-transcribed Limbu plus some Latin |
212
+
213
+ Aggregate generated synthesis-text inventory after this sweep:
214
+
215
+ ```text
216
+ manifests: 20
217
+ samples: 228212
218
+ duplicates removed/reported by converters: 45258
219
+ script_counts:
220
+ Devanagari: 202560
221
+ Mixed_Devanagari_Latin: 10337
222
+ Latin: 6521
223
+ Limbu: 8772
224
+ Mixed_Limbu_Devanagari: 14
225
+ Mixed_Limbu_Latin: 3
226
+ Other: 5
227
+ ```
228
+
229
+ Interpretation:
230
+
231
+ - Limbu/Sirijonga has true Unicode Limbu rows from Limdic and can render with
232
+ Namdhinggo after filtering `script == Limbu`.
233
+ - Most Gurung, Dhimal, Magar, and Rai rows are Devanagari-transcribed language
234
+ text; these should render with a Devanagari font unless/until native-script
235
+ sources are found.
236
+ - Bantawa, Khaling, Koyu, and Thulung include substantial Latin phonetic data.
237
+ Those rows are useful for corpus/dictionary modeling, but they are not native
238
+ script OCR targets unless the research question explicitly includes romanized
239
+ field notation.
240
+
241
+ ## Limbu Synthetic Smoke
242
+
243
+ First attempt rendered mixed-script rows with Namdhinggo and exposed tofu boxes
244
+ for Devanagari/Latin rows. That output is not training-valid. The corrected
245
+ smoke render uses only `script == Limbu`.
246
+
247
+ Correct command:
248
+
249
+ ```bash
250
+ uv run ocrtech render-synthesis-text-lines \
251
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl \
252
+ --out data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only \
253
+ --font-path data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Regular.ttf \
254
+ --limit 120 \
255
+ --degradation-profile clean \
256
+ --degradation-profile scan \
257
+ --degradation-profile low_light \
258
+ --degradation-profile uneven_light \
259
+ --degradation-profile camera_left \
260
+ --degradation-profile phone_photo
261
+
262
+ uv run ocrtech audit-rendered-synthesis-lines \
263
+ data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/manifests/rendered-line-manifest.jsonl \
264
+ --label-file data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/labels/train.txt \
265
+ --out outputs/cdjk-limbu-rendered-smoke-audit-2026-06-16-limbu-only
266
+ ```
267
+
268
+ Result:
269
+
270
+ ```text
271
+ passed=True samples=120 skipped=0 warnings=0
272
+ audit: passed=True samples=120 claim_eligible=0 issues=0 warnings=0
273
+ degradation_counts:
274
+ camera_left: 20
275
+ clean: 20
276
+ low_light: 20
277
+ phone_photo: 20
278
+ scan: 20
279
+ uneven_light: 20
280
+ ```
281
+
282
+ Visual spot check confirmed actual Limbu glyphs in the corrected render.
283
+
284
+ ## Repeatable Pattern for Each Language
285
+
286
+ 1. Inventory copied source root with `audit-synthesis-resources`.
287
+ 2. Verify actual file schemas before conversion.
288
+ 3. Use a schema-specific converter or explicit field mapping.
289
+ 4. Run `audit-synthesis-text-manifest`.
290
+ 5. Filter by script before selecting a script-specific font.
291
+ 6. Verify font renderability and font readiness.
292
+ 7. Render train/eval with deterministic degradations.
293
+ 8. Run `audit-rendered-synthesis-lines`.
294
+ 9. Keep all synthetic rows marked `claim_evidence_eligible=false`.
295
+ 10. Evaluate claims only on real held-out sources such as verified Gorkhapatra
296
+ pages.
297
+
298
+ ## Next Checklist
299
+
300
+ - [ ] Add explicit field-mapped conversion recipes for Gurung, Dhimal, and Rai
301
+ dialect corpora.
302
+ - [ ] Add Magar converter support for `mgp/source/mscore` monolingual rows and
303
+ `mgp/npi` lexicon rows.
304
+ - [ ] Build a script/font matrix for each language before rendering.
305
+ - [ ] Create train/eval splits with `split-synthesis-text` before large-scale
306
+ rendering.
307
+ - [ ] Run large synthetic rendering on a machine with enough free disk, not on
308
+ this local 33GiB-limited workspace.
309
+ - [ ] Link rendered synthetic bundles to PaddleOCR recognizer recipes only
310
+ after font readiness and rendered audits pass.
ocr-tech-mps-integration/docs/CLAIM_REVIEW.md ADDED
@@ -0,0 +1,333 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Claim Review
2
+
3
+ `review-claim` is the portfolio-level gate for a SOTA claim. `validate-claim` can tell you whether one experiment passed its configured gates. It cannot tell you whether you have enough evidence across the internal Nepali/English pack and public suites to say "better than what is out there." That is what `review-claim` is for.
4
+
5
+ ## What It Audits
6
+
7
+ `review-claim` checks that:
8
+
9
+ - every required experiment slot is backed by a real `experiment.json`
10
+ - every referenced eval manifest and benchmark report still exists at the
11
+ recorded path, and the eval manifest matches the experiment's recorded
12
+ SHA-256
13
+ - successful benchmark rows are bound back to sample ids and input paths in the
14
+ reviewed eval manifest, and row slice labels must match the manifest entry
15
+ - the expected datasets or manifest patterns are present
16
+ - required baselines were actually run
17
+ - required metric families are present on successful rows for the required baselines
18
+ - required metric families are present on the slices they are supposed to
19
+ substantiate when `required_metric_slices` is configured
20
+ - benchmark reports do not contain error rows when the slot requires clean runs;
21
+ status counts are derived from `benchmark/report.json`, not trusted from the
22
+ copied `experiment.json` summary
23
+ - required validation status is present for internal claim-grade experiments,
24
+ and the referenced `validation.json` agrees with `experiment.json`
25
+ - required validation gates passed for the baseline/metric/slice comparisons
26
+ declared by `required_validation_gates`
27
+ - when configured, candidate fallback diagnostics are present and the aggregate
28
+ fallback trigger/failure rates stay under the slot limits
29
+ - required slices exist for the internal Nepali/English pack
30
+ - candidate experiments all point to the same candidate model config
31
+ - the candidate model card matches experiment hashes and passes `audit-model`,
32
+ with or without an explicit `--model-config`
33
+ - if candidate experiments record a fallback model card, that card exists,
34
+ matches the experiment SHA-256, and passes `audit-model`
35
+ - when configured, the candidate model card itself records the same fallback
36
+ component in `provenance.fallback_model_card`
37
+ - when configured, the candidate model card proves it includes a trained local
38
+ recognizer with artifacts, train/eval manifest hashes, and training-summary
39
+ provenance
40
+ - when configured, the candidate model card references a passing admission
41
+ validation report before it can enter portfolio review
42
+ - routed candidates such as `quality_select_composite`, `quality_ranked_ensemble`, and `script_select_composite` were calibrated on a separate eval manifest, not on one of the reviewed experiment manifests
43
+ - routed candidates also fail review if their selected calibration trial collapses to one engine on every calibration sample
44
+ - routed candidates also fail review if a reviewed benchmark shows they are exactly indistinguishable from one of their component baselines on paired holdout CER
45
+
46
+ Metric coverage is a hard gate because the target claim is a document-understanding
47
+ claim, not a plain OCR claim. A required experiment can now declare
48
+ `required_metrics`; each metric must appear as a finite value on at least one
49
+ successful row for every required baseline in that slot. For the internal
50
+ Nepali/English document pack, the SOTA config requires text OCR metrics
51
+ (`cer`, `wer`), document reconstruction
52
+ (`document_reconstruction_similarity`), table reconstruction
53
+ (`table_cell_f1`), reading order (`reading_order_pair_accuracy`), and figure/caption metrics
54
+ (`figure_detection_f1`, `figure_caption_cer`). Public table/document slots
55
+ declare their own metric set.
56
+
57
+ For claim-grade slots, also declare `required_metric_slices`. This prevents a
58
+ metric from satisfying the portfolio check on the wrong document type. For
59
+ example, `table_cell_f1` must be finite on a `table` or document-parsing row,
60
+ and `figure_caption_cer` must be finite on a `figure` row.
61
+
62
+ Metric presence is not enough. When a required slot includes `candidate` and
63
+ one or more external baselines, `review-claim` also requires paired metric
64
+ coverage on the same sample ids. If `required_metric_slices` is configured, the
65
+ pairing is checked on those slices; otherwise it is checked across all rows.
66
+ This prevents a SOTA review from passing because candidate metrics and baseline
67
+ metrics both exist somewhere in the report but never compare the same documents.
68
+
69
+ Sample coverage is also a hard gate. Every `require_baselines` entry must have a
70
+ successful benchmark row for every sample in the reviewed eval manifest. This
71
+ prevents a report from pointing at a hard eval pack while only running an easy
72
+ subset.
73
+
74
+ When a slot declares `require_validation_status`, `review-claim` reads the
75
+ referenced validation report and verifies `claim_status`. For `validated`, it
76
+ also requires `passed: true`. The copied status inside `experiment.json` is not
77
+ enough on its own. The validation artifact must also carry provenance linking it
78
+ to the same benchmark report, eval manifest, validation config, model cards,
79
+ and train manifest hashes referenced by the experiment.
80
+
81
+ Use `required_validation_gates` for claim-grade slots. Metric coverage proves
82
+ that a benchmark emitted a metric; passing validation gates prove the candidate
83
+ beat a named baseline under the configured sample count, slices, margins, and
84
+ statistical checks. Review requires each matched gate to carry quantitative
85
+ paired evidence, not just `passed: true`: positive `pairs`, finite candidate
86
+ and baseline means, finite absolute improvement, and win/loss/tie counts that
87
+ sum to the pair count. It also re-checks the threshold fields embedded in the
88
+ gate result, including `min_pairs`, margins, `min_pair_win_rate`, `max_p_value`,
89
+ and `require_ci_above_zero`, so a hand-edited report cannot mark a weak gate as
90
+ passed without satisfying its own configured test.
91
+
92
+ The SOTA validation configs also use `min_pair_win_rate` on improvement gates.
93
+ This is checked as wins divided by all paired samples, not only non-tied pairs.
94
+ It prevents an average-only improvement, or a single win with many ties, from
95
+ supporting a broad "better OCR/document engine" claim.
96
+
97
+ `required_validation_gates` is only valid on slots that also declare
98
+ `require_validation_status`. Each required gate baseline must also appear in
99
+ `require_baselines`, and each gate metric must appear in `required_metrics`.
100
+
101
+ For claim-grade slots, set `required_validation_config` to the checked-in
102
+ validation config path. `review-claim` hashes that file and requires
103
+ `experiment.json` to record the same `validation_config_sha256`. This prevents a
104
+ portfolio from passing with an easier local validation config that happens to
105
+ contain a subset of the expected gates.
106
+
107
+ For hybrid candidates, use `require_candidate_fallback_metrics`,
108
+ `max_candidate_fallback_trigger_rate`, and
109
+ `max_candidate_fallback_trigger_rate_by_slice`, and
110
+ `max_candidate_fallback_failed_rate`. These checks make fallback dependency
111
+ auditable. A candidate that wins mostly by routing pages to the fallback is a
112
+ different claim from a local-first engine with selective fallback, even if both
113
+ produce strong output metrics. The per-slice trigger limits matter because a
114
+ low aggregate fallback rate can still hide misuse on ordinary English or
115
+ Nepali text pages.
116
+
117
+ When an experiment records `environment.fallback_min_quality_score`,
118
+ `review-claim` also expects candidate rows to include
119
+ `fallback_primary_quality_score` and `fallback_min_quality_score` whenever
120
+ `require_candidate_fallback_metrics` is enabled. That keeps quality-gated
121
+ fallback policies auditable per sample instead of only recording the threshold
122
+ once in `experiment.json`.
123
+
124
+ Fallback model cards are audited separately from the primary candidate card.
125
+ When `experiment.json` records `fallback_model_config` and
126
+ `fallback_model_config_sha256`, `review-claim` verifies the hash and runs
127
+ `audit-model` on that fallback card. The SOTA profile also sets
128
+ `require_fallback_model_present: true`,
129
+ `require_candidate_fallback_provenance: true`,
130
+ `require_shared_fallback_model: true` and
131
+ `require_fallback_model_audit_passed: true`, so a portfolio cannot silently mix
132
+ unaudited fallback components across required experiment slots or omit fallback
133
+ provenance while still claiming the selective-fallback engine.
134
+
135
+ The candidate card must also declare the fallback component it was built to use:
136
+
137
+ ```json
138
+ {
139
+ "provenance": {
140
+ "fallback_model_card": {
141
+ "path": "runs/fallback/model-card.json",
142
+ "sha256": "..."
143
+ }
144
+ }
145
+ }
146
+ ```
147
+
148
+ `review-claim` hashes that model card, compares it with every reviewed
149
+ `fallback_model_config_sha256`, and audits the fallback card. This closes the
150
+ gap where an experiment could record a fallback at benchmark time while the
151
+ candidate card itself remained ambiguous about which selective-fallback engine
152
+ was being claimed.
153
+
154
+ That second guardrail matters for exactly the case we hit on `k2`: a rendered-document `script_select_composite` calibrated on `claimpack-rendered-v2` chose `primary_script_threshold=0.0`, meaning the route degenerated to "always pick Surya." Separation from the final holdout is not enough there; the candidate is still not a meaningful new routed model.
155
+
156
+ The holdout-equivalence guardrail covers the next failure mode after calibration separation: even when calibration was done on a separate split, a routed candidate can still reduce to one component baseline on the reviewed holdout. If the paired benchmark rows show `0` wins, `0` losses, and all ties on CER against a component baseline, `review-claim` now fails `candidate-distinctness`.
157
+
158
+ For the SOTA track, set `require_trained_recognizer: true`. A direct
159
+ `paddleocr` or `hf_vision_encoder_decoder` candidate passes this check only when
160
+ the model card has recorded artifacts plus `provenance.train_manifest`,
161
+ `provenance.eval_manifest`, and `provenance.training_summary` with SHA-256
162
+ hashes. Review resolves those paths from the model-card location and verifies
163
+ the files still match the recorded hashes. Composite candidates must point to
164
+ the trained component with:
165
+
166
+ ```json
167
+ {
168
+ "provenance": {
169
+ "trained_recognizer_model_card": {
170
+ "path": "runs/recognizer/package/model-card.json",
171
+ "sha256": "..."
172
+ }
173
+ }
174
+ }
175
+ ```
176
+
177
+ This prevents a wrapper around Tesseract, Surya, or a VLM from satisfying the
178
+ paper objective by benchmark routing alone.
179
+
180
+ For recognizer branch admission, set `require_model_admission: true` and record
181
+ the admission validation report in the model card:
182
+
183
+ ```json
184
+ {
185
+ "metrics": {
186
+ "admission_validation": {
187
+ "path": "outputs/validation-next-recognizer-hard50/validation.json",
188
+ "sha256": "...",
189
+ "required_status": "validated"
190
+ }
191
+ }
192
+ }
193
+ ```
194
+
195
+ This catches the failure mode we hit with the PaddleOCR branches: the model can
196
+ have artifacts and train/eval provenance, but still be empirically rejected by
197
+ the hard recognizer gate. Rejected admission reports are summarized with their
198
+ failed requirements and gate deltas, so review output can show why a recognizer
199
+ was rejected without manually opening the raw validation JSON. Admission reports
200
+ are also bound back to `provenance.candidate_model_config`; review rejects a
201
+ passing admission artifact if it points at a different model id, backend, base
202
+ model, or artifact set.
203
+
204
+ For document-engine SOTA claims, metric presence is not enough. The
205
+ `configs/claim-review-sota-v1.json` portfolio requires passing validation gates
206
+ for the named claim dimensions:
207
+
208
+ - OCR: CER and WER gates on Nepali/English slices where applicable
209
+ - tables: `table_cell_f1`
210
+ - reading order: `reading_order_pair_accuracy`
211
+ - figures: `figure_detection_f1` and `figure_caption_cer`
212
+ - reconstruction: `document_reconstruction_similarity`
213
+
214
+ The corresponding validation configs must contain matching gates. The
215
+ `tests/test_claim_review_sota_config.py` checks pin this relationship so a
216
+ future config edit cannot silently downgrade figure/caption or WER from a
217
+ passing gate to a reported-only metric.
218
+
219
+ The SOTA profile also requires exported structured-output artifacts from the
220
+ candidate benchmark rows:
221
+
222
+ - every required slot: `document.json`, `document.md`, and `quality.json`
223
+ - table slices: table CSV and HTML exports under `tables/`
224
+ - figure slices: `figures/metadata.json` and exported figure image files
225
+
226
+ For current benchmark reports, `review-claim` uses the candidate row's
227
+ `output_artifacts` records and verifies each recorded path, SHA-256 hash, and
228
+ byte size before inspecting content. Older reports without `output_artifacts`
229
+ fall back to path inference from the benchmark report directory and candidate
230
+ row input stem. JSON artifacts must parse with the expected top-level shape,
231
+ Markdown/table/image files must be nonempty, and figure metadata must contain
232
+ entries. This means a claim cannot be supported only by scalar metrics, stale
233
+ files, or swapped artifacts if the human-readable Markdown, machine-readable
234
+ JSON, tables, or figures were not actually produced as usable artifacts by the
235
+ reviewed benchmark row.
236
+
237
+ ## Example Config
238
+
239
+ ```json
240
+ {
241
+ "require_trained_recognizer": true,
242
+ "require_model_admission": true,
243
+ "require_fallback_model_present": true,
244
+ "require_candidate_fallback_provenance": true,
245
+ "require_shared_fallback_model": true,
246
+ "require_fallback_model_audit_passed": true,
247
+ "required_experiments": [
248
+ {
249
+ "name": "internal-nepali-pack",
250
+ "datasets": ["ocrtech-eval-pack"],
251
+ "require_validation_status": "validated",
252
+ "require_baselines": ["candidate", "tesseract", "stock-paddle", "glm-ocr", "paddleocr-vl"],
253
+ "required_metrics": ["cer", "wer", "document_reconstruction_similarity", "table_cell_f1", "reading_order_pair_accuracy", "figure_detection_f1", "figure_caption_cer"],
254
+ "required_metric_slices": {
255
+ "cer": ["nepali", "english"],
256
+ "wer": ["nepali", "english"],
257
+ "document_reconstruction_similarity": ["table", "reading_order", "figure"],
258
+ "table_cell_f1": ["table"],
259
+ "reading_order_pair_accuracy": ["reading_order"],
260
+ "figure_detection_f1": ["figure"],
261
+ "figure_caption_cer": ["figure"]
262
+ },
263
+ "required_validation_gates": [
264
+ {"baseline": "tesseract", "metric": "cer", "slices": ["nepali"]},
265
+ {"baseline": "stock-paddle", "metric": "cer", "slices": ["nepali"]},
266
+ {"baseline": "glm-ocr", "metric": "cer", "slices": ["nepali"]},
267
+ {"baseline": "surya", "metric": "table_cell_f1", "slices": ["table"]},
268
+ {"baseline": "paddleocr-vl", "metric": "reading_order_pair_accuracy", "slices": ["reading_order"]}
269
+ ],
270
+ "require_candidate_fallback_metrics": true,
271
+ "max_candidate_fallback_trigger_rate": 0.35,
272
+ "max_candidate_fallback_failed_rate": 0.0,
273
+ "required_slices": ["nepali", "english", "table", "reading_order", "figure", "scan"]
274
+ },
275
+ {
276
+ "name": "ocrbench-v2",
277
+ "datasets": ["ocrbench-v2"],
278
+ "require_validation_status": "validated",
279
+ "required_metrics": ["cer", "wer"],
280
+ "require_baselines": ["candidate", "glm-ocr", "paddleocr-vl"]
281
+ },
282
+ {
283
+ "name": "omnidocbench",
284
+ "datasets": ["omnidocbench"],
285
+ "require_validation_status": "validated",
286
+ "required_metrics": ["cer", "document_reconstruction_similarity", "table_cell_f1", "reading_order_pair_accuracy"],
287
+ "require_baselines": ["candidate", "paddleocr-vl"]
288
+ }
289
+ ]
290
+ }
291
+ ```
292
+
293
+ The config is intentionally explicit. If two experiments match one slot, the review fails. Tighten the config until every required slot maps to one concrete artifact.
294
+ `eval_manifest_patterns` use shell-style wildcards, so a slot such as
295
+ `"*Real5-OmniDocBench*"` matches a manifest path containing that benchmark name.
296
+
297
+ The checked-in SOTA profile is paired with slot-specific validation configs:
298
+
299
+ - `configs/validation-internal-sota-v1.json`
300
+ - `configs/validation-public-omnidocbench-sota-v1.json`
301
+ - `configs/validation-public-real5-sota-v1.json`
302
+
303
+ Those configs are expected to produce the passed gates referenced by
304
+ `required_validation_gates`.
305
+
306
+ ## Run
307
+
308
+ ```bash
309
+ ocrtech review-claim \
310
+ --experiment outputs/internal-claim/experiment.json \
311
+ --experiment outputs/ocrbench-v2/experiment.json \
312
+ --experiment outputs/omnidocbench/experiment.json \
313
+ --config claim-review.json \
314
+ --model-config runs/recognizer/package/model-card.json \
315
+ --out outputs/claim-review
316
+ ```
317
+
318
+ ## Outputs
319
+
320
+ ```text
321
+ outputs/claim-review/
322
+ claim-review.json
323
+ claim-review.md
324
+ ```
325
+
326
+ If this report fails, the claim is not review-ready.
327
+
328
+ Do not hand-edit copied `experiment.json` files to point at convenient local
329
+ paths. If an experiment was run remotely, sync the benchmark report, manifest,
330
+ references, model card, and validation report together, or regenerate the
331
+ experiment report from the synced paths. `review-claim` intentionally fails on
332
+ stale paths instead of guessing because path rewrites without hashes destroy
333
+ claim provenance.
ocr-tech-mps-integration/docs/CLAUDE_HANDOFF.md ADDED
@@ -0,0 +1,613 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Claude Handoff: Nepal Multilingual OCR
2
+
3
+ Goal: build and publish a research-grade PaddleOCR/PP-Structure OCR engine for
4
+ at least ten languages spoken in Nepal, using Gorkhapatra/Naya Nepal pages as
5
+ real validation evidence, PDF-native and legacy-font extraction for weak labels,
6
+ and script-aware synthetic data for training and ablation.
7
+
8
+ Updated: 2026-06-16
9
+ Repo: `Ampixa/ocr-tech`
10
+ Visibility: private at handoff time
11
+ Branch: `main`
12
+
13
+ This handoff is for a follow-on Claude/Codex agent. The user is steering a
14
+ research-grade OCR project, not a demo. The target is a PaddleOCR-centered
15
+ document OCR engine whose claims are tied to real validation and reproducible
16
+ training data, not synthetic-only benchmarks.
17
+
18
+ ## Operating Rules
19
+
20
+ - Use `uv run ...` for project commands.
21
+ - Use `apply_patch` for manual edits.
22
+ - Do not commit generated corpora, rendered images, model runs, or outputs
23
+ unless explicitly requested; `data/`, `outputs/`, and `runs/` are ignored
24
+ intentionally.
25
+ - Do commit reproducibility configs, docs, tests, and source code changes.
26
+ - GPG signing fails on this machine; use:
27
+
28
+ ```bash
29
+ git -c commit.gpgsign=false commit -m "<message>"
30
+ ```
31
+
32
+ - Do not commit remote passwords or credentials. Hosts are documented below,
33
+ but credentials must stay in the conversation/session, not tracked files.
34
+ - Do not claim SOTA or model quality unless backed by a passed real validation
35
+ audit and benchmark.
36
+ - Synthetic OCR data is never claim evidence. It is training/ablation material.
37
+ - Validate schemas before writing converters. Do not infer fields from file
38
+ names.
39
+
40
+ ## Current Research Goal
41
+
42
+ Build a PaddleOCR/PP-OCR + PP-Structure-style document engine for at least ten
43
+ languages spoken in Nepal.
44
+
45
+ Core scope:
46
+
47
+ - OCR recognition
48
+ - reading order
49
+ - tables
50
+ - figures/captions
51
+ - Markdown output
52
+ - JSON output
53
+ - optional VLM fallback for hard figures/pages
54
+
55
+ Current research angle:
56
+
57
+ > Low-resource Nepal language OCR adaptation using PDF-native weak labels,
58
+ > legacy font conversion, real Gorkhapatra validation, and script-aware
59
+ > synthetic robustness.
60
+
61
+ The real claim should be about measurable OCR/structure improvement on held-out
62
+ Gorkhapatra/Naya Nepal pages, not about "understanding everything."
63
+
64
+ ## Repo State
65
+
66
+ Important recent commits:
67
+
68
+ ```text
69
+ 22454e2 Document cdjk language corpus migration
70
+ 1f98430 Record migrated synthesis text sweep
71
+ ```
72
+
73
+ The repo was checked with:
74
+
75
+ ```bash
76
+ jq . configs/cdjk-language-corpus-migration-v1.json >/dev/null
77
+ git diff --check
78
+ uv run pytest tests/test_synthesis_resources.py -q
79
+ ```
80
+
81
+ All passed before this handoff was written.
82
+
83
+ At handoff time, the only unrelated visible untracked path was:
84
+
85
+ ```text
86
+ .playwright-mcp/
87
+ ```
88
+
89
+ That is browser scratch output and should not be committed.
90
+
91
+ ## Most Important Existing Docs
92
+
93
+ Read these before acting:
94
+
95
+ - `docs/ACTIVE_RESEARCH_CHECKLIST.md`
96
+ - `docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md`
97
+ - `docs/CDJK_LANGUAGE_CORPUS_MIGRATION.md`
98
+ - `docs/SYNTHESIS_RESOURCE_AUDIT.md`
99
+ - `docs/FONT_ASSETS.md`
100
+ - `docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md`
101
+ - `docs/NEPAL_MULTILINGUAL_OCR_GOAL.md`
102
+
103
+ ## What Has Been Built
104
+
105
+ ### CLI Surface
106
+
107
+ The repo already has many `ocrtech` commands. Most relevant now:
108
+
109
+ ```text
110
+ ocrtech audit-gorkhapatra-language-pages
111
+ ocrtech prepare-gorkhapatra-language-page-pack
112
+ ocrtech prepare-gorkhapatra-language-page-review
113
+ ocrtech audit-gorkhapatra-language-page-review
114
+ ocrtech extract-gorkhapatra-language-page-pdf-text
115
+ ocrtech convert-limbu-legacy
116
+ ocrtech audit-synthesis-resources
117
+ ocrtech prepare-limbu-limdic-text
118
+ ocrtech prepare-limbu-unicode-text
119
+ ocrtech prepare-tamang-text
120
+ ocrtech prepare-magar-text
121
+ ocrtech prepare-toolkit-parallel-text
122
+ ocrtech audit-synthesis-text-manifest
123
+ ocrtech split-synthesis-text
124
+ ocrtech render-synthesis-text-lines
125
+ ocrtech render-synthesis-text-split
126
+ ocrtech audit-rendered-synthesis-lines
127
+ ocrtech audit-rendered-degradation-split
128
+ ocrtech train-recognizer
129
+ ```
130
+
131
+ ### Limbu PDF-Native Extraction
132
+
133
+ Source rule for every Gorkhapatra/Naya Nepal language:
134
+ `https://gorkhapatraonline.com/publications/naya-nepal` is only the discovery
135
+ index for identifying which language/date was posted. Do not use the Naya Nepal
136
+ blog/card image as OCR, eval, or training evidence. Fetch the full paper from
137
+ the Gorkhapatra epaper date-range endpoint, for example
138
+ `https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-16&pdfdaterange2=2026-06-23&slug=gorkhapatra`,
139
+ then render the matching PDF page and extract/convert the text layer from that
140
+ same PDF page. Record PDF URL/hash/page number, rendered image hash, text
141
+ extraction provenance, and conversion map hash. Classify each language page as
142
+ `unicode_native`, `legacy_font`, `devanagari_transcription`, `latin_notation`, or
143
+ `no_text_layer`; manual transcription/native-reader work is only the fallback
144
+ when the PDF has no usable text layer or the font cannot yet be decoded.
145
+
146
+ The current successful Gorkhapatra Limbu path:
147
+
148
+ ```bash
149
+ uv run ocrtech extract-gorkhapatra-language-page-pdf-text \
150
+ data/limbu-gorkhapatra-language-page-pack-2026-06-16/manifests/gorkhapatra-language-page-pack.jsonl \
151
+ --candidate-language लिम्बू \
152
+ --out data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16
153
+ ```
154
+
155
+ Known result:
156
+
157
+ ```text
158
+ passed=True samples=2/2 spans=776 target_spans=304 fonts=10 warnings=0
159
+ sample_id=gorkhapatra-language-page-0001-01
160
+ pdf_page_number=6
161
+ target_font=NamdhinggoSILL
162
+ target_spans=304
163
+ embedded_font=UFNWLJ+NamdhinggoSILL
164
+ ```
165
+
166
+ Page conclusion:
167
+
168
+ - page 6 is the Limbu/Sirijonga page
169
+ - page 7 is the same-date non-Limbu page
170
+
171
+ The PDF text layer exposes legacy ASCII-like Namdhinggo text, not Unicode
172
+ Limbu. Example:
173
+
174
+ ```text
175
+ legacy: kfMG g' ;fK;SF[ yf]af]cf
176
+ unicode: ᤐᤠ᤺ᤴ ᤏᤢ ᤛᤠᤵᤛᤰᤖᤧ ᤌᤥᤒᤥᤀᤠ
177
+ ```
178
+
179
+ The converter is implemented in `src/ocrtech/limbu_legacy.py` using the vendored
180
+ SIL map:
181
+
182
+ ```text
183
+ data/mappings/limbu-legacy/Limbu.map
184
+ ```
185
+
186
+ ### Limbu Review Dashboard
187
+
188
+ Generated dashboard path:
189
+
190
+ ```text
191
+ data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html
192
+ ```
193
+
194
+ Correct local file URL:
195
+
196
+ ```text
197
+ file:///Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html
198
+ ```
199
+
200
+ This is the correct browser UI for the Gorkhapatra PDF-native Limbu page. It
201
+ shows the full rendered PDF image with bounded boxes; clicking a box selects the
202
+ matching right-side row with converted text and the corresponding rendered span
203
+ crop. Do not substitute the PDF-native verification-bundle HTML for this task;
204
+ that was the wrong/broken UI.
205
+
206
+ Local server used:
207
+
208
+ ```bash
209
+ uv run python -m http.server 8765 --bind 127.0.0.1
210
+ ```
211
+
212
+ Dashboard URL:
213
+
214
+ ```text
215
+ http://127.0.0.1:8765/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html
216
+ ```
217
+
218
+ Important bug already fixed in the generated dashboard: clicking a bounding box
219
+ must not re-render or scroll the image/page back to the top. The fix removed
220
+ automatic `scrollIntoView` from bounding-box selection.
221
+
222
+ If this dashboard is regenerated by code later, preserve this behavior:
223
+
224
+ ```js
225
+ function selectSpan(id){
226
+ selected=id;
227
+ document.querySelectorAll('.box.selected,.span.selected').forEach(el => el.classList.remove('selected'));
228
+ document.querySelectorAll(`[data-span="${id}"],[data-row="${id}"]`).forEach(el => el.classList.add('selected'));
229
+ }
230
+ ```
231
+
232
+ ### Migrated Corpus
233
+
234
+ Source host:
235
+
236
+ ```text
237
+ cdjk@100.110.148.43
238
+ machine: cdjks-MacBook-Pro.local
239
+ external volume: /Volumes/TRANSCEND
240
+ ```
241
+
242
+ Do not write generated data back to that host. Treat it as a source for corpus
243
+ and dictionaries unless the user explicitly asks otherwise.
244
+
245
+ Tracked migration policy:
246
+
247
+ ```text
248
+ configs/cdjk-language-corpus-migration-v1.json
249
+ ```
250
+
251
+ Ignored local copy:
252
+
253
+ ```text
254
+ data/external-language-resources/cdjk-100.110.148.43/
255
+ ```
256
+
257
+ Copied roots:
258
+
259
+ ```text
260
+ dhimal-toolkit
261
+ gurung-language-toolkit
262
+ limbu-speech-toolkit
263
+ ltk
264
+ magar-toolkit
265
+ rai-speech-toolkit
266
+ ```
267
+
268
+ Migration result:
269
+
270
+ ```text
271
+ 2307 text/resource files transferred
272
+ 1.6G local size
273
+ resource audit: passed=True roots=6/6 files=2308 warnings=0
274
+ ```
275
+
276
+ ### Prepared Synthesis Text
277
+
278
+ Generated synthesis text is ignored under `data/synthesis-text/`, but the
279
+ documented state is:
280
+
281
+ ```text
282
+ 20 manifests
283
+ 228212 total samples
284
+ 45258 duplicate rows removed/reported by converters
285
+ 202560 Devanagari rows
286
+ 10337 Mixed_Devanagari_Latin rows
287
+ 6521 Latin rows
288
+ 8772 true Unicode Limbu rows
289
+ 14 Mixed_Limbu_Devanagari rows
290
+ 3 Mixed_Limbu_Latin rows
291
+ 5 Other rows
292
+ ```
293
+
294
+ Important generated manifests:
295
+
296
+ ```text
297
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/limbu-limdic-text.jsonl
298
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl
299
+ data/synthesis-text/cdjk-tamang-2026-06-16/tamang-text.jsonl
300
+ data/synthesis-text/cdjk-magar-2026-06-16/magar-text.jsonl
301
+ data/synthesis-text/cdjk-gurung-2026-06-16/toolkit-parallel-text.jsonl
302
+ data/synthesis-text/cdjk-dhimal-parallel-2026-06-16/toolkit-parallel-text.jsonl
303
+ data/synthesis-text/cdjk-dhimal-mono-2026-06-16/toolkit-parallel-text.jsonl
304
+ data/synthesis-text/cdjk-magar-parallel-extra-2026-06-16/toolkit-parallel-text.jsonl
305
+ data/synthesis-text/cdjk-magar-mono-extra-2026-06-16/toolkit-parallel-text.jsonl
306
+ data/synthesis-text/cdjk-rai-*-2026-06-16/toolkit-parallel-text.jsonl
307
+ ```
308
+
309
+ All synthesis text audits passed with `claim_eligible=0`.
310
+
311
+ Important interpretation:
312
+
313
+ - True Unicode Limbu exists from Limdic and should render with Namdhinggo after
314
+ filtering `script == Limbu`.
315
+ - Most Gurung, Dhimal, Magar, Tamang, and Rai rows are Devanagari-transcribed
316
+ language text, not native-script OCR.
317
+ - Bantawa, Khaling, Koyu, and Thulung include substantial Latin phonetic data.
318
+ These are useful for language resources but should not be called native-script
319
+ OCR data.
320
+
321
+ ### Synthetic Rendering
322
+
323
+ Corrected Limbu-only smoke render:
324
+
325
+ ```bash
326
+ uv run ocrtech render-synthesis-text-lines \
327
+ data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl \
328
+ --out data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only \
329
+ --font-path data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Regular.ttf \
330
+ --limit 120 \
331
+ --degradation-profile clean \
332
+ --degradation-profile scan \
333
+ --degradation-profile low_light \
334
+ --degradation-profile uneven_light \
335
+ --degradation-profile camera_left \
336
+ --degradation-profile phone_photo
337
+ ```
338
+
339
+ Audit:
340
+
341
+ ```bash
342
+ uv run ocrtech audit-rendered-synthesis-lines \
343
+ data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/manifests/rendered-line-manifest.jsonl \
344
+ --label-file data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/labels/train.txt \
345
+ --out outputs/cdjk-limbu-rendered-smoke-audit-2026-06-16-limbu-only
346
+ ```
347
+
348
+ Result:
349
+
350
+ ```text
351
+ passed=True samples=120 claim_eligible=0 issues=0 warnings=0
352
+ degradation_counts:
353
+ camera_left: 20
354
+ clean: 20
355
+ low_light: 20
356
+ phone_photo: 20
357
+ scan: 20
358
+ uneven_light: 20
359
+ ```
360
+
361
+ Important lesson: an earlier mixed-script render produced tofu boxes because
362
+ Devanagari/Latin rows were rendered with a Limbu font. Always filter by script
363
+ before choosing a font.
364
+
365
+ ## Current Strategic Decision
366
+
367
+ The user asked what next. The right order is:
368
+
369
+ 1. Build Gorkhapatra validation for all target languages.
370
+ 2. Create language/script/font matrix from actual Gorkhapatra PDF evidence.
371
+ 3. Generate split-safe synthetic data per script/font bucket.
372
+ 4. Research and execute PaddleOCR recognizer fine-tuning.
373
+ 5. Keep PP-Structure/layout mostly stock until recognizer evidence says
374
+ detection/layout fine-tuning is required.
375
+
376
+ Do not scale synthetic data first without real validation. It will create a
377
+ large training set but no credible paper claim.
378
+
379
+ ## Immediate Next Task
380
+
381
+ Build a reusable all-language Gorkhapatra validation generator.
382
+
383
+ It should turn this manual Limbu success into a repeatable command/workflow:
384
+
385
+ ```text
386
+ Naya Nepal publication card
387
+ -> date/language/article metadata
388
+ -> matching epaper PDF
389
+ -> candidate language-page pages
390
+ -> rendered page image
391
+ -> PDF-native spans/fonts/bboxes/codepoints
392
+ -> optional legacy conversion by script/font
393
+ -> review dashboard
394
+ -> reviewed JSON/CSV annotations
395
+ ```
396
+
397
+ Suggested CLI shape:
398
+
399
+ ```text
400
+ ocrtech prepare-gorkhapatra-language-validation
401
+ ```
402
+
403
+ Likely arguments:
404
+
405
+ ```text
406
+ --language-label <Nepali label>
407
+ --publication-pages <N>
408
+ --epaper-url <date-range endpoint>
409
+ --out <data/gorkhapatra-validation-...>
410
+ --target-font <optional font name>
411
+ --legacy-converter <optional converter id, e.g. limbu-namdhinggo>
412
+ --render-pages
413
+ --download-pdfs
414
+ ```
415
+
416
+ Output should include:
417
+
418
+ ```text
419
+ validation.json
420
+ validation.md
421
+ samples/*.pdf-text.json
422
+ samples/*.pdf-text.txt
423
+ spans.csv
424
+ fonts/
425
+ page-images/
426
+ span-renders/
427
+ dashboard.html
428
+ review.csv
429
+ review.jsonl
430
+ ```
431
+
432
+ Dashboard requirements:
433
+
434
+ - Full page image with clickable bbox overlay.
435
+ - Right panel with selected span, PDF-rendered crop, raw extracted text,
436
+ converted Unicode text when available, font, bbox, codepoints.
437
+ - Review status controls:
438
+ - `accept`
439
+ - `needs_fix`
440
+ - `bad_segment`
441
+ - `drop`
442
+ - `wrong_page`
443
+ - `wrong_language`
444
+ - Must preserve image/page scroll when clicking bbox.
445
+ - Must not ask the user to transcribe text if PDF-native extraction/conversion
446
+ already produces usable Unicode text.
447
+ - Must save/export annotations as JSONL/CSV.
448
+
449
+ ## Language Validation Targets
450
+
451
+ At least ten target languages are desired. Current likely sources:
452
+
453
+ - Limbu/Sirijonga
454
+ - Tamang
455
+ - Magar
456
+ - Gurung
457
+ - Dhimal
458
+ - Newar/Newa
459
+ - Bantawa/Rai
460
+ - Chamling/Rai
461
+ - Kulung/Rai
462
+ - Athpariya/Rai
463
+ - Khaling/Rai
464
+ - Yakkha/Rai
465
+ - Sunwar/Rai
466
+ - Lohorung/Rai
467
+ - Dumi/Rai
468
+ - Thulung/Rai
469
+ - Koyu/Rai
470
+
471
+ But Gorkhapatra validation must decide actual script form:
472
+
473
+ - Unicode native script
474
+ - legacy font encoding
475
+ - Devanagari transcription
476
+ - Latin notation
477
+ - unsupported/no text layer
478
+
479
+ Do not assume native Unicode just because the language has a script.
480
+
481
+ ## Training Research Direction
482
+
483
+ Use primary sources before changing training code. Current understanding:
484
+
485
+ - PaddleOCR/PP-OCR remains the primary engine family.
486
+ - Start by fine-tuning recognizer, not layout/detection.
487
+ - Use PP-Structure/PP-StructureV3-style parsing for document layout and
488
+ structured output.
489
+ - Consider PaddleOCR-VL only as fallback/teacher/comparator, not the default
490
+ small local engine.
491
+
492
+ Recognizer-first rationale:
493
+
494
+ - The repo already has line/synthesis text and rendered line labels.
495
+ - Recognition turns cropped text images into text.
496
+ - Detection/layout claims require bbox/layout ground truth, which is not yet
497
+ mature across languages.
498
+ - Fine-tuning detection before measuring detection failures is premature.
499
+
500
+ Training prerequisites before a serious run:
501
+
502
+ - Split synthesis text with `split-synthesis-text`.
503
+ - Exclude real validation/reference text hashes from synthetic training.
504
+ - Render train/eval with font readiness and degradation audits.
505
+ - Keep English/Nepali non-regression sets.
506
+ - Package PaddleOCR recognizer training recipe.
507
+ - Evaluate on Gorkhapatra real pages and held-out synthetic rows.
508
+
509
+ ## Remote Machines
510
+
511
+ Known hosts from the session:
512
+
513
+ ```text
514
+ k2@100.97.238.103
515
+ cdjk@100.110.148.43
516
+ cdjk@100.117.21.47
517
+ ```
518
+
519
+ Do not commit passwords. The user provided credentials in chat previously, but
520
+ tracked docs must not contain them.
521
+
522
+ Important storage findings:
523
+
524
+ - `cdjk@100.110.148.43` has `/Volumes/TRANSCEND`, about `3.6Ti` total and
525
+ `2.0Ti` available at inventory time. Use as source for corpus/dictionary
526
+ resources unless instructed otherwise.
527
+ - `cdjk@100.117.21.47` root was full at inspection time:
528
+
529
+ ```text
530
+ / 915G 865G 3.7G 100%
531
+ /mnt/tts_cache_nvme 234G 150G 72G 68%
532
+ ```
533
+
534
+ Do not dump large generated data to a nearly-full root filesystem.
535
+
536
+ ## Source Files To Inspect Before Editing
537
+
538
+ For Gorkhapatra flow:
539
+
540
+ ```text
541
+ src/ocrtech/source_audit.py
542
+ src/ocrtech/cli.py
543
+ src/ocrtech/limbu_legacy.py
544
+ tests/test_source_audit.py
545
+ tests/test_limbu_legacy.py
546
+ ```
547
+
548
+ For synthesis:
549
+
550
+ ```text
551
+ src/ocrtech/synthesis_resources.py
552
+ tests/test_synthesis_resources.py
553
+ configs/cdjk-language-corpus-migration-v1.json
554
+ ```
555
+
556
+ For fonts/language registry:
557
+
558
+ ```text
559
+ src/ocrtech/font_assets.py
560
+ src/ocrtech/language_registry.py
561
+ configs/namdhinggo-font-assets-v1.json
562
+ configs/nepal-font-assets-v1.json
563
+ languages/nepal-v1.json
564
+ ```
565
+
566
+ For training:
567
+
568
+ ```text
569
+ src/ocrtech/training.py
570
+ src/ocrtech/models.py
571
+ docs/MODELS.md
572
+ docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md
573
+ ```
574
+
575
+ ## Known Pitfalls
576
+
577
+ - The user is frustrated by fake OCR candidates and manual busywork. Do not ask
578
+ them to validate bad OCR text when PDF-native text extraction can provide the
579
+ answer.
580
+ - For Limbu, PDF text was extractable but legacy encoded. The correct path was
581
+ PDF-native text -> Namdhinggo legacy conversion -> Unicode display.
582
+ - Dashboard bbox clicks must not reset scroll.
583
+ - Do not treat Devanagari-transcribed language data as native script OCR.
584
+ - Do not render mixed-script rows with a single script-specific font.
585
+ - Do not collapse mixed schemas into one converter command by dropping
586
+ validation. Split sources by schema instead.
587
+ - Do not publish synthetic-only claims.
588
+
589
+ ## Recommended Next Commit
590
+
591
+ Implement the reusable validation generator and tests:
592
+
593
+ 1. Refactor current generated dashboard logic into source code.
594
+ 2. Add a CLI command that accepts a manifest/PDF-text extraction dir and emits
595
+ dashboard + review CSV/JSONL.
596
+ 3. Preserve current Limbu dashboard features:
597
+ - full page image
598
+ - bbox overlay
599
+ - PDF-rendered span crop
600
+ - converted Unicode text
601
+ - raw legacy text
602
+ - font metadata
603
+ - stable scroll behavior
604
+ 4. Add fixture tests for dashboard generation and annotation schema.
605
+ 5. Document command in `docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md`.
606
+ 6. Run focused tests:
607
+
608
+ ```bash
609
+ uv run pytest tests/test_source_audit.py tests/test_limbu_legacy.py -q
610
+ git diff --check
611
+ ```
612
+
613
+ Then commit with GPG signing disabled and push.
ocr-tech-mps-integration/docs/CODEX_HANDOFF.md ADDED
The diff for this file is too large to render. See raw diff
 
ocr-tech-mps-integration/docs/DATASETS.md ADDED
@@ -0,0 +1,381 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Dataset Workflow
2
+
3
+ The dataset workflow is deliberately strict. If train/eval contamination gets in here, every later validation result is suspect.
4
+
5
+ ## Prepare
6
+
7
+ Convert raw labels to JSONL:
8
+
9
+ ```bash
10
+ ocrtech prepare-data labels.csv \
11
+ --out data/manifests \
12
+ --dataset-name nepali-lines \
13
+ --split all \
14
+ --image-field image_path \
15
+ --text-field text \
16
+ --strict-chars
17
+ ```
18
+
19
+ If the source is noisy and you need an explicit reject list instead of failing at the first bad row:
20
+
21
+ ```bash
22
+ ocrtech prepare-data labels.csv \
23
+ --out data/manifests \
24
+ --dataset-name nepali-lines \
25
+ --split all \
26
+ --image-field image_path \
27
+ --text-field text \
28
+ --skip-invalid
29
+ ```
30
+
31
+ That writes:
32
+
33
+ - `data/manifests/nepali-lines-all.jsonl`
34
+ - `data/manifests/nepali-lines-all-prepare.json`
35
+ - `data/manifests/nepali-lines-all-prepare.md`
36
+ - `data/manifests/nepali-lines-all-rejects.jsonl` when any rows are skipped
37
+
38
+ Verified Hugging Face sources can be converted directly:
39
+
40
+ ```bash
41
+ ocrtech prepare-data gauravgiri/nepali-ocr-dataset \
42
+ --hf \
43
+ --split train \
44
+ --limit 100 \
45
+ --skip-invalid \
46
+ --out data/manifests
47
+
48
+ ocrtech prepare-data prashant0919/nepali-synthetic-ocr-lines \
49
+ --hf \
50
+ --split train \
51
+ --limit 100 \
52
+ --out data/manifests
53
+
54
+ ocrtech prepare-data nvidia/OCR-Synthetic-Multilingual-v1 \
55
+ --hf \
56
+ --hf-subset en \
57
+ --split test \
58
+ --limit 1000 \
59
+ --out data/manifests-english-test
60
+ ```
61
+
62
+ Before those conversion commands, record a schema inspection so the training
63
+ corpus has evidence that fields were verified from the actual source:
64
+
65
+ ```bash
66
+ ocrtech inspect-hf-dataset gauravgiri/nepali-ocr-dataset \
67
+ --split train \
68
+ --limit 3 \
69
+ --out outputs/hf-inspect-training-v1
70
+
71
+ ocrtech inspect-hf-dataset prashant0919/nepali-synthetic-ocr-lines \
72
+ --split train \
73
+ --limit 3 \
74
+ --out outputs/hf-inspect-training-v1
75
+
76
+ ocrtech inspect-hf-dataset nvidia/OCR-Synthetic-Multilingual-v1 \
77
+ --hf-subset en \
78
+ --split test \
79
+ --limit 3 \
80
+ --out outputs/hf-inspect-training-v1
81
+ ```
82
+
83
+ Verified on `k2` on `2026-06-15`:
84
+
85
+ | dataset | split/subset | mode | inferred image | inferred text | result |
86
+ | --- | --- | --- | --- | --- | --- |
87
+ | `gauravgiri/nepali-ocr-dataset` | `train` | `datasets.load_dataset` | `image` | `text` | pass |
88
+ | `prashant0919/nepali-synthetic-ocr-lines` | `train` | HF repo `metadata.jsonl` | `file_name` | `text` | pass |
89
+ | `nvidia/OCR-Synthetic-Multilingual-v1` | `test/en` | HDF5 `line_bboxes` | `images + annotations.line_bboxes[].bbox` | `annotations.line_bboxes[].text` | pass |
90
+
91
+ Artifacts:
92
+
93
+ - `outputs/hf-inspect-training-v1/gauravgiri__nepali-ocr-dataset-train-inspect.json`
94
+ - `outputs/hf-inspect-training-v1/prashant0919__nepali-synthetic-ocr-lines-train-inspect.json`
95
+ - `outputs/hf-inspect-training-v1/nvidia__OCR-Synthetic-Multilingual-v1__en-test-inspect.json`
96
+
97
+ ## Public Benchmarks
98
+
99
+ Public document parsing benchmarks are tracked separately from training data.
100
+ Do not mix them into recognizer/corrector training manifests. They are final
101
+ claim evidence.
102
+
103
+ List the known registry:
104
+
105
+ ```bash
106
+ ocrtech list-public-benchmarks
107
+ ```
108
+
109
+ Current target order:
110
+
111
+ 1. `opendatalab/OmniDocBench` for broad document parsing.
112
+ 2. `PaddlePaddle/Real5-OmniDocBench` for real-world physical distortions.
113
+ 3. `allenai/olmOCR-bench` for PDF-to-Markdown/unit-test style evaluation.
114
+
115
+ Prepare the verified OmniDocBench schema:
116
+
117
+ ```bash
118
+ ocrtech prepare-public-benchmark opendatalab/OmniDocBench \
119
+ --out data/public/omnidocbench \
120
+ --limit 100 \
121
+ --language english \
122
+ --require-tables
123
+ ```
124
+
125
+ This converter is schema-bound. It expects the verified Hugging Face layout:
126
+ `OmniDocBench.json` as a list of rows, each row with `page_info.image_path`,
127
+ `page_info.page_attribute`, and `layout_dets`. Table references come from
128
+ `layout_dets[].html`; text references come from ordered `layout_dets[].text`
129
+ plus table HTML text. Images are resolved under `images/`.
130
+
131
+ Before benchmarking a public pack, audit its manifest:
132
+
133
+ ```bash
134
+ ocrtech audit-public-benchmark data/public/omnidocbench/manifests/omnidocbench-eval.jsonl \
135
+ --benchmark opendatalab/OmniDocBench \
136
+ --out outputs/public-omnidocbench-audit
137
+ ```
138
+
139
+ Each public manifest row must carry provenance:
140
+
141
+ - `metadata.public_benchmark = true`
142
+ - `metadata.benchmark_name`
143
+ - `metadata.source_url`
144
+ - `metadata.reference_path`
145
+ - `metadata.slices` containing `public`
146
+
147
+ This is intentionally stricter than `prepare-data`. The agent must verify the
148
+ actual downloaded dataset schema before writing a converter for any public
149
+ benchmark. No schema guessing.
150
+
151
+ Current claim-sized public OmniDocBench table pack on `k2`:
152
+
153
+ - manifest: `data/public/omnidocbench-table100/manifests/omnidocbench-eval.jsonl`
154
+ - prepare report: `data/public/omnidocbench-table100/public-benchmark-prepare.json`
155
+ - audit: `outputs/public-omnidocbench-table100-audit/public-benchmark-audit.json`
156
+ - audit status: passed
157
+ - samples: `100`
158
+ - slices: `public=100`, `document_parsing=100`, `english=100`, `table=100`,
159
+ `table_hard=32`, `figure=22`
160
+
161
+ This is the first public OmniDocBench manifest in the repo that can satisfy the
162
+ configured `100`-page public document gate and the `40`-page public table gate.
163
+
164
+ For the current Nepali starter corpus, merge verified sources with the built-in command instead of shell concatenation:
165
+
166
+ ```bash
167
+ ocrtech merge-manifests \
168
+ data/manifests/gauravgiri__nepali-ocr-dataset-train.jsonl \
169
+ data/manifests/prashant0919__nepali-synthetic-ocr-lines-train.jsonl \
170
+ --out data/manifests/nepali-mixed-all.jsonl
171
+ ```
172
+
173
+ Do not pretend all HF datasets look the same. This repo only has verified adapters for the schemas we checked:
174
+
175
+ - `gauravgiri/nepali-ocr-dataset`: `image`, `text`
176
+ - `prashant0919/nepali-synthetic-ocr-lines`: `train/metadata.jsonl` plus image files
177
+ - `cfilt/RoundTripOCR-nepali`: `ocr`, `correct`, `font`
178
+ - `nvidia/OCR-Synthetic-Multilingual-v1`: HDF5 shards with `line_bboxes`
179
+
180
+ Known adapters also stamp verified slice metadata so later audits and experiments do not depend on hand-added tags:
181
+
182
+ - `gauravgiri/nepali-ocr-dataset`: `nepali`, `real`
183
+ - `prashant0919/nepali-synthetic-ocr-lines`: `nepali`, `synthetic`
184
+ - `nvidia/OCR-Synthetic-Multilingual-v1 --hf-subset en`: `english`, `synthetic`, `line_crop`, `en`
185
+
186
+ Verified on `k2` on June 14, 2026:
187
+
188
+ - `gauravgiri/nepali-ocr-dataset --limit 100 --skip-invalid`: `97` prepared, `3` rejected due to empty `text`
189
+ - `prashant0919/nepali-synthetic-ocr-lines --limit 100`: `100` prepared, `0` rejected
190
+ - merged starter corpus: `197` samples, no merge warnings
191
+ - deterministic split: `167` train / `30` eval, leakage check passed
192
+
193
+ Each manifest row contains:
194
+
195
+ - `sample_id`
196
+ - `dataset`
197
+ - `split`
198
+ - `image_path`
199
+ - `text`
200
+ - `sha256`
201
+ - `metadata.text_sha256`
202
+ - `metadata.sample_sha256`
203
+
204
+ ## Audit
205
+
206
+ Before splitting or training:
207
+
208
+ ```bash
209
+ ocrtech audit-manifest data/manifests/nepali-lines-all.jsonl \
210
+ --out outputs/manifest-audit \
211
+ --require-slice nepali \
212
+ --require-slice english \
213
+ --strict-chars
214
+ ```
215
+
216
+ The audit checks:
217
+
218
+ - missing input files
219
+ - image SHA-256 mismatches
220
+ - duplicate sample IDs
221
+ - duplicate paths and hashes
222
+ - slice counts
223
+ - script counts
224
+ - unsupported characters when requested
225
+
226
+ ## Split
227
+
228
+ Create train/eval manifests with deterministic grouping:
229
+
230
+ ```bash
231
+ ocrtech split-manifest data/manifests/nepali-lines-all.jsonl \
232
+ --out data/splits \
233
+ --eval-ratio 0.15 \
234
+ --seed 13 \
235
+ --stratify-by slices
236
+ ```
237
+
238
+ The splitter groups entries by `metadata.sample_sha256` before assigning splits by default. If the same image+text sample appears twice, it stays in one split.
239
+
240
+ For rendered packs with multiple degraded variants of the same underlying text, group by `text_sha256` instead. This keeps clean, blur, scan, low-contrast, and skew variants of the same target text on the same side of the train/eval boundary:
241
+
242
+ ```bash
243
+ ocrtech split-manifest data/eval-pack-table-figure-dev-v1/manifests/ocrtech-eval-pack.jsonl \
244
+ --out data/splits-table-figure-dev-v1 \
245
+ --eval-ratio 0.2 \
246
+ --seed 17 \
247
+ --stratify-by document_type \
248
+ --group-by text_sha256
249
+ ```
250
+
251
+ Supported grouping keys are:
252
+
253
+ - `sample_signature`
254
+ - `text_sha256`
255
+ - `image_sha256`
256
+ - `image_path`
257
+ - `sample_id`
258
+
259
+ Outputs:
260
+
261
+ ```text
262
+ data/splits/
263
+ train.jsonl
264
+ eval.jsonl
265
+ split-summary.json
266
+ split-summary.md
267
+ ```
268
+
269
+ The split summary records the grouping key. For the table/figure development pack, the verified `k2` split produced:
270
+
271
+ - `60` train samples
272
+ - `20` eval samples
273
+ - `16` text groups
274
+ - `group_by = text_sha256`
275
+ - leakage check passed
276
+
277
+ ## Rebalance
278
+
279
+ Use `rebalance-manifest` only on the training split when a slice is obviously underrepresented and you want a reproducible upsampling step instead of an undocumented training-time sampler choice:
280
+
281
+ ```bash
282
+ ocrtech rebalance-manifest data/splits/train.jsonl \
283
+ --out data/splits/train-rebalanced.jsonl \
284
+ --slice english \
285
+ --slice nepali \
286
+ --seed 13
287
+ ```
288
+
289
+ Outputs:
290
+
291
+ ```text
292
+ data/splits/
293
+ train-rebalanced.jsonl
294
+ train-rebalanced-rebalance.json
295
+ train-rebalanced-rebalance.md
296
+ ```
297
+
298
+ This command duplicates rows with explicit provenance in metadata:
299
+
300
+ - `rebalance_duplicate`
301
+ - `rebalance_source_sample_id`
302
+ - `rebalance_slice`
303
+
304
+ Do not rebalance the unsplit combined manifest. That would duplicate samples before train/eval assignment and corrupt the benchmark.
305
+
306
+ ## Train
307
+
308
+ Use only the split outputs:
309
+
310
+ ```bash
311
+ ocrtech train-recognizer \
312
+ --train-manifest data/splits/train.jsonl \
313
+ --eval-manifest data/splits/eval.jsonl \
314
+ --out runs/recognizer
315
+ ```
316
+
317
+ For generated document packs, do not train a recognizer on whole rendered pages unless the model is explicitly document-level. The current full-page HF branch failed badly. Extract line/cell/caption crops first, then split those crop rows with text-level grouping:
318
+
319
+ ```bash
320
+ ocrtech extract-crops \
321
+ data/eval-pack-table-figure-dev-v1/manifests/ocrtech-eval-pack.jsonl \
322
+ --out data/crops-table-figure-dev-v1 \
323
+ --crop-type line \
324
+ --crop-type table-cell \
325
+ --crop-type figure-caption
326
+
327
+ ocrtech split-manifest data/crops-table-figure-dev-v1/manifests/crops.jsonl \
328
+ --out data/splits-crops-table-figure-dev-v1 \
329
+ --eval-ratio 0.2 \
330
+ --seed 17 \
331
+ --stratify-by document_type \
332
+ --group-by text_sha256
333
+ ```
334
+
335
+ The crop extractor only reads explicit renderer layout metadata from the reference JSON. If that metadata is absent, it fails instead of guessing bboxes.
336
+
337
+ Before training, build the recognizer corpus from verified manifests with
338
+ explicit bucket targets:
339
+
340
+ ```bash
341
+ ocrtech build-recognizer-corpus \
342
+ data/manifests/english-lines.jsonl \
343
+ data/manifests/nepali-lines.jsonl \
344
+ data/splits-crops-table-figure-dev-v1/train.jsonl \
345
+ --out data/splits/recognizer-train-balanced.jsonl \
346
+ --target-latin-only 200 \
347
+ --target-devanagari-only 200 \
348
+ --target-mixed 100 \
349
+ --target-real 100 \
350
+ --target-synthetic 100
351
+ ```
352
+
353
+ This command upsamples existing examples only. If a requested bucket is missing
354
+ from the inputs, it fails; add source data instead of weakening the audit gate.
355
+
356
+ For OCR correction from the verified CFILT dataset:
357
+
358
+ ```bash
359
+ ocrtech train-corrector \
360
+ --hf-dataset cfilt/RoundTripOCR-nepali \
361
+ --hf-split train \
362
+ --limit 1000 \
363
+ --out runs/corrector-cfilt
364
+ ```
365
+
366
+ Do not point training at the combined manifest.
367
+
368
+ ## Validation
369
+
370
+ Pass the training manifests into claim validation:
371
+
372
+ ```bash
373
+ ocrtech validate-claim \
374
+ --benchmark-report outputs/benchmark/report.json \
375
+ --eval-manifest data/eval-pack/manifests/ocrtech-eval-pack.jsonl \
376
+ --train-manifest data/splits/train.jsonl \
377
+ --config validation.json \
378
+ --out outputs/validation
379
+ ```
380
+
381
+ The validation step checks train/eval leakage again. That is intentional redundancy.
ocr-tech-mps-integration/docs/EVAL_HARNESS_GOAL.md ADDED
@@ -0,0 +1,437 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Evaluation Harness Goal
2
+
3
+ Updated: 2026-06-20
4
+
5
+ ## Current Status
6
+
7
+ This is a parked claim-safety design, not the active product work. The active
8
+ 2026-06-23 objective is Limbu book OCR: real book pages in, editable Unicode
9
+ book text out, with line structure preserved and unresolved text surfaced for
10
+ human review. Do not add generic eval-harness features from this document unless
11
+ they directly unblock that OCR loop. For the current Limbu work, the useful
12
+ evaluation surface is deliberately thin: frozen real-book page acceptance checks,
13
+ artifact audits, page/line counts, missed/extra/blank-line rates, review-packet
14
+ coverage, and reviewed-only training-pack export validation.
15
+
16
+ ## Goal
17
+
18
+ The goal is a claim-grade OCR evaluation harness for the Nepal/diaspora recognizer
19
+ suite that makes weak publication claims difficult by default, not merely inconvenient:
20
+ each language/script result must be scored only from a frozen, versioned eval pack whose
21
+ source images, labels, manifest rows, source-document/page/bbox provenance, split role,
22
+ domain, ground-truth trust evidence, claim eligibility, canonical pack-content digest,
23
+ and script/grapheme coverage are
24
+ auditable; each run must record the exact model, checkpoint, config, dictionary,
25
+ prediction file, normalization profile, command, timestamp, machine, pack-content digest,
26
+ and artifact hashes;
27
+ each metric must be recomputed from canonical predictions as corpus micro edit-distance
28
+ CER over both Unicode codepoints and UAX #29 extended grapheme clusters, with exact-line
29
+ accuracy, matched/unmatched rows, worst examples, confidence intervals, and paired
30
+ comparison tests when claims compare systems; recognizer-only oracle-crop scores must
31
+ remain separate from page-level OCR scores that include detection, crop lineage, reading
32
+ order, and alignment; and the harness must fail or hard-warn on missing images, duplicate
33
+ IDs, low match rate, train/dev/test leakage, checkpoint selection on test data, synthetic
34
+ or transliterated data presented as real evidence, machine-converted labels presented as
35
+ human-verified evidence, human GT without review/agreement evidence, eval packs too small
36
+ for the claimed conclusion, uncovered script inventories, and aggregate averages that
37
+ hide per-language, per-script, per-domain, or per-GT-status failures. A reviewer opening
38
+ one output directory should be able to decide what was scored, against which ground truth,
39
+ how trustworthy that ground truth is, whether the test set was frozen and unseen for
40
+ selection, what normalization and metric were used, how uncertain the number is, what the
41
+ failures look like, what script material was not covered, and whether the result is safe
42
+ for a paper claim.
43
+
44
+ ## Requirements
45
+
46
+ - **Eval packs are versioned evidence objects.** Each pack must include `images/`,
47
+ a label file, a manifest JSONL, and a summary/datasheet containing source documents,
48
+ page/bbox provenance, language, script, domain, split role, GT source, GT trust
49
+ level, known leakage status, claim eligibility, and a `pack_content_sha256` over the
50
+ canonical row identity/text/provenance fields so pack drift is detectable across
51
+ normalization, scoring, audit, and comparison.
52
+ - **Metrics are canonical and micro-averaged.** The harness must compute
53
+ total-edit-distance / total-reference-length CER, not per-line macro averages.
54
+ It must report both codepoint CER and grapheme-cluster CER, plus exact-line accuracy,
55
+ matched/unmatched counts, prediction match rate, total reference units, and
56
+ worst-line examples.
57
+ - **Point estimates are not enough for claims.** Every claim-facing metric must include
58
+ confidence intervals computed by resampling the scoring units, and same-pack system
59
+ comparisons must include a paired test over per-row errors. The report must show the
60
+ effective sample size per language/script/domain and hard-warn when a pack is too
61
+ small for the claimed conclusion.
62
+ - **PaddleOCR training metrics are not claim metrics.** PaddleOCR `RecMetric` reports
63
+ exact-line `acc` and `norm_edit_dis` as one minus the average per-sample normalized
64
+ Levenshtein distance; this may be useful during training, but it is not the
65
+ corpus-level micro CER used for claims.
66
+ - **Recognition and page OCR are separate tasks.** Oracle-crop recognizer results are
67
+ valid recognizer evidence only. Page-level OCR claims require page images, detector
68
+ output, crop/reading-order/alignment logic, and a separate end-to-end score.
69
+ - **Unicode handling is explicit.** Text is normalized with a named profile, defaulting
70
+ to NFC with whitespace preserved. The audit must reject a stale or unsupported
71
+ profile that no longer matches the metric recomputation path. Metrics must also
72
+ report how many raw GT/reference and prediction strings changed under that profile,
73
+ and the audit must recompute those counts from canonical row artifacts so hidden
74
+ normalization effects cannot disappear into a single CER. Grapheme metrics use
75
+ Unicode extended grapheme clusters; any script-specific tailoring must be recorded.
76
+ - **Human verification is first-class metadata.** Machine-converted GT may be scored,
77
+ but the output must label it as machine-converted. Paper-grade human-reading claims
78
+ require either fully reviewed GT or a documented native-reader calibration sample.
79
+ Human-verified GT must record who reviewed it, how much was double-reviewed, and the
80
+ observed disagreement or correction rate; a trust label without review evidence is not
81
+ claim evidence.
82
+ - **Script coverage is evidence, not decoration.** Each eval pack must report the
83
+ codepoint, grapheme-cluster, digit, punctuation, conjunct/combining-mark, and
84
+ language-specific inventory coverage it exercises, plus gaps against the intended
85
+ script inventory. A low CER on a pack that omits common graphemes or conjuncts must not
86
+ be eligible for broad script support claims.
87
+ - **Reports are inspectable.** Each run should emit `eval-run.json`, `metrics.json`,
88
+ `metrics.md`, `predictions.tsv`, source-audited `unmatched.tsv`, `worst-lines.csv`,
89
+ `worst-pages.csv`, `sample-review.csv`, `sample-review.html`, `confusions.json`, `command.txt`, and a
90
+ human-readable `claim-safety.md` report.
91
+ Worst-example rows must carry the row identity, hashes, language/script/domain/GT/split,
92
+ claim eligibility, source-document/page/bbox provenance, and reference-artifact trust
93
+ fields needed to inspect a failure without reverse-engineering the rest of the run.
94
+ - **Reporting is per language/script/domain.** Shared recognizers, such as the
95
+ Devanagari model, must still report Majhi, Magar, Sunuwar-Devanagari, Magahi,
96
+ Bhojpuri, Thami, Wambule, and Darai separately. Native-script targets such as
97
+ Limbu, Sunuwar/Jenticha, Kirat Rai, Lepcha, Tirhuta, Gurung Khema, and Newa must
98
+ stay separate from fallback or synthetic-only results.
99
+
100
+ ## Current Implementation Slice
101
+
102
+ - `tools/run_recognizer_eval.py` scores an existing recognizer prediction file
103
+ against a PaddleOCR-style label file. It records non-empty model identity plus config/checkpoint/dictionary,
104
+ SHA-256 identities for local checkpoint/config/dictionary artifacts and ties those
105
+ identities back to the visible run fields, prediction, label, manifest,
106
+ summary/datasheet, script inventory, train/dev manifest, command, machine platform/Python metadata,
107
+ normalization, eval-pack version, frozen-pack declaration, selection declaration,
108
+ per-row crop image SHA-256 identities and source-document/page/bbox provenance in
109
+ `predictions.tsv` and `worst-lines.csv`, deterministic row-
110
+ bootstrap 95% confidence intervals for codepoint CER, grapheme CER, and exact-line
111
+ accuracy, observed and missing required script codepoint/grapheme inventory coverage,
112
+ and claim-safety warnings, and emits a direct `claim-safety.md` report in
113
+ the score directory. A claim-safe recognizer run now requires those artifacts to be
114
+ readable local files whose hashes can be audited and at least 30 scored rows per
115
+ emitted language/script/domain/GT/split slice so tiny or pooled smoke packs cannot
116
+ pass as paper evidence. Declared manifest crop-image hashes must
117
+ match actual crop bytes; missing script inventory files or missing required
118
+ codepoints/graphemes are claim-safety blockers, and rows declared `human_verified`
119
+ must carry human review evidence such as reviewer or review-artifact metadata.
120
+ - `tools/audit_recognizer_eval_pack.py` audits a recognizer oracle-crop eval pack
121
+ before scoring. It verifies label/image/manifest/summary integrity, duplicate
122
+ labels, manifest identifiers, and manifest image paths/basenames, manifest text/hash consistency, source-document/
123
+ page/bbox coverage, language/script/domain/GT/split metadata, claim eligibility,
124
+ frozen-pack declaration, unseen-for-selection declaration, and minimum scoring-unit
125
+ counts overall and per language/script/domain/GT/split slice.
126
+ - `tools/normalize_recognizer_eval_pack.py` converts existing PaddleOCR-style
127
+ recognizer packs into a self-contained canonical pack with copied images, normalized
128
+ `eval.txt`, `manifests/recognizer-eval.jsonl`, and `summary.json`. It preserves legacy
129
+ provenance fields, fills only explicitly provided defaults, hashes copied images, and
130
+ fails on missing images, duplicate labels, duplicate manifest IDs or image
131
+ paths/basenames, or manifest/label text mismatches.
132
+ - `configs/recognizer-eval-pack-normalization-v1.json` records reproducible
133
+ normalization recipes for the current real recognizer packs: Newa, Sunuwar, Lepcha,
134
+ Limbu, Kirat Rai, and shared Devanagari. `tools/run_recognizer_pack_normalization.py`
135
+ validates those recipes, supports dry-runs, materializes selected packs through the
136
+ normalizer, writes per-pack `audit/recognizer-eval-pack-audit.{json,md}` artifacts,
137
+ and records pack audit results in `normalization-run.json`.
138
+ - `configs/eval-target-coverage-v1.json` records the intended language/script coverage
139
+ sets: the broader first-ten document OCR targets and the current real recognizer
140
+ packs. `tools/validate_eval_target_coverage.py` compares those targets against the
141
+ normalization and eval-run recipe registries so missing packs, missing eval recipes,
142
+ non-claim-ready targets, and shared-pack language/script mismatches that could hide
143
+ per-language failures are explicit gaps. Target `eval_status` values are constrained
144
+ to known states, and only `registered_real_pack` can become coverage-ready; partial,
145
+ shared, missing, or invented status labels cannot silently pass as claim support.
146
+ - `tools/run_page_ocr_eval.py` scores saved page-level OCR `document.json` outputs
147
+ against structured references. It emits an explicit `eval_mode=page_level_ocr`
148
+ artifact, separates page OCR from recognizer oracle-crop metrics, records model/run
149
+ provenance with local checkpoint/config/dictionary SHA-256 identities, records
150
+ scored source image and `document.json` SHA-256 identities plus the embedded
151
+ `document.source_path` in `page-metrics.csv`, rejects claim-safe runs whose
152
+ `document.source_path` does not match the eval-manifest input, records and audits
153
+ the page pack `pack_content_sha256`, recomputes page text micro CER over codepoints
154
+ and grapheme clusters,
155
+ audits `page-metrics.csv` row identity back to the eval manifest, resolved
156
+ `document.json`, scorer metadata defaults, row claim eligibility, and parsed
157
+ reference fields, reports
158
+ auditable per-language/script/domain/GT/split page slices recomputed from
159
+ `page-metrics.csv`, existing document metrics such as reading-order pair accuracy,
160
+ table scores, figure scores, reconstruction
161
+ metrics, page text exact accuracy, canonical page prediction/reference text
162
+ pairs, per-row reference path/hash/trust artifacts, source-document/page/bbox provenance columns, reference coverage,
163
+ missing document/reference rows, row-level source-document/page/bbox/
164
+ claim-eligibility and language/script/domain/GT metadata coverage, row-level
165
+ claim eligibility, line-bbox IoU matching diagnostics when
166
+ references provide explicit reading-order bboxes,
167
+ worst-page review CSV/HTML artifacts, deterministic page-bootstrap 95% confidence
168
+ intervals for page text CER, observed and missing required script codepoint/grapheme
169
+ inventory coverage from reference text, and train/dev/eval leakage checks with recorded
170
+ train-manifest hashes. Page GT declared as `machine_converted_spot_checked` requires
171
+ an eligible native-reader calibration artifact, matching the recognizer-side trust
172
+ gate. Page claim-safe runs require at least 30 scored pages per emitted
173
+ language/script/domain/GT/split slice, and page score
174
+ directories also emit `claim-safety.md` so they can be reviewed without a recipe
175
+ wrapper. Duplicate page manifest sample IDs or image paths and stale manifest image
176
+ hashes fail before scoring, and missing script inventory files or missing required
177
+ codepoints/graphemes are claim-safety blockers. Page rows declared `human_verified`
178
+ must also carry human review evidence rather than relying on a trust label alone.
179
+ - `tools/run_eval_output_command.py` materializes canonical eval outputs through a
180
+ recorded external command before scoring. In `recognizer` mode it writes one
181
+ prediction file from a label pack; in `page` mode it runs once per eval-manifest row
182
+ and requires each sample to produce `document.json`. It records the exact expanded
183
+ command(s), command template, invocation, timestamp, stdout/stderr log paths, output
184
+ hashes/sizes, per-page input image paths/hashes, model/checkpoint/config/dictionary metadata, machine details, and
185
+ failure details with machine/invocation and label or eval-manifest hashes when a
186
+ command does not produce the required artifact. When
187
+ `--model-config` is provided without
188
+ `--command`, it uses the repo's candidate engine defaults: recognizer mode calls
189
+ `tools/predict_recognizer_eval.py`, while page mode calls `ocrtech parse-doc`.
190
+ - `tools/predict_recognizer_eval.py` turns a candidate model card into a canonical
191
+ recognizer prediction file for oracle-crop eval packs. It reads the label file,
192
+ runs the audited `candidate` engine on each crop, writes `image<TAB>prediction<TAB>
193
+ confidence`, and records model-card, label, output, machine, and command hashes.
194
+ - `tools/prepare_native_spotcheck_bundle.py` creates native-reader review bundles from
195
+ recognizer labels, and `tools/apply_native_spotcheck_reviews.py` ingests completed
196
+ review CSVs into calibration artifacts. The calibration step validates statuses,
197
+ emits reviewed-label and manifest-override artifacts, measures machine-converted label
198
+ CER against corrected/native-reviewed text, requires reviewer identity on reviewed
199
+ rows, and records whether the spot-check is eligible evidence for treating a pack as
200
+ `machine_converted_spot_checked`. Scorers bind that calibration evidence to the
201
+ current run's language, script, domain, `machine_converted` source GT status, and
202
+ `machine_converted_spot_checked` recommendation. Bundle summaries record source file
203
+ hashes, machine metadata, timestamp, and review artifact paths.
204
+ - `tools/build_recognizer_leakage_manifest.py` turns recognizer train/dev label files
205
+ into JSONL leakage manifests with sample IDs, image hashes, text hashes, and combined
206
+ sample hashes. These manifests are the standard input for scorer-side train/dev/eval
207
+ leakage checks; text-only mode exists for corpora where image files are intentionally
208
+ unavailable. Page-derived crop manifests must also preserve `source_sample_id`, and
209
+ the page scorer/audit treat train `source_sample_id` overlap with eval page
210
+ `sample_id` as hard leakage.
211
+ - `docs/EVAL_HARNESS_RUNBOOK.md` gives the checked operational recipe for chaining
212
+ normalization, leakage-manifest generation, output materialization, recognizer scoring,
213
+ native-reader calibration, and page-level scoring. It documents the exact claim-safety
214
+ flags and the distinction between repo candidate model-card runs and external
215
+ PaddleOCR/HF command templates.
216
+ - `configs/eval-harness-recipes-v1.json` records concrete current recipe instances for
217
+ Newa, Sunuwar, Lepcha, Limbu, Kirat Rai, shared Devanagari, and packaged HF/Paddle
218
+ recognizer smoke paths. `tools/validate_eval_harness_recipes.py` verifies that
219
+ referenced local paths exist, runs the canonical recognizer eval-pack audit for each
220
+ path-complete recognizer pack, statically audits page eval manifests for source-image
221
+ hashes, page provenance, row claim eligibility, human-review evidence, and
222
+ claim-ready references plus minimum overall and per-slice scoring-unit counts, and
223
+ separates structural recipe validity from claim readiness
224
+ blockers, including missing, malformed, or recipe-contradicting `summary_file` metadata
225
+ for language/script/task/dataset/domain/GT/split/version/eligibility,
226
+ missing or malformed `script_inventory_file`, incomplete row provenance,
227
+ pack-audit warnings or issues, missing, malformed, ineligible, or mismatched recipe-level
228
+ native spot-check calibration when default or audited row GT is `machine_converted_spot_checked`,
229
+ scorer-supported normalization profile declarations,
230
+ external command templates missing required input/output placeholders or containing
231
+ unsupported placeholders, unresolved external prediction command templates, commands still marked as
232
+ requiring operator resolution, unhashable checkpoint/config/dictionary artifacts,
233
+ malformed or empty `leakage_sources`, declared
234
+ checkpoint selection on the test pack, and declared known leakage.
235
+ - `tools/run_eval_recipe.py` is the recipe-level recognizer orchestrator. It validates
236
+ the selected recipe, builds or wires train/dev leakage manifests, materializes
237
+ predictions through the recorded runner, scores the canonical predictions, and writes
238
+ `recipe.json`, `recipe-validation.json`, `recipe-run.json`, root-level `audit.json`,
239
+ root-level `claim-safety.md`, materialization logs, leakage artifacts, and the scorer
240
+ output under one reviewable run directory. The root claim-safety report carries the
241
+ score-mode-specific recognizer or page facts from the recipe, score, and audit JSON
242
+ files, including script-inventory coverage, metadata contradictions, uncertainty
243
+ intervals, leakage status, and the score-mode review artifact list, so it can be
244
+ audited for stale reviewer-facing summaries, root `recipe.json` is tied back to the selected
245
+ recipe in the hashed registry file, root `recipe-validation.json` is tied back to
246
+ `recipe-run.json` and recomputed from the current validator, recipe-run
247
+ timestamp/machine/command provenance is checked, root
248
+ materialization/audit/claim-safety output pointers must point at the run directory's
249
+ own artifacts, root audit summaries must preserve current score-audit issue lists,
250
+ warnings, claim-safe status, and score-dir identity, root score summary paths and
251
+ hashes must match `score/metrics.json` and `score/eval-run.json`, root train-manifest
252
+ path/hash identities must match the score eval-run leakage inputs, the hashed recipe
253
+ registry file must sit under the declared recipe root, and the root
254
+ prediction/materialization paths, command template, output report, and materialized
255
+ model/checkpoint/config/dictionary metadata are tied back to the scored prediction
256
+ file hash and score eval-run metadata. Failed recipe attempts write
257
+ `recipe-run-error.json` with timestamp, machine, command, recipe validation,
258
+ registry hash, root, output directory, exception type, message, and traceback so a
259
+ failed run is still attributable.
260
+ - `tools/compare_recognizer_eval_runs.py` compares two completed recognizer oracle-crop
261
+ score directories on the same frozen eval pack. It pairs canonical `predictions.tsv`
262
+ rows by `sample_id`, verifies eval-pack/run identity including `pack_content_sha256`
263
+ and row GT identity, emits
264
+ `comparison.json`, `comparison.md`, `paired-rows.tsv`, and `command.txt`, reports candidate and
265
+ baseline micro CER over codepoints and graphemes, paired wins/losses/ties over
266
+ row-level CER, deterministic paired-row bootstrap confidence intervals for the CER
267
+ improvement, and an exact two-sided sign-test p-value. Its audit mode checks run
268
+ provenance, recomputes the comparison, and rejects stale JSON, Markdown, command,
269
+ or paired-row artifacts.
270
+ - `tools/compare_page_ocr_eval_runs.py` does the same for completed page-level OCR
271
+ score directories, using `page-eval-run.json`, `page-metrics.csv`, and
272
+ `text-pairs.tsv`. It verifies page eval-pack identity and page/reference text row
273
+ identity including `pack_content_sha256`, emits `comparison.json`,
274
+ `comparison.md`, `paired-pages.tsv`, and `command.txt`, reports
275
+ paired page text codepoint/grapheme CER improvements, deterministic paired-page
276
+ bootstrap intervals, and an exact two-sided sign-test p-value, and can audit
277
+ command/provenance plus the comparison artifacts back to the current score
278
+ directories.
279
+ - `tools/audit_eval_run.py` audits completed recognizer or page score directories, or a
280
+ recipe-run root with `score/`, for required artifacts, run metadata, metric arithmetic,
281
+ timestamp and command-file consistency, machine platform/Python provenance, recorded
282
+ input-file hash consistency, scorer-supported normalization profile consistency,
283
+ normalization-change summary recomputation from recognizer `predictions.tsv` or page
284
+ `text-pairs.tsv`,
285
+ model-artifact hash and run-field consistency,
286
+ recognizer `predictions.tsv` source-row recomputation from the recorded
287
+ label/prediction/manifest files, row-level
288
+ edit-distance and per-language/script/domain/GT/split slice recomputation,
289
+ recognizer crop-image and manifest-declared image-hash verification, recognizer source label/prediction
290
+ recomputation for label-line, matched-line, unmatched-label, unmatched-prediction,
291
+ match-rate metrics, deterministic bootstrap uncertainty intervals, and script
292
+ inventory recomputation from recognizer GT rows, recognizer
293
+ `unmatched.tsv` identity/hash/score recomputation from the source
294
+ label and prediction files, recognizer `confusions.json` recomputation from canonical
295
+ predictions, page `reference-artifacts.tsv` recomputation from scored reference
296
+ files, page `text-pairs.tsv` recomputation from scored document/reference text,
297
+ page `confusions.json` recomputation from those canonical text pairs,
298
+ page text CER row and total
299
+ recomputation from canonical page text pairs, page row provenance recomputation
300
+ from source manifests, scored source-image/
301
+ `document.json` hash verification, page bootstrap uncertainty interval recomputation,
302
+ script inventory recomputation from page reference text,
303
+ and page-metric identity recomputation from manifest/document/reference sources,
304
+ run-level language/script/domain/GT/split
305
+ list recomputation from canonical row artifacts, page
306
+ `line-alignment.csv` row-content and
307
+ detection/alignment recomputation from scored document/reference files, including
308
+ predicted/reference line IDs, bboxes, text, IoU, and per-line text CER, page
309
+ recognizer normalizer/defaulted-metadata count checks against the recorded
310
+ summary/datasheet,
311
+ direct-run summary/datasheet metadata consistency checks against run language,
312
+ script, domain, GT, split, eval-pack version, frozen/unseen, default
313
+ claim-eligibility, and task fields,
314
+ dictionary coverage recomputation from recognizer GT rows or page reference text,
315
+ with GT/reference codepoints outside the recorded dictionary treated as a
316
+ claim-safety blocker,
317
+ `missing.tsv` recomputation from the eval
318
+ manifest, document outputs, and reference files, slice totals, duplicate scored sample/image IDs
319
+ in completed output artifacts, ambiguous recognizer manifest image keys,
320
+ duplicate page manifest image paths, page manifest
321
+ image-hash mismatches, leakage-check presence and leakage-overlap recomputation
322
+ from train/eval sources, native spot-check calibration summary checks against the
323
+ recorded calibration artifact, metadata-contradiction recomputation from source
324
+ provenance signals, provenance/metadata coverage recomputation from source
325
+ manifests, page reference-coverage and manifest/scored/missing/claim-eligible count
326
+ recomputation from source artifacts, durable
327
+ claim-safety blockers
328
+ such as metadata contradictions, missing manifests or summary/datasheets, unrun
329
+ leakage checks, leakage overlaps, low match rate, unsafe domains, untrusted GT,
330
+ missing provenance, missing uncertainty evidence, too-small claim-facing sample size,
331
+ missing script inventory evidence, missing required script codepoints/graphemes,
332
+ zero-length recognizer GT rows or page text references, recognizer row-level claim
333
+ ineligibility, and incomplete page coverage,
334
+ warning/claim-safety
335
+ consistency, `metrics.md` consistency against `metrics.json`, recomputed
336
+ worst-example CSVs, reviewer CSV order/fields, and reviewer HTML contents, direct
337
+ score-directory claim reports whose core lines are checked against JSON run facts,
338
+ recipe-root claim reports whose recognizer/page score facts are checked against the
339
+ nested score directory, and materialization artifacts whose timestamp, machine platform/Python
340
+ and invocation metadata, JSON records, model/checkpoint/config/dictionary metadata,
341
+ static recipe-level dictionary coverage checks against recognizer GT or page
342
+ reference codepoints,
343
+ per-record commands/log paths, command-template file,
344
+ Markdown report, recognizer prediction path/hash or page document-output directory,
345
+ and output hashes are checked against the score
346
+ eval-run facts plus claim-report artifacts whose top-level report is also checked
347
+ against recipe and score JSON facts.
348
+ - The scorer computes corpus micro CER over NFC codepoints and Unicode extended
349
+ grapheme clusters, exact-line accuracy, matched/unmatched rows, per-language/script/
350
+ domain/GT/split slice metrics, worst lines/pages, review CSV/HTML, and recognizer/page
351
+ confusion counts.
352
+ - Claim-safety warnings now cover missing manifest or summary, missing eval-pack
353
+ version, missing script inventory files, missing required script codepoints or
354
+ graphemes, GT/reference codepoints outside the recorded dictionary, unhashable model
355
+ artifacts, zero-length recognizer GT rows or page text references, non-frozen or selection-used eval packs, low
356
+ prediction match rate, mixed languages/scripts/domains/GT statuses/splits, unsafe
357
+ domains, untrusted GT, metadata contradictions such as synthetic/transliterated
358
+ provenance declared as real material or machine-converted provenance declared as
359
+ `human_verified`, missing source-document/page/bbox/claim-eligibility provenance,
360
+ explicit row-level claim ineligibility, missing page row source-document/page/bbox/
361
+ claim-eligibility/language/script/domain/GT metadata, normalizer-filled metadata defaults from
362
+ canonical pack summaries, missing human-review evidence for `human_verified` rows, missing,
363
+ mismatched, or ineligible native spot-check calibration when
364
+ `machine_converted_spot_checked` is claimed, known leakage, unrun leakage checks, and
365
+ train/dev/eval overlaps from provided JSONL manifests.
366
+ - A Newa recognizer artifact was produced at
367
+ `outputs/evals/newa-prachalit/newa-native-rescore-2026-06-19-zenodo/`. It is
368
+ intentionally **not** claim-safe: GT is machine-converted, the pack is not declared
369
+ unseen for selection, page provenance and claim eligibility are missing, and no
370
+ train/dev leakage manifest was provided.
371
+ - The same Newa eval pack has a standalone audit at
372
+ `outputs/evals/newa-prachalit/newa-real-eval-pack-audit-2026-06-19/`. Basic pack
373
+ integrity passes (`3180/3180` labels matched to manifest rows and images), but
374
+ claim readiness fails because the pack is not declared unseen for selection, has
375
+ no page metadata, lacks row-level claim eligibility, and uses machine-converted GT.
376
+
377
+ ## Remaining Gaps
378
+
379
+ - Resolve remaining remote-only checkpoint/package locations in
380
+ `configs/eval-harness-recipes-v1.json` and replace placeholder external commands with
381
+ runnable per-family commands. The recipe registry exists and validates local paths,
382
+ and `tools/run_eval_recipe.py` can execute resolved recognizer recipes, but several
383
+ language-specific PaddleOCR checkpoints are not packaged locally or hash-verifiable,
384
+ so the validator must keep them out of claim-ready status.
385
+ - Extend page-level evaluation with live detector output capture and crop lineage from
386
+ model runners; the current page harness scores saved structured documents and emits
387
+ per-line IoU/alignment rows when references provide explicit line bboxes, but it does
388
+ not yet run the detector/recognizer pipeline itself.
389
+ - Archive and review the materialized canonical eval packs under `outputs/eval-packs/`.
390
+ The six current real recognizer packs now exist with per-pack audits, but all six are
391
+ structurally non-claim-ready because of machine-converted GT, missing or defaulted
392
+ provenance/claim metadata, and for some packs selection-status blockers.
393
+ - Fill the target coverage gaps reported by `tools/validate_eval_target_coverage.py`.
394
+ The current coverage config validates, but is intentionally not coverage-ready:
395
+ English, Tamang, Magar, Wambule, Bantawa/Rai, Kulung, Yakkha, and Tharu/Rana Tharu
396
+ still lack registered real eval packs, and the current registered real-pack recipes
397
+ remain non-claim-ready because model commands, GT trust, provenance, calibration, or
398
+ row eligibility blockers remain unresolved. A target also remains non-coverage-ready
399
+ unless its `eval_status` is exactly `registered_real_pack`; use `partial_real_recognizer_pack`,
400
+ `shared_devanagari_pack`, or `missing_real_eval_pack` only when the resulting gap is
401
+ intended and visible.
402
+ - Generate and wire train/dev leakage manifests for each actual recognizer family and
403
+ pass them to scoring runs. The manifest builder exists, but each family still needs
404
+ its concrete train/dev label paths recorded in the run recipes, including page-origin
405
+ `source_sample_id` for crops selected from full-page eval material.
406
+ - Apply native-reader spot-check/calibration to the actual language eval packs, with
407
+ enough reviewed rows per script/domain to justify any `machine_converted_spot_checked`
408
+ upgrade. The harness now has artifacts and scorer wiring for this, but the reviewed
409
+ human evidence still has to be collected.
410
+
411
+ ## Research Basis
412
+
413
+ - OCR-D states that OCR quality evaluation requires representative ground truth,
414
+ realistic images, manual transcriptions, localization of errors by document/page/
415
+ region, and repeatable workflows:
416
+ <https://ocr-d.de/en/spec/ocrd_eval.html>
417
+ - The OCR evaluation survey by Neudecker et al. describes CER as edit-distance errors
418
+ divided by reference character count and highlights that OCR evaluation depends on
419
+ alignment, reading order, layout, and tool-specific normalization choices:
420
+ <https://www.primaresearch.org/www/assets/papers/HIP21_CNeudecker_OcrEvalSurvey.pdf>
421
+ - Unicode UAX #29 defines extended grapheme clusters as the default programmatic
422
+ approximation of user-perceived characters, which is necessary for scripts where one
423
+ visible unit can span multiple codepoints:
424
+ <https://www.unicode.org/reports/tr29/>
425
+ - PaddleOCR's `RecMetric` implementation averages normalized Levenshtein distance per
426
+ sample and returns `norm_edit_dis`; this is not the same quantity as micro CER:
427
+ <https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/main/ppocr/metrics/rec_metric.py>
428
+ - Datasheets for Datasets motivates documenting dataset motivation, composition,
429
+ collection, and recommended use, which maps directly onto eval-pack provenance:
430
+ <https://arxiv.org/abs/1803.09010>
431
+ - Model Cards for Model Reporting motivates model documentation with benchmarked
432
+ evaluation, intended use, limitations, and evaluation procedure:
433
+ <https://arxiv.org/abs/1810.03993>
434
+ - Recent OCR evaluation survey work warns that benchmark design can make historical
435
+ and marginalized documents invisible when evaluation emphasizes convenient modern
436
+ layouts rather than the actual target material:
437
+ <https://arxiv.org/html/2603.25761v1>
ocr-tech-mps-integration/docs/EVAL_HARNESS_RESEARCH_GOAL.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Research-Backed Eval Harness Goal
2
+
3
+ Updated: 2026-06-20
4
+
5
+ ## Sources Read
6
+
7
+ - Web/paper search performed on 2026-06-20 before drafting this goal.
8
+ - OCR-D quality assurance guidance: representative ground truth, realistic images,
9
+ manual transcription, page/region localization, CER/WER definitions, reading-order
10
+ and layout caveats, IoU/object-detection evaluation, and NFC normalization guidance:
11
+ <https://ocr-d.de/en/spec/ocrd_eval.html>
12
+ - Neudecker et al., "A survey of OCR evaluation tools and metrics" (HIP 2021):
13
+ OCR evaluation is tool- and normalization-sensitive, and text accuracy depends on
14
+ alignment, reading order, layout, and the exact edit-distance metric:
15
+ <https://dl.acm.org/doi/10.1145/3476887.3476888>
16
+ - Santos, "OCR Evaluation Tools for the 21st Century" (ComputEL 2019): OCR evaluation
17
+ for modern and endangered-language work needs Unicode-capable character/word
18
+ reporting rather than ASCII-era assumptions:
19
+ <https://doi.org/10.33011/computel.v1i.345>
20
+ - Unicode Standard Annex #29: extended grapheme clusters are the default
21
+ programmatic approximation of user-perceived characters:
22
+ <https://www.unicode.org/reports/tr29/>
23
+ - PaddleOCR `RecMetric`: `norm_edit_dis` is one minus a per-sample average of
24
+ normalized Levenshtein distance, not corpus micro CER:
25
+ <https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/main/ppocr/metrics/rec_metric.py>
26
+ - Leifert et al., "End-to-End Measure for Text Recognition" (ICDAR 2019): end-to-end
27
+ text recognition evaluation must account for recognition plus segmentation/alignment,
28
+ not just isolated text strings:
29
+ <https://doi.org/10.1109/ICDAR.2019.00-16>
30
+ - Beyene and Dancy, "A Survey of OCR Evaluation Methods and Metrics and the
31
+ Invisibility of Historical Documents" (arXiv 2026 preprint): modern OCR and document
32
+ understanding evaluations overfit to modern/institutional benchmark genres, often
33
+ miss historical/community archive layout failures, and can make marginalized
34
+ materials invisible even when headline CER/WER looks acceptable:
35
+ <https://arxiv.org/html/2603.25761v1>
36
+ - Springmann et al., "Automatic quality evaluation and (semi-) automatic improvement
37
+ of OCR models for historical printings": GT is scarce, manual lines can change
38
+ results materially, and confidence/proxy measures are not replacements for true GT:
39
+ <https://arxiv.org/abs/1606.05157>
40
+ - Gebru et al., "Datasheets for Datasets": datasets need explicit motivation,
41
+ composition, collection, processing, intended use, and limitations:
42
+ <https://arxiv.org/abs/1803.09010>
43
+ - Mitchell et al., "Model Cards for Model Reporting": model releases need benchmarked
44
+ performance, intended use, limitations, evaluation procedure, and per-condition
45
+ reporting:
46
+ <https://arxiv.org/abs/1810.03993>
47
+
48
+ ## Dense Goal Paragraph
49
+
50
+ Build a claim-grade OCR evaluation harness for the Nepal/diaspora recognizer suite
51
+ whose default output makes an unsafe paper claim obvious before anyone can quote a
52
+ number: every supported target, including the shared Devanagari languages
53
+ Majhi, Magar, Sunuwar-Devanagari, Magahi, Bhojpuri, Thami, Wambule, and Darai plus
54
+ native-script targets such as Limbu/Sirijonga, Sunuwar/Jenticha, Newa/Prachalit,
55
+ Tirhuta, Lepcha, Kirat Rai, Gurung Khema, and any later Tamang, Bantawa/Rai, Kulung,
56
+ Yakkha, Tharu/Rana Tharu, or English scope, must be evaluated as its own
57
+ language/script/domain slice from a frozen, versioned, real-image eval pack with
58
+ image bytes, labels, manifest rows, source-document/page/bbox provenance, split role,
59
+ domain, ground-truth source and trust evidence, native-reader calibration when GT is
60
+ machine-converted, row claim eligibility, script inventory coverage, and leakage
61
+ declarations; every run must record exact model/checkpoint/config/dictionary or model
62
+ card, prediction artifact, normalization profile, Unicode and grapheme-segmentation
63
+ behavior, command, timestamp, machine, tool versions, and artifact hashes; every
64
+ reported metric must be recomputable from canonical row artifacts as corpus micro
65
+ edit-distance CER over both Unicode codepoints and UAX #29 extended grapheme clusters,
66
+ not PaddleOCR `norm_edit_dis` or any per-line macro surrogate, with exact-line
67
+ accuracy, matched and unmatched rows, missing-prediction accounting, confusion
68
+ summaries, worst examples, reviewer artifacts, deterministic confidence intervals,
69
+ and paired same-pack tests for model comparisons; recognizer oracle-crop results must
70
+ remain separate from page-level OCR results, where page claims additionally require
71
+ detection/crop lineage, reading-order and layout alignment, line-bbox/IoU or explicit
72
+ reference matching, table/figure/text reconstruction metrics where relevant,
73
+ historical/newspaper degradation and multi-column failure visibility, and visible
74
+ coverage of missing documents or references; the harness must fail or hard-warn on
75
+ missing or mutated files, duplicate IDs or image paths, stale hashes, unsupported or
76
+ asymmetric normalization, low match rate, unrun or positive train/dev/test leakage,
77
+ checkpoint selection on test data, synthetic or rendered data presented as real
78
+ evidence, transliterated or machine-converted labels presented as human-verified
79
+ evidence, human GT without review/agreement evidence, too-small or script-incomplete
80
+ packs, aggregate scores that hide per-language/script/domain/GT failures, modern-
81
+ benchmark averages that do not exercise the target community archives, and any
82
+ hand-edited report whose JSON/TSV/CSV artifacts no longer recompute; a reviewer
83
+ opening one output directory must be able to answer, without trusting prose, exactly
84
+ what was scored, against which GT, how trustworthy that GT is, whether the test set was
85
+ frozen and unseen for selection, which Unicode, script, layout, provenance, and archive
86
+ coverage assumptions moved the number, how uncertain the number is, what failed, whose
87
+ material is still unsupported, and whether the result is safe to cite in a paper.
88
+
89
+ ## Non-Negotiable Requirements
90
+
91
+ - Real held-out eval packs are evidence objects, not folders of images. They need
92
+ immutable manifests, hashes, provenance, split status, GT trust, domain labels,
93
+ claim eligibility, script inventory coverage, and a datasheet-style summary.
94
+ - Recognizer and page OCR are different tasks. Oracle-crop CER may support a recognizer
95
+ claim only; page OCR claims need detector output, crop lineage, reading order,
96
+ layout alignment, and missing-page/reference accounting.
97
+ - Metrics must be recomputed from emitted row artifacts. Aggregate JSON is not a
98
+ source of truth unless `predictions.tsv`, `page-metrics.csv`, alignment rows, and
99
+ references recompute to the same values.
100
+ - Unicode is part of the metric contract. The harness must record normalization,
101
+ Unicode/grapheme implementation behavior, codepoint CER, grapheme CER, and missing
102
+ required script inventory items.
103
+ - Claim-facing reports need uncertainty and comparison discipline. Point CER alone is
104
+ not enough; small packs need hard warnings, and model-vs-model claims need paired
105
+ same-pack analysis.
106
+ - Ground truth quality must be visible. Machine conversion and transliteration are
107
+ scoreable, but they are not human-verified evidence unless a native-reader
108
+ calibration artifact supports that upgrade.
109
+ - The harness must be hostile to silent omissions. Missing predictions, duplicate
110
+ rows, leaked train/dev examples, stale hashes, skipped pages, unsupported
111
+ normalization, and stale Markdown should fail audit or block claim safety.
ocr-tech-mps-integration/docs/EVAL_HARNESS_RUNBOOK.md ADDED
@@ -0,0 +1,803 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Eval Harness Runbook
2
+
3
+ Updated: 2026-06-20
4
+
5
+ This is the operational path for producing a claim-grade OCR eval directory. The
6
+ goal is not to get a number quickly; it is to make the number hard to misread.
7
+ Each run should leave enough artifacts for a reviewer to answer what was scored,
8
+ which model produced it, which ground truth was used, whether the pack was frozen
9
+ and unseen for selection, what leakage checks ran, and why the result is or is not
10
+ safe for a paper claim.
11
+
12
+ ## Recipe-First Eval
13
+
14
+ Use this path when a registered recipe exists for either `recognizer_oracle_crop`
15
+ or `page_level_ocr`. Recipes keep validation, leakage checks, output
16
+ materialization, scoring, root reports, and audit metadata under one output
17
+ directory.
18
+
19
+ ### Registry Validation
20
+
21
+ When a recipe exists in `configs/eval-harness-recipes-v1.json`, use it as the
22
+ default entry point. This keeps validation, leakage checks, prediction
23
+ materialization, and scoring under one output directory.
24
+
25
+ Before scoring legacy source packs, validate or run the normalization registry:
26
+
27
+ ```bash
28
+ uv run python tools/run_recognizer_pack_normalization.py \
29
+ --config configs/recognizer-eval-pack-normalization-v1.json \
30
+ --root . \
31
+ --all \
32
+ --out outputs/eval-packs/normalization-runs/<run-id> \
33
+ --dry-run \
34
+ --require-valid
35
+ ```
36
+
37
+ Drop `--dry-run` only when you intentionally want to materialize canonical packs
38
+ under `outputs/eval-packs/<pack-version>/`. A materialized pack gets its own
39
+ `audit/recognizer-eval-pack-audit.json` and `.md`; read those before wiring the
40
+ pack into a claim run.
41
+
42
+ Then validate intended language/script coverage against the normalization and
43
+ eval-run recipe registries:
44
+
45
+ ```bash
46
+ uv run python tools/validate_eval_target_coverage.py \
47
+ --coverage-config configs/eval-target-coverage-v1.json \
48
+ --normalization-config configs/recognizer-eval-pack-normalization-v1.json \
49
+ --eval-recipes configs/eval-harness-recipes-v1.json \
50
+ --root . \
51
+ --out outputs/evals/target-coverage.json \
52
+ --require-valid
53
+ ```
54
+
55
+ Do not add `--require-coverage-ready` until every target in the selected claim
56
+ scope has a normalized eval pack, runnable eval recipe, and no claim blockers.
57
+ If a target points at a shared aggregate pack, the validator must show matching
58
+ target language/script metadata or an explicit gap; do not use a shared pack
59
+ average as evidence for an individual language claim. Coverage target status is
60
+ also gated: `eval_status` must be one of the known states, and only
61
+ `registered_real_pack` is allowed to become coverage-ready. Use
62
+ `partial_real_recognizer_pack`, `shared_devanagari_pack`, or
63
+ `missing_real_eval_pack` when a target is deliberately not ready for claims.
64
+ For book OCR targets, set `requires_page_eval_recipe` and
65
+ `requires_book_candidate_evidence` on the target set or target. That makes coverage
66
+ depend on a page-level recipe with book-candidate evidence from the scoring/admission
67
+ lane, not just an oracle-crop recognizer score or a loose page replay. The validator
68
+ also reports whether the linked book candidate is operationally admitted and
69
+ claim-safe, so non-claim-safe Sirijonga and Devanagari book evidence show up as
70
+ separate gaps.
71
+
72
+ ```bash
73
+ uv run python tools/validate_eval_harness_recipes.py \
74
+ --recipes configs/eval-harness-recipes-v1.json \
75
+ --root . \
76
+ --require-valid
77
+ ```
78
+
79
+ Each recipe `eval_pack` must declare `normalization_profile:
80
+ NFC_preserve_whitespace`, matching the scorer/auditor recomputation contract, and a
81
+ readable, parseable `summary_file` and `script_inventory_file`. The recipe validator
82
+ checks the summary JSON object and inventory JSON shape before a recipe is considered
83
+ structurally valid, and a summary without dataset/version/eval-pack metadata blocks
84
+ claim readiness. Summary/datasheet declarations for language, script, dataset, task,
85
+ domain, GT status, split, pack version, frozen/unseen status, and default claim
86
+ eligibility must agree with the recipe metadata; contradictions block claim readiness.
87
+ Do not paper over a pack that needs another normalization policy; add
88
+ scorer support and audit coverage first, then change the recipe.
89
+ The script inventory is a JSON object with `required_codepoints` and
90
+ `required_graphemes` lists. Codepoints may be written as `U+XXXX` values or literal
91
+ single characters; graphemes are literal strings. The scorer records observed coverage
92
+ from GT/reference text, and claim-safe runs fail if any required item is missing.
93
+ When a runner dictionary is a readable local file, the recipe validator also compares
94
+ its non-whitespace codepoints to recognizer GT or page reference text; out-of-vocabulary
95
+ GT/reference codepoints block claim readiness before scoring.
96
+ If the recipe default or audited row metadata uses `machine_converted_spot_checked`
97
+ GT, the recipe must also declare a readable JSON `eval_pack.spotcheck_calibration_file`;
98
+ the recipe validator checks its eligibility/recommendation/source-status fields and
99
+ language/script/domain consistency, and recipe execution passes that artifact into the
100
+ scorer, where it is hashed and audited.
101
+
102
+ ### Pack Content Digest Contract
103
+
104
+ Every claim-facing eval pack summary/datasheet must include `pack_content_sha256`.
105
+ This is not a cosmetic version string. It is the immutable content identity that lets
106
+ scoring, audit, and comparison prove that the score still belongs to the same source
107
+ pack.
108
+
109
+ For recognizer oracle-crop packs, the digest binds each canonical row's sample ID,
110
+ source image hash, NFC-normalized GT text hash, language, script, domain, GT status,
111
+ split, and claim eligibility. For page-level OCR packs, the digest binds each canonical
112
+ page row's sample ID, input image hash, reference artifact hash, reference text hash,
113
+ reference fields, reference trust metadata, source-document/page/bbox provenance,
114
+ language, script, domain, GT status, split, and claim eligibility.
115
+
116
+ Claim-safe scoring must fail when the summary digest is missing or stale. Completed
117
+ score directories must record the recomputed value in `metrics.json` and in the
118
+ mode-specific run file (`eval-run.json` for recognizer, `page-eval-run.json` for page
119
+ OCR). Audit must recompute the digest from current source artifacts and reject any
120
+ mismatch in the run JSON, metrics JSON, or summary/datasheet. Same-task comparison
121
+ tools must treat the digest as eval-pack identity, so a candidate and baseline cannot
122
+ be compared as claim-safe if their `pack_content_sha256` values differ.
123
+
124
+ When a test or fixture intentionally edits pack content before scoring, regenerate the
125
+ summary digest before running the scorer. When content changes after scoring, do not
126
+ refresh the digest; that is tampering or drift, and audit should fail.
127
+
128
+ For `page_level_ocr` recipes, `eval_pack.eval_manifest` is the canonical page
129
+ manifest and materialization writes per-page `document.json` files under the root
130
+ `documents/` directory. External page command templates must write either
131
+ `{document_json}` directly or write `document.json` below `{output_dir}`. The recipe
132
+ validator statically audits page manifests for readable source images and hashes,
133
+ source-document/page/bbox provenance, row claim eligibility, human-review evidence for
134
+ `human_verified` rows, parseable references, claim-ready reference metadata, and
135
+ minimum scoring-unit counts overall and per language/script/domain/GT/split slice; any
136
+ page audit issue or warning blocks claim readiness. For path-complete recognizer packs,
137
+ the recipe validator also runs
138
+ `tools/audit_recognizer_eval_pack.py`; any pack-audit issue or warning becomes a
139
+ claim-readiness blocker, including overall and language/script/domain/GT/split
140
+ sample-size warnings below the claim minimum. `leakage_sources` must also be
141
+ parseable and non-empty; page-level leakage manifests must satisfy the page manifest
142
+ schema, and recognizer label leakage sources must contain valid rows with existing
143
+ images. A recipe is not claim-ready merely because its label, manifest, summary,
144
+ script inventory, and image paths exist.
145
+
146
+ For Limbu, keep the pack counts straight. The canonical materialized recognizer pack
147
+ `outputs/eval-packs/limbu-gorkhapatra-real-v1/` has 1,829 labels and 1,829 manifest
148
+ rows, not 39. The number 39 belongs to an older operational page run's unresolved
149
+ review queue and must not be described as the available Limbu eval set. The current
150
+ Limbu recognizer pack is structurally auditable but not claim-ready: it is
151
+ machine-converted, has 0 claim-eligible rows, is not declared unseen for selection, and
152
+ is missing source-document/page/bbox provenance. The separate CDC Maths PDF text-layer
153
+ pack has 81 silver rendered-line rows, split 69 train / 12 eval; use it for recognizer
154
+ bootstrap/training experiments, not for claim-safe evaluation.
155
+ For `page_level_ocr` recipes with `ocr_output_kind=live_detector`, the recipe must
156
+ also declare `runner.output_audit` bound to `{output_dir}` so page scoring is paired
157
+ with an operational artifact audit. Limbu page recipes must run `audit-limbu-output`
158
+ and record strict flags including `--require-capture-prep`, prepared-image quality
159
+ thresholds (`--min-capture-prepared-width`, `--min-capture-prepared-height`,
160
+ `--min-capture-prepared-entropy`, `--min-capture-prepared-luminance-stddev`, and
161
+ `--min-capture-prepared-edge-stddev`), `--require-no-pending-review`,
162
+ `--require-no-dropped-image-lines`, output quality/volume gates
163
+ (`--min-line-count`, `--min-average-line-confidence`, and `--min-quality-score`), and
164
+ the script-specific `--require-script` value for their target, for example
165
+ `limbu_sirijonga` for Sirijonga-script Limbu or `devanagari_limbu` for
166
+ Devanagari-written Limbu. This keeps page scoring tied to the operational artifact
167
+ audit instead of allowing a claim-ready registry entry for an OCR bundle that would
168
+ fail provenance, capture-quality, output-volume, output-quality, review-resolution, or
169
+ complete-line-retention gates.
170
+
171
+ Then run the selected recipe:
172
+
173
+ ```bash
174
+ uv run python tools/run_eval_recipe.py \
175
+ --recipes configs/eval-harness-recipes-v1.json \
176
+ --recipe-id <recipe-id> \
177
+ --root . \
178
+ --out outputs/evals/<run-id> \
179
+ --timeout-seconds 3600 \
180
+ --min-match-rate 0.99
181
+ ```
182
+
183
+ Add `--require-claim-safe` only when the recipe validator reports no claim
184
+ blockers and the scorer is expected to emit no warnings. If the recipe uses an
185
+ external command template, replace the placeholder command in the recipe first;
186
+ unresolved templates are rejected by default. Recognizer external templates must include
187
+ `{label_file}`, `{data_dir}`, and `{prediction_file}`; page external templates must
188
+ include `{input_path}` plus `{document_json}` or `{output_dir}`. Unknown placeholders
189
+ are validation errors, so command templates must use only the mode-supported
190
+ placeholders documented by `tools/run_eval_output_command.py`. Recipes that declare
191
+ `runner.selected_checkpoint_on_test` or non-empty `known_leakage` remain structurally
192
+ valid but are blocked from claim readiness because the scorer will warn on those same
193
+ facts at runtime. Runner `checkpoint`, `config`, and `dictionary` values must point to
194
+ readable local files, not directories, because claim-safe scores record and audit their
195
+ SHA-256 hashes.
196
+ Recognizer and page eval-pack summaries must also carry `pack_content_sha256`, the
197
+ canonical digest over sample ID, input/reference hashes, normalized GT text or
198
+ structured-reference text, provenance, language, script, domain, GT status, split, and
199
+ claim eligibility. The scorers recompute this digest from the rows they actually
200
+ scored; claim-safe runs fail when the summary digest is missing or stale.
201
+
202
+ The recipe runner writes the root-level files a reviewer should open first:
203
+ `recipe-run.json`, `audit.json`, and `claim-safety.md`. For page-level recipes with
204
+ `runner.output_audit`, execution runs that audit once per materialized page output
205
+ directory before scoring and records `output-audit/output-audit-run.json` plus
206
+ `output-audit/command-template.txt` and Markdown with commands, logs, return codes,
207
+ and available audit artifact hashes. The runner honors
208
+ `runner.output_audit.result_filename` when recording the per-page audit JSON; if it is
209
+ omitted, the default is `limbu-output-audit.json`. The root audit requires each
210
+ recorded per-page audit JSON to still exist with the recorded SHA-256. It also
211
+ requires the output-audit command template sidecar when `recipe-run.json` records an
212
+ output-audit command, rejects a missing or stale sidecar, and replays each per-page
213
+ output-audit command from that template using the recorded output directory. It
214
+ also replays the recorded stdout/stderr log SHA-256 values for each per-page audit
215
+ command. It requires `output-audit/output-audit-run.md` and checks that the Markdown
216
+ report still reflects the recorded count, per-page output audit rows, log hashes, and
217
+ recorded audit artifact hashes.
218
+ The report summarizes what was scored, the GT trust
219
+ metadata, frozen/unseen declarations, leakage status, micro-CER metrics, output-audit
220
+ status, warnings, and review artifact locations. The root report is audited against
221
+ `recipe-run.json`, the score eval-run JSON, and `metrics.json`, with mode-specific
222
+ expectations for recognizer oracle-crop and page-level OCR score facts, so stale
223
+ top-level summaries fail audit.
224
+ The recipe-root audit also checks that `materialize/output-command-run.json` matches
225
+ the materialization summary embedded in `recipe-run.json`, that
226
+ `materialize/command-template.txt` and `materialize/output-command-run.md` match the
227
+ recorded command/output summary, that materialization timestamp, machine platform/
228
+ Python metadata, invocation, per-record command, and stdout/stderr log paths and
229
+ SHA-256 values are present and replay against current log bytes. It also replays
230
+ materialization record commands from the recorded command template using the recorded
231
+ recognizer or page materialization fields; if those fields are missing, the root audit
232
+ fails instead of skipping command replay. For recognizer roots, the root `predictions.txt` path and hash must be the
233
+ same prediction artifact recorded by the score eval-run JSON. For page roots,
234
+ each materialization record must carry the source page image path/hash, the recorded
235
+ hash is rechecked against current bytes, `outputs.documents_dir` must match the score
236
+ eval-run `documents_dir`, and materialized `document.json` outputs must live under that directory. Recipe-root materialization
237
+ model/checkpoint/config/dictionary metadata is also checked against the score eval-run
238
+ metadata.
239
+ If materialization fails before a score directory exists, keep
240
+ `materialize/output-command-run-error.json`: it carries the timestamp, machine,
241
+ invocation, model/checkpoint/config/dictionary metadata, command template, and label
242
+ or eval-manifest path/hash used for the failed attempt. If the recipe wrapper fails,
243
+ keep `recipe-run-error.json`: it records the timestamp, machine, command, recipe
244
+ registry hash, selected recipe validation, root, output directory, exception type,
245
+ message, and traceback.
246
+ The underlying recognizer or page score directory also writes its own
247
+ `claim-safety.md`, so direct scorer runs are reviewable without a recipe wrapper.
248
+
249
+ Audit the finished run directory before reporting the number:
250
+
251
+ ```bash
252
+ uv run python tools/audit_eval_run.py \
253
+ --run-dir outputs/evals/<run-id> \
254
+ --out outputs/evals/<run-id>/audit.json \
255
+ --require-audit-pass
256
+ ```
257
+
258
+ The audit recomputes recognizer row edits and aggregate micro-CER from
259
+ `score/predictions.tsv`; a metric file that is internally ratio-consistent but does
260
+ not match the emitted GT/prediction rows must fail audit. The run machine metadata
261
+ must include concrete `platform` and `python` values, and `normalization_profile`
262
+ must match the scorer-supported `NFC_preserve_whitespace` profile used by the audit
263
+ recomputation path. The audit also recomputes `normalization_summary` from
264
+ `predictions.tsv` for recognizer runs and `text-pairs.tsv` for page runs; those
265
+ fields count raw GT/reference and prediction strings changed by NFC, so
266
+ normalization-heavy scores remain visible instead of being hidden behind normalized
267
+ CER. The audit
268
+ also reconstructs `predictions.tsv` row identity, GT text, prediction text,
269
+ prediction score, crop hash, language/script/domain/GT/split, and row
270
+ claim-eligibility fields from the recorded label, prediction, and manifest sources.
271
+ The same file carries
272
+ language/script/domain/GT/split, claim-eligibility, and crop image SHA-256 values, so
273
+ recognizer slice metrics, claim-eligibility provenance, and crop-image identity are
274
+ recomputed from the emitted row artifact, not trusted as separate aggregates.
275
+ `worst-lines.csv` is also audited against those canonical prediction rows, including
276
+ crop hashes, trust metadata, claim eligibility, and source-document/page/bbox
277
+ provenance, so a failure sample cannot be made more claim-friendly by editing the
278
+ review artifact after scoring.
279
+ Recognizer label-line count, scored-line count, matched-line count, unmatched label
280
+ count, unmatched prediction count, and prediction match rate are recomputed from the
281
+ recorded source label and prediction files.
282
+ Recognizer `normalizer_counts` in `metrics.json` and `eval-run.json` are also checked
283
+ against the recorded summary/datasheet `counts`, so defaulted pack metadata cannot be
284
+ hidden by editing both score JSON files after scoring.
285
+ Recognizer and page summary/datasheet files are semantically audited too: if a
286
+ hash-valid summary declares language, script, domain, GT status, split, eval-pack
287
+ version, frozen/unseen status, default claim eligibility, or task metadata that
288
+ contradicts the scored run metadata, the audit fails.
289
+ Run-level language/script/domain/GT/split lists in `eval-run.json` are also checked
290
+ against the unique values in `predictions.tsv`.
291
+ `predictions.tsv` also carries source-document/page/bbox provenance columns, and the
292
+ audit reconstructs those values from the recorded label/manifest sources.
293
+ For page runs, the audit also recomputes `line-alignment.csv` row contents from the
294
+ scored `document.json` and reference JSON, including predicted/reference line IDs,
295
+ line bboxes, text, IoU, and per-line text CER, then recomputes line detection counts,
296
+ precision/recall/F1, mean IoU, and aggregate metric means from `line-alignment.csv`
297
+ and `page-metrics.csv`.
298
+ The audit also reconstructs `page-metrics.csv` row identity from the eval manifest,
299
+ resolved `document.json`, embedded `document.source_path`, scorer metadata defaults,
300
+ and parsed reference fields, so
301
+ language/script/domain/GT/split, source-document/page/bbox provenance,
302
+ `claim_eligible`, and `reference_fields` cannot be changed after scoring without
303
+ failing audit.
304
+ Run-level language/script/domain/GT/split lists in `page-eval-run.json` must match
305
+ the unique values in `page-metrics.csv`.
306
+ `reference-artifacts.tsv` carries the exact reference path, SHA-256, scoreable
307
+ reference fields, text hash, and reference trust metadata for each scored page, and
308
+ the audit recomputes it from the eval manifest and reference files.
309
+ `worst-pages.csv` is audited against `page-metrics.csv` plus
310
+ `reference-artifacts.tsv`, including input/document hashes, language/script/domain/GT
311
+ metadata, claim eligibility, source-document/page/bbox provenance, reference hashes,
312
+ and reference trust fields.
313
+ `page-metrics.csv` carries raw page text edit/ref counts and exact text-match flags,
314
+ `text-pairs.tsv` carries the exact prediction/reference page text used for those
315
+ counts, and the audit recomputes the text-pair artifact from scored `document.json`
316
+ text plus reference text before rolling it into page text micro-CER,
317
+ text exact-accuracy totals, and per-language/script/domain/GT/split page slices.
318
+ It also records source page image and `document.json` SHA-256 values, which the audit
319
+ rechecks so a page score cannot point at mutated OCR outputs after the fact.
320
+ Train/dev/eval leakage reports are recomputed from the recorded train manifests and
321
+ canonical eval rows/manifests, including checked state, row counts, and overlap sets.
322
+ For page-derived crop training or checkpoint-selection data, include `source_sample_id`
323
+ on train rows; scorer and audit treat overlap between train `source_sample_id` and eval
324
+ page `sample_id` as hard leakage even when crop image hashes and line text hashes do not
325
+ match the full page row directly.
326
+ Native spot-check calibration summaries embedded in run and metric JSON are checked
327
+ against the recorded calibration artifact.
328
+ Metadata contradictions are recomputed from manifest/reference provenance signals and
329
+ canonical row domain/GT fields.
330
+ Recognizer provenance coverage and page row metadata coverage counts are recomputed
331
+ from their source manifests, and page reference coverage is recomputed from
332
+ `page-metrics.csv`. Page manifest/scored/missing/claim-eligible counts are recomputed
333
+ from the eval manifest, document outputs, references, and claim-eligibility metadata.
334
+ Recognizer and page score directories also carry deterministic bootstrap uncertainty
335
+ summaries in `metrics.json`; the audit recomputes those intervals from
336
+ `predictions.tsv` or `page-metrics.csv`. Claim-safe runs currently require at least
337
+ 30 scored rows/pages, so one-page smoke checks and tiny oracle-crop packs must remain
338
+ non-claim-safe even when their point CER is perfect.
339
+ Recognizer and page score directories also carry script inventory summaries in
340
+ `metrics.json` and run JSON; the audit recomputes observed codepoints/graphemes from
341
+ recognizer GT rows or page reference text and verifies the inventory file hash. A
342
+ missing inventory file, a stale inventory summary, or missing required
343
+ codepoints/graphemes remains a claim-safety blocker even when CER is low.
344
+ They also carry `dictionary_coverage` in `metrics.json`; the audit rehashes the
345
+ recorded dictionary file, recomputes the non-whitespace GT/reference codepoints, and
346
+ blocks claim-safe status when the ground truth contains codepoints outside the
347
+ dictionary vocabulary. `metrics.md` and `claim-safety.md` must surface the same
348
+ dictionary coverage, normalization-change, and zero-length GT/reference counters,
349
+ and the audit treats stale Markdown as an output-directory integrity failure.
350
+ Recognizer metrics record `zero_length_gt_lines`, and page metrics record
351
+ `zero_length_text_references`; the audit recomputes both from canonical rows and
352
+ blocks claim-safe status when any scored GT/reference text is empty.
353
+ The audit also derives claim-safety blockers from durable run facts, so a tampered
354
+ `claim_safe=true` field without matching warnings still fails if the artifacts show
355
+ metadata contradictions, a missing manifest or summary/datasheet, an unrun leakage
356
+ check, leakage overlaps, low recognizer match rate, unsafe GT/domain status, missing
357
+ provenance, missing uncertainty evidence, missing script inventory evidence, missing
358
+ required script inventory items, zero-length GT/reference text, too-small claim-facing
359
+ sample size, recognizer row-level claim ineligibility, or incomplete page coverage.
360
+ The direct and recipe-root `claim-safety.md` reports are also checked against their
361
+ JSON run and metric facts, using recognizer or page score fields according to
362
+ `eval_mode`, including score timestamp, machine, command provenance, hashed
363
+ checkpoint/config/dictionary artifact identity, normalization-change,
364
+ dictionary-coverage, script-inventory gaps, metadata contradictions, uncertainty
365
+ intervals, zero-length GT/reference counters, and per-language/script/domain/GT/split
366
+ slice metrics, so stale or aggregate-only reviewer-facing summaries fail audit instead
367
+ of merely having the right section headings. Direct and recipe-root score reports must
368
+ also list the review artifacts a reviewer needs to open, including `command.txt`,
369
+ canonical row artifacts, worst-example CSVs, sample-review CSV/HTML files, confusions,
370
+ and page alignment/reference artifacts.
371
+ For recipe-root audits, `recipe.json` must still match the selected recipe in the
372
+ hashed source registry and `recipe-validation.json` must still match
373
+ `recipe-run.json`; the embedded validation is also recomputed from the current recipe
374
+ registry and eval-pack audit. Recipe-run timestamp, machine, command, and root output
375
+ pointers are audited too. Root `audit.json` and the embedded recipe-run audit summary must
376
+ preserve current score-audit issues, warnings, claim-safe status, and score-dir
377
+ identity, and the recipe-run score summary must point to and hash the current
378
+ `score/metrics.json` plus the mode-specific score run file (`score/eval-run.json`
379
+ for recognizer roots, `score/page-eval-run.json` for page roots). The root train-manifest
380
+ path/hash list must also match the score eval-run leakage inputs, and the hashed
381
+ recipe registry must be under the declared recipe root. Do not hand-edit those files
382
+ after scoring.
383
+ For page-level Limbu book-candidate recipes that use
384
+ `tools/score_limbu_book_ocr_candidate.py`, the recipe validator also audits the
385
+ declared `evidence` chain. It reads the score wrapper, admission report, metrics, and
386
+ review-coverage report; verifies recorded SHA-256 values; requires champion metrics;
387
+ requires review-coverage counts for missed and extra lines; and blocks candidates with
388
+ champion regressions or uncovered review regions. It also parses the recipe
389
+ `command_template` and requires the scorer command to bind the same `--book-output`,
390
+ `--book-audit`, `--champion-metrics`, and `--review-coverage-report` paths recorded in
391
+ the evidence chain. A command string or free-text claim blocker is not enough evidence
392
+ for current-best registration, and stale evidence cannot be attached to a weaker replay
393
+ command.
394
+ The score-level `metrics.md` report is checked against `metrics.json` for core
395
+ counts, CER values, slices, reference coverage, metric means, and warnings.
396
+ Recognizer `unmatched.tsv` row identity, label image hashes, prediction scores, and
397
+ source-level match/count metrics are recomputed from
398
+ the recorded source label and prediction files, recognizer `confusions.json` is
399
+ recomputed from canonical predictions, page `confusions.json` is recomputed from
400
+ canonical `text-pairs.tsv` prediction/reference text, and page `missing.tsv` is
401
+ recomputed from the eval manifest, document outputs, and reference files.
402
+ Worst-example CSVs, reviewer CSV order and fields, and reviewer HTML samples are also
403
+ checked against the canonical scored rows, so stale or hand-edited failure samples do
404
+ not satisfy reviewability.
405
+ Completed score artifacts are checked for duplicate scored sample IDs, image paths,
406
+ document paths, and missing-row keys; duplicate IDs are not only a pre-normalization
407
+ pack failure.
408
+
409
+ For paper-claim numbers, also add `--require-claim-safe`.
410
+
411
+ ## Recognizer Oracle-Crop Eval
412
+
413
+ Use this manual path for line-crop recognizers only. Do not use it for page-level
414
+ OCR claims.
415
+
416
+ ### Manual Path
417
+
418
+ 1. Normalize the eval pack into the canonical schema.
419
+
420
+ ```bash
421
+ uv run python tools/normalize_recognizer_eval_pack.py \
422
+ --label-file data/<pack>/eval.txt \
423
+ --data-dir data/<pack> \
424
+ --manifest-file data/<pack>/manifest.jsonl \
425
+ --summary-file data/<pack>/summary.json \
426
+ --out outputs/eval-packs/<language>-<pack-version> \
427
+ --dataset <dataset-id> \
428
+ --language <language> \
429
+ --script <script> \
430
+ --domain real_print \
431
+ --gt-status machine_converted \
432
+ --split eval \
433
+ --eval-pack-version <pack-version> \
434
+ --frozen-eval-pack \
435
+ --unseen-for-selection \
436
+ --no-claim-eligible
437
+ ```
438
+
439
+ If the normalizer reports missing source document, page, bbox, GT status, or claim
440
+ eligibility, do not hide it with defaults. Fix the pack or keep the result
441
+ non-claim-safe.
442
+ If a recognizer manifest row declares an image `sha256`, the scorer verifies that it
443
+ matches the current crop bytes. A stale declared hash fails scoring and fails later
444
+ audit if an older score directory points at a mutated manifest.
445
+
446
+ 2. Build train/dev leakage manifests for every model family used to select or
447
+ train the checkpoint.
448
+
449
+ ```bash
450
+ uv run python tools/build_recognizer_leakage_manifest.py \
451
+ --label data/<train-pack>/train.txt::data/<train-pack>::train \
452
+ --label data/<train-pack>/eval.txt::data/<train-pack>::dev \
453
+ --out outputs/leakage/<model-family>-train-dev.jsonl \
454
+ --summary outputs/leakage/<model-family>-train-dev.summary.json \
455
+ --source <model-family> \
456
+ --language <language> \
457
+ --script <script> \
458
+ --domain real_print \
459
+ --gt-status machine_converted
460
+ ```
461
+
462
+ Use `--text-only` only when the corpus intentionally has no images. A text-only
463
+ manifest can catch text leakage, but it cannot catch duplicate crop images.
464
+ If crops were cut from a page eval candidate, the leakage manifest must preserve the
465
+ origin page as `source_sample_id`; otherwise a line-crop pack can look clean under exact
466
+ image/text hash checks while still being selected from the eval page.
467
+
468
+ 3. Materialize predictions through a recorded command.
469
+
470
+ For repo candidate model cards, omit `--command` and provide `--model-config`:
471
+
472
+ ```bash
473
+ uv run python tools/run_eval_output_command.py \
474
+ --mode recognizer \
475
+ --model-config outputs/<model-package>/model-card.json \
476
+ --out outputs/evals/<run-id>/materialize \
477
+ --system-name <system-name> \
478
+ --model-name <model-name> \
479
+ --checkpoint <local-checkpoint-file> \
480
+ --config <local-training-or-inference-config-file> \
481
+ --dictionary <local-dictionary-file> \
482
+ --label-file outputs/eval-packs/<language>-<pack-version>/eval.txt \
483
+ --data-dir outputs/eval-packs/<language>-<pack-version> \
484
+ --prediction-file outputs/evals/<run-id>/predictions.txt \
485
+ --timeout-seconds 3600
486
+ ```
487
+
488
+ For external PaddleOCR/HF runners, pass `--command` explicitly. The command must
489
+ write `{prediction_file}` in `image<TAB>prediction<TAB>confidence` format and may
490
+ use only `{label_file}`, `{data_dir}`, `{prediction_file}`, and `{out_dir}`.
491
+
492
+ ```bash
493
+ uv run python tools/run_eval_output_command.py \
494
+ --mode recognizer \
495
+ --command 'bash scripts/predict_external_recognizer.sh --labels {label_file} --data {data_dir} --out {prediction_file}' \
496
+ --out outputs/evals/<run-id>/materialize \
497
+ --system-name <system-name> \
498
+ --model-name <model-name> \
499
+ --checkpoint <local-checkpoint-file> \
500
+ --config <local-training-or-inference-config-file> \
501
+ --dictionary <local-dictionary-file> \
502
+ --label-file outputs/eval-packs/<language>-<pack-version>/eval.txt \
503
+ --data-dir outputs/eval-packs/<language>-<pack-version> \
504
+ --prediction-file outputs/evals/<run-id>/predictions.txt \
505
+ --timeout-seconds 3600
506
+ ```
507
+
508
+ 4. Score the predictions.
509
+
510
+ ```bash
511
+ uv run python tools/run_recognizer_eval.py \
512
+ --label-file outputs/eval-packs/<language>-<pack-version>/eval.txt \
513
+ --data-dir outputs/eval-packs/<language>-<pack-version> \
514
+ --prediction-file outputs/evals/<run-id>/predictions.txt \
515
+ --out outputs/evals/<run-id>/score \
516
+ --manifest-file outputs/eval-packs/<language>-<pack-version>/manifests/recognizer-eval.jsonl \
517
+ --summary-file outputs/eval-packs/<language>-<pack-version>/summary.json \
518
+ --script-inventory-file outputs/eval-packs/<language>-<pack-version>/script-inventory.json \
519
+ --train-manifest outputs/leakage/<model-family>-train-dev.jsonl \
520
+ --model-name <model-name> \
521
+ --checkpoint <local-checkpoint-file> \
522
+ --config <local-training-or-inference-config-file> \
523
+ --dictionary <local-dictionary-file> \
524
+ --language <language> \
525
+ --script <script> \
526
+ --domain real_print \
527
+ --gt-status machine_converted \
528
+ --split eval \
529
+ --eval-pack-version <pack-version> \
530
+ --frozen-eval-pack \
531
+ --unseen-for-selection \
532
+ --no-selected-checkpoint-on-test \
533
+ --no-claim-eligible \
534
+ --min-match-rate 0.99
535
+ ```
536
+
537
+ Only add `--require-claim-safe` when every warning is expected to be absent. If
538
+ the GT status is `machine_converted_spot_checked`, also pass
539
+ `--spotcheck-calibration-file`. Recipe runs can declare the same artifact under
540
+ `eval_pack.spotcheck_calibration_file` so the calibration evidence is part of the
541
+ versioned recipe instead of an invisible operator-only flag.
542
+ For a claim-safe recognizer run, `--checkpoint`, `--config`, and `--dictionary`
543
+ must point to readable local files. The scorer records their SHA-256 hashes in
544
+ `eval-run.json`, and `tools/audit_eval_run.py` re-verifies those hashes and checks
545
+ that the visible run fields still match the hashed artifact identities.
546
+ The scorer also records whether all non-whitespace GT codepoints are present in the
547
+ dictionary; missing GT codepoints are a claim-safety warning and are independently
548
+ recomputed during audit.
549
+ The same audit checks hash-valid summary/datasheet metadata against the direct run's
550
+ recorded language/script/domain/GT/split, version, frozen/unseen, default
551
+ claim-eligibility, and task metadata.
552
+ `--script-inventory-file` must also point to a readable local JSON file; it is hashed in
553
+ `eval-run.json`, summarized in `metrics.json`, and audited against the emitted GT rows.
554
+
555
+ After scoring, audit the score directory:
556
+
557
+ ```bash
558
+ uv run python tools/audit_eval_run.py \
559
+ --run-dir outputs/evals/<run-id>/score \
560
+ --out outputs/evals/<run-id>/score/audit.json \
561
+ --require-audit-pass
562
+ ```
563
+
564
+ ### Same-Pack Recognizer Comparisons
565
+
566
+ When a paper claim compares two recognizers, do not compare separate aggregate numbers
567
+ by eye. Score both systems on the same eval pack, audit both score directories, then
568
+ run the paired comparison tool:
569
+
570
+ ```bash
571
+ uv run python tools/compare_recognizer_eval_runs.py \
572
+ --candidate-score-dir outputs/evals/<candidate-run>/score \
573
+ --baseline-score-dir outputs/evals/<baseline-run>/score \
574
+ --candidate-name <candidate-name> \
575
+ --baseline-name <baseline-name> \
576
+ --out outputs/evals/<comparison-run> \
577
+ --min-pairs 30 \
578
+ --require-claim-safe
579
+ ```
580
+
581
+ The tool pairs rows by `sample_id`, checks that eval-pack identity including
582
+ `pack_content_sha256` and row GT identity match, and emits `comparison.json`,
583
+ `comparison.md`, `paired-rows.tsv`, and
584
+ `command.txt`. It reports codepoint and grapheme micro-CER deltas, paired
585
+ wins/losses/ties, deterministic paired-row bootstrap confidence intervals for the
586
+ improvement, and a two-sided exact sign-test p-value. A comparison is claim-safe only
587
+ when both input score directories are claim-safe, the rows are fully paired on the same
588
+ eval pack, the bootstrap lower bound is above zero, the sign test passes the configured
589
+ alpha, the candidate actually improves the baseline, and comparison timestamp/machine/
590
+ command provenance is present and internally consistent.
591
+
592
+ Audit the comparison directory before using it in a claim:
593
+
594
+ ```bash
595
+ uv run python tools/compare_recognizer_eval_runs.py \
596
+ --audit-only \
597
+ --comparison-dir outputs/evals/<comparison-run> \
598
+ --require-claim-safe
599
+ ```
600
+
601
+ For page-level OCR system comparisons, use the page comparison tool instead of the
602
+ recognizer comparison tool:
603
+
604
+ ```bash
605
+ uv run python tools/compare_page_ocr_eval_runs.py \
606
+ --candidate-score-dir outputs/evals/<candidate-page-run> \
607
+ --baseline-score-dir outputs/evals/<baseline-page-run> \
608
+ --candidate-name <candidate-name> \
609
+ --baseline-name <baseline-name> \
610
+ --out outputs/evals/<page-comparison-run> \
611
+ --min-pairs 30 \
612
+ --require-claim-safe
613
+ ```
614
+
615
+ It pairs pages by `sample_id`, checks page eval-pack identity including
616
+ `pack_content_sha256` plus page/reference text identity, and emits `comparison.json`,
617
+ `comparison.md`, `paired-pages.tsv`, and `command.txt` for page text
618
+ codepoint/grapheme CER deltas, paired wins/losses/ties, paired-page bootstrap
619
+ intervals, and a two-sided exact sign-test p-value. Audit it the same way:
620
+
621
+ ```bash
622
+ uv run python tools/compare_page_ocr_eval_runs.py \
623
+ --audit-only \
624
+ --comparison-dir outputs/evals/<page-comparison-run> \
625
+ --require-claim-safe
626
+ ```
627
+
628
+ ## Native-Reader Spot Check
629
+
630
+ Machine-converted labels may be scored, but they are not human-verified evidence.
631
+ Use this path to create auditable calibration evidence.
632
+
633
+ ```bash
634
+ uv run python tools/prepare_native_spotcheck_bundle.py \
635
+ --label-file outputs/eval-packs/<language>-<pack-version>/eval.txt \
636
+ --data-dir outputs/eval-packs/<language>-<pack-version> \
637
+ --manifest-file outputs/eval-packs/<language>-<pack-version>/manifests/recognizer-eval.jsonl \
638
+ --prediction-file outputs/evals/<run-id>/predictions.txt \
639
+ --out outputs/spotchecks/<language>-<pack-version> \
640
+ --sample-size 50 \
641
+ --seed 20260619 \
642
+ --title '<language> native-reader spot check'
643
+ ```
644
+
645
+ After a reviewer exports the completed CSV:
646
+
647
+ ```bash
648
+ uv run python tools/apply_native_spotcheck_reviews.py \
649
+ --review-file outputs/spotchecks/<language>-<pack-version>/reviewed-spotcheck.csv \
650
+ --out outputs/spotchecks/<language>-<pack-version>/calibration \
651
+ --language <language> \
652
+ --script <script> \
653
+ --domain real_print \
654
+ --source-gt-status machine_converted \
655
+ --min-usable-rows 30 \
656
+ --max-machine-grapheme-cer 0.02 \
657
+ --max-unusable-rate 0.05
658
+ ```
659
+
660
+ If the calibration output says `claim_evidence_eligible=false`, do not mark the
661
+ pack `machine_converted_spot_checked`. Completed rows without reviewer identity
662
+ are not eligible calibration evidence; fix the review export rather than
663
+ overriding the gate. The scorer also requires the calibration artifact's language,
664
+ script, domain, `source_gt_status=machine_converted`, and
665
+ `recommended_pack_gt_status=machine_converted_spot_checked` to match the scored run.
666
+
667
+ ## Page-Level OCR Eval
668
+
669
+ Use this path for end-to-end page claims. Recognizer-only scores cannot support a
670
+ page OCR claim because detection, crop lineage, and reading order are absent.
671
+
672
+ 1. Materialize page outputs through a recorded command.
673
+
674
+ ```bash
675
+ uv run python tools/run_eval_output_command.py \
676
+ --mode page \
677
+ --model-config outputs/<page-model-package>/model-card.json \
678
+ --out outputs/page-evals/<run-id>/materialize \
679
+ --system-name <system-name> \
680
+ --model-name <model-name> \
681
+ --checkpoint <local-checkpoint-file> \
682
+ --config <local-training-or-inference-config-file> \
683
+ --dictionary <local-dictionary-file> \
684
+ --eval-manifest data/<page-eval-pack>/manifests/eval.jsonl \
685
+ --documents-dir outputs/page-evals/<run-id>/documents \
686
+ --timeout-seconds 3600
687
+ ```
688
+
689
+ For an external runner, pass `--command` and write `{document_json}` for each
690
+ input page. Page commands may use only `{input_path}`, `{sample_id}`, `{output_dir}`,
691
+ `{document_json}`, `{documents_dir}`, and `{out_dir}`.
692
+
693
+ 2. Score saved `document.json` outputs.
694
+
695
+ ```bash
696
+ uv run python tools/run_page_ocr_eval.py \
697
+ --eval-manifest data/<page-eval-pack>/manifests/eval.jsonl \
698
+ --documents-dir outputs/page-evals/<run-id>/documents \
699
+ --ocr-output-kind live_detector \
700
+ --out outputs/page-evals/<run-id>/score \
701
+ --summary-file data/<page-eval-pack>/summary.json \
702
+ --script-inventory-file data/<page-eval-pack>/script-inventory.json \
703
+ --model-name <model-name> \
704
+ --checkpoint <local-checkpoint-file> \
705
+ --config <local-training-or-inference-config-file> \
706
+ --dictionary <local-dictionary-file> \
707
+ --eval-pack-version <pack-version> \
708
+ --frozen-eval-pack \
709
+ --unseen-for-selection \
710
+ --no-selected-checkpoint-on-test \
711
+ --train-manifest outputs/leakage/<model-family>-train-dev.jsonl
712
+ ```
713
+
714
+ For direct live-detector page scoring, claim-safe mode requires a passing output audit
715
+ for every scored page output directory. By default the scorer looks for
716
+ `limbu-output-audit.json` with `passed=true` beside each `document.json`; use
717
+ `--require-output-audit`, `--output-audit-filename`, and
718
+ `--output-audit-passed-field` when scoring a different page-output audit format.
719
+ The page auditor replays those per-page audit artifacts from `page-metrics.csv` and
720
+ fails if a recorded `page_output_audit` summary no longer matches the current files,
721
+ so deleting or editing an output audit after scoring invalidates the score directory.
722
+
723
+ A paper-safe page claim requires reference text plus page/line layout references
724
+ that let the scorer report detection/alignment/reading-order behavior. If line
725
+ bboxes are absent, the result is not a full page-level OCR claim. Each page
726
+ manifest row must also carry explicit `language`, `script`, `domain`, and
727
+ `gt_status` metadata plus source-document, page, bbox, and claim-eligibility
728
+ provenance; CLI defaults are recorded as fallbacks, not as row-level claim evidence.
729
+ The page summary must carry a `pack_content_sha256` over the canonical page row,
730
+ input-image, reference-artifact, reference-text, and provenance identity; claim-safe
731
+ scoring rejects summaries whose digest is missing or stale.
732
+ Page manifest `sample_id` and `image_path` values must be unique,
733
+ and any declared row `sha256` must match the actual input image bytes; duplicate IDs,
734
+ duplicate input images, or stale image hashes fail before scoring. If page GT status
735
+ is `machine_converted_spot_checked`, pass
736
+ `--spotcheck-calibration-file`; without an eligible calibration artifact the
737
+ page result is not claim-safe.
738
+ For a claim-safe page run, `--checkpoint`, `--config`, and `--dictionary` must point
739
+ to readable local files whose SHA-256 hashes can be recorded and audited.
740
+ `--script-inventory-file` must point to a readable local JSON file; the scorer records
741
+ which required codepoints/graphemes were covered by reference text and the auditor
742
+ recomputes that summary from the eval manifest and references.
743
+ `--ocr-output-kind` must identify what kind of document outputs were scored. Use
744
+ `live_detector` for normal end-to-end page OCR outputs, `oracle_line_boxes` for
745
+ diagnostic runs where reference/PDF line boxes are fed to recognizers, and
746
+ `sidecar_fixture` only for tests or synthetic fixtures. For Limbu, the detector
747
+ diagnostic requires scoring the same frozen page pack once as `live_detector` and once
748
+ as `oracle_line_boxes`, then comparing the two score directories with
749
+ `tools/compare_page_ocr_eval_runs.py`.
750
+
751
+ Audit the page score directory:
752
+
753
+ ```bash
754
+ uv run python tools/audit_eval_run.py \
755
+ --run-dir outputs/page-evals/<run-id>/score \
756
+ --out outputs/page-evals/<run-id>/score/audit.json \
757
+ --require-audit-pass
758
+ ```
759
+
760
+ ## Claim-Safe Gates
761
+
762
+ A recognizer or page result is not paper-safe unless all of these are true:
763
+
764
+ - Eval pack is frozen, versioned, and declared unseen for model/checkpoint
765
+ selection.
766
+ - The score records a manifest and summary/datasheet for the eval pack.
767
+ - The score records a script inventory file, its hash, observed codepoint/grapheme
768
+ coverage, and no missing required codepoints or graphemes.
769
+ - Manifest row IDs and input image paths are unique, recognizer manifest image
770
+ paths/basenames cannot collide with basename-based prediction matching, and declared
771
+ source-image hashes match the current files; stale recognizer crop hashes, duplicate
772
+ page manifest IDs or image paths, stale page manifest image hashes, and duplicate
773
+ completed score artifact IDs fail audit.
774
+ - Every scored row has source-document, page, bbox, language, script, domain,
775
+ split, GT status, and claim eligibility metadata, and every scored row is claim
776
+ eligible.
777
+ - GT status is `human_verified` or backed by eligible native-reader calibration
778
+ as `machine_converted_spot_checked`.
779
+ - Rows declared `human_verified` carry human review evidence such as `reviewed_by`,
780
+ `review_artifact`, double-review, agreement, or correction-rate metadata; the trust
781
+ label alone is not claim evidence.
782
+ - The domain is real target material, not synthetic or transliterated evidence.
783
+ - Row/reference provenance does not contradict the declared domain or GT status:
784
+ synthetic/transliterated provenance cannot be claimed as real material, and
785
+ machine-converted provenance cannot be claimed as `human_verified`.
786
+ - Train/dev leakage manifests were passed and checked.
787
+ - Model artifacts are hash-verifiable: recognizer and page runs require checkpoint,
788
+ config, and dictionary files, and
789
+ the run fields must match the hashed artifact identity entries.
790
+ - Model identity is recorded: recognizer and page runs require a non-empty model name,
791
+ and page runs require a non-empty system name.
792
+ - No hard leakage exists by sample ID, image hash, image path, or combined
793
+ sample hash.
794
+ - The prediction match rate is at least the configured threshold.
795
+ - No scored recognizer GT row or page text reference is zero-length.
796
+ - Headline reporting is per language/script/domain/GT status, not a pooled
797
+ average hiding weak languages, and every emitted language/script/domain/GT/split
798
+ slice has at least the claim-facing minimum scoring units.
799
+ - Same-pack recognizer comparisons use `tools/compare_recognizer_eval_runs.py` and pass
800
+ its paired bootstrap/sign-test gates; separate aggregate score directories are not
801
+ enough evidence for a comparative claim.
802
+ - The score directory contains `claim-safety.md` with `What Was Scored`, `Ground
803
+ Truth And Pack`, `Metrics`, and `Warnings` sections.