Add 1000 files (ocr-tech/tests/test_audit_limbu_book_missed_line_review_coverage.py ...)
d82db76 verified |
Download ocr-tech-mps-integration/docs/CODEX_HANDOFF.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 319 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/docs/CODEX_HANDOFF.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech-mps-integration/docs/CODEX_HANDOFF.md
-
curl -L -o CODEX_HANDOFF.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/docs/CODEX_HANDOFF.md
319 kB
# Codex Handoff: Limbu Book OCR
Updated: 2026-06-29
Repo: `Ampixa/ocr-tech` · Branch: `main` · Visibility: private
This is for Codex acting as critical reviewer + next implementer. The prior agent
(Claude) consults you at milestones specifically to catch blind spots it
rationalizes past. **Be skeptical of every number below and every claim of "fixed."**
The living source of truth is `docs/RECOGNIZER_GOAL_PROGRESS.md`; this handoff is the
fast path into the current state.
2026-06-29 Limbu end-to-end OCR update: the immediate goal is a Limbu-first OCR
system for real page images/PDFs, not a generic multilingual plan and not a YOLO-first
page demo. The Space payload at `spaces/limbu-ocr/` now has `Line OCR`,
`Paddle Page OCR`, `Paddle Book OCR`, image-line fallback tabs, `YOLO Probe`, and
runtime tabs. The production page/book path uses PaddleOCR full-page text detection,
orders detected text lines, routes each crop through the bundled Sirijonga and
Devanagari-written Limbu Paddle recognizers, and returns Unicode text plus
crop/overlay evidence. `Paddle Book OCR` accepts ordered page image files or PDFs and
returns a ZIP with `book.txt`, `book.json`, annotated pages, and line crops. YOLO
remains only an optional probe. Verification on this turn:
`python3 -m py_compile spaces/limbu-ocr/app.py`, a Python 3.11 import/artifact +
synthetic line-segmentation smoke, PaddleOCR API introspection against
`paddleocr==3.7.0`, one real Gorkhapatra page Paddle smoke with 306 detections, a
capped `recognize_page_paddle(max_lines=5)` run, and focused repo tests
`tests/test_image_lines.py tests/test_parse_limbu_book_pages.py tests/test_run_limbu_book_ocr.py`
with result `49 passed, 1 skipped`. During testing, the book runner/capture-prep
interface was fixed so newer capture-audit fields are forwarded and older
programmatic callers do not crash on missing optional namespace fields. Next step is
testing the uploaded Space revision on real Limbu pages, not expanding eval harness
abstractions. Uploaded Space commit:
`https://huggingface.co/spaces/ampixa/limbu-ocr/commit/bda8e8b5ebaf476e7a52cd56996f525fd7687d56`.
The Space is private; authenticated direct-domain access returned HTTP 200 while
unauthenticated access returned HTTP 404.
2026-06-25 OCR Space update: the Limbu-only OCR Hugging Face Space payload is staged
at `spaces/limbu-ocr/` and documented in `docs/LIMBU_OCR_SPACE_HANDOFF.md`. It bundles
the current Sirijonga line recognizer, Devanagari-written Limbu recognizer, and YOLO
line detector behind a self-contained Gradio app. The app passed `py_compile` and a
Python 3.11 lightweight import/artifact health check. It has been uploaded to
`https://huggingface.co/spaces/ampixa/limbu-ocr` at commit
`76f54073ed75d1a4df6ea3d63fb06cd871383b95`, and the runtime URL
`https://ampixa-limbu-ocr.hf.space/` returned HTTP 200. Current HF CPU quota permits
three active CPU Spaces; the intended active set is `ampixa/limbu-ocr`,
`ampixa/limbu-translator`, and `ampixa/limbu-tts`. `ampixa/real-nepali-tts`,
`ampixa/gurung-tts`, and `voidash/bantawa-tts-demo` are paused. Do not resume broad
language inventory from here; the immediate path is OCR deployment/testing and then
real-page validation.
Recognizer training correction: also read
`docs/LIMBU_RECOGNIZER_TRAINING_HANDOFF.md` before discussing Limbu training data,
CER, or the active G6 run. The `1,866` Gorkhapatra train rows are not the whole
Limbu corpus; the active combined pack has `48,507` train rows plus `255` held-out
Gorkhapatra eval rows and is training with checkpoint selection by `norm_edit_dis`.
2026-06-24 update: the next recognizer experiment is now a targeted-fabrication
ablation, not more generic synthetic volume. The new generator is
`tools/build_limbu_targeted_fabrication_pack.py`; it renders verified Limbu text,
oversamples current failure lines/confusion characters, matches real Gorkhapatra line
widths, applies phone-photo degradation, rejects non-rendering fonts, rejects blank
crops, and writes training-only manifests. The first component is
`outputs/limbu-targeted-fabrication-v1-2026-06-24/` with `9,996` fabricated line crops
and `4` render rejects from the ink guard. Important correction: an earlier pass under
this same output name used extracted Gorkhapatra PDF subset fonts and generated blank
images; the corrected pack uses upstream renderable Namdhinggo fonts under
`data/fonts/namdhinggo/Namdhinggo-3.100/`. The combined train/eval pack that includes
it is `outputs/limbu-combined-synth-real-gorkhapatra-targeted-v1-2026-06-24/` with
`58,467` train rows and `255` held-out Gorkhapatra eval rows after `49` exact eval-text
overlap drops; both manifests passed `ocrtech audit-manifest`. Treat this as an
ablation input only. Do not claim OCR quality from synthetic rows.
2026-06-24 Unicode/diacritic coverage update: read
`docs/LIMBU_UNICODE_DIACRITIC_STATS.md` before generating more Sirijonga text. The
Limbu block has `68` assigned codepoints, but the targeted combined pack above covered
only `61/68`; missing were `U+1909`, `U+190A`, `U+191A`, `U+191D`, `U+191E`,
`U+1932`, and `U+1940`. A HarfBuzz/Pillow shaping probe over `21,891` Limbu component
cases rendered cleanly with Namdhinggo, but that only proves the font is permissive;
it does not validate linguistic/order sanity. The new corpus-stat tool is
`tools/analyze_limbu_unicode_corpus_stats.py`; the new coverage text generator is
`tools/build_limbu_unicode_coverage_text.py`. It was corrected to emit
`base+subjoined+vowel`, `base+vowel+KEMPHRENG+final`, and to include `U+1932` in
finals instead of orphaning it. The coverage text pack
`data/synthesis-text/limbu-sirijonga-unicode-coverage-v1-2026-06-24/` has `1,936`
source rows and zero generated order warnings; the rendered training-only component
`outputs/limbu-unicode-coverage-fabrication-v1-2026-06-24/` has `3,872` rendered rows,
zero render rejects, and nonblank pixel stats. The combined pack with both targeted and
Unicode coverage is
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-2026-06-24/` with
`62,339` train rows and `255` eval rows; it covers `68/68` assigned Limbu codepoints
and has no codepoint below `1,000` occurrences after dedupe. Train/eval manifest audits
passed. Treat `unicode-coverage-fabrication-v1` as a separate ablation component, not
claim evidence.
2026-06-24 active training update: the targeted+Unicode combined pack is running on
the AWS G6 L4 instance `i-0fa19dff7b233a353` at `3.219.242.144`, workspace
`/opt/ocrtech`, PID `11133`. Run directory:
`runs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-24/`.
It uses existing pack + targeted fabrication + Unicode coverage, base checkpoint
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams`,
dictionary `data/dictionaries/limbu-sirijonga-v3.txt`, batch `64`, LR `0.00001`,
`15` epochs, and `main_indicator=norm_edit_dis`. First eval at step `200` saved a
warmup best with `norm_edit_dis=0.04608837027816626` and `acc=0.0`; do not interpret
that as CER. The dictionary v3 update is append-only over v2 because the new pack
requires seven additional Limbu chars: `U+1909`, `U+190A`, `U+191A`, `U+191D`,
`U+191E`, `U+1932`, and `U+1940`. After completion, score the exported best checkpoint
on the frozen Gorkhapatra recognizer eval before claiming any improvement.
2026-06-24 completed training result: reject this targeted+Unicode checkpoint. The run
completed on the G6 L4 instance, the completion audit passed with `issue_count=0` and
one regex warning, and Paddle selected epoch `14` with
`norm_edit_dis=0.8278816454641351`, `acc=0.1450980335255673`. The exported/package-best
score is
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-best-eval-2026-06-24/score/`:
codepoint CER `0.024696356275303644` (`244/9880`), grapheme CER
`0.053276955602537` (`252/4730`), exact-line accuracy `0.4666666666666667` on `255`
matched lines. It is worse than the real-heavy champion and worse than the older
combined baseline; see paired comparisons under
`outputs/limbu-targeted-unicode-v1-best-vs-real-heavy-champion-comparison-2026-06-24/comparison/`
and
`outputs/limbu-targeted-unicode-v1-best-vs-combined-baseline-comparison-2026-06-24/comparison/`.
Do not promote this package into book OCR. Keep the real-heavy model as the current
Sirijonga recognizer champion and treat this run as evidence that the current targeted
synthetic/Unicode supplement is hurting real-print performance.
2026-06-24 controlled ablation result: the regression has now been isolated enough to
make a decision. The grid is
`runs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`; scores and paired
comparisons are under `outputs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`.
All cells were exported, packaged, predicted, scored against the champion's exact
frozen Gorkhapatra eval files, and paired against the real-heavy champion. Results:
control v2 replay LR `1e-6` scored codepoint CER `0.011234817813765181`, grapheme CER
`0.02452431289640592`, exact `0.7333333333333333`; this is one edit better than the
champion and not statistically meaningful. v2 plus `256` capped targeted rows scored
worse (`0.01194331983805668` codepoint, `0.026004228329809725` grapheme). v2 LR
`1e-5` tied codepoint CER but worsened grapheme CER (`0.026004228329809725`) despite
exact `0.7372549019607844`. Both v3 dictionary cells failed catastrophically around
`0.98` codepoint CER and `0.0` exact-line accuracy because the dictionary expansion
resizes the recognition head. Do not promote any ablation checkpoint. The next serious
recognizer move is either stay on v2 and improve real/book-domain line data, or create
an explicit full-Unicode training/rehead plan; do not mix dictionary expansion into a
synthetic-data ablation again.
## Goal
Build a usable Limbu book OCR system: scanned or photographed real Limbu book pages in,
editable Unicode book text out, with page and line structure preserved. It must support
both Sirijonga Limbu and Devanagari-written Limbu. The current bottleneck is page
layout, reading order, missed-line recovery, source-page triage, and review burden with
the current recognizers held fixed. Keep evaluation minimal and tied to acceptance
checks on frozen real-book pages: page CER/WER, line precision/recall/F1, reading-order
accuracy, blank/extra/missed rates, and review-packet coverage.
Do not expand the eval harness as its own project. Use metrics only when they prove
whether the book OCR loop got better. The next useful work should make one of these
true: cleaner input page selection, stronger capture prep, better line preservation,
fewer unresolved rows per real Limbu page, safer correction application, or a measured
improvement over the current diagnostic baselines.
## Critical Gorkhapatra Source Rule
For all Naya Nepal/Gorkhapatra language data, the Naya Nepal publication page
(`https://gorkhapatraonline.com/publications/naya-nepal`) is discovery only. Use it
to identify the language/date/post, but do not use the blog/card image as OCR,
training, or eval evidence. The authoritative source is the full Gorkhapatra epaper
PDF from the date-range endpoint, for example:
`https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-16&pdfdaterange2=2026-06-23&slug=gorkhapatra`.
The correct text-image pair is the rendered PDF page plus the text layer extracted
from that same PDF page, with PDF URL/hash/page number, rendered image hash, text
extraction command/version, and legacy-conversion map hash recorded. For every
language, classify the PDF text layer as `unicode_native`, `legacy_font`,
`devanagari_transcription`, `latin_notation`, or `no_text_layer`; build/reuse an
explicit converter only when the font/text layer requires it. Manual transcription is
the fallback for `no_text_layer` or undecoded fonts, not the default path. See
`docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md` before changing this path.
The browser annotation/review UI already exists. Use the PDF span dashboard:
`data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html`.
It shows the full rendered PDF page, clickable bounded spans, and the right-side
text/rendered-crop panel. Do not use
`data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-review.html`;
that local file was identified as the wrong/broken UI and deleted. The newer book
unresolved review station is `tools/prepare_limbu_book_unresolved_review_packet.py`.
## Current Limbu Pipeline Pivot
The active local work has narrowed to a Limbu-first operational OCR pipeline that must
handle both Devanagari-written Limbu and Sirijonga-script Limbu. Do not frame this as
"video to page"; moving-camera material must be reduced to still page inputs before it
enters the OCR path. The target is a script-aware OCR path with capture cleanup,
Devanagari/Sirijonga routing,
recognizer-only handling for existing line crops, review queues, line crops, output
manifests, and audits. The current docs/config are
`docs/LIMBU_FIRST_OCR_PIPELINE.md` and `configs/limbu-first-ocr-pipeline-v1.json`.
Count sanity check: do not repeat the mistake of treating `39` as the size of the
Limbu eval/data surface. In the current config, `39` is the unresolved review-row count
from one older operational Gorkhapatra page-eval run, paired with 85 dropped image-line
rows. It is review debt for that run, not the corpus size and not the canonical eval
pack size. The materialized recognizer eval pack
`outputs/eval-packs/limbu-gorkhapatra-real-v1/` currently has 1,829 label rows and
1,829 manifest rows; its audit passes structurally but `claim_ready=false` because it
is machine-converted, not declared unseen for selection, has 0 claim-eligible rows, and
lacks source-document/page/bbox provenance. Separately, the CDC Maths PDF text-layer
silver pack at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/`
has 81 rendered-line crops, split into 69 train and 12 eval rows after grouping by
`sample_signature`. Treat those 81 rows as silver training/bootstrap data, not
claim-safe ground truth.
Current 2026-06-23 correction: the user pushed back that eval-harness work was being
overdone. Treat that as authoritative direction. The freshest real-book operational
smoke is
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/`,
run with `/tmp/ocrtech-paddle311/bin/python`. It processed 2 unique source pages, kept
692 lines, wrote 599 review rows, and passed the book artifact audit. The priority
queue/dashboard are just operator views over the same review work: 304 high-priority
empty/no-script transcription-or-drop rows, 15 mixed-script verification rows, and 280
low-confidence spot checks. The canonical queue remains `book-line-review-queue.tsv` in
page/reading order; `book-line-priority-review-queue.tsv` is only a sorted working copy.
This evidence says the next product work is reducing review burden and recovering real
page lines, not building more generalized scoring infrastructure.
Current 2026-06-23 PDF book intake bridge: `tools/build_limbu_book_page_manifest.py`
now accepts either `--image-dir` or `--pdf`. The PDF path renders selected book pages to
PNG, writes the normal OCR page manifest, and records source PDF path/hash, PDF page
number/index, rendered image hashes, and DPI per row. This is an operational intake
feature, not an eval-harness feature: real PDFs can now enter the same
capture/OCR/review loop as photographed pages. Real smoke on the local CDC Maths Limbu
PDF:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-pages.jsonl`
renders PDF pages 8..10 at 220 DPI into three 1870x2420 PNG pages with no source
attention flags. Passing the manifest through capture prep produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/capture-prep/prepared-page-manifest.jsonl`
with 3 pages and 0 failed capture audits. That prepared manifest has already been fed
through the book wrapper below; keep this paragraph as intake provenance, not as the
next action.
That front door has now been run through the full book wrapper using the current v4
router, without changing recognizers:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/`.
It processes 3 pages end-to-end, emits editable `book.md`, review queues, page-layout
overlays, page triage, layout-gap review, and a passing book audit. Current counts:
93 preserved lines, 21 book-line review rows, 3 page-layout rows, 3 page-triage rows,
9 layout-gap rows, `operator_attention_page_count=0`, and `book_audit_passed=True`.
The review queue now separates page furniture more honestly: 9 high-priority
`inspect_non_text_or_drop_empty_detection` rows, 3 medium
`inspect_non_text_or_drop_low_confidence_detection` rows, 1 high
`transcribe_or_drop_empty_detection` row, and 8 low-confidence spotchecks. The latest
line-review contact sheet is:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png`.
Thirteen obvious non-text rows have since been dropped in the v2 cleaned projection
documented later in this handoff. The remaining real work on this artifact is the
8-row text-shaped low-confidence review packet, followed by audited application and
reviewed-only training export. Do not go back to the 21-row queue unless you are
rebuilding the cleaned projection from scratch.
Current 2026-06-23 triage cleanup: `tools/apply_limbu_book_page_triage.py` now filters
all page-scoped review queues after explicit page rejection: aggregate book review,
book-line review, priority book-line review, and layout-gap review. It also recomputes
counts/provenance, clears stale dashboard paths, and regenerates filtered canonical,
priority, and focused book-line review dashboards from the kept-page queues.
The real p2-rejected check at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/manual-triage-check/triaged-p1-only/`
keeps only `gorkhapatra-unique-current-p0001`, reduces the review surface from 599 rows
to 230 rows, reports priority counts 6 high / 4 medium / 220 low, and passes
`tools/audit_limbu_book_output.py` with zero issues. This is source/page triage and
operator-noise reduction, not OCR-quality improvement.
Current 2026-06-23 review guidance cleanup: `tools/parse_limbu_book_pages.py` now writes
`book_line_bbox` and `book_line_aspect_ratio` into `book-line-review-queue.tsv` and
classifies large square/near-square empty detections as
`inspect_non_text_or_drop_empty_detection`. This keeps portrait/page-furniture boxes out
of the text-transcription bucket without dropping anything automatically. Real rerun:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/`
passes the book audit; the p2-rejected projection
`.../manual-triage-check/triaged-p1-only/` passes `audit_limbu_book_output` and leaves
the kept page with 5 empty visible-text transcription/drop rows, 1 non-text inspect/drop
row, 4 mixed-script verification rows, and 220 low-confidence spot checks. This is
review-packet quality work, not a CER/WER improvement.
Current 2026-06-23 layout review packet: `tools/run_limbu_book_ocr.py` now writes
`page-layout-review/page-layout-review-manifest.tsv`,
`page-layout-review/page-layout-review-dashboard.html`, and one full-page overlay PNG
per OCR page. The overlays draw line boxes and reading-order indices over the prepared
page image, color-coded green accepted text / amber review text / red empty OCR /
purple structural text. Real run:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-layout-overlay/`
passes the book audit and has 2 layout rows / 2 overlays. Page 1 is 323 lines, 6 empty
OCR boxes, 230 review rows, reading order `wide_column_right_to_left`. The first overlay
was visually checked and is nonblank; use it to inspect missed boxes and reading-order
problems before crop-level transcription. This is layout-debug/review evidence, not OCR
quality improvement.
Current 2026-06-23 training-loop bridge: `tools/export_limbu_book_training_pack.py`
exports a reviewed `book-run.json` projection into line crops, PaddleOCR-style labels,
a JSONL recognizer manifest, dictionary, and script inventory. By default it exports
only lines that carry explicit book-line review, split-review, or layout-gap insertion
metadata; unreviewed OCR rows are excluded so pending recognizer output does not become
training truth. The exporter now revalidates that reviewed metadata before writing
labels: status must be accepted/corrected, reviewer must be present and non-machine,
and the evidence-specific reviewed text must be present and match the exported line.
Candidate insertions require explicit `corrected_text`, so replayed machine candidate
text cannot become recognizer GT. The runner README now points from review application to this exporter:
`uv run python tools/export_limbu_book_training_pack.py <run>-reviewed/final-book --out
<run>-training-pack-reviewed`. A real bootstrap check on the latest p1-only triaged
projection used `--include-unreviewed` at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/training-pack-bootstrap-p1-unreviewed/`;
it produced 317 crops/labels, skipped 6 empty rows, and counted 221 Sirijonga, 27
Devanagari, and 69 mixed rows. That artifact is explicitly marked
`gt_status_counts={"unreviewed_ocr_projection": 317}` and is not ground truth. The
reviewed-export command correctly refused an older final-book projection with no
accepted review rows, which means the next real blocker is completing accepted line/gap
review, not adding more harness.
Current 2026-06-23 training pivot after scope correction: stop building eval harness
surface and use real Limbu text sources to train. `tools/export_limbu_pdf_text_training_pack.py`
now exports converted PDF text-layer lines as rendered full-line crops plus PaddleOCR
labels, without using current OCR predictions as labels. This is deliberately marked
silver, not human-reviewed gold: `gt_status=pdf_text_layer_silver`, `claim_eligible=false`.
Real export from CDC Maths PDF pages 11..15:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/`.
It has 81 rows from 82 PDF text-layer rows, skipping 1 no-Limbu row; script buckets are
64 Sirijonga and 17 mixed Sirijonga/Devanagari rows. A leakage-clean split was created
under `.../splits/` with 69 train and 12 dev rows, grouped by sample signature after a
sample-id split correctly failed on duplicate crop/text leakage. The generic PP-OCRv5
silver run at
`runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/` is a negative result: using
the 55-character pack dictionary reinitialized the recognition head, finished at dev
`norm_edit_dis=0.0903799184964279`, packaged cleanly, but produced poor runtime
predictions. Do not route that package into book OCR except as evidence that same-head
warm-start matters.
The better candidate is the v4 Sirijonga warm-start run at
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/`, generated with
`data/dictionaries/limbu-sirijonga-v2.txt`, base checkpoint
`outputs/limbu-first-router-v1/source-archives/limbu/checkpoints/best_accuracy.pdparams`,
20 CPU epochs, batch size 8, and LR `0.00002`. It loaded the base checkpoint without
head-shape mismatch warnings. The best Paddle dev checkpoint is epoch 14:
`acc=0.49999958333368055`, `norm_edit_dis=0.7922192746296381`; the final epoch drifted
lower, so package `best_accuracy`, not `latest`. Export/package/audit are complete under
`export-best/` and `package-best/`, with the model card at
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/package-best/model-card.json`.
Runtime smoke on held-out crop `cdc-maths-p0011-pdf-line-0001.png` produced the exact
silver label at confidence `0.9353305101394653`. The narrow 12-row silver runtime score
is at `.../eval-best/score/`: codepoint CER `0.012618296529968454`, grapheme CER
`0.023529411764705882`, exact-line accuracy `0.75`, 12/12 matched predictions, no
dictionary coverage gaps. This is **not claim-safe**: it is PDF-text silver, only 12
rows, not unseen for checkpoint selection, and has known train/dev text overlap. Use it
only as admission evidence that the candidate is worth trying on real book pages.
That frozen-page router comparison has now been run and rejects global promotion. A
router variant at
`outputs/limbu-router-v5-ppocr-devanagari-v4warm-sirijonga-silver-2026-06-23/model-card.json`
keeps the Devanagari primary and routing policy fixed while replacing only the
Sirijonga secondary with the warm-start package; its model audit passes. On the CDC
Maths pages 11..15, the v5 run
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v5-v4warm-silver-cdc-compare/`
passes the book audit and preserves the same 189 lines, but review rows increase from
58 to 68. PDF-text coverage is unchanged: 68 covered lines, 13 missing/partial
candidates, 1 skipped no-Limbu row. A quick geometry-aligned PDF-text check showed only
a tiny aggregate text gain (1730 -> 1708 codepoint edits over 1841 reference
codepoints) with several long-line regressions. On the real-print Gorkhapatra two-page
run
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-router-v5-v4warm-silver-compare/`,
the v5 router again passes the book audit and preserves the same 692 lines, but review
rows increase from 599 to 665; the usable Limbu page alone rises from 230 to 296 review
rows. The comparison artifact is
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/router-v5-v4warm-admission-comparison.json`.
Decision: keep router v4 as the operational default. Use the v5 warm-start recognizer
as training evidence or a selective probe only; do not replace the router with it.
Current 2026-06-23 layout/missed-line probe: after the user pushed back on overdoing
eval harness work, the next check was deliberately narrow and operational. Added
`tools/sweep_limbu_pdf_text_line_detector_params.py` with focused coverage in
`tests/test_sweep_limbu_pdf_text_line_detector_params.py`; it is a detector-only
geometry diagnostic, not an OCR-quality harness. On CDC Maths pages 11..15, the sweep
at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/`
shows current defaults at 68/81 covered Limbu source lines, 13 missing/partial
candidates, and 189 boxes. The best recall config reaches 76/81 and 5 candidates but
emits 335 boxes, so do not promote it blindly. A balanced config
(`horizontal_kernel=23`, `vertical_kernel=3`, `min_width=15`, `min_height=10`,
`min_area=50`, `max_height=100`) was run through the real book OCR path with router v4
at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/`;
it passes audit, improves PDF-text coverage to 73/81 and 8 candidates, but increases
OCR lines from 189 to 214 and review rows from 58 to 60, with 2 operator-attention
pages. Decision: keep current defaults, but use this balanced detector as the next
real-page probe where missed lines are visibly more costly than a small review-row
increase.
The runner now exposes detector probes directly as operational switches:
`--detector-profile current|balanced-pdf-text|current-plus-balanced-rescue`. `current`
remains the default. `balanced-pdf-text` materializes
`<run>/profiled-page-manifest/book-pages.jsonl`, merges the balanced detector settings
into page `parse_kwargs`, records the profile and manifest hashes in
`limbu-book-ocr-run.json`, and refuses conflicting row-level detector settings instead
of silently overriding them. `current-plus-balanced-rescue` keeps the current detector
as primary and adds the balanced settings only as an additive rescue pass. Use these
for controlled real page probes, e.g.:
```bash
/tmp/ocrtech-paddle311/bin/python tools/run_limbu_book_ocr.py \
--image-dir /path/to/book-page-photos \
--out outputs/limbu-live-book-smoke-YYYY-MM-DD/my-book-balanced-probe \
--skip-duplicate-images \
--detector-profile balanced-pdf-text
```
The switch has also been checked on the real-print Gorkhapatra two-unique-page smoke
with stable page IDs by reusing the current baseline raw manifest. Artifact:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-balanced-detector-profile-probe-stable-pageids/`.
It passes audit and records source manifest SHA
`b8bb7138a12c42c1a919339bbfad5165f5162ab768f4c8c00874f777b364384c` plus profiled
manifest SHA `b8f1be0a77ba7e86bf34cd15c3bd4592cbdbca252ca4c00ae20c30bf49e42010`.
Compared with `gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard`, totals
move 692 -> 690 lines and 599 -> 596 review rows, but layout-gap warnings rise 1 -> 5.
On the usable Limbu page, lines move 323 -> 325, review rows 230 -> 231, Sirijonga
classifications 286 -> 290, and other rows 6 -> 5. Treat this as safe probe evidence,
not default-promotion evidence.
For visual inspection of that profile delta, use the new packet:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-balanced-detector-line-geometry-delta-review/`.
It was generated by `tools/compare_limbu_book_line_geometry.py` and compares current
default vs stable `balanced-pdf-text` by line-box IoU. It reports 679 matched lines,
11 candidate-only lines, and 13 baseline-only lines; on the usable Limbu page, 317
matched, 8 candidate-only, and 6 baseline-only. The packet includes
`line-geometry-delta-review.tsv`, `line-geometry-delta-review.html`, copied crops, and
full-page context images. It now also has
`line-geometry-delta-contact-sheet.png` with SHA
`b164f9d3e01a64c74081272a6923921442b4a4023fa3142360fa2268e51ffeb6`. A spot check of
`crops/added_in_candidate-gorkhapatra-unique-current-p0001-0094-img-line-0094.png`
shows real visible page text, but this remains fragment-level evidence. Review the
24-row packet before changing defaults. Current visual read: the profile adds small
real Sirijonga fragments on the usable page, but it also removes larger baseline
text/header detections, so keep `balanced-pdf-text` opt-in only.
There is now a better opt-in profile: `--detector-profile current-plus-balanced-rescue`.
It keeps current detector defaults as primary and uses the balanced settings only as a
rescue pass, so it can add missed fragments without replacing the larger baseline
boxes. Real stable-page-ID Gorkhapatra run:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-current-plus-balanced-rescue-profile-probe-stable-pageids/`.
It passes audit with 703 lines / 606 review rows / 1 layout-gap row. On the usable
Limbu page: 323 -> 331 lines, 230 -> 234 review rows, 286 -> 292 Sirijonga rows, and
the `possible_missed_line_gap` flag disappears. Source-pass counts are primary 323 +
rescue-1 8 on the usable page, primary 369 + rescue-1 3 on the rejected page. Geometry
delta packet:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-current-plus-balanced-rescue-line-geometry-delta-review/`,
with 692 matched default lines, 11 candidate-only rescue lines, and 0 removed lines.
Contact sheet SHA:
`be12597dc1f32ce446b0f122af530cd04b512dd9861db1b33ab9643f6d381544`. This is the best
line-preservation probe so far, but keep it opt-in until checked on the next actual
book/photo batch.
The same additive rescue profile was checked on CDC Maths pages 11..15 at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-current-plus-balanced-rescue-cdc-compare/`.
It passes audit with 219 lines / 65 review rows / 14 layout-gap rows. Compared with
current default, it preserves all 189 baseline lines and adds 30 rescue boxes; compared
with the replacement `balanced-pdf-text` run, it gets the same PDF-text coverage
improvement, 68/81 -> 73/81 covered source lines, without deleting any baseline boxes.
Geometry delta packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/current-plus-balanced-rescue-line-geometry-delta-review/`,
with 189 matched default lines, 30 candidate-only rescue lines, and 0 removed lines.
Contact sheet SHA:
`838d1868b445f9ea8a11cf6e34db95fadcb9cb092098ff2321f2bec5680af22b`. This reinforces
the current rule: additive rescue is the next opt-in line-preservation probe; it is not
a reason to build more generic eval harness or change defaults before the next real
book/photo batch.
The runner README now includes a `Detector profile:` section for every run. It records
the active profile, whether it is default/additive/replacement, its usage note, and the
profiled manifest path. This is operator workflow guidance so tomorrow's runs do not
mistake additive rescue evidence for default-promotion evidence.
For default `current` runs, `limbu-book-ocr-run.json` and the README now also include a
ready-to-run `detector_profile_probe_command` that reruns the exact source page manifest
with `--detector-profile current-plus-balanced-rescue` into a sibling output directory.
They also include `detector_profile_probe_compare_command`, which runs
`tools/compare_limbu_book_line_geometry.py` against the current run and the sibling
rescue run. This removes copy-paste reconstruction from the missed-line probe workflow
while keeping the default profile unchanged.
The generated run README now also points directly to
`docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md` from the "After review" section. That is the
reviewed TSV -> audited final book -> reviewed-only training pack handoff. The immediate
human blocker remains accepted/corrected rows from that workflow, not more runner
plumbing or generic metrics.
Fresh runner outputs with nonzero review rows and no page-triage source-reject/reshoot
blockers now include `browser_review_station_command` in `limbu-book-ocr-run.json` and
a `Browser review station:` section in the README. The command builds a pending
unresolved-review HTML/TSV packet from the run's focused review queue using
`tools/prepare_limbu_book_unresolved_review_packet.py`. The run summary also includes
`browser_review_station_apply_command`, which expects the exported
`book-unresolved-review-packet.reviewed.tsv` to be saved in the browser review station
directory and applies it with `tools/apply_limbu_book_unresolved_review_packet.py`
using `--packet-dir` provenance. If the reviewed rows should become training input, the
run summary also includes `browser_review_station_training_pack_command`, exporting
`browser-review-station-applied/final-book` through
`tools/export_limbu_book_training_pack.py`. Clean runs with 0 review rows do not emit
these browser-station commands, and pages that should be rejected or reshot must be
handled in page triage before line review. This closes the setup gap between "OCR run
completed" and "reviewer can accept/correct lines"; it still creates no labels until
the exported reviewed TSV is audited and applied. The runner test now executes the
generated command's target packet builder on a synthetic reviewable low-confidence Limbu
run and checks the generated apply/training-pack command paths, so this path is
verified beyond string matching.
The browser review station also carries layout-gap evidence now. If a run has
`layout_gap_review_row_count > 0`, the generated command passes
`layout-gap-review-queue.tsv` as `--missing-line-review-queue`. The unresolved packet
builder treats each pending layout-gap row as a `candidate_missing_line`, derives a
page-wide bbox from `from_y`/`to_y`, copies the gap crop, renders highlighted page
context, and preserves the neighboring OCR-line context. This is the narrow
missed-line recovery handoff the book-OCR goal needs: the reviewer sees suspected
omissions in the same browser station as low-confidence text, without promoting any
candidate text until reviewed TSV application/audit.
The generated layout-gap browser packet now has an end-to-end application test that
uses the generated apply-command paths. A synthetic suspicious-gap runner case builds
the browser packet, edits the exported TSV as a reviewer would
(`review_status=accepted`, explicit `corrected_text`, reviewer), saves it at the
`browser_review_station_apply_command` `--review-packet` path, then applies it through
`tools/apply_limbu_book_unresolved_review_packet.py` with `--packet-dir` provenance.
The final book projection contains the inserted Unicode line with
`candidate_line_review_insertion.row_id` equal to the original `gap_row_id`. This proves
the browser station is not a dead-end review surface for missed lines; it feeds the
existing audited final-book path.
Current 2026-06-23 layout-gap ink guidance: `tools/run_limbu_book_ocr.py` now adds
dark-pixel ink stats to `layout-gap-review-queue.tsv`: dark-pixel count, density,
ink bbox, row-band classification, candidate-text band count, horizontal-rule count,
hash-bound candidate text-band crop paths/hashes, full-page gap-context image
paths/hashes, priority, and recommended action. The context image highlights the
suspicious y-band on the full prepared page, so reviewers can reject section rules,
headers, and spacing without interpreting a thin crop in isolation. For text-like gaps,
the dashboard now shows the full gap crop, highlighted page context, and focused
candidate-band crops, so the reviewer does not have to transcribe from a full-width
spacing band. This keeps the missed-line queue focused on gap bands that contain likely
text-shaped ink instead of treating every large geometric space equally. The layout-gap
dashboard and README expose the same priority/action counts. Re-running the packet
writer on the latest p1-only triaged projection at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/layout-gap-review-ink-guidance/`
produced 1 gap row, priority counts `{"low": 1}`, action counts
`{"likely_rule_or_spacing": 1}`, and the row records `5737` dark pixels with density
`0.038029`, `0` candidate text bands, `3` horizontal-rule bands, and empty
candidate-band crop lists (`[]`). This corrects the earlier geometry-only warning: the
p1 gap is likely rule/spacing, not an actual missed body line. Page triage now consumes
this ink-guided gap summary before assigning
operator flags: the same p1 check at
`.../triaged-p1-only/ink-guided-page-triage-check/` keeps
`suspicious_gap_count=1` for auditability but writes
`candidate_text_gap_count=0`, `horizontal_rule_gap_count=1`, removes
`possible_missed_line_gap`, and recommends `review_low_confidence_lines` instead of
layout-gap inspection.
Current 2026-06-23 layout-gap application hardening: accepted layout-gap insertions now
carry candidate text-band crop paths/hashes into inserted-line metadata and insertion
pairs when those focused crops exist. Both `tools/apply_limbu_book_layout_gap_reviews.py`
and `tools/audit_limbu_book_layout_gap_review_application.py` validate those optional
candidate-band crop hashes, in addition to the full gap crop hash. This is not another
eval-harness feature; it protects the review-to-training bridge from detached TSV edits
or stale evidence after a reviewer inserts a missed line. The audit also cross-checks
`layout-gap-insertion-pairs.jsonl` against the inserted document lines and metadata, so
a self-consistent pair-file hash is not enough if the pair text/crop evidence no longer
describes the actual edited book projection.
Current 2026-06-23 review-burden cleanup: `tools/parse_limbu_book_pages.py` now emits
`book-line-focused-review-queue.tsv` and `book-focused-review-dashboard.html` beside the
canonical full queue and priority queue. The focused queue keeps every critical/high/
medium row and only a deterministic, evenly spaced sample of
`spotcheck_low_confidence_text` rows per page (default 25 via
`--focused-low-confidence-limit-per-page`). This does not remove any canonical review
row, and it does not change OCR text; it gives the operator a bounded low-confidence
spotcheck surface while preserving exhaustive flags for later review/application. Page
triage filters the focused queue when pages are rejected, and
`tools/audit_limbu_book_output.py` hash/card-count audits the focused queue/dashboard.
`tools/apply_limbu_book_review_queues.py` records the focused queue and summary in its
top-level application report for provenance, but still applies the canonical queue.
The book-output audit verifies the focused queue is a true subset of the canonical
book-line review queue, not merely a smaller TSV with a matching hash.
The combined review wrapper now refuses an explicit focused queue as
`--book-line-review-queue`; use the canonical queue for application.
`tools/merge_limbu_book_focused_review_queue.py` now closes the operator-loop gap:
after a reviewer edits the bounded focused TSV, the tool overlays only editable review
fields (`review_status`, `corrected_text`, `reviewer`, `notes`, and split fields when
present) onto the full canonical queue. It refuses detached focused rows, header
mismatches, duplicate row identities, and any non-review/provenance field difference,
so crop paths/hashes and OCR metadata cannot drift while focused edits are promoted
back into an application-safe full queue.
Real refresh with stable page IDs:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-focused-review-view-stable-pageids/`
was run under `/tmp/ocrtech-paddle311/bin/python` with the v4 router unchanged,
duplicate-source skipping, and `--page-id-prefix gorkhapatra-unique-current-p`. It
keeps the same operational shape as the prior two-page run: `2` pages, `692` preserved
lines, `599` canonical book-line review rows, and a passing book-output audit. The
focused queue is `369` rows: all `319` must-review rows plus `50` sampled
low-confidence rows, omitting `230` low-confidence spot checks from the bounded working
view. After explicitly rejecting the known non-Limbu page
`gorkhapatra-unique-current-p0002`, the p1-only projection at
`.../manual-triage-check/triaged-p1-only/` passes `audit_limbu_book_output` with zero
issues and leaves `230` canonical rows but only `35` focused operator rows (`10`
must-review rows plus `25` low-confidence spot checks). The triaged projection now has
fresh filtered dashboards too: canonical `230` cards, priority `230` cards, and focused
`35` cards, with no rejected-page card in the focused dashboard. This is the current
useful review packet for the real Limbu page; it is review-burden reduction, not
OCR-quality improvement.
No-op merge proof on the current pending p1-only focused queue writes
`.../manual-triage-check/triaged-p1-only/book-line-review-queue.focused-merged.tsv`:
`230` output rows, `35` focused rows consumed, `0` changed rows, and a merge report
beside it. After focused review is filled, the same command is the handoff point into
`tools/apply_limbu_book_review_queues.py --book-line-review-queue <merged-full-queue>`.
Codex then used the focused contact sheets to make only obvious visual drop decisions,
not Limbu text transcriptions: footer/contact metadata `img-line-0179`,
date/header metadata `img-line-0180`, portrait detection `img-line-0193`, masthead
`img-line-0001`, website/page furniture `img-line-0002`, and decorative page
ornament/icon `img-line-0039`.
The edited focused queue is
`.../manual-triage-check/triaged-p1-only/book-line-focused-review-queue.codex-obvious-drops-v2.tsv`;
the merged full queue is
`.../manual-triage-check/triaged-p1-only/book-line-review-queue.codex-obvious-drops-v2-merged.tsv`
with `230` output rows and `6` changed rows. Applying that full queue with
`tools/apply_limbu_book_review_queues.py --skip-page-triage` writes
`.../manual-triage-check/triaged-p1-only/reviewed-obvious-drops-v2/final-book/`: `6`
book-line drops, `317` remaining lines, `0` layout-gap insertions, and
`audit_limbu_book_output` passes with zero issues. Reviewed training-pack export
correctly refuses this drop-only projection because there are no accepted text
corrections; drops must not become recognizer labels.
The next operational artifact is a pending-only focused review packet, not a larger eval
harness. It was prepared from the edited focused queue at
`.../manual-triage-check/triaged-p1-only/pending-focused-review-packet-v2/` using
`tools/prepare_limbu_book_unresolved_review_packet.py --book-line-review-queue
.../book-line-focused-review-queue.codex-obvious-drops-v2.tsv` and audited with
`tools/audit_limbu_book_unresolved_review_packet.py`: `29` packet rows, `6`
non-pending focused rows skipped, item counts `{"existing_book_line": 29}`, all rows
`existing_low_confidence_line`, suggested action `correct_existing_line_or_accept`,
`audit_passed=true`, `issues=0`. The packet HTML has been regenerated as an editable
review station at
`.../pending-focused-review-packet-v2/book-unresolved-review-packet.html`: each crop
has `review_status`, prefilled `corrected_text`, optional split fields, reviewer,
notes, a source-page context image with the active line highlighted, neighboring OCR
lines in document order, local browser autosave, filters for all rows / blank OCR /
prefilled rows / pending / reviewed / preflight issues, per-row quick actions, and a
browser-side "Export reviewed TSV" button that emits the same applicator schema as
`book-unresolved-review-packet.tsv`. Candidate missing-line rows now also get
geometry-based neighboring OCR lines around their proposed insertion point, so
reviewers can judge missed-line crops against local document order instead of the crop
alone.
The quick actions can accept current text, mark corrected text, drop existing lines,
keep pending, and jump to the next pending row; the same active-row actions are also
available from the keyboard: Ctrl/Cmd+Enter marks a nonblank correction,
Ctrl/Cmd+Shift+Enter accepts nonblank text, Alt+D drops an existing line, Alt+P returns
it to pending, and Alt+N jumps to the next pending row. Accept/correct refuses blank
text in the browser UI. For nonblank OCR rows, `corrected_text` is prefilled as an
editable starting point while `review_status` remains `pending`; this is not training
truth until the reviewer explicitly accepts/corrects the row and fills a reviewer name.
Browser autosave only protects in-progress editing and is guarded by the current packet
signature, so stale local drafts from older regenerated packets are ignored; the
exported TSV remains the review handoff artifact. Export is browser-preflighted for
missing reviewer names, blank accepted text, machine-looking reviewer names, invalid
statuses, and invalid drop/reject rows, but the command-line packet audit remains
authoritative.
`tools/audit_limbu_book_unresolved_review_packet.py` now accepts `--packet-tsv`, and
`tools/apply_limbu_book_unresolved_review_packet.py` now accepts `--packet-dir`, so an
exported `book-unresolved-review-packet.reviewed.tsv` can be audited/applied against
the original packet provenance without overwriting the source TSV. See
`docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md` for the exact commands.
`tools/apply_limbu_book_reviewed_packet_to_training_pack.py` is now the preferred
post-review handoff: it applies the reviewed TSV, audits the final book, and exports
the reviewed-only training pack in one command, failing if no reviewed rows are
exported. This is the next human review surface. Once those rows are accepted/corrected
in the exported/reviewed packet TSV, run that handoff wrapper against the already-cleaned
`reviewed-obvious-drops-v2/final-book/` base. Codex also ran a non-claim mechanical
smoke with one copied OCR row marked accepted in `/tmp`: it exposed stale chainable
document-hash recording in the book-line/candidate applicators, which is now fixed; the
rerun passed reviewed packet audit, application audit, final book-output audit, and
exported `training_rows=1`. The smoke artifact was deleted and is not training data,
and the same contract is now covered by a strict synthetic wrapper integration test. The
wrapper now writes a `README.md` in the handoff directory with artifact paths and the
next `split-manifest` / `train-recognizer` commands, while keeping the boundary explicit
that this is not a CER/WER quality claim. A
pending-only proof at
`.../pending-focused-review-packet-v2/application-pending-on-reviewed-drops-base/`
applies `0` rows, keeps final line count `317`, records
`book_line_review_row_count=29`, and passes both
`tools/audit_limbu_book_unresolved_review_packet_application.py` and
`tools/audit_limbu_book_output.py`. The packet splitter now carries packet crop
path/hash into `source_crop_paths` / `source_crop_sha256s` on the generated book-line
review TSV, so accepted packet corrections retain crop evidence for training
provenance. `tools/audit_limbu_book_unresolved_review_packet.py --allow-reviewed` is
now the reviewed-packet preflight, and `tools/apply_limbu_book_unresolved_review_packet.py`
runs it before splitting or applying. Pending-only packets still pass as no-ops, but
accepted/corrected rows require explicit non-machine reviewer text and explicit
`corrected_text` or `split_corrected_texts`; drop/reject rows require reviewer identity
and must be existing book lines. The current pending proof records
`reviewed_application_ready=false`, which is correct because no human text labels exist
yet. The unified application audit now also verifies the packet-audit file record,
`review_packet_audit_passed`, `reviewed_application_ready`, accepted/drop status
counts, and requires `reviewed_application_ready=true` whenever the application changes
book text. The unified applicator now runs that application audit itself after writing
the final report, writes `book-unresolved-review-packet-application-audit.json`, and
fails the command if the audit does not pass. Do not restart generic eval-harness work
before this text review produces labels.
The reviewed training-pack exporter now runs `tools/audit_limbu_book_output.py` on the
source book output before it writes labels and records that audit in
`source-book-output-audit/`. A real retry against
`reviewed-obvious-drops-v2/final-book/` wrote a passing source-book audit at
`.../reviewed-obvious-drops-v2/training-pack-reviewed-after-source-audit/source-book-output-audit/`
(`passed=true`, `issues=[]`) and then correctly refused the export with "no training
rows were exported" because the projection still has only drops, not accepted text
corrections. This is the desired state before human review: source consistency is
proven, but no labels are minted.
The same exporter now recognizes `candidate_line_review_insertion` metadata as
explicit review evidence, in addition to book-line corrections/splits and layout-gap
insertions. This matters for missed-line recovery: accepted candidate/missing lines
inserted through the unified review packet will now become reviewed training rows
instead of being silently skipped after application.
Direct candidate-line review application now also validates crop evidence whenever a
review row supplies `crop_path` or `crop_sha256`: both fields must be present, the crop
must exist, and the SHA-256 must match before insertion. The inserted-line metadata and
`candidate-line-insertion-pairs.jsonl` now preserve `crop_path` and `crop_sha256`, so
reviewed missed-line training rows stay tied to the crop evidence that justified them.
Current 2026-06-23 reviewed-projection audit hardening: applying book-line corrections
or layout-gap insertions now clears inherited focused queues and review dashboards from
the chainable `book-run.json`, because the canonical review rows may have changed and
the old focused/dashboard views can become stale. The chain writers also recompute
`source_book_document_json_sha256` whenever they rewrite `source_book_document_json`.
`tools/audit_limbu_book_output.py` now checks `document.md` and `document.body.md`
beside the recorded `document_json`, while still using `output_dir` for the original
OCR page bundle/manifests/crops. This fixed the real reviewed-obvious-drops-v2 audit
failure mode.
Historical 2026-06-21/2026-06-22 state:
Critical 2026-06-22 correction: do not keep working as if the Devanagari PP-OCRv5
training run is still the blocker. It was intentionally stopped at the
`best_accuracy` checkpoint after the dev curve regressed while the CPU run still had
hours left. The stopped-run completion audit passed with `ready_for_export=true`, the
model was exported and packaged at
`outputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-stopped-2026-06-22/package/model-card.json`
(SHA-256 `b05ca0e4a74f1818307bb547780b49969a60e93d273dda7a78e342cfdbd0d788`), and
the sealed 31-line recognizer recipe scored codepoint CER `0.118219`, grapheme CER
`0.256822`, and exact-line accuracy `0.290323` at
`outputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-stopped-2026-06-22/sealed-heldout-eval/recipe-run-paddle-external/score`.
This is not claim-safe because the pack and GT are still draft evidence, but it is
enough for engineering direction. The same recognizer has now been run on the three
real CDC Devanagari pages with the existing image-line detector:
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-stopped-epoch11`
scores page codepoint CER `0.480602`, grapheme CER `0.697389`, WER `0.872727`, line
F1 `0.400301` mean / `0.350877` micro, and matched-line codepoint CER `0.221018`.
That replaced the old image-line Tesseract page result (`0.865400` codepoint CER,
`0.872156` matched-line CER). A directed detector sweep then tested simple morphology
cleanup with the recognizer held fixed. The selected detector
`h31/v5/min_width80/max_height80` lives at
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80`
and scores page codepoint CER `0.458432`, grapheme CER `0.645161`, WER `0.824242`,
line F1 `0.593750`, and matched-line codepoint CER `0.177895`. It reduces predictions
from `83` to `33`, but still matches only `19/31` reference lines. The paired
comparison against the original PP-OCRv5 detector is non-claim-safe but positive:
codepoint CER improvement `0.022169`, grapheme CER improvement `0.052227`, paired
pages `3`. The updated line-failure analysis at
`outputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-devanagari-cdc-ppocrv5-detector-h31-v5-minw80-maxh80`
shows `19` matched rows, `12` missed references, `14` extra predictions, and dominant
error mass from missed reference text (`47.58%`). So the next Devanagari work is
targeted missed-line recovery/layout modeling with this recognizer held fixed. More
packaging, another blind recognizer loop, or another broad morphology sweep is the
wrong next move unless the page-level failure analysis changes.
Follow-up rescue-pass tests confirm that warning: a compact long-line rescue improved
mean line F1 (`0.605257` -> `0.626903`) but worsened page codepoint CER (`0.458432` ->
`0.464766`), while broader rescue passes regressed to `0.537609` and `0.832146` by
reintroducing noisy extra predictions. Evidence and a CDC geometry review packet live at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/`. The review packet has 3
page images and 31 line boxes, with browser review page
`devanagari-cdc-linebox-geometry-review-packet/line-box-review.html`.
The obvious next hypothesis, splitting wide connected components on vertical whitespace,
has also been tested. The option is now wired in the standalone materializer and
operational `parse-limbu-doc` image-line path, but all CDC split-wide variants were
no-ops: detector source counts stayed `{"primary": 33}` and page codepoint CER stayed
`0.458432` with the same metric SHA-256
`b1d993e9fcf09a7cff18d31c3c7e43ee2ff34cd8a57eaae96526e9f977d4c5cb`. Do not repeat
simple wide-component splitting; inspect the geometry packet and implement
page-region/row recovery for the missed areas. The concrete target list is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/devanagari-cdc-page-region-recovery-targets.md`:
12 misses split into 3 no-overlap rows, 4 overmerged rows, 2 partial detections, 1
low-IoU refinement, and 2 page markers. Existing-row repair has now been tested and
rejected too: same-row merge gap80 regressed page codepoint CER to `0.463975`,
detected-row split gap24/min700 regressed to `0.467933`, and detected-row split min900
was a no-op. The detected-row split switches are now available in both the standalone
materializer and the operational `parse-limbu-doc` / book-manifest image-line path, but
the CDC evidence still rejects them as a default. Next work should recover absent rows
and short page markers from page regions. Region-filtered rescue is the first positive
step: rescue-pass parsers now
accept `region_min_x_ratio`, `region_max_x_ratio`, `region_min_y_ratio`, and
`region_max_y_ratio`, and the tight bottom-long + top-title CDC diagnostic improves
page codepoint CER from `0.458432` to `0.452890`, grapheme CER from `0.645161` to
`0.637481`, mean line F1 from `0.605257` to `0.635724`, and matched references from
`19` to `21` with 36 detected lines. The score artifact is
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80-region-rescue-bottom-long-top-title-tight/metrics.json`
(SHA-256 `1c787c10392edf4f8a27dac018b5406b3a960b963a419aa4a64e9ec1c0acc0af`).
The conservative page-marker follow-up is a page-CER tie and the current line-structure
best: codepoint CER stays `0.452890`, mean line F1 rises to `0.659689`, and matched
references rise to `22` with 37 detections by recovering the social-page `२` marker
(score SHA-256 `19846ebae905411387ccbb88ee9520021387aac9f3debcc4ce81e88a87e1c836`).
The runtime image-line path and standalone materializer now tag conservative
bottom-margin digit-only small-box detections as `structural_role=page_marker`; the
fresh h20 structural rerun marks exactly the recovered social-page `२` marker and keeps
the same metric SHA-256, so it is metadata-only for current scoring. Do not treat this
as script-count exclusion. `export_document` now writes a separate `document.body.md`
that omits structural blocks while keeping full `document.md` and `document.json`
auditable; the one-page operational social artifact
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/operational-parse-social-page-marker-structural`
proves the recovered marker remains in full markdown, is excluded from body markdown,
is bound in `limbu-output-manifest.json`, and passes `audit_limbu_output`.
The current h20 unresolved failure surface also has a textless review packet at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/`:
24 crop rows, split into 9 missed-reference regions and 15 extra-prediction regions.
Reference text is not written to the review JSONL/HTML; 2 exact machine-text echoes of
hidden reference strings are suppressed, and the latest leak check found zero hidden
reference strings in the packet. It is covered by
`tools/audit_unresolved_page_review_packet.py`; the current audit artifact
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/unresolved-region-review-audit.json`
passes with 24 rows, 0 hidden-reference leaks, and 0 issues (SHA-256
`33105c0144c5cfa56ee20554a6aee82434f9c4cb6c19a60cc9533882a918db5b`). Treat this as
a review handoff artifact, not a solved OCR-quality improvement.
A targeted h20 + same-row merge gap120 follow-up was tested for the remaining
top-title/fragment near misses and rejected. It raises mean line F1 to `0.694815`, but
page codepoint CER regresses to `0.465558`, grapheme CER to `0.654378`, matched-line
CER to `0.190031`, and reading-order pair accuracy to `0.571429`; the refreshed
failure analysis still has 9 unmatched references and dominant missed-reference error
mass. Do not chase broader same-row merging as the next default path.
The first book-shaped wrapper has also landed as `tools/parse_limbu_book_pages.py`. It
reads an ordered JSONL page manifest, runs the existing `parse_limbu_document` path for
each page, and writes per-page OCR bundles plus aggregate `book.md`, `book.body.md`,
`book-review-queue.tsv`, `book-run.json`, and a scorer-compatible `score-documents/`
projection. The persistent sidecar smoke artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-book-sidecar-smoke`
(2 pages, 4 lines, 1 aggregate review row, 0 page audit failures, `book-run.json`
SHA-256 `7f9da4cec883e0fc9e72e16f62c1ac633046daea4c86091d25129643376f8819`). More
importantly, the selected h20 structural CDC diagnostic has now been replayed through
the book wrapper using real CDC page images and selected materialized page documents.
Replay prep lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-replay-inputs`
(summary SHA-256 `c28d0511bc42b855125c8e6a2063715ff15a0d085869a2107e32e7dfb4f411cf`);
book output lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay`
(`book-run.json` SHA-256 `9654cebe89ab68618cf5bf36419e5bd3a9513195faf9d4f9eb4ce5481adb3e8d`).
It has 3 pages, 35 lines, 27 aggregate review rows, 0 page audit failures, and score
projection metrics of page codepoint CER `0.446556`, grapheme CER `0.626728`,
matched-line codepoint CER `0.175130`, and matched-line reading-order pair accuracy
`0.600000` over 3/3 pages with no missing docs/references or source-path mismatches
(metrics SHA-256 `1a353b2d06b89caf081d190ad79add2eee57d1d51b70cb6144be40e53d082d75`).
The replay prep drops blank selected-output lines by default and records them; in this
CDC replay it dropped 2 science-page blanks (`img-line-0006`, `img-line-0007`). The
scorer now exposes the objective's line-rate surface directly: 31 reference lines, 35
predictions, 22 matches, 9 missed references, 13 extra predictions, 0 blank
predictions, micro line precision/recall/F1 `0.628571`/`0.709677`/`0.666667`, and
missed/extra/blank rates `0.290323`/`0.371429`/`0.000000`. Treat this as real-page
replay/packaging/scoring evidence, not fresh OCR inference.
The replay exposed and fixed a body-export bug: sidecar-replayed structural
`page_marker` lines could be grouped into normal text blocks and leak into
`book.body.md`; `src/ocrtech/structure.py` now emits structural lines as separate
blocks first. This is editable-text progress, but not finished OCR quality: 27 review
rows remain and matched-line CER is unchanged.
The lower-height h10 marker pass recovers more single-digit markers and reaches `23`
matched references, but it regresses page codepoint CER to `0.454473`, grapheme CER to
`0.639017`, WER to `0.844707`, and reading-order pair accuracy to `0.538462`; keep it
as diagnostic evidence, not the default. Page markers need explicit structural/review
semantics rather than being blindly mixed into page text order.
The h10 marker pass composed with source-break post-recognition line assembly is also
rejected as the default. It gets recall to `0.806452` and missed references to `6`, but
page codepoint CER is only `0.452890`, precision drops to `0.675676`, extra predictions
rise to `12`, two blank predictions appear, and reading-order pair accuracy is
`0.553191`. Score:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-cdc-h10-page-markers-line-assembly-gap160-width500/metrics.json`
(SHA-256 `6ccf1bce5263f0eed1fc966a6d2f9c3c6567534fdd750e7a05549462a12b6c57`).
New operational detector parity work: `parse-limbu-doc` now supports
`min_aspect_ratio` / `max_aspect_ratio` in primary and rescue image-line detector
passes, and opt-in `image_line_split_detected_tall_*` settings. The geometry probe on
the science page split the known over-tall `rescue-1` box `[319, 1856, 1075, 115]`
into `[319, 1856, 1052, 37]` and `[317, 1944, 1051, 29]`. The Paddle follow-up has
now been scored and rejected as a default path. It improves line recall/F1 to
`0.838710` / `0.776119` and cuts missed references to `5`, but page codepoint CER
regresses to `0.478226`, grapheme CER to `0.674347`, WER to `0.882958`, matched-line
CER to `0.191242`, and reading-order pair accuracy is only `0.578431`. The book audit
passes (`ceb9d681e4ee50ac48880f41b5c0dc25e084cd4cfcc43b0774d573366634210e`), but the
candidate gate rejects it (`admitted_operational=false`, admission SHA-256
`2d56eaaf0071fa17341d9905aaa7bc46d5bb89fa40c1428e936645e7727665dc`) because it
regresses against the `0.458432` baseline and misses the `0.60` reading-order
threshold. Treat tall split as review/recall evidence, not selected OCR text. Probe:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/detected-tall-split-geometry-probe.json`
(SHA-256 `51b364b4211f40d54714c903b9038fab284fc83e7853efa85ccc3ebb32225c64`).
The exact Paddle follow-up command is documented at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/RUN_ON_PADDLE.md`
(SHA-256 `f3cb56df021fe20982746b1a657a8ab80248bb019694ef5bfcf04c4f5e776a37`) and uses
manifest SHA-256 `a6657c2f9cb7be1ba7e8c89c91650bf9ab18c3071a7042b89580a3a10f54dfad`.
The review-only reuse path is now implemented too:
`tools/prepare_candidate_line_review_packet.py` compares selected score documents to a
secondary candidate document set and crops candidate-only lines below the selected-line
IoU threshold; `tools/audit_candidate_line_review_packet.py` verifies the packet. The
detected-tall-vs-current-best packet has 4 rows (maths marker `१`, two science
bottom-row split candidates, and science marker `२`) at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-only-review-packet/`.
Packet SHA-256 is `f20438d05d158377387a05df9197cc3a6e7e9afec573a60668c1d429d1b83128`;
audit SHA-256 is `919615e48fe66c24810e6137b5bb65dd64b348ebfee0b5817af38b47f38c2f40`
and passes with 0 issues. Use this pattern for rejected high-recall candidates:
surface missing-line evidence for review while keeping selected OCR text stable.
Accepted candidate-only rows can now be inserted with
`tools/apply_candidate_line_reviews.py`; it writes a separate book projection,
score-documents, and `candidate-line-insertion-pairs.jsonl` with provenance, never
mutating the selected OCR run. The real detected-tall packet is still pending-only:
the smoke output at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-line-review-application-pending/`
has `review_rows=4`, `applied_count=0`, `skipped_count=4`, audit SHA-256
`95521bc3c5ad86bd6cdf19350cfa1323a972ce817ee363451ff79349ccc4f69f`, and preserves
selected `book.md` / `book.body.md` byte-for-byte. `tools/audit_candidate_line_review_application.py`
independently audits these application outputs; the real pending application audit
passes with 0 issues, SHA-256
`0e6addc547c1ef8df3ae9fe54d6599a16341fb2d57168d3b785da4cbe38cf452`. This completes
the review loop mechanically. The pending application projection has also been scored
and admitted through the existing book candidate gate using the application audit:
metrics SHA-256
`184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244`, operational
admission SHA-256 `eb0e559908046322343a7507a0143ae2d724876c069eb2528f0d1950085fd5b5`,
`admitted_operational=true`, `claim_safe=false`. Because no candidate rows were
accepted, the score is byte-identical to the current h20 source-break line-assembly
candidate. Actual missing-line recovery still depends on accepted human review or a
better automatic selector.
One automatic selector has now been tried and rejected as a default:
`tools/select_candidate_line_reviews.py` implements `page_marker_v2`, a conservative
bottom-margin digit-only structural page-marker policy with confidence, selected-line
IoU, size, and duplicate-text guards. On the real detected-tall packet it selected only
the maths marker `१` and rejected the other 3 rows; selection report SHA-256
`928aaecd9a102c5615e1085b3d29a80188d419c35e17bbfc9c743876f7e20872`, selected JSONL
SHA-256 `8152ba724a67a4ef25296c6dcd47a02c7050262356065d6c0c0979fa4f366867`, rejected
JSONL SHA-256 `a3a571ad12c8efade971d0b89baa0ef9b1345cb15d97beba3ec284f5089b3e85`.
The applied projection passes independent application audit (SHA-256
`b0e377bec4f0fca9f4506fa9126a65c57014adf7d419a04e08f98d50bf3b8743`) and improves line
recall/F1 to `0.806452` / `0.769231`, but it regresses page codepoint CER to
`0.448139` and drops reading-order pair accuracy to `0.553191`; admission rejects it
with SHA-256 `9cd826a51b38aa29facab4ec332a4084421ac25bf26365dc04f954e6f1b5a76f`
(`admitted_operational=false`). Current best remains h20 source-break line assembly.
Do not spend the next cycle blindly inserting page markers; the issue is structural
export/scoring semantics, not lack of a marker selector.
To make that visible, `tools/run_page_ocr_eval.py` now emits structural-line
diagnostics in addition to the existing all-line metrics. It counts non-text
`metadata.structural_role` predictions, matched/unmatched structural predictions,
exact structural text matches, and per-role prediction totals. A replay of the current
best with these metrics has one exact matched structural marker (metrics SHA-256
`f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`). The v2
auto-marker replay has two exact matched structural markers (metrics SHA-256
`2634d0f70f6dac3cac08844ea1530aff08c26271a92a7eb2c725a879415ce5b9`) while still
showing page CER `0.448139` and reading order `0.553191`. The correct interpretation:
the marker signal is real, but the editable body-text candidate is worse.
There is now a body-line review-ceiling diagnostic too:
`tools/select_candidate_line_reviews_by_reference_overlap.py`. It is intentionally
oracle-labeled: it selects candidate-only review rows by overlap with currently
unmatched reference lines, not by deployable OCR evidence. On the detected-tall packet
it selected exactly one text row, the science bottom line, with IoU `0.860889` to
`limbu-devanagari-cdc-science-p0009-l0009` (selection SHA-256
`d11dcdc1cc0b07626e6d85b97356c86cadd670796b9e926708e2be7585344d05`). If the selected
row is inserted with reference text as the corrected text, score improves to page
codepoint CER `0.420428`, line recall/F1 `0.806452` / `0.769231`, and reading order
`0.634409` (metrics SHA-256
`247c4186059047fc9de16019f358172bc46261de497127581f63335f986cb521`). If the same row
is inserted with its candidate OCR text, page codepoint CER is only `0.445764` and
matched-line codepoint CER worsens to `0.191413` (metrics SHA-256
`21fd79e6a0645c0084cac959743354f52c224b28b15e4e60be7f56e3ac664d75`). The application
audit now detects oracle/reference-assisted review rows and sets
`operational_review_safe=false`; `tools/admit_limbu_book_ocr_candidate.py` rejects
such applications even when metrics pass. Admission SHA-256 values:
`c8f69e7d011c217e74218ffe280e901420ab99ebefa7f329344af776a6e779c7` for reference
text and `2a97b7f01425efdd514887324c8c23c6b4dd46c48376b99c2299fad736d88b45` for
candidate text. Current best remains h20 source-break line assembly; this diagnostic
only proves that a reviewer-correction lane can recover a real missed body line.
For the actual non-oracle review lane, use
`tools/prioritize_candidate_line_reviews.py`. It ranks candidate-only rows from packet
fields only and keeps every row pending. The real queue at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/`
has 4 rows: 2 high-priority body-line candidates followed by 2 structural marker
checks. Hashes: prioritization JSON
`ef3fc75efc52597c97ea2e28196164b31ef3ceb18414ef5b47e559cc4343e51a`, JSONL
`9a53ad3d3ada7e61211d5406ce35c9ed9a10e86a41ea09ea0f4edcb3766ad49e`, TSV
`753adcd61f70f490b0429d6ff9d25ff88b0bc1ac8ec20a2f9ae21ec1f514fa19`, HTML
`5fbd7f39801ac7e29b0503314475ef26e6e61345b26330720bf840ad1e2dcdef`. The queue now has
an independent audit at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/prioritized-candidate-line-review-audit.json`
(SHA-256 `d2fe0e33e3bd5f78698632892581651a3e95332936b7e9ddfa926c1ecc3fcd22`): it
passes with 0 issues, 4 source/JSONL/TSV rows, no oracle/reference-assisted rows, and
`operational_review_queue_safe=true`. The first two rows are the science bottom-row
text candidates and should be reviewed with action `transcribe_visible_line_or_reject`;
do not apply them without corrected text from a reviewer. Rerun
`tools/audit_prioritized_candidate_line_reviews.py` only after regenerating or manually
touching the pre-review queue; it expects all rows to stay pending. After reviewer
edits, use `tools/audit_candidate_line_review_application.py` on the application output.
There is also a non-oracle crop probe for the two body-line rows:
`tools/prepare_candidate_line_text_crop_probe.py`. The real artifact at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-text-crop-probe/`
processed 2 body-line candidates and refined both. Rank 1 was top-bleed-trimmed from
`1100x85` to `1100x63`; rank 2 was trimmed left of a detected horizontal rule from
`1099x77` to `466x77`, removing the rule/illustration before recognition. Hashes:
report `7f1dfc09d4cf612e7516dbdea07eb6bc19608e637ad11efe41e8a8112393c79a`, JSONL
`a7d7cced3cdca1a223aa6c6d00ccc70acabee90183107eac6b70ff61b630f011`, HTML
`10db1161085a49e722ad373b88ff9656aaf87b5423dad2ec88e8f1c48f1f18b6`, label file
`47b3594dac7a97f516d21f271c1fe1e794f6d621bfc17539d1cf6b8e540ef4e0`, refined rank-1
crop `4e97bca5580d613777199395db9e091e248cc79035a1063a284a896ec9632594`, refined
rank-2 crop `4f8221c4552b937e8e0182a4ecf9f1062a9b1056ec377bfd4ffe00e99c020d5a`. The
Devanagari PP-OCRv5 recognizer was run locally over the refined crops in both
`.venv-paddle313` and `/tmp/ocrtech-paddle311`; both produced the same prediction TSV
SHA-256 `734b208860106d84a7dbd7473e18927293a29b51e8b67298e173c5b2c626bbee` with 2 rows
and 0 failures. A comparison artifact at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-crop-recognizer-probe-comparison/`
shows it is not promotable: max probe confidence `0.469282`, rank-1 confidence improves
`0.251062 -> 0.469282`, rank-2 confidence improves only `0.363574 -> 0.390961`,
prediction text changes twice, and
`automatic_application_safe=false` (report SHA-256
`18cdfb099ae6d93ee96ee350faec0305837349d65142ebaaad05db1553c04eb7`, JSONL SHA-256
`f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d`). Do not promote
the crop probe without reviewed text or a better recognizer path that passes
page-level admission.
The TSV workflow is now wired into application: `tools/apply_candidate_line_reviews.py`
accepts `--candidate-review-file` pointing at either JSONL or TSV, while keeping the
old `--candidate-review-jsonl` alias. `tools/audit_candidate_line_review_application.py`
also reads TSV review files. A real pending TSV application smoke lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review-application-pending/`.
It applied 0 rows because all prioritized rows are still pending, preserved selected
`book.md` / `book.body.md` byte-for-byte, passed application audit
(`e36036c0e3a9177e7c526465cf16c4a066f54ae37fe920bfb97c1a3bb9278c5c`), scored with
current-best metrics (`f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`),
and passed operational admission
(`453226159011a944a606e0c951c5b7dceda3bee73c76f44d195c44e2d61672c1`,
`claim_safe=false`). This means the next real movement is to review/accept ranked
body-line rows, not to build more plumbing.
The split-worksheet review lane now has the same score/admission closure:
`tools/score_limbu_book_ocr_candidate.py` scores any candidate book projection with
`tools/run_page_ocr_eval.py`, then runs `tools/admit_limbu_book_ocr_candidate.py`.
Admission now accepts unified unresolved-review application audits only by verifying
that the audit's final-book identity matches the candidate `book-run.json` SHA-256;
normal book audits still require `output_dir` equality. The application audit tool now
records `final_book_output_dir` and `output_dir` for `application/final-book/`. The
real pending split-worksheet application was scored/admitted at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-worksheet-review-application-pending`:
wrapper report SHA-256 `f546cb6e4cf007c1a32b8e6799ce3203d1060eafd327cc86a72921217a59a832`,
metrics SHA-256 `f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`,
page-eval-run SHA-256 `3356fb45524953c8837b6ab237c701866285adf11661101de9e634de8b4c153b`,
and admission SHA-256 `7fadfc25c0191fa71abb9d93e6266c0e0078fea015df7a02bd067d02556db966`.
It is operationally admitted and claim-unsafe, with the same metric values as current
best: codepoint CER `0.446556`, grapheme CER `0.626728`, WER `0.839243`, line F1
`0.750000`, missed/extra/blank rates `0.225806` / `0.272727` / `0.000000`, and
reading order `0.611765`. The hash differs from the older current-best score because
the eval metadata records the reviewed-worksheet candidate path/system name. This
wrapper closes the post-review gate; it is not OCR improvement by itself.
Remaining failure analysis lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-failure-analysis-devanagari-cdc-region-rescue-bottom-long-top-title-tight/`;
missed reference text remains dominant, so do not declare book OCR solved.
- `parse-limbu-doc` now accepts project-relative `prepare-limbu-capture` metadata
paths; the previous binding logic double-prefixed relative prepared paths.
- `PaddleOcrEngine` now supports `recognition_mode=auto`, using full PaddleOCR for page
images and lower-level PaddleOCR `TextRecognition` for wide/short line crops.
- `outputs/limbu-first-router-v1/model-card.json` opts the Sirijonga secondary into
that auto mode and now points at the exported v4 component under
`components/limbu-sirijonga-paddle-v4/`.
- The stronger v4 checkpoint archive
`outputs/limbu-recognizer-v4-2026-06-18/limbu-v4.tgz`
(`a0ec317d053402f0966e3b87f3d489b64f33810cc99167f85c0c267101a22cae`) has been
exported with PaddleOCR `tools/export_model.py` from `/tmp/PaddleOCR-v3.7.0` and
packaged at `outputs/limbu-v4-paddle-package-2026-06-21/package/model-card.json`.
The stale `stronger_unexported_checkpoint` advisory was removed from the router.
- The router now has a Devanagari primary guard:
`primary_guard_script=devanagari`, `primary_guard_threshold=0.5`. This prevents a
Sirijonga recognizer hallucination from replacing a line that Tesseract already sees
as clearly Devanagari.
- Automatic Limbu post-correction profiles now have an audit/admission gate:
`ocrtech audit-limbu-post-correction-profile`. Profiles with no admission evidence
can still run but are marked experimental and `audit-limbu-output` warns they are not
claim-ready. Profiles that claim `admission.status=validated` must point at a hashed
eval artifact, frozen eval-pack identifier, metric, and non-regressing before/after
values; otherwise `parse-limbu-doc` fails.
- Accepted Limbu review correction pairs now have a deterministic derivation lane:
`ocrtech derive-limbu-post-correction-profile`. It reads one or more
`limbu-correction-pairs.jsonl` files, hashes them into a derivation report, drops
unchanged rows, enforces `--min-support`, rejects conflicting noisy-to-clean mappings,
and emits exact-match rules in `limbu-post-correction-profile.json`. The output is
deliberately `admission.status=experimental`; held-out admission is still required
before any automatic correction is claim-ready.
- Limbu post-correction profiles now also have a held-out scorer:
`ocrtech score-limbu-post-correction-profile`. It applies the exact runtime profile
rules to held-out noisy/clean correction pairs, writes JSON/JSONL/Markdown eval
artifacts, reports codepoint micro CER before/after, fails on aggregate regression or
any regressed held-out row, and prints `eval_run_sha256` for
`admission.eval_run_sha256`.
- Passing post-correction evals can now be stamped into a new admitted profile with
`ocrtech admit-limbu-post-correction-profile`. It refuses failed evals, profile/eval
SHA mismatches, missing frozen eval-pack IDs, aggregate regressions, and row
regressions, then writes `limbu-post-correction-profile-admitted.json`, an admission
report, and an audit directory. Do not hand-edit profile admission fields.
- Reviewed Limbu correction pairs now have a frozen pack builder:
`ocrtech build-limbu-correction-pair-pack`. It canonicalizes noisy/clean rows, rejects
duplicate sample IDs and empty text, writes deterministic train and heldout JSONL
splits, records source file hashes, and emits `pack_content_sha256`. Use the train
split for derivation and the heldout split for profile scoring.
- Frozen correction-pair packs now have an independent replay audit:
`ocrtech audit-limbu-correction-pair-pack`. It checks summary shape, train/heldout
file hashes, row counts, split labels, duplicate sample IDs, row text shape,
available source file hashes, and recomputed `pack_content_sha256`.
- `limbu-output-manifest.json` now records hashed input provenance for model cards,
fallback model cards, capture-prep metadata, post-correction profiles, review source
documents, and review TSVs when those inputs are supplied. `audit-limbu-output`
recomputes those hashes, so stale model/profile/review inputs fail the bundle audit.
- Real Gorkhapatra candidate page run, now treated as Namdhinggo/Sirijonga-script
evidence rather than Devanagari-written Limbu evidence:
`outputs/limbu-dual-script-real-run-2026-06-21/gorkhapatra-devanagari-ocr-v4-guarded`.
Capture/output audits pass; 98 OCR lines, 22 review lines, average line confidence
0.821, quality score 0.896. Routing selected Tesseract for 7 guarded Devanagari
lines and PaddleOCR for 91 Sirijonga-classified lines. The directory name is a stale
operational label; accepted visual review and PDF target-font evidence point to
Namdhinggo/Sirijonga Limbu, not Devanagari-written Limbu.
- A current-schema rerun of that real Sirijonga page now exists at
`outputs/limbu-dual-script-real-run-current-2026-06-21/gorkhapatra-sirijonga-ocr-v4-current`,
with fresh capture prep at
`outputs/limbu-dual-script-real-run-current-2026-06-21/gorkhapatra-sirijonga-prep`.
Capture audit passed; auto-deskew skipped for insufficient near-horizontal line
evidence. The OCR output has 92 lines, 4 tables, 25 review rows, and script counts
`limbu_sirijonga=79`, `devanagari_limbu=11`, `mixed_limbu_devanagari=1`, `other=1`.
The strict capture/Sirijonga audit passes at
`strict-audit-capture-sirijonga/limbu-output-audit.json` (SHA-256
`67fe54feab8f636996de78c36cb303e60c5799bdce2fbeca082a9cd2997ce564`). The final
strict policy audit with `--require-no-pending-review` fails, correctly, because
`review_line_count=25`. Do not present this as finished digitization until the review
queue is resolved. The current-schema output has also been scored against a
derivative current-input copy of the non-claim Gorkhapatra draft reference at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/score-current-router-v4-current`.
It is still bad full-page OCR: codepoint CER 0.823802, grapheme CER 0.859202, line
F1 0.010127, and 2/303 matched reference lines. The audit has no source-path
mismatch but remains non-claim-safe, with the only hard audit issue being that the
train/dev leakage check was not run; the larger claim blockers are one-page sample
size, unreviewed machine-converted GT, non-frozen pack, and non-claim eligibility.
`compare-current-router-vs-live-v4-current` shows the fresh run is slightly worse
than the old guarded live-detector score, not an improvement.
Replaying the best image-line route on the same fresh prepared input gives the
current best non-oracle diagnostic. The unpruned route
`current-image-line-box-documents-width480-wide-rtl` uses Otsu/ink morphology boxes,
a 23x3 kernel, 12 px crop padding, the width-480 Sirijonga line recognizer, and wide
right-to-left order; `auto_layout` selects that same order from image features. It
scores codepoint CER 0.761144, grapheme CER 0.804042, line F1 0.833866, 261/303
matched references, and matched-line codepoint CER 0.106076. The pruned
precision-first variant `current-image-line-box-documents-width480-wide-rtl-prune-conf085`
keeps 199/323 lines using `drop_empty`, confidence >= 0.85, and Sirijonga/mixed
script filtering; it scores codepoint CER 0.730446, grapheme CER 0.763956, precision
1.0, recall 0.656766, and matched-line codepoint CER 0.036114. This is useful for
review triage, not a solved full-page path, because it drops too much content.
`parse-limbu-doc` now has `--line-detection-mode image-line` to run this class of
detector/crop recognizer path operationally. It writes `image-line-ocr-run.json`,
`image-line-crops/manifest.jsonl`, hashes the image-line crops in the Limbu output
manifest, and still goes through the normal script-aware review queue and audit
path. Final audits for image-line outputs should include
`--require-no-dropped-image-lines`; use `--image-line-filter-*` only for explicit
precision-first triage runs.
Operational evidence now exists under
`outputs/limbu-operational-image-line-current-2026-06-21`: the unpruned run keeps
323 lines, drops 0 detected lines, has 96 review rows, passes capture/Sirijonga
audit with the no-dropped-lines policy enabled (SHA-256
`ec5e62a13be875a86f16cb1725f7cbf586153adbc243d1df1adc3ede2a121300`), and fails
final strict policy on pending review rows; its score is codepoint CER 0.761144 and
line F1 0.833866. The pruned `conf085` run keeps 199/323 lines and has zero review
rows, but the earlier strict pass is now superseded by
`--require-no-dropped-image-lines`: the current final strict audit fails with
`removed_line_count=124` (SHA-256
`a7527a6fe61c23bc37598ade9f8a8d9429b188a1616398e3f76c4f3639735fa4`). This is
correctly classified as a precision-first partial output, not complete page OCR; its
line recall is only 0.656766 against the draft reference.
- Gorkhapatra Limbu Sirijonga draft page eval now exists at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21`. It was
built from PDF-native `NamdhinggoSILL` spans with the `limbu-namdhinggo` converter,
using the existing accepted review row for `gorkhapatra-language-page-0001-01`.
The draft has 303 reference lines, 36 spans with unmapped conversion issues, and is
explicitly non-claim (`gt_status=machine_converted_unreviewed`,
`claim_evidence_eligible=false`). The existing guarded live-detector OCR score is
audited and non-claim-safe: codepoint CER 0.815643, grapheme CER 0.857214, line
detection F1 0.009975, and 2/303 matched reference lines. The oracle-line-box
diagnostic using the same v4 recognizer over 303 PDF-derived reference boxes is also
audited and non-claim-safe: codepoint CER 0.029268, grapheme CER 0.060792, line F1
1.0, matched-line codepoint CER 0.030031, matched-line grapheme CER 0.063993, and
303/303 matched lines. The paired comparison shows a 0.786375 absolute codepoint-CER
improvement over live detection. A tuned morphology detector now reaches line F1
0.833866 and mean IoU 0.879406 on the same page, but page text is still bad
(codepoint CER 0.763835, grapheme CER 0.828226) and matched-line text is also poor
(codepoint CER 0.285785, grapheme CER 0.551916). The new matched-line reading-order
metric is 0.606543 over 33,930 aligned-line pairs; the older raw
`reading_order_pair_accuracy` remains 0.0 only because live predicted IDs are
`img-line-*` while references are `span-*`. This is the clearest local evidence that
same-font Gorkhapatra Sirijonga recognition is strong when boxes are oracle-quality,
while live full-page OCR still needs better live crop/line recognition and better
aligned-line ordering; it does not fill the Devanagari-written Limbu requirement.
- Real Sikkim Herald Sirijonga crop run:
`outputs/limbu-dual-script-real-run-2026-06-21/sikkim-sirijonga-crop-ocr-v4-guarded`.
Capture/output audits pass and the line is recognized as Sirijonga Unicode with
confidence 0.797. Diagnostic codepoint CER against the demo GT is about 0.095, a
real improvement over the earlier deployed component's roughly 0.302, but it is still
a one-line diagnostic and remains in review.
- Real Sikkim Herald Sirijonga page run:
`outputs/limbu-dual-script-real-run-2026-06-21/sikkim-sirijonga-page-ocr-v4-guarded`.
Capture/output audits pass; 22 detected lines, 20 review lines, average confidence
0.635, quality score 0.619. The page still fails the quality gate.
This is operational evidence that the page path runs, not evidence that it is good.
- Sikkim Herald Sirijonga draft page eval now has both live-detector and
oracle-line-box diagnostics, plus an image-morphology line-detector prototype. The draft pack is
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1` and is not
claim evidence. Live detector scoring has codepoint CER 0.891560, grapheme CER
0.897412, and 1/86 matched lines. The image-line detector prototype generated by
`tools/materialize_page_image_line_boxes.py` detects 125 lines, matches 59/86
references, gets line detection F1 0.559242, codepoint CER 0.460255, and grapheme
CER 0.576710. Oracle-line-box scoring, generated by
`tools/materialize_page_oracle_line_boxes.py` with the v4 component forced to
`recognition_mode=line`, has 86/86 matched lines, codepoint CER 0.274289, and
grapheme CER 0.548983. The comparison shows detection/layout is a major failure,
the morphology detector is a useful but insufficient prototype, and the remaining
oracle-box CER is still too high, so Sikkim-register recognizer data and reference
verification are also needed.
- There is still no verified Devanagari-written Limbu page reference in the local
evidence set. The Gorkhapatra accepted Limbu page is Sirijonga/Namdhinggo; the
two-script goal is therefore imbalanced until a real Devanagari-written Limbu page is
found or transcribed.
- Local Devanagari-written Limbu text exists at
`data/synthesis-text/cdjk-rai-limbu-2026-06-16/toolkit-parallel-text.jsonl`
(`language=limbu`, `script=Devanagari`, source `bible_brain/LIFWBT_Limbu`), but it is
text-only and license/claim status is pending. A synthetic rendered Devanagari smoke
page now exists at
`outputs/limbu-page-eval-drafts/limbu-devanagari-smoke-page-2026-06-21`; the
sidecar-perfect score/audit pass consistency, but `claim_safe=false` by design
because it is synthetic, one page, not frozen, and not human verified.
- `tools/inventory_limbu_page_sources.py` is now the source-readiness gate for the
Devanagari side. The current report at
`outputs/limbu-page-source-inventory-2026-06-21` now has 36 records, 3 real
Devanagari page candidates, 3 Devanagari text-only sources, 1 synthetic Devanagari
smoke pack, and `ready_for_real_devanagari_page_eval_draft=true`. The curriculum
`Maths`, `Science`, and `Social` JSON files contain Devanagari Limbu line text and
now resolve to mirrored CDC Limbu PDFs fetched from `lib.moecdc.gov.np`: Maths ref
3757 SHA-256 `25fb14113adeb8ca474de70d26bd53455ae7b424de9e92eb43d9028b11e4d186`,
Science ref 3756 SHA-256
`c24178a1789cad5cd209f9dccc8de54c7aa35cf28d7e5966221ccdd2ff2ee97e`, and Social
ref 3758 SHA-256
`d32366b3a4f80b44817ba633cf25bc3395d3c3df7a418f7999c902d71e938502`. With
`--require-real-devanagari-page`, the tool exits 0. This is source readiness only,
not claim-grade OCR evidence.
- A real Devanagari-written Limbu CDC curriculum page draft now exists at
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21`, built
by `tools/build_limbu_devanagari_curriculum_page_eval_draft.py`. It renders real CDC
PDF pages and uses curriculum JSON line text for Maths page 8, Science page 9, and
Social page 9. The pack has 3 pages, 31 total text lines, pack content SHA-256
`d587e9ee5cc2983f761142c18de071c2578218d41341b5b2058409e1294ef333`, manifest
SHA-256 `2a708b3dbd444a05329343482452648e1cc3ecc34954860c95e2935737c10522`, and
script-inventory SHA-256
`ad1f497fdfc8bb172948db8644c967f55c0559f9a1140a7ee64a5a978d032462`. The
sidecar-perfect eval at `score-sidecar-perfect` scores 3/3 pages with codepoint CER
0, grapheme CER 0, text exact accuracy 1.0, and reading-order pair accuracy 1.0, but
the audit intentionally fails claim safety because line-bbox coverage is 0, layout
references are absent, the pack is not frozen, and GT is
`curriculum_text_unreviewed`. This is the first real Devanagari page draft, not a
claim-grade Devanagari eval.
- The current Devanagari primary path was measured on that CDC draft with
`parse-limbu-doc --engine tesseract` (`nep+eng`, PSM 6). Outputs live at
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21/live-tesseract-documents`;
all three `audit-limbu-output` checks pass, so the bundles are internally auditable.
The score at `score-live-tesseract-primary` is intentionally non-claim-safe but
decisive: 3/3 pages scored, codepoint CER 0.897862, grapheme CER 1.390169, exact page
accuracy 0, and all 58 detected lines entered review. Per-page codepoint CER is
Maths 0.867220, Science 0.908257, Social 0.927536. A quick PSM/upscale sanity check
did not produce a meaningful rescue; the best tested setting was still about 0.87
codepoint CER. Do not trust the current `nep+eng` Tesseract primary as the
Devanagari-written Limbu solution.
- PDF-native line geometry is not enough to turn the CDC pages into line-box references:
PyMuPDF exposes only 2, 4, and 2 text lines on the selected pages while the
curriculum references have 14, 9, and 8 lines. A morphology line detector plus
Tesseract crop-recognition diagnostic now exists at
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21/image-line-box-documents-tesseract`
and `score-image-line-tesseract`. It detected 83 line-like boxes, had 12 empty crop
predictions, and scored codepoint CER 0.865400 / grapheme CER 0.937020. The paired
comparison against full-page Tesseract at `compare-image-line-vs-fullpage-tesseract`
shows only a non-claim-safe 0.032462 absolute codepoint-CER improvement. Cropping
helps slightly, but the Devanagari recognizer/data problem remains unsolved.
- Correction to the previous bullet: plain PDF text extraction exposes too few lines,
but PDF word groups are usable as unreviewed machine-aligned line boxes for the three
CDC pages. The line-box pack at
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21`
has pack content SHA-256
`e696a92dee7ac49b4e81659aee7fb344af4c8fa0d4741c6cd43ef074cd052200` and
`line_box_source=pdf_words_machine_aligned`. Science drops two private-use marker
PDF lines and Social drops two extra trailing PDF lines during alignment. The
sidecar-perfect line-box score audits cleanly (`audit_passed=True`,
`claim_safe=False`) with line detection F1 1.0 and matched-line CER 0. Full-page
Tesseract on the same pack remains terrible: page codepoint CER 0.897862, grapheme
CER 1.390169, line detection F1 0.385568, and matched-line codepoint CER 0.869087.
The image-line Tesseract diagnostic finds 83 boxes with 12 empty crop predictions and
scores page codepoint CER 0.865400, grapheme CER 0.937020, line detection F1 0.400301,
and matched-line codepoint CER 0.872156. The paired text comparison again shows only
a non-claim-safe 0.032462 absolute codepoint-CER improvement. Treat these CDC pages
as a legacy visual-register OCR problem; `nep+eng` Tesseract is not a viable
Devanagari-written Limbu primary.
- A recognizer-level CDC Devanagari crop pack now exists at
`outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crops-2026-06-21`,
built by `tools/build_limbu_devanagari_line_crop_recognizer_pack.py`. It converts the
line-box page references into 31 real image line crops with Unicode Devanagari labels
and raw legacy PDF strings in row metadata. Pack content SHA-256 is
`6d76ab654a0c210a1291f397f2274878d116713a91010ccc91a62cf892ea09b0`; manifest SHA-256
is `97b2a148345ee8cd7a9f8322a16a8f3339776cbe1b1e8b75ceeed86932333d5f`. The
sidecar-perfect recognizer score has 31/31 matched lines, codepoint CER 0, grapheme
CER 0, and exact line accuracy 1.0. The diagnostic `nep+eng` Tesseract crop score has
31/31 matched predictions, codepoint CER `0.852632`, grapheme CER `0.857143`, and
exact line accuracy `0.0`. This isolates recognizer failure from page detection:
even oracle/machine-aligned crops are not readable by generic Tesseract.
`tools/analyze_limbu_devanagari_legacy_target.py` then compared those same
predictions against the raw legacy PDF strings stored in the row metadata. The
analysis at `legacy-target-analysis-tesseract-nep-eng` reports Unicode codepoint CER
`0.852632`, raw legacy codepoint CER `0.880213`, and
`legacy_string_ocr_shortcut_supported=false`. Do not lead with a "recognize legacy
Latin then transduce" workaround; Tesseract is not even closer to the raw legacy
string. The practical next step is a same-register visual recognizer/data path.
- The same-register data path is now concrete. `tools/build_limbu_devanagari_cdc_line_crop_corpus.py`
now builds the current training corpus at
`outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21`.
All 281 local CDC text pages align with PDF word groups, yielding 6,446 raw line
crops before leakage filtering. The earlier 501-row dev split leaked duplicate
image/sample hashes against train, so do not tune on
`limbu-devanagari-cdc-line-crop-corpus-2026-06-21`. The dev-clean corpus first
removes train/dev rows that overlap the held-out eval split by image hash, sample
hash, or text hash, then removes dev rows that overlap train by the same keys. It
leaves 5,551 train, 258 dev, and 31 sealed eval rows across 254/24/3 pages. Corpus
pack SHA-256: `36f10b8b123b792f643e206ca59e3bfd9ce03d5357b5de7ed1cdaba2b92145d1`;
eval split SHA-256 is unchanged:
`6d76ab654a0c210a1291f397f2274878d116713a91010ccc91a62cf892ea09b0`.
`audit-train-dev-limbu-specific` passes with leakage check `pass` and real train/dev
rows; `audit-train-dev-vs-heldout-eval` also passes for the combined train+dev
manifest against the 31-row eval split. The held-out eval Tesseract baseline from the
identical eval split remains codepoint CER `0.852632`, grapheme CER `0.857143`, exact
line accuracy `0.0`, and the corpus eval legacy-target analysis again says
`legacy_string_ocr_shortcut_supported=false`.
- A portable PP-OCRv5-mobile Devanagari-written Limbu training bundle now exists at
`runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21`. It packages
the dev-clean corpus train/dev manifests and images, the PP-OCRv5 mobile recipe, and
a passing Limbu-specific corpus audit. Archive:
`limbu-devanagari-cdc-ppocrv5-mobile-devclean-training-bundle.tar.gz`, SHA-256
`01b3689603309c13cb31ef31ff7dc0def65b556a92269d33483ee11da6e8c14d`.
Bundle manifest SHA-256:
`95322aa852b14662e22c53ddbc1ae8346f37a51a7d1ae8348ec4f7a5fbf8da9f`.
The archived YAML was verified to use portable paths under
`training-recipes/ppocrv5-mobile-devclean/` and `bundle_assets/images/`; run command
after extraction is
`python PaddleOCR/tools/train.py -c training-recipes/ppocrv5-mobile-devclean/paddle_recognizer.yml`.
- A train/dev-only CPU smoke for that bundle now exists at
`outputs/limbu-devanagari-cdc-ppocrv5-training-smoke-2026-06-21`. It was prepared by
`tools/prepare_paddleocr_training_smoke.py` with 32 train rows and 8 dev rows,
`sealed_eval_used=false`, `epoch_num=1`, `batch_size=2`, `num_workers=0`, and empty
`pretrained_model` to avoid a network/cache-dependent smoke. The PaddleOCR run
completed on CPU under `/tmp/ocrtech-paddle311/bin/python` plus `/tmp/PaddleOCR-v3.7.0`,
saw 16 train iterations and 4 validation iterations, and saved `best_accuracy`,
`latest`, and `iter_epoch_1` checkpoints. `tools/audit_paddleocr_training_smoke.py`
reports `audit_passed=true`, `issues=0`, and `sealed_eval_used=false`; smoke summary
SHA-256 is `998ef85358e1858c22bac5966ad00290c2e1809b5d1497d61bea38b94ebc73e7`,
smoke audit SHA-256 is `811950bacfb19bcc936fc84ed89cb617a6732aaa6ef6529cef19acfac37284a6`,
and best checkpoint SHA-256 is
`3b41eab11423fa1ac3d53bcffd1c8108f211818a9adb7257bf5de25745115535`. This proves the
bundle can train from train/dev without touching the sealed eval split; it is not
quality evidence and should not be admitted.
- The eval recipe registry now has a sealed held-out admission slot for this branch:
`limbu-devanagari-cdc-ppocrv5-devclean-heldout` in
`configs/eval-harness-recipes-v1.json`. It points at the 31-row eval split from the
dev-clean corpus, uses `train-manifest.jsonl` (train+dev) only as the leakage source,
and validates structurally with zero issues. It is intentionally not claim-ready:
prediction command and checkpoint are unresolved, GT is
`curriculum_text_unreviewed`, the pack is not frozen, and row-level claim eligibility
is false. After training, fill in the checkpoint/model-card/prediction command and
run `tools/run_eval_recipe.py` against this recipe; do not tune on this eval split.
- The Devanagari branch now has an executable training/admission guardrail:
`configs/limbu-devanagari-ppocrv5-admission-plan-v1.json`, validated by
`tools/validate_recognizer_training_admission_plan.py`. The current validation output
at `outputs/limbu-devanagari-ppocrv5-admission-plan-validation-2026-06-21` reports
`plans=1 valid=True training_ready=True heldout_admission_ready=0`, JSON SHA-256
`6ecc4001c7a278aeaa1a3bf6cb3415fbc5c57b66a897ae29b38a37b0c7d1aa19`. It checks the
training bundle archive and manifest hashes, exact train/dev/eval counts, no
train/dev/eval overlap by sample/image/text hashes, train+dev leakage-source identity,
and that checkpoint selection is restricted to train/dev while eval is explicitly
forbidden. Run this validator before any GPU training run and again before filling in
the held-out admission recipe.
- A concrete full-training launcher for that guardrail now exists at
`outputs/limbu-devanagari-cdc-ppocrv5-training-job-2026-06-21`, prepared by
`tools/prepare_recognizer_training_job.py`. It writes
`recognizer-training-job.json` (SHA-256
`b241f1f0bb7a9011429598fd3c92e5aaffced5a1af6bcb796a81bf5927b611d8`) and executable
`run-training.sh` (SHA-256
`06a2787a34a1da197c78242c4d05c9b318322789de453af08c54f3b03be56831`). The launcher
extracts the portable bundle into `workspace/` and runs PaddleOCR with
`/tmp/ocrtech-paddle311/bin/python` plus `/tmp/PaddleOCR-v3.7.0`. The job manifest
records `training_ready=true`, `heldout_admission_ready=false`, 5,551 train rows,
258 dev rows, and the 31-line sealed eval split with `used_for_training=false` and
`archive_member_hits=[]`. One warning is expected: the training bundle manifest
records the passing corpus audit, but the audit JSON is not itself packaged inside
the tarball. This launcher is not a trained checkpoint and is not admission evidence.
- The full-training launcher now has a runtime pre-training preflight:
`tools/preflight_recognizer_training_job.py`. The real run at
`outputs/limbu-devanagari-cdc-ppocrv5-training-runtime-preflight-2026-06-21`
extracted the bundle into an isolated audit workspace, parsed the packaged PaddleOCR
YAML, verified the dictionary and checkpoint directory path, checked all train/dev
label rows against extracted crop files, rechecked sealed eval hashes and archive
exclusion, and imported Paddle/PaddleOCR with the recorded runtime. It reports
`ready_to_start_training=true`, `issues=0`, `warnings=0`, train rows `5551`, dev rows
`258`, missing images `0`, malformed label rows `0`, Python `3.11.13`, Paddle
`3.3.1`, and PaddleOCR `3.7.0`. JSON SHA-256:
`417b8fcb177edd038fe6739a50311656ef2e2930ef8b0ec910361b8b75594289`; Markdown
SHA-256: `fdd24527d29f5576709427622f3c8b092461bb7ce2c77cc9533e971889392686`.
This is a launch-readiness gate only; it does not replace the post-training
completion audit or held-out admission.
- A current-host Devanagari training launch is now running. The launcher was prepared
by `tools/prepare_recognizer_training_runtime_launch.py` at
`outputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21`.
Because this machine is CPU-only (`compiled_with_cuda=false`), it records explicit
overrides: `Global.use_gpu=False`,
`Global.save_model_dir=training-recipes/ppocrv5-mobile-devclean/checkpoints-current-host-cpu`,
`Train.loader.num_workers=0`, and `Eval.loader.num_workers=0`. Launch manifest
SHA-256: `c56e9211058ff0df87e595fd926b607057339bd96b466468779c13c3eb90d27b`;
launcher SHA-256: `1c58b3815ae1fc3e66026c7c51d6c81948749243d93a4d64afa3607dec77e4fd`.
It is running in tmux session `limbu_devanagari_ppocrv5_training_20260621`; verified
progress reached `epoch: [11/20], global_step: 7000` with CPU ETA around `7:33:52`.
Use `tmux attach -t limbu_devanagari_ppocrv5_training_20260621` to inspect it, or
tail `outputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21/training.log`.
This is still not a completed checkpoint or admission result.
- A structured training status monitor now exists:
`tools/monitor_recognizer_training_run.py`. The current snapshot at
`outputs/limbu-devanagari-cdc-ppocrv5-training-status-2026-06-21` reports
`status=running`, `ready_for_completion_audit=false`, `issues=0`, `warnings=0`,
latest progress `epoch: [11/20], global_step: 7000`, `acc=0.375`,
`norm_edit_dis=0.778184`, `ips=2.44833`, ETA `7:33:52`, latest eval
`acc=0.20542634862688572`, `norm_edit_dis=0.6081954332798136`, best eval
`acc=0.20542634862688572`, `norm_edit_dis=0.6081954332798136` at epoch 11,
and `next_action=continue_monitoring_training`.
The monitor now records training-log age against a 900-second stale threshold; this
snapshot has `warnings=0`. Training is still active, so do not run
completion/export/admission yet. JSON SHA-256:
`d0c2a2bebe9a6a9d1bd5938092247e9ff968714b8708d16d49735943930f4a50`; Markdown
SHA-256: `9d991c83d98ad85934e05954a88d9761cfd6773c9c232b343456410f3ce20b8c`.
Re-run it with
`uv run python tools/monitor_recognizer_training_run.py --launch-manifest outputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21/recognizer-training-runtime-launch.json --root /Users/cdjk/github/llm/ocr-tech --out outputs/limbu-devanagari-cdc-ppocrv5-training-status-2026-06-21 --parse-last-lines 4000`.
- The post-training completion gate is now explicit:
`tools/audit_recognizer_training_completion.py` audits the completed launcher
workspace, checks the selected `best_accuracy` checkpoint, and only then writes the
export/package/sealed-eval commands plus a resolved `runner.type=model_config`
held-out recipe. The current real audit at
`outputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-2026-06-21` is red:
`ready_for_export=false`, `issues=5`, JSON SHA-256
`3fe15749a5c2a83ce9a764c8cc98f9cfe8b107b33d7eb2be728cf831346ef516`. The issues are
expected before the long training run: no extracted/trained workspace, no config in
workspace, no checkpoint, and no dictionary path from config. After running
`outputs/limbu-devanagari-cdc-ppocrv5-training-job-2026-06-21/run-training.sh`, run
this audit again before exporting, packaging, or touching the sealed eval recipe.
- A generic Devanagari `deva-v2` checkpoint-export/package/eval record now exists at
`outputs/limbu-devanagari-primary-candidate-deva-v2-export-preflight-2026-06-21`.
It was generated from `outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz`
(archive SHA-256
`7e81660674468f99b2940224fe5ef3e7c9d062d6a05fd3cf4dabf52ac9b4e1df`), with extracted
checkpoint SHA-256
`ddea1db34e89e2c1cdde53d5364eb9156b26dbe8e0a5e67db4c26dd9c97e6676` and corrected
export recipe SHA-256 `6e18a8cfcf8ebab247be910f9566b30327f10dcd3aef2270ec3979960f7b86c3`.
`/tmp/PaddleOCR-v3.7.0` was restored at commit
`b03f46425e8ff4442b268ce449e3eef758146cd4`; export required installing the source
tree requirements into `/tmp/ocrtech-paddle311` and overriding
`Global.character_dict_path` plus `Global.use_gpu=false`. The packaged candidate
model-card SHA-256 is `0c41a9811484349805cd608410da3626b795c1601a7ba014c0e23bca69c0efd0`
and model-card audit passes. It predicts the 31 CDC held-out rows with 0 runtime
failures but scores codepoint CER `0.835628`, grapheme CER `0.858748`, exact-line
accuracy `0.0`, and `claim_safe=false`. The score bundle audit passes with zero
issues and `claim_safe=false` at
`outputs/limbu-devanagari-primary-candidate-deva-v2-export-preflight-2026-06-21/score-cdc-heldout-audit`
(SHA-256 `8a538059a0320902196b3a1125edd97b34f21cb9afb4f32826f24c4c08135edf`).
This is a failed generic baseline, not a Devanagari primary. The same-register CDC
PP-OCRv5 training bundle remains the next serious Devanagari path.
- Sirijonga page materialization now has an explicit reading-order switch.
`tools/materialize_page_image_line_boxes.py` accepts
`--reading-order top_to_bottom|column_major`; default remains the historical
top-to-bottom row-major ordering, while `column_major` orders detected line boxes down
each detected column before moving right. Unit coverage exists in
`tests/test_materialize_page_image_line_boxes.py::test_detect_line_boxes_can_order_two_column_pages_column_major`.
`tools/compare_limbu_sirijonga_detector_reading_order.py` now compares these modes
against reference line boxes without invoking a recognizer. On the current real
Sirijonga drafts it is neutral, not a win: Gorkhapatra tuned morphology stays at
matched-line reading-order pair accuracy `0.606543` for both modes, and Sikkim stays
at `0.992402` for both modes. Do not promote `column_major` as a default; use the
comparison tool to test future detector variants.
- Sirijonga detector tuning now has a parameter-sweep tool:
`tools/sweep_limbu_sirijonga_detector_params.py`, covered by
`tests/test_sweep_limbu_sirijonga_detector_params.py`. It ranks morphology configs
without OCR recognition. On Gorkhapatra, the sweep
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/detector-param-sweep-v1`
evaluated 1,680 configs and improved detector-only line F1 from `0.833866` to
`0.844660` by using `horizontal_kernel=23`, `vertical_kernel=3`, `max_height=80`,
`min_area=50`, `bbox_source=ink`, `threshold=otsu`, and `top_to_bottom`. It still
matches only `261/303` lines and leaves matched-line reading-order accuracy at
`0.606543`. Because the 315 candidate boxes are a strict subset of the previous
323-line tuned materialization, `tools/filter_page_image_line_box_documents.py`
reused existing OCR outputs with `--max-height 80` and scored them at
`score-image-line-detector-v4-tuned-maxheight80-filtered`; codepoint CER stayed
`0.763835` and grapheme CER moved slightly worse from `0.828226` to `0.828392`.
Treat this as a geometry cleanup, not a text-quality win.
- `tools/analyze_page_line_failures.py` now joins `line-alignment.csv` with an
image-line crop manifest and buckets matched lines by geometry/text quality. On the
Gorkhapatra tuned image-line score, the diagnostic output
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-image-line-detector-v4-tuned`
reports `138/261` matched lines as high-IoU (>=`0.85`) but bad-text, compared with
only `11` low-geometry bad-text matches. The high-IoU bad-text bucket has mean IoU
`0.893282`, mean recognizer confidence `0.445911`, mean codepoint reference length
`45.17`, mean crop width `330.88` px, mean crop aspect ratio `6.73`, and mean
grapheme CER `0.678634`. Width bands confirm the recognizer-width signal: <=256 px
matched crops have codepoint CER `0.132605` and confidence `0.805627`, while 769+ px
crops have codepoint CER `0.412653` and confidence `0.219722`.
- `tools/prepare_limbu_sirijonga_width_ablation.py` now prepares the next real
Sirijonga measurement instead of another detector tweak. The Gorkhapatra ablation
pack is
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/recognizer-width-ablation-v1`:
233 high-IoU matched crops, band counts `{"000-256": 20, "257-384": 202,
"385-512": 2, "513-768": 4, "769-plus": 5}`, and model-card variants for
`[3,48,320]`, `[3,48,480]`, `[3,48,640]`, `[3,48,960]`, and `[3,48,1280]`.
`/tmp/ocrtech-paddle311/bin/python` has been restored with `paddle` 3.3.1 and
`paddleocr` 3.7.0, and `run-width-ablation.sh` has scored all variants with zero
prediction failures. Width 480 is best on this non-claim diagnostic pack: codepoint
CER `0.100651`, grapheme CER `0.169567`, exact line accuracy `0.278970`. Width 320
scores codepoint CER `0.269910`; width 640 is close but worse at `0.108741`; width
960 is `0.212755`; width 1280 collapses to `0.442740`. The width-480 page
materialization at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/image-line-box-documents-v4-tuned-width480`
proves the setting helps line recognition but is not enough for page OCR:
matched-line codepoint CER improves from `0.285785` to `0.105977`, matched-line
grapheme CER improves from `0.551916` to `0.178678`, but full-page codepoint CER
regresses slightly from `0.763835` to `0.770900` and matched-line reading-order
accuracy stays `0.606543`. A layout-specific `wide_column_right_to_left` ordering
diagnostic improves Gorkhapatra matched-line reading-order pair accuracy from
`0.606543` to `0.763985` with unchanged line F1 `0.833866`, but it regresses the
Sikkim draft from `0.992402` to `0.753361`, so it is a selectable layout mode, not a
global Sirijonga default. Gorkhapatra width-480 page materialization with that order
scores full-page codepoint CER `0.761228` and grapheme CER `0.803876`, a small
one-page improvement over width-480 top-to-bottom (`0.770900` / `0.804207`) while
matched-line codepoint CER remains `0.105977`. `tools/filter_page_ocr_documents.py`
now adds non-oracle live-line pruning by empty text, confidence, script, and geometry.
`tools/sweep_page_ocr_line_pruning.py` now wraps that filter with page eval and
paired comparison, so threshold selection is auditable. The latest pass added
`tools/audit_page_ocr_line_filter.py`; it replays filtered documents from the source
OCR documents and checks line reports, source hashes, and kept/removed counts. The
pruning sweep now runs this audit for each threshold candidate. On Gorkhapatra width-480
wide-RTL output, the automated sweep at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/ocr-line-pruning-sweep-width480-wide-rtl-v1`
improves page codepoint CER through threshold `0.85`: unpruned `0.761228`, 0.50
`0.746678`, 0.60 `0.744996`, 0.70 `0.739697`, 0.80 `0.738436`, 0.85 `0.730362`;
0.90 over-prunes and regresses to `0.760387`. The 0.85 candidate keeps 199/323
lines, removes 124, replay-audits cleanly, and is still one-page non-claim evidence. Sikkim is the
counterexample: its automated sweep at
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/ocr-line-pruning-sweep-image-line-v4-v1`
shows 0.85 pruning hurts page codepoint CER from `0.460255` to `0.694308`, while
mild 0.70 pruning improves it to `0.444063` with 107/125 lines kept and replay-audits
cleanly. `tools/admit_page_ocr_line_pruning_threshold.py` now combines these sweeps,
admits only thresholds that are common across pages, audit-clean, and non-regressing
under both codepoint and grapheme CER, and records the exact per-page documents,
score, comparison, and audit hashes for the admitted threshold. The real run at
`outputs/limbu-sirijonga-pruning-threshold-admission-2026-06-21` selects `conf070`:
Gorkhapatra improves codepoint CER by `0.021531`, Sikkim improves by `0.016192`, and
`conf085` is rejected because it regresses Sikkim by `-0.234053` codepoint CER.
JSON SHA-256:
`cc8b0b8986c273bafbc73fc74ffc706dd28eeb93343c921aa4572411aae06429`; Markdown
SHA-256: `9e8672e71d58ae501ddd7024acf34a074ead09b12627cf81f1bff6e3b398b6e5`.
`tools/audit_admitted_page_ocr_candidate.py` now audits residual failures for that
admitted candidate. The real `conf070` residual audit at
`outputs/limbu-sirijonga-conf070-residual-audit-2026-06-21` reports 296 matched
lines, 93 unmatched references, 51 unmatched predictions, 13 near-miss unmatched
references at IoU >= 0.30, 0 high-IoU unmatched references at IoU >= 0.50, and 26
matched high-text-error rows. Gorkhapatra is now mostly recall-limited: 238 matched,
65 missed references, 2 prediction-only lines, precision `0.991667`. Sikkim remains
both recall- and false-line-limited: 58 matched, 28 missed references, 49
prediction-only lines, precision `0.542056`. The refreshed audit also replays the
admitted per-page score/comparison/filter artifact hashes and reports
`artifact_integrity_passed=true`. JSON SHA-256:
`28b0edcb15eaf4a3860e2560872ecd419ad5226f88964bf04feaa58104149f64`; Markdown
SHA-256: `178ce581a9829294da482916880ce9a19df0e103a4f272a4b7b7e11ab44e2516`.
The refreshed audit also buckets residual geometry: Gorkhapatra misses are mostly
short references (51/65) centered across middle/bottom body regions, while Sikkim
has medium-width missed references (22/28, bottom-heavy) and many short
prediction-only fragments (47/49).
Next step:
keep width 480 for Sirijonga line recognition, use `conf070` as the current
cross-page operational pruning candidate, then handle Gorkhapatra missed-line recall
and Sikkim false-line/layout classification separately. Do
not choose 960/1280, split lines, or hard-code a confidence threshold before
addressing the page assembly failure.
Latest follow-up: `tools/filter_page_ocr_documents.py`,
`tools/audit_page_ocr_line_filter.py`, and `tools/sweep_page_ocr_line_pruning.py`
now support page-relative width/height ratio filters. The real `min_width_ratio=0.05`
sweep pair lives under each Sirijonga draft page directory as
`ocr-line-pruning-sweep-minwidth005-v1`. Cross-page admission at
`outputs/limbu-sirijonga-pruning-threshold-admission-minwidth005-2026-06-21`
selects `conf070`, improving Gorkhapatra codepoint CER to `0.739361` and Sikkim to
`0.423454`; JSON SHA-256
`808a2e08fd853fbc1c612b398e37fe4903808a141a642b82d3338a3f64e632e4`. Residual audit
at `outputs/limbu-sirijonga-conf070-minwidth005-residual-audit-2026-06-21` shows
the tradeoff: prediction-only rows fall from 51 to 28 and Sikkim precision rises to
`0.682927`, but missed references rise from 93 to 97 and Sikkim recall drops to
`0.651163`; JSON SHA-256
`969868c2eac5decc1a2a4a82848f0c051adbd04c5092d7d010b31698b818e2f5`. Treat it as
precision-oriented review triage, not complete page OCR.
Operational replay of that admitted recipe now exists at
`outputs/limbu-operational-image-line-current-2026-06-21/gorkhapatra-width480-minwidth005-conf070`:
323 detected image lines, 238 kept, 85 removed, and 39 review rows. The
capture/Sirijonga audit passes, but the strict final/no-dropped-lines audit fails
with `39` unresolved review rows and `85` dropped rows, which is the correct
behavior for a triage output. Exact page-eval scoring of this operational output is
in
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/score-operational-image-line-width480-minwidth005-conf070`:
codepoint CER `0.7393608074011775`, grapheme CER `0.7682623819778035`, metrics
SHA-256 `14d0da1bf80e5afb876adfa6191501f2f18039dc86014123d7cf0d64551c4176`, audit
SHA-256 `37c6d0e0efb74d3c61b7a5bd9f3ce5bf4caf0487d47d96ba1f42e7aabfb8b6a9`. The
eval audit now passes structurally but remains non-claim: the pack is unfrozen, one
page, machine-converted/unreviewed, and leakage checking against
`recognizer-width-ablation-v1/manifest.jsonl` records 233 checked rows with
`source_sample_id` overlap on `gorkhapatra-limbu-sirijonga-0001-01`.
- `tools/analyze_page_line_failures.py` now also reports unmatched-reference and
prediction-only geometry plus best unmatched near-miss pairs. It also reports
attributed codepoint error mass by source and now joins operational image-line crop
manifests that have line ids but no `sample_id`. The current operational
Gorkhapatra width-480 unpruned analysis at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-operational-width480-unpruned-v4`
has 261 matched rows, 42 unmatched references, 62 unmatched predictions, and only 15
high-IoU bad-text matches. Only 5/42 unmatched references have any unmatched
prediction at IoU >= 0.30, and 0/42 have one at IoU >= 0.50. The dominant error
source is still `missed_reference_text` at 1,416/2,919 attributed codepoints (0.485).
The v4 overlap diagnostic checks each unmatched reference against every predicted
box, including already matched predictions: 28/42 have no overlapping prediction at
all, 14/42 best-overlap a prediction-only row, 5/42 reach IoU >= 0.30, and 0/42
reach IoU >= 0.50. This rules out matched-line swallowing as the primary cause.
The same center-band pattern is present without pruning: 31/106 center references are
unmatched with 1,116 missed-reference codepoints, while left is 3/97 and right is
8/100; top combined bands are middle/center (20/42, error mass 810), top/center
(6/19, error mass 233), and middle/right (5/38, error mass 163). Result SHA-256 is
`f540fe59bf23b4f9b81c67b8e21f6827a563336d999781132ca4a2b23732c4a0`; report
SHA-256:
`a51ddce081e8d0989ce13bbaa90a575aa4887a5e69a2b70c0745f8d7d0c7b467`. The refreshed `minwidth005-conf070`
analysis at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-operational-width480-minwidth005-conf070-v4`
has 236 matched rows, 67 unmatched references, 2 unmatched predictions, and only 2
high-IoU bad-text matches; the dominant codepoint error source is
`missed_reference_text` at 2,422/2,888 attributed codepoints (0.839). Its overlap
diagnostic is even stricter: 65/67 unmatched references have no overlapping
prediction at all, and only 2/67 reach IoU >= 0.30. The analyzer also reports
missed-reference spatial bands: the center horizontal band has 42/106
references unmatched and 1,757 missed-reference codepoints, versus left 9/97 and
right 16/100; the top combined miss bands are middle/center (20/42, error mass 810),
bottom/center (14/45, error mass 664), and top/center (8/19, error mass 283). Result
SHA-256: `1da537b596b8c6e2b010998fc183289a8e632c75237d509c783b3977ba9e1c65`;
report SHA-256:
`d9fef331d97b94877f39b02b8ddf29445d97e96d44cfcaf16e93f6b15ba993ba`. Lowering
matching thresholds or splitting matched boxes is not the main rescue. A direct pixel
check of the top unmatched center-column references showed the text is visible in the
prepared image; the failed rows sit inside a rejected Otsu/morphology connected
component at `[995,401,326,1404]`, so the failure is over-tall component rejection,
not missing ink.
- Adaptive thresholding was tested as a bounded detector-only branch and rejected for
now. The best small probe, `adaptive_mean:41:11` with the operational 23x3 morphology
settings, raised matches from 261 to 263 but produced 320 predictions and F1
`0.844302`; as a rescue pass on the previous `max_height=80` tuned route it still
lowered F1 below the no-rescue `0.844660` baseline. Do not wire adaptive thresholding
as a default without stronger cross-page evidence.
- The useful Gorkhapatra recall fix is tall-component splitting. `src/ocrtech/image_lines.py`,
`tools/materialize_page_image_line_boxes.py`, and
`tools/compare_limbu_sirijonga_detector_reading_order.py` now expose opt-in
`split_tall_components` / `--split-tall-components` with row-projection settings
`split_tall_row_min_ink=2` and `split_tall_max_row_gap=4`; split rows carry
`detector_source_pass=primary-split-tall`. Detector-only comparison at
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/compare-detector-split-tall-maxh80-v1`
uses Otsu, ink boxes, 23x3 morphology, `max_height=80`, detector padding 2, and
tall splitting. It reaches 285/303 matches, 339 predictions, precision `0.840708`,
recall `0.940594`, F1 `0.887850`, mean IoU `0.869405`, and wide-RTL matched
reading-order pair accuracy `0.712824`. Result SHA-256:
`06c242545dce5462dcfb25773b71045c1c3cde026843751d62d2c3a7f44a07c3`; report SHA-256:
`f9dd7e679f733937a5b793e45e38eb02394c8b3fb80fc3f8eb039b3ac770e82c`;
alignment TSV SHA-256:
`44feb8ba76840f85b008c0a1c773b3a033d1b11637e3f002ceebcdab23ae7414`.
Full OCR materialization has now been run and rejects split-tall as an admitted page
route by itself. `image-line-box-documents-v4-split-tall-width480-wide-rtl` was
materialized with the v4 width-480 line recognizer, `recognition_mode=line`, 339
lines, and 0 empty predictions. Run SHA-256:
`80808cefbc3804b0385a6c2a2fe16522ebe1a7c307b9447210006905d1712c96`; crop manifest
SHA-256: `7e6b62b25ef17687897034ed70eb1bf09c3d50a407d8a786707825cbd4a99cc5`;
document SHA-256:
`da1a1664aee6beb1381ed338f884033c44e5c3a17d9bc9b610f86f778b97563b`. Scoring at
`score-image-line-detector-v4-split-tall-width480-wide-rtl` gives page codepoint CER
`0.791421` and grapheme CER `0.831042`, worse than the non-split wide-RTL score
`0.761228` / `0.803876`; metrics SHA-256:
`db52ef81012890dbe77179b9deadb72adb04f90c9228cffd2b68860ae5902b18`, run SHA-256:
`33977edad6ff145a24891ecfda9b990f0895074c5043ef902a4ba46f92c8e990`.
The paired comparison at
`compare-split-tall-width480-wide-rtl-vs-tuned-width480-wide-rtl` reports negative
absolute improvement: codepoint `-0.030193`, grapheme `-0.027166`; comparison
SHA-256 `86906240daae771ba960cbd725b19210bde3eb08d4a2d3c4d95e37ca9610835e`.
The failure shape changed in the right direction but exposes the next bottleneck:
line-failure analysis at `line-failure-analysis-split-tall-width480-wide-rtl-v1`
shows 285 matches, 18 unmatched references, 54 unmatched predictions, and
`missed_reference_text` error mass falls to 425, but page assembly is now dominated
by matched bad text/extra prediction text and worse reading order (`0.712824` vs
prior wide-RTL `0.763985`). Analysis SHA-256:
`b3d8266031538bce0c63f38f6db4e2f485ac2e18fe26f9fcda4bc3408f05524d`; report
SHA-256 `eef5b226a0f60f203501a6c0c2ab48166fd5e159bf91f559df70a4f5c79a64e0`.
Pruning without fixing order does not rescue split-tall enough: the split-tall confidence sweep at
`ocr-line-pruning-sweep-split-tall-width480-wide-rtl-v1` has best threshold `0.85`
with codepoint CER `0.740875` and grapheme CER `0.768759` (sweep SHA-256
`085765fe1264ec1b45e879cf2677539eb1436f208a5a2e6a01e9b88cea1d1e5f`), still worse
than the older non-split Gorkhapatra-only conf085 result `0.730362` / `0.763956`.
The clean comparison at `compare-split-tall-conf085-vs-tuned-width480-wide-rtl-conf085`
gives codepoint improvement `-0.010513` and grapheme `-0.004804` (comparison SHA-256
`4cc4b640ca0ab5e1c6beaac2badeae0f1acf1378674b3c4ccc9a8d67eda92068`).
A source-pass-aware ordering mode was added:
`wide_column_right_to_left_split_tall_last`. It keeps the existing wide-RTL order
for primary boxes and places `*-split-tall` boxes after primary boxes. Detector-only
comparison at `compare-detector-split-tall-wide-rtl-last-v1` improves split-tall
matched-line reading-order accuracy from `0.712824` to `0.765654` with the same
285/303 matches and F1 `0.887850`; comparison SHA-256:
`cce99e017e0e3565c50e56d8886588910d567e94685d8f302e703f155347c636`. Full OCR at
`image-line-box-documents-v4-split-tall-width480-wide-rtl-last` scores codepoint CER
`0.778638` and grapheme CER `0.820938`, better than plain split-tall but still worse
than non-split wide-RTL before pruning. With the same non-oracle conf sweep,
`ocr-line-pruning-sweep-split-tall-wide-rtl-last-v1` reaches best conf085 codepoint
CER `0.730614` and grapheme CER `0.763624` (sweep SHA-256
`869381446576317d8258f36557bec4e7da0171e3f613ada59f4c8e44a9b3ca6f`). Against the
previous non-split conf085 Gorkhapatra result, the clean comparison at
`compare-split-tall-wide-rtl-last-conf085-vs-tuned-width480-wide-rtl-conf085` is a
near-tie: codepoint `-0.000252`, grapheme `+0.000331`; comparison SHA-256
`cfbb25555feab3af1d4863f4f654efd11b94bb18dde047b0b9d2890a330a71f9`. Sikkim
cross-page detector validation now rejects global admission. Without splitting,
`compare-detector-split-last-no-split-v1` keeps 59/86 Sikkim matches and F1
`0.559242`, but drops matched-line reading-order accuracy from `0.992402` to
`0.753361` (comparison JSON SHA-256
`014bdc6c9fd08d7f82b8292cdcfe0e0a6a08dd2a2a6a6e3a79024e51d7acc813`). With
`--split-tall-components`, `compare-detector-split-tall-last-v1` adds one prediction
without adding matches, lowering F1 to `0.556604` and preserving the same ordering
regression to `0.753361` (comparison JSON SHA-256
`77d2bd30cfc2ed0a2827704c0bf21b51ec3698af3aedb1cc3459382530a9b6fc`). Full OCR
materialization was intentionally skipped because the detector-only Sikkim gate
already fails. Keep `wide_column_right_to_left_split_tall_last` as a
Gorkhapatra/layout-specific diagnostic, not a global Sirijonga route.
- A safer gated branch now exists: `auto_layout_split_tall_last`. It detects once
without tall splitting, uses the existing auto-layout features, and only reruns
tall splitting plus split-last ordering if the page qualifies as a wide-column
layout. Detector-only Gorkhapatra output at
`compare-detector-auto-layout-split-tall-last-v1` keeps 285/303 matches, F1
`0.887850`, and raises matched-line reading-order accuracy from `0.712824` to
`0.765654` (comparison JSON SHA-256
`6364dd87e617b3b76236ea37d71b08fab216d5c5ffb8a3b08d6b03560581237b`). Sikkim output
at its matching `compare-detector-auto-layout-split-tall-last-v1` selects
top-to-bottom, keeps 59/86 matches, avoids the extra split false-positive, improves
F1 back to `0.559242`, and preserves matched-line reading-order accuracy at
`0.992402` (comparison JSON SHA-256
`5aa325d176af4a85dda65a3e3684ae9c80bc7c9a7485e0cb7e0729f8fddf20e7`). Full OCR was
also materialized and scored. Gorkhapatra
`image-line-box-documents-v4-auto-split-tall-width480` emits 339 lines, selects
`wide_column_right_to_left_split_tall_last`, scores codepoint CER `0.778638` and
grapheme CER `0.820938`, and exactly matches the explicit split-last score
(`compare-auto-split-tall-vs-split-tall-wide-rtl-last` JSON SHA-256
`9326ef4b2e33964d985c734b34ba824977658887ee3ee6f61f7ab8cb724c3b61`). Sikkim
`image-line-box-documents-v4-auto-split-tall` emits 125 primary lines, selects
`top_to_bottom`, scores codepoint CER `0.460255` and grapheme CER `0.576710`, and
exactly matches the existing image-line baseline (`compare-auto-split-tall-vs-image-line-v4`
JSON SHA-256 `50b845f80d1212eb2c751ba43070880602c70c506f6682e9f8e1ca0266802e7e`).
Do not admit this as final OCR policy yet: it is a safer candidate branch, but
unpruned Gorkhapatra page CER remains worse than the non-split wide-RTL route, and
all current page evidence is still draft, one-page, and not claim safe.
- The gated branch has now also been tested through the pruning/admission layer.
Gorkhapatra `ocr-line-pruning-sweep-auto-split-tall-minwidth005-v1` has a local
best at conf085, codepoint CER `0.730614` / grapheme CER `0.763624`, but Sikkim
conf085 regresses to codepoint CER `0.702650` / grapheme CER `0.731978`.
Cross-page admission at
`outputs/limbu-sirijonga-auto-split-tall-pruning-threshold-admission-2026-06-21`
selects conf070 instead; admission JSON SHA-256
`2bc16296a6295a03482ef30a01729c5ae7b37278c6623ca38d6431d86a3665ed`. The selected
gated conf070 scores Gorkhapatra codepoint CER `0.753406` / grapheme CER `0.782508`
and Sikkim codepoint CER `0.423454` / grapheme CER `0.511091`. Direct paired
comparison against the already-admitted non-split minwidth005 conf070 route rejects
promotion: Sikkim is tied, but Gorkhapatra regresses by `0.014045` codepoint CER and
`0.014245` grapheme CER. Comparison JSON SHA-256:
`f26b0911a4ba92bce47ea53bee0af6f88a5ef93f61f6361586daacd74d814b70`. Keep
auto-split-tall as a recall diagnostic/alternative branch, not the admitted
operational route.
- Same-row component merge is now implemented as an opt-in detector stage in
`src/ocrtech/image_lines.py`, `src/ocrtech/pipeline.py`, `src/ocrtech/cli.py`,
`tools/materialize_page_image_line_boxes.py`, and
`tools/compare_limbu_sirijonga_detector_reading_order.py`. Defaults are disabled;
the tested candidate uses `y_tolerance=8`, `max_gap=120`,
`max_center_delta=300`, `max_width=420`. Detector-only F1 improves Sikkim to
`0.875000` (`90` preds, `77/86` matches; JSON SHA-256
`379960a430a943e6076499eff5a744e084f736134a2e0579ad900e97e9dd7aa8`) and
Gorkhapatra to `0.875000` (`289` preds, `259/303` matches; current-schema JSON
SHA-256 `1b2fdcdb08375ac5896487b66711787e5c59a7566dc88b60fe6e396ca2903421`).
Full OCR plus the same `drop_empty`/Sirijonga/`min_width_ratio=0.05` sweep selects
`conf070` at
`outputs/limbu-sirijonga-same-row-merge-pruning-threshold-admission-2026-06-21`
(JSON SHA-256 `970a1ad210423d430687d1666f190aa1c4795f516e9998bc4c544a086e45d32f`).
Do not globally promote it: selected row-merge `conf070` improves Sikkim against
the admitted minwidth005 `conf070` route (`0.412169` vs `0.423454` codepoint CER),
but regresses Gorkhapatra (`0.744239` vs `0.739361`). Keep it as a conditional
fragmented top-to-bottom page repair candidate, not a wide-column operational
default.
- The conditional same-row gate has now been implemented and measured as
`merge_same_row_auto_fragmented_top_to_bottom`. It resolves layout on unmerged boxes,
only applies to pages selected as `top_to_bottom`, and requires at least 8 lines and
18% line-count reduction from the proposed merge. Sikkim auto-gated detector output
stays at `90` predictions / `77` matches / F1 `0.875000` with reading-order accuracy
`0.990089` (JSON SHA-256
`f5a50caaac7ee1eaea2bac4b39917cdb04e5ce8c1c622b620e9dad63e8e5b1f1`);
Gorkhapatra auto-gated output stays on the non-merge wide-column path with `323`
predictions / `261` matches / F1 `0.833866` (JSON SHA-256
`da5d57bb77fd1eb2b1f29a69f8b491623631d24e9a8dfccd06a1e7491cbd0d21`). Cross-page
pruning admission at
`outputs/limbu-sirijonga-auto-same-row-merge-pruning-threshold-admission-2026-06-21`
selects `conf070` (JSON SHA-256
`2784674a4b8ff7382086a9d1c544c25729df5f938a792b21b67d81298c9bfeed`). Direct
comparison against the previous admitted minwidth005 `conf070` route improves
Sikkim (`0.412169` vs `0.423454` codepoint CER) and ties Gorkhapatra (`0.739361`
both). Treat this as the current stronger two-page operational candidate, but not a
claim-safe final policy.
- Operational `parse-limbu-doc --line-detection-mode image-line` now accepts repeatable
`--image-line-rescue-detector-pass key=value,...` plus
`--image-line-merge-iou-threshold` and the disabled-by-default
`--image-line-merge-same-row-*` geometry gates, carries `detector_source_pass` through line
metadata and crop manifests, records detected/kept/removed source-pass counts in
`image-line-ocr-run.json`, and `audit-limbu-output` replays that field and the count
summaries against `document.json` and the crop manifest. Regression coverage:
`tests/test_image_lines.py::test_detect_line_boxes_with_rescue_pass_adds_non_duplicate_components`,
`tests/test_cli.py::test_cli_parse_limbu_doc_passes_post_correction_profile`, and
existing image-line output audit tests.
- `tools/materialize_page_image_line_boxes.py` and
`tools/compare_limbu_sirijonga_detector_reading_order.py` now support auditable
repeatable `--rescue-detector-pass` morphology passes. The pass is merged with the
primary detector by IoU, records `detector_source_pass` on lines/crop rows, and is
exposed to detector-only comparison before spending recognizer time. On the current
Gorkhapatra page, the measured rescue variants are rejected. The broad
`h15,v1,min_width=20` pass raises matches from 261 to 281 and recall to 0.927393,
but predictions jump to 702 and F1 collapses to 0.559204. The best conservative
variant tested, `h41,v1,min_width=250,min_area=250,max_height=80,merge_iou=0.30`,
gives only 264 matches and recall 0.871287 while lowering precision to 0.774194,
F1 to 0.819876, and reading-order accuracy to 0.756596; comparison SHA-256
`088fdca2fe6865f66998bdd10a41908240232a271ce5567a62a4528e7c02c431`. Do not
promote multi-pass morphology without a better precision gate.
- The rescue detector now also supports `min_aspect_ratio` and `max_aspect_ratio`
gates, but the first aspect-gated Gorkhapatra pass is still rejected. The best tested
aspect-gated variant, `h41,v1,min_width=250,min_aspect_ratio=8,merge_iou=0.30`,
has 341 predictions, 264 matches, precision 0.774194, recall 0.871287, F1 0.819876,
and reading-order accuracy 0.756596, below the no-rescue baseline; comparison
SHA-256 `75934afe974a8189b97b18f61c82dde1d5d87bdbf9a540c93c6ae4aac0f75156`.
- A geometry-only Sirijanga line-box review packet now exists at
`outputs/limbu-sirijonga-line-box-review-packet-2026-06-21`. It was built by
`tools/prepare_line_box_review_packet.py` from the current Gorkhapatra and Sikkim
page eval manifests and audited by `tools/audit_line_box_review_packet.py`.
It covers 2 real draft pages and 389 seeded line boxes, with
`line-box-review-audit.json` reporting `audit_passed=true` and 0 issues. The packet
deliberately excludes reference transcription: use it for non-native geometry and
reading-order review only. `tools/apply_line_box_review_packet.py` now ingests
completed geometry reviews into detector/layout references with empty text and
`claim_evidence_eligible=false`; it blocks pending rows by default. Running it on
the current packet correctly blocks because all 389 rows are still `pending`.
Passing this audit does not make the machine-converted Sirijonga text claim-grade.
- The Sikkim detector sweep
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/detector-param-sweep-v1`
evaluated 900 configs and did not beat the current image-line detector family:
best F1 remains `0.559242`, with `125` predictions for `86` references and
`59` matches. Do not spend tomorrow treating a simple morphology parameter tweak or
`column_major` as the Sikkim fix; the Sikkim path needs better segmentation logic,
same-register recognition data, and reference verification.
- The previously referenced `/tmp/ocrtech-paddle311/bin/python` runtime is absent in
this shell. A refreshed full-router preflight at
`outputs/limbu-first-router-v1/runtime-preflight-2026-06-21-current` fails under the
repo `uv` environment because Python is 3.14.2 and both `paddle` and `paddleocr` are
missing. It also now fails `component-admission:devanagari_primary`: the router
model card explicitly records the current Tesseract primary as failed on the CDC
Devanagari diagnostic. Restore a Paddle-compatible Python runtime before claiming
current full Sirijonga router measurements, but do not call the whole router ready
until the Devanagari primary is replaced or re-admitted with evidence.
- Attempted inspection of the remote 4 TB SSD at `cdjk@100.110.148.43` failed with SSH
permission denied (`publickey,password,keyboard-interactive`). Do not assume the
remote has or lacks Devanagari-written Limbu pages until access works.
## Operating rules (hard-won; do not relitigate)
- **`norm_edit_dis` is NOT CER.** PaddleOCR's `norm_edit_dis` is a per-sample-normalized
macro average; on short-line datasets it runs ~4–8× above true micro CER. Always score
with `tools/eval_metrics.py` (micro codepoint + grapheme-cluster UAX#29) fed by
`infer_rec.py`. Harness: `/home/ubuntu/re_score.py` on the GPU instance. **Eyeball the
predictions vs GT before trusting any number.**
- **Synthetic eval is not claim evidence.** Real-crop CER only. Label every number
real vs synthetic.
- **Same-font is the lever.** Real *same-font/same-domain* training data beats model
tweaks. Different-font real data HURTS. Proven 3× now (Limbu, Devanagari, Sunuwar).
- **Labels are machine-converted** (legacy→Unicode or transliteration). A low CER can
mean the model learned the converter's biases, not that the text is human-correct.
Native-reader verification is still owed on all scripts.
- AWS SSO (`aws sso login --profile playaround`) **must be run in a plain terminal**,
NOT via this session's command runner — it auto-backgrounds the device-code poll and
fails with `invalid_grant`.
## Verified scorecard (micro CER, this session)
Real newspaper print — STRONG:
```
Limbu/Sirijonga (native) graph 0.041 cp 0.021 real Gorkhapatra, 303 crops
8 Devanagari (one model) graph 0.075–0.129 real Gorkhapatra
Majhi .075 Magar .078(LEAKED) Sunuwar-Deva .080 Magahi .083
Bhojpuri .101 Thami .120 Wambule .121 Darai .129
Sunuwar (Jenticha, native) graph=cp 0.121 real Herald, 429 crops *** NEW ***
```
Synthetic / weak:
```
Tirhuta (native) graph 0.343 cp 0.250 synthetic
Newa/Prachalit translit graph 0.613 cp 0.418 synthetic (transliterated text)
Newa native retrain PENDING re-score (see Open #1)
```
Blocked (no same-font training data): Lepcha, Kirat Rai (see Open #3).
## What this session did
1. **Metric correction.** The whole scorecard had reported `1−norm_edit_dis` as CER.
Re-scored all models with micro CER → Limbu/Devanagari are ~4–8× *better* than
reported; Newa/Tirhuta confirmed weak. Memory of the gotcha saved.
2. **Newa v4 width experiment.** Hypothesis: 480px squish was crushing long Prachalit
lines. Widened to 640px, disabled RecConAug, retrained. **Verdict: width alone did
NOT rescue Newa** (cp 0.628 at epoch 8, undertrained but plateauing, worse than v3).
Confirms YOUR earlier call: the bottleneck is transliteration/data fidelity, not
pixels. Checkpoint preserved (`runs/newa-v4-wide-2026-06-18/`).
3. **Sunuwar same-font harvest + retrain.** Verified the Sunuwar legacy→Unicode
converter applies uniformly across all 25 Herald editions (87–90% byte-confirmed).
Harvested **3,010 real same-font crops** from 22 editions (`tools/harvest_sunuwar_train_crops.py`),
held the 3 mukhia editions out as the 429-crop eval (clean by-edition split).
Fine-tuned the synth model on them → **CER 0.342 → 0.121 (−65%)**, exact 0.7%→18.4%.
Data-floor thesis confirmed for a third script.
4. **Newa native corpus.** Assembled **4,456 real *native* Prachalit lines**
(`data/real-text/newa-real-native.txt`, from nepal-lipi.com + newa-all-clean +
nepal-lipi-extra) to replace the 463k transliterated salad, rendered (6,776 imgs,
2 fonts), retrained 30 epochs. Micro CER PENDING.
5. **Devanagari→Gurung Khema transliterator** (`tools/devanagari_to_gurungkhema.py`).
Aksharamukha has NO Khema target; map hand-derived from Unicode 16 names + Pandey
L2/22-157 (U+1612F THOLHOMA = virama). Validated 0 dotted-circles on real Gurung
text. Caveat: self-built, NOT native-validated.
## Open questions — CRITICALLY REVIEW THESE
1. **Newa native result is uncaptured.** The retrain runner logged only `norm_edit_dis`
for Newa (no micro-CER re-score — a wiring gap), then the instance auto-stopped.
Mid-training `norm_edit_dis` was 0.239 at epoch 11 on a *native* held-out eval —
which looks WEAK, but that eval is harder/cleaner than the transliterated one, so
it's not comparable. **Need: restart instance, re-score `runs/newa-native/checkpoints/
best_accuracy` with micro CER on held-out native AND on the Zenodo real handwritten
pack.** Is native text actually helping, or is Newa still broken? Don't assume it helped.
2. **Is Sunuwar 0.121 real or converter-noise-limited?** The converter is ~89% byte-
confirmed, so GT has ~11% machine-conversion noise. Model+GT may share that bias.
Needs a native-reader spot-check of ~50 crops to bound true CER. Also: would more
editions / higher byte-coverage push it lower, or is 0.12 the converter's floor?
3. **Lepcha / Kirat Rai are blocked.** Lepcha's shape converter returns 0 on all 26
non-eval editions (different un-decoded font); Kirat Rai's typeface was never cracked
(~70/4,343 lines). Same-font training data isn't available for them without new
font-decoding work. Is that decoding worth it, or are these languages parked?
4. **`magar` selection leakage** (one of 8 Devanagari packs was the deva-v2 selection
set). It sits mid-range so it isn't inflating the headline, but it's not a clean test.
5. **No detector.** All numbers are recognition-only on oracle crops. Page-level OCR
needs a text detector; detector misses are the largest absent error source.
6. **Gurung Khema** is a synthetic-only, self-transliterated artifact with no real eval
— defensible as "first-ever research model" only, NOT a quality claim.
## Pending tasks (priority order)
1. Re-score Newa native (Open #1) — restart, `re_score.py`, micro CER, report.
2. Native-reader label verification spot-checks (Sunuwar, Devanagari, Limbu).
3. Push Sunuwar lower: harvest is capped at 89% byte-confirm; investigate the unmapped
bytes / add more editions; consider mixing synth+same-font.
4. Lepcha/Kirat Rai font decoding (deferred by user).
5. Build a text detector for page-level eval.
6. Newa v5: the real fix is better/native-validated text + real printed-register data,
not shape (v4 proved shape insufficient).
## Infrastructure
```
GPU: i-0fa19dff7b233a353 g6.xlarge us-east-1 profile=playaround (SSO)
STATE: stopped (EBS preserved). g6 capacity is intermittent —
start may need retry on InsufficientInstanceCapacity.
SSH: ~/.ssh/ocrtech-g6-training-20260615.pem (public IP changes each start;
last 44.211.88.63 — re-query after start)
```
On the EBS (preserved):
```
/home/ubuntu/native-retrain/ sunuwar-samefont + newa-native checkpoints, retrain.log
/home/ubuntu/staged-train/ synth corpora (sunuwar/kiratrai/lepcha), real eval packs,
dicts, run_queue.sh (the prior queue runner)
/home/ubuntu/deva-train/pretrain/devanagari_PP-OCRv5_mobile_rec_pretrained.pdparams (warm-start)
/home/ubuntu/{limbu,newa,tirhuta,deva}-train/ the strong/older models — DO NOT TOUCH
/home/ubuntu/re_score.py, /home/ubuntu/eval_metrics.py (micro-CER harness)
```
**Devanagari shared recognizer + Limbu model are off-limits for the native-expert work** —
all retraining is additive in `native-retrain/`.
## Key new files (committed except gitignored data/)
```
tools/eval_metrics.py micro CER (codepoint + grapheme)
tools/harvest_sunuwar_train_crops.py same-font crop harvester (the winning pattern)
tools/build_sunuwar_real_eval_pack.py 429-crop real eval pack
tools/devanagari_to_gurungkhema.py novel Devanagari→Khema transliterator
src/ocrtech/{sunuwar,lepcha,lepcha_herald,kiratrai}_legacy.py Herald converters + tests
data/ (gitignored): sunuwar-train-pack/, *-real-eval-pack/, real-text/newa-real-native.txt
docs/RECOGNIZER_GOAL_PROGRESS.md living scorecard — READ FIRST
```
## Current eval-harness status
The newer claim-grade harness work lives in `docs/EVAL_HARNESS_GOAL.md` and
`docs/EVAL_HARNESS_RUNBOOK.md`. The harness now separates recognizer oracle-crop
scoring from page-level OCR, records artifact hashes and run commands, emits
direct `claim-safety.md` reports, rejects metadata contradictions, and audits
metrics from row artifacts instead of trusting aggregate JSON. Scorers now require and
hash a `script_inventory_file` for claim-safe runs, record observed/missing required
codepoint and grapheme coverage, and hard-block claims when required inventory items
are absent. The auditor rejects a
run whose machine metadata lacks concrete platform/Python values or whose
`normalization_profile` no longer matches the scorer-supported `NFC_preserve_whitespace`
metric recomputation profile; the recipe validator also requires each eval pack to
declare that same profile, a readable and parseable summary/datasheet, a readable and
parseable `script_inventory_file`, and a readable
`spotcheck_calibration_file` whose eligibility/recommendation/source-status fields and
language/script/domain match whenever recipe-default or audited row GT is
`machine_converted_spot_checked` before a recipe can be claim-ready. Summary/datasheet
language, script, task, dataset, domain, GT, split, pack-version, frozen/unseen, and
default claim-eligibility declarations must also agree with the recipe metadata. Runner
checkpoint/config/dictionary values must also be readable local files, not directories,
so the recipe validator matches the scorer's hash-verifiability gate, and readable
runner dictionaries are checked against recognizer GT or page reference codepoints
before a recipe can be claim-ready. For path-complete
recognizer packs it now runs `audit_recognizer_eval_pack.py` and turns any pack-audit
issue or warning into a claim-readiness blocker; page recipes get a static
eval-manifest/reference audit for source image hashes, source-document/page/bbox
provenance, row claim eligibility, human review evidence, claim-ready references, and
minimum overall/per-slice scoring-unit counts. Recognizer pack audits apply the same
overall and language/script/domain/GT/split minimum before a recipe can be claim-ready,
`leakage_sources` are parsed and must be non-empty, page leakage manifests must satisfy
the page manifest schema, external command templates must include their required eval
input and output placeholders and must not contain unsupported placeholders, and recipe-declared checkpoint selection on the test pack
or known leakage is also a claim-readiness blocker, so empty manifests, missing page
evidence, input-disconnected commands, or acknowledged test-set contamination cannot
pass recipe registration as claim-ready.
Rows declared `human_verified` now also need explicit human review evidence such as
reviewer or review-artifact metadata; a trust label by itself is a claim blocker.
It also checks that checkpoint/config/dictionary run fields still match their hashed
artifact identity entries and that claim-safe runs carry non-empty model/system identity.
Recognizer slices
and crop-image hashes are recomputed from `predictions.tsv`, `unmatched.tsv` carries
source-audited missing label/prediction identity, label image hashes, and prediction
scores, recognizer manifest image paths/basenames are rejected when they would make
basename-based label/prediction matching ambiguous, `worst-lines.csv` carries audited row identity, crop hash, trust metadata,
claim eligibility, and source-document/page/bbox provenance for the worst recognizer
failures, and `predictions.tsv` rows are reconstructed from recorded label/prediction/manifest sources; page text row
counts, page text totals, line-alignment row contents, aggregate page metric means,
and per-language/script/domain/GT/split page slices are recomputed from scored
documents, references, `page-metrics.csv`, and `line-alignment.csv`. Page
`page-metrics.csv` row identity is reconstructed from the eval manifest, resolved
`document.json`, embedded `document.source_path`, scorer metadata defaults, row claim
eligibility, source-document/page/bbox provenance, and parsed reference fields; recognizer `predictions.tsv` also
carries source-audited row provenance and claim eligibility,
page `reference-artifacts.tsv` carries source-audited reference path/hash/field/trust
identity, page `text-pairs.tsv` carries the exact prediction/reference text used for
page text CER, exact-match recomputation, and page confusion recomputation, and page
`worst-pages.csv` carries audited
input/document hashes, row provenance, claim eligibility, and reference-artifact trust
fields for the worst page failures, run-level language/script/domain/GT/split lists
are checked against canonical row artifacts, undersized per-slice sample counts are
claim blockers so aggregates cannot hide weak languages/scripts, page text exact-match accuracy is carried
in `page-metrics.csv` and recomputed into aggregate/slice metrics, deterministic bootstrap uncertainty summaries are recomputed from
`predictions.tsv` and `page-metrics.csv`, script inventory summaries are recomputed from
recognizer GT rows or page reference text, and source-image and `document.json` SHA-256
values are rechecked. The auditor
now recomputes train/dev/eval leakage reports from recorded train manifests and
canonical eval rows/manifests, including overlap sets, checked state, and row counts,
and it checks embedded native spot-check calibration summaries, including
language/script/domain/source-GT/recommended-GT binding, against the recorded
calibration artifact. Metadata contradictions are recomputed from manifest/reference
provenance signals and canonical row domain/GT fields; recognizer provenance coverage
and page row source-document/page/bbox/claim-eligibility/language/script/domain/GT
metadata coverage counts are recomputed from source manifests, recognizer
normalizer/defaulted-metadata counts are checked against the recorded summary/datasheet,
hash-valid direct-run summary/datasheet language/script/domain/GT/split/version,
frozen/unseen, default claim-eligibility, and task declarations are checked against
the score run metadata,
dictionary coverage is recomputed from recognizer GT rows or page reference text and
the recorded dictionary file so out-of-vocabulary GT/reference codepoints block
claim-safe status,
zero-length recognizer GT rows and page text references are counted and recomputed so
empty references cannot hide inside CER denominators,
and page
reference coverage is recomputed from `page-metrics.csv`; page
manifest/scored/missing/claim-eligible counts are recomputed from source artifacts. It
also derives claim-safety blockers from the durable metrics/run metadata, so missing
manifests or summary/datasheets, unrun leakage checks, leakage overlaps, metadata
contradictions, unsafe domains or GT, low match rate, missing provenance, recognizer
row-level claim ineligibility, missing uncertainty evidence, missing script inventory
evidence, missing required script codepoints/graphemes, zero-length GT/reference text,
too-small claim-facing sample size, and incomplete page coverage cannot be hidden by editing `claim_safe` or `warnings`.
Direct `claim-safety.md` reports are checked against the JSON run/metric facts, so
stale human-facing summaries no longer pass just because their sections exist; recipe
root `claim-safety.md` reports are checked against recipe-run and score JSON facts too,
with recognizer/page-specific expected lines based on `eval_mode`.
Recipe-root `recipe.json` must match the selected recipe in the hashed source registry,
`recipe-validation.json` must match `recipe-run.json`'s embedded validation, and that
embedded validation is recomputed from the current recipe registry plus eval-pack
audit.
Recipe-run timestamp, machine platform/Python, command, and root output pointers for
materialization, audit, and claim-safety artifacts are audited.
Recipe failures now emit `recipe-run-error.json` with timestamp, machine, command,
recipe registry hash, selected recipe validation, root/output directory, exception
type, message, and traceback.
Root `audit.json` and `recipe-run.json` audit summaries are checked for current
issues, warnings, claim-safe status, and score-dir identity.
Recipe-run score summary paths and hashes are checked against `score/metrics.json`
and the mode-specific score run file: `score/eval-run.json` for recognizer roots or
`score/page-eval-run.json` for page roots.
Recipe-run train-manifest path/hash identities are checked against the score eval-run
leakage inputs.
Recipe-run `root` must be an existing directory, and the hashed recipe registry must
live under that declared root.
Recipe-root materialization JSON, timestamp, machine platform/Python and invocation
metadata, per-record commands/log paths, `command-template.txt`, `output-command-run.md`,
recognizer `outputs.prediction_file`, page `outputs.documents_dir`, and materialization
record hashes are tied back to the scored prediction file path/hash or scored page
document directory, and materialization model/checkpoint/config/dictionary metadata is
checked against the score eval-run.
Materialization failures emit `output-command-run-error.json` with the same run
identity plus label or eval-manifest path/hash, so failed external-command attempts
remain attributable.
Target coverage validation constrains `eval_status` to known states and now reports
non-claim-ready status even when concrete pack/recipe blockers already exist. Only
`registered_real_pack` targets can become coverage-ready, so partial, shared, missing,
or invented status labels cannot be used as silent claim evidence. The registry also
has a dedicated `limbu-first-two-script-ocr` target set requiring both
`limbu/Limbu` and `limbu/Devanagari` before a two-script Limbu OCR claim, and that
target set now requires `page_eval_recipe_id` links so recognizer-only readiness
cannot satisfy the full page-capture goal.
Score-level `metrics.md` reports are checked against `metrics.json` as well.
Recognizer `unmatched.tsv` plus label-line, matched-line, unmatched-label,
unmatched-prediction, and match-rate metrics are recomputed from source
label/prediction files; recognizer `confusions.json` is recomputed from canonical
scored rows, declared recognizer manifest crop hashes and image-key uniqueness are
checked against current crop bytes, page `confusions.json` is recomputed from canonical `text-pairs.tsv`, and page
`missing.tsv` is recomputed from manifest, document, and reference availability.
Worst-example CSVs, reviewer CSV order/fields, and reviewer HTML samples are audited
against canonical scored rows so the visible failure samples cannot silently go stale.
Completed score artifacts are also checked for duplicate sample IDs, image paths,
document paths, and missing-row keys, so duplicate-ID problems cannot be hidden after
normalization. Page scoring also rejects duplicate eval-manifest `sample_id` and
`image_path` values and stale declared page image hashes before scoring, and the audit
reader reports duplicate page manifest image paths or source-image hash mismatches if
an older or hand-edited score directory points at one.
Recognizer-vs-recognizer claims now have a paired comparison artifact:
`tools/compare_recognizer_eval_runs.py` pairs two completed recognizer score
directories by `sample_id`, requires same eval-pack and row GT identity, reports
candidate/baseline micro CER deltas for codepoints and graphemes, paired
wins/losses/ties, deterministic paired-row bootstrap confidence intervals, and an exact
two-sided sign-test p-value. It writes `comparison.json`, `comparison.md`, and
`paired-rows.tsv`, and its `--audit-only` mode recomputes those artifacts so stale
comparison summaries fail.
## Limbu-first operational OCR slice
User redirected the next production push to Limbu rather than Sunuwar because upcoming
digitization work needs a pipeline for imperfect page captures that may contain both
Devanagari-written Limbu and Sirijonga-script Limbu. A first operational slice now
exists, but it is not a page-level CER claim. The infrastructure is two-script shaped,
while the measured local page evidence is currently Sirijonga-heavy; a real
Devanagari-written Limbu page reference is still missing.
- `src/ocrtech/engines.py` script routing now supports `script=limbu` /
`sirijonga` / `limbu-sirijonga` by measuring U+1900..U+194F codepoint ratio. The
`script_select_composite` backend also supports `routing_signal_engine=secondary`
or `both`, which is important for Limbu because a Devanagari primary may not emit
Sirijonga codepoints needed to trigger a switch.
- `src/ocrtech/pipeline.py` exposes `parse_limbu_document`, which wraps the normal
parser and annotates every OCR line with Sirijonga, Devanagari, and Latin ratios,
a script class, review reasons, and an explicit `post_correction=not_configured`
marker. `--capture-prep-metadata` binds `limbu-capture-prep.json` into
`document.json` and `limbu-pipeline-audit.json`, including the metadata file hash
plus raw/prepared image hashes. It now also reruns `audit-limbu-capture` into the OCR
output directory under `capture-prep-audit/`, fails stale or incomplete capture
bundles, and fails if the OCR input does not match the audited `prepared_image_path`.
- `src/ocrtech/pipeline.py` also exposes `apply_limbu_review_corrections`, and
`ocrtech apply-limbu-review-corrections` applies accepted rows from
`limbu-review-queue.tsv` back into a corrected `document.json`/`document.md`. It
writes `limbu-correction-audit.json` plus `limbu-correction-pairs.jsonl` for future
Limbu-specific corrector training, and fails on duplicate accepted targets or empty
accepted corrections. Corrected-row script-class inference now carries forward the
source OCR document's `script_ratio_threshold` and records it in
`limbu-correction-audit.json`, so correction pairs cannot silently use a different
script policy than the OCR run that produced the review queue.
- Both Limbu OCR and review-correction stages now write `limbu-output-manifest.json`
with stage, run context, source hash when available, document counts, expected
artifact paths, SHA-256 hashes, sizes, and missing-artifact list.
`audit-limbu-output` now replays manifest `source_path` / `source_sha256` and binds
`document_source_path` plus `source_path` to `document.json.source_path`. For OCR bundles,
it also binds manifest `provenance.capture_prep_metadata` and
`provenance.post_correction_profile` to the exact paths and hashes embedded in
`limbu-pipeline-audit.json` / `limbu-post-correction-audit.json`. For
review-correction bundles, it also binds manifest
`provenance.input_document` and `provenance.review_queue` to the exact paths and
hashes recorded in `limbu-correction-audit.json`, so a manifest cannot point at
different valid files while the stage replay uses the real inputs.
- `parse-limbu-doc` now writes line-crop provenance under `line-crops/`: a JSONL row
per OCR line with bbox, script class, review flags, crop path/hash when the source is
croppable, and explicit warnings when crops cannot be generated. Valid crop PNGs are
included in `limbu-output-manifest.json`.
- `parse-limbu-doc` also writes `limbu-review-dashboard.html`, a static review contact
sheet for queued lines that shows the line crop when available, page/line id, script
class, confidence, review reasons, OCR text, and the TSV row shape needed for accepted
corrections. The dashboard is included in `limbu-output-manifest.json`.
- Review queue confidence triage is now script-aware. `parse-limbu-doc` still supports
the global `--low-confidence-threshold`, but can override it with
`--sirijonga-low-confidence-threshold`, `--devanagari-low-confidence-threshold`,
`--mixed-script-low-confidence-threshold`, and `--other-script-low-confidence-threshold`.
`limbu-pipeline-audit.json`, `limbu-line-audit.jsonl`, `limbu-review-queue.tsv`, and
the dashboard record the effective threshold so later review/audit can replay why a
Sirijonga or Devanagari line entered the queue. Review threshold and
`script_ratio_threshold` inputs are now validated as finite ratios in `[0, 1]`.
`audit-limbu-output` applies the same validation to embedded
`limbu-pipeline-audit.json` thresholds before replay, so a refreshed manifest hash
cannot hide invalid script-classification policy. Regressions:
`tests/test_pipeline.py::test_parse_limbu_doc_rejects_non_finite_or_boolean_review_thresholds`
and
`tests/test_pipeline.py::test_parse_limbu_doc_rejects_invalid_script_ratio_threshold`,
plus
`tests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_embedded_script_ratio_threshold_after_manifest_refresh`.
- `audit-limbu-output` now also replays queued confidence values against
`limbu-line-audit.jsonl`. A tampered review queue that keeps the same line id/text but
changes confidence now fails audit; regression:
`tests/test_pipeline.py::test_audit_limbu_output_rejects_review_queue_confidence_mismatch`.
- `audit-limbu-output` now replays the human-facing
`limbu-review-dashboard.html` cards against `limbu-review-queue.tsv` and
`line-crops/manifest.jsonl`, including displayed confidence, text, TSV copy row, and
crop link. This catches dashboard tampering even after the output manifest hash is
refreshed; regression:
`tests/test_pipeline.py::test_audit_limbu_output_rejects_review_dashboard_confidence_mismatch_after_manifest_refresh`.
- `parse-limbu-doc` now has a guarded deterministic Limbu post-correction lane via
`--post-correction-profile`. The JSON profile must declare `profile_id`,
`language=limbu`, `script_scope`, and exact/token rules whose replacements contain
Limbu or Devanagari text. `script_scope` and rule-level `script_class` values are
constrained to `limbu_sirijonga`, `devanagari_limbu`,
`mixed_limbu_devanagari`, or `other`; regressions:
`tests/test_pipeline.py::test_audit_limbu_post_correction_profile_rejects_unknown_script_scope`
and
`tests/test_pipeline.py::test_audit_limbu_post_correction_profile_rejects_unknown_rule_script_class`.
Every OCR run writes `limbu-post-correction-audit.json` and
`limbu-post-correction-lines.jsonl`; without a profile these artifacts explicitly say
`status=not_configured`, and with a profile they record the profile SHA-256, rule ids,
original/corrected text, and changed count.
- `build-limbu-correction-pair-pack` should be the first step after accumulating
reviewed correction pairs. It writes `train-limbu-correction-pairs.jsonl`,
`heldout-limbu-correction-pairs.jsonl`, and `limbu-correction-pair-pack.json` with
frozen-pack status and a pack-content digest. The pack remains calibration evidence,
not page OCR ground truth. Accepted review rows, pack build, pack audit, derivation,
and held-out scoring now constrain correction-pair `script_class` values to the
same vocabulary as post-correction profiles: `limbu_sirijonga`, `devanagari_limbu`,
`mixed_limbu_devanagari`, or `other`. Missing row values are inferred from clean
text, but explicit typos such as `sirijonga` fail instead of being silently hidden.
The audit replay path uses the same validation, so legacy or hand-written review
queues with unsupported accepted-row script labels cannot pass replay just because
their hashes match.
Regressions:
`tests/test_pipeline.py::test_apply_limbu_review_corrections_rejects_unknown_script_class`,
`tests/test_pipeline.py::test_limbu_review_correction_replay_rejects_unknown_script_class`,
`tests/test_pipeline.py::test_build_limbu_correction_pair_pack_rejects_unknown_script_class`,
`tests/test_pipeline.py::test_audit_limbu_correction_pair_pack_rejects_unknown_script_class`,
`tests/test_pipeline.py::test_derive_limbu_post_correction_profile_rejects_unknown_script_scope`,
`tests/test_pipeline.py::test_derive_limbu_post_correction_profile_rejects_unknown_pair_script_class`,
and
`tests/test_pipeline.py::test_score_limbu_post_correction_profile_rejects_unknown_pair_script_class`.
- `audit-limbu-correction-pair-pack` should pass before using the pack for derivation
or held-out post-correction scoring. It catches stale train/heldout files and digest
mismatches after pack creation.
- `derive-limbu-post-correction-profile` bridges reviewed correction bundles into that
profile format. It refuses ambiguous mappings and writes both JSON and Markdown
derivation reports, but it does not produce a validated profile; run
`audit-limbu-post-correction-profile` and attach held-out evidence before treating an
automatic profile as more than triage.
- `score-limbu-post-correction-profile` is the missing evidence generator for that
admission step. It produces `limbu-post-correction-profile-eval.json` plus per-row
JSONL, and the CLI prints the stable eval JSON SHA-256. Copy that hash, the frozen
eval-pack identifier, metric `text_cer_codepoint_micro`, and before/after values into
profile `admission` only when the scorer passes.
- `admit-limbu-post-correction-profile` now does that copy safely. It writes a new
admitted profile rather than mutating the source file, and immediately audits the
result with `audit-limbu-post-correction-profile`.
- `ocrtech audit-limbu-output` audits a Limbu OCR or review-correction output
directory by recomputing manifest artifact hashes/sizes, checking missing artifacts,
validating document counts and source-path binding, checking OCR line/post-correction/review
queue/dashboard/crop counts, verifying capture-prep audit status, verifying
post-correction profile hashes and crop hashes when they exist, binding OCR manifest
provenance to the embedded capture/profile audit records, rerunning the
capture-prep audit from the embedded metadata path into `capture-prep-replay-audit/`,
replaying each OCR line's embedded `document.json` `metadata.limbu_pipeline`
script ratios, script class, review flag, review reasons, and effective threshold
from the current line text/confidence after validating the embedded replay
thresholds, replaying OCR-stage embedded
`post_correction` metadata from `limbu-post-correction-lines.jsonl` and
`limbu-post-correction-audit.json`, and replaying review-correction bundles from the
original `document_path` plus
`review_queue_path` recorded in `limbu-correction-audit.json`. The replay verifies
source hashes, manifest provenance bindings, the source script-ratio threshold,
applied/changed/skipped counts, corrected `document.json` text,
`limbu-correction-pairs.jsonl` rows, and the corrected line's embedded
`metadata.limbu_pipeline` review flags plus `post_correction`
reviewer/status/original/corrected-text metadata, so corrected text, training pairs,
and the embedded correction story are not accepted on count checks alone. A refreshed
manifest hash no longer hides tampered embedded line metadata; regressions:
`tests/test_pipeline.py::test_audit_limbu_output_replays_document_line_limbu_metadata_after_manifest_refresh`
`tests/test_pipeline.py::test_audit_limbu_output_replays_post_correction_line_metadata_after_manifest_refresh`,
and
`tests/test_pipeline.py::test_audit_limbu_output_replays_review_correction_line_metadata_after_manifest_refresh`.
It writes `limbu-output-audit.json` and `.md`, and exits nonzero on stale or
incomplete bundles.
- `ocrtech audit-limbu-output --require-reviewer-for-corrections` is now available for
strict corrected-bundle audits. It fails accepted correction rows whose `reviewer`
field is blank; regression:
`tests/test_pipeline.py::test_audit_limbu_output_strict_policy_requires_reviewer_for_corrections`.
- `ocrtech audit-limbu-output` now has strict operational policy flags for real page
digitization: `--require-capture-prep`, prepared-image quality thresholds
(`--min-capture-prepared-width`, `--min-capture-prepared-height`,
`--min-capture-prepared-entropy`, `--min-capture-prepared-luminance-stddev`, and
`--min-capture-prepared-edge-stddev`), `--require-no-pending-review`, repeated
`--require-script` values such as `limbu_sirijonga` and `devanagari_limbu`, and
repeated `--require-script-count SCRIPT=COUNT` gates for minimum per-script line
counts. Capture-threshold policy inputs are validated as finite non-negative numbers
before audit work starts; regression:
`tests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_capture_quality_policy`.
Required script policy inputs are normalized/deduplicated, boolean count values are
rejected instead of being treated as `1`, and invalid script names fail before audit
work starts; regressions:
`tests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_required_script_policy`
and
`tests/test_pipeline.py::test_audit_limbu_output_normalizes_required_script_policy`.
It also has `--min-line-count`, `--min-average-line-confidence`, and
`--min-quality-score`, which gate strict acceptance on audited `document.json`
quality metadata rather than only provenance; `--min-line-count` also rejects boolean
API values instead of accepting Python's `True == 1` behavior. These are opt-in so diagnostics can
still be artifact-audited, but real Limbu page outputs should use them to require
replayed capture provenance, prepared-image quality, enough OCR text, minimum output
quality, no unresolved review rows, and expected script coverage.
- `ocrtech audit-limbu-capture --require-metadata-path-self` is now available for
strict capture-prep audits. It fails when `metadata_path` inside
`limbu-capture-prep.json` does not resolve to the exact metadata file being audited;
without the flag this remains a portability warning. Regression:
`tests/test_capture.py::test_audit_limbu_capture_can_require_metadata_path_self_binding`.
- `ocrtech audit-limbu-output --require-capture-metadata-path-self` now forwards that
same metadata self-binding requirement into the OCR output's capture-prep replay.
Regression:
`tests/test_pipeline.py::test_audit_limbu_output_can_require_capture_metadata_path_self_binding`.
A fresh two-script smoke at
`outputs/limbu-strict-output-policy-smoke-2026-06-21` passes that strict policy;
`strict-audit/limbu-output-audit.json` SHA-256 is
`2fc71f191ba581d58f59480b54695a9297778f3c54a2a63c8a7da339d618142e`.
- `src/ocrtech/cli.py` adds `ocrtech parse-limbu-doc`, emitting normal parser
outputs plus `limbu-pipeline-audit.json`, `limbu-line-audit.jsonl`,
`limbu-post-correction-audit.json`, `limbu-review-queue.tsv`, and
`limbu-review-dashboard.html`; it also adds `ocrtech apply-limbu-review-corrections`.
- `src/ocrtech/capture.py` and `ocrtech prepare-limbu-capture` preserve the raw
capture, apply EXIF transpose, optional manual crop, optional four-corner
perspective rectification via `--perspective-quad`, optional automatic page-corner
detection via `--auto-detect-page`, optional automatic deskew via `--auto-deskew` /
`--max-auto-deskew-degrees`, optional manual rotation, grayscale, and autocontrast,
and write `limbu-capture-prep.json` with source/prepared hashes, source/prepared
image-quality metrics (size, luminance spread, edge spread, entropy), timestamp,
operation list, and CLI run context when available. Auto-deskew now tries
near-horizontal Hough line evidence first and falls back to row-projection scoring
over text ink for fragmented glyph rows; it records either the detected/applied angle
or a skipped status when evidence is insufficient.
- `ocrtech audit-limbu-capture` now audits a prepared capture bundle or
`limbu-capture-prep.json`: it verifies raw-copy and prepared-image hashes, validates
capture operation metadata, replays image-quality metrics for available images, writes
`limbu-capture-audit.json` / `.md`, fails stale or missing prepared artifacts, and
warns rather than fails if the original source path has moved after the raw copy was
preserved. It also supports opt-in prepared-image quality gates:
`--min-prepared-width`, `--min-prepared-height`, `--min-prepared-entropy`,
`--min-prepared-luminance-stddev`, and `--min-prepared-edge-stddev`; chosen
thresholds are recorded as `quality_policy`.
- `docs/LIMBU_FIRST_OCR_PIPELINE.md` documents the command, output contract, model
shape, capture cleanup requirements, and claim boundary.
- `configs/limbu-first-ocr-pipeline-v1.json` records the intended pipeline profile
and current local Limbu/Devanagari asset references.
- `src/ocrtech/limbu_runtime.py` and `ocrtech preflight-limbu-pipeline` now provide a
machine-readiness gate for the packaged router: router model-card audit, nested
component model-card audits, Pillow/OpenCV/NumPy/Paddle/PaddleOCR imports,
Python executable/version, Tesseract binary discovery, and required `nep`/`eng`
language data. The report is written to `limbu-runtime-preflight.json` and `.md`.
Use `--require-validated-components --required-component-role devanagari_primary
--required-component-role sirijonga_secondary` for the strict two-script gate: this
fails warning-only `experimental`/`pending` components and missing component roles,
so a dependency-ready runtime cannot be mistaken for an admitted Limbu router. The
preflight report now records the enforced policy and fails malformed
`component_admission` list entries instead of silently filtering them out.
- `outputs/limbu-first-router-v1/` now contains an auditable router package:
`model-card.json` is a `script_select_composite` candidate with Tesseract
`nep+eng` as the Devanagari primary and packaged PaddleOCR Sirijonga as the
secondary. The Sirijonga component now uses the exported v4 package copied into
`components/limbu-sirijonga-paddle-v4/`; both the component and router model-card
audits pass. The router keeps `routing_signal_engine=secondary` for Sirijonga
discovery but adds `primary_guard_script=devanagari` to stop secondary hallucinations
from replacing clearly Devanagari primary lines. The default project `.venv` still
uses Python 3.14 and fails Paddle /
PaddleOCR imports, but a Python 3.11 runtime at `/tmp/ocrtech-paddle311` imports
`paddle==3.3.1` and `paddleocr==3.7.0`; with Homebrew Tesseract plus manually added
`nep.traineddata`, the Python dependency checks pass under
`/tmp/ocrtech-paddle311`. The full router preflight should still fail until the
Devanagari primary is replaced or admitted with evidence; the current router records
`devanagari_primary` as failed. A minimal candidate-engine smoke parse also completed under
`outputs/limbu-first-router-v1/runtime-smoke-py311/out`, and
`audit-limbu-output` passed for that smoke bundle.
- `tools/build_limbu_devanagari_smoke_page_eval_draft.py` builds a non-claim synthetic
Devanagari-written Limbu page-eval draft from the local real text corpus. The current
pack at `outputs/limbu-page-eval-drafts/limbu-devanagari-smoke-page-2026-06-21`
renders 14 lines from 8 source rows with Noto Sans Devanagari, records source/font/
image/reference/manifest/script-inventory hashes, and writes a sidecar-perfect
document for harness regression. The score at `score-sidecar-perfect` has codepoint
CER 0, grapheme CER 0, line F1 1.0, matched-line reading-order pair accuracy 1.0 over
91 pairs, no missing required graphemes, and `audit_passed=true`; it remains
`claim_safe=false` and is not real-page evidence.
- `tools/inventory_limbu_page_sources.py` inspects known Limbu page/text candidates:
curriculum JSON/PDF references, Sikkim Herald Limbu PDFs, Gorkhapatra Namdhinggo
listings, local text-only Devanagari sources, and the synthetic smoke pack. It writes
`limbu-page-source-inventory.json` and `.md` and can hard-fail with
`--require-real-devanagari-page` when no real Devanagari page source is present.
Verification for this slice:
`uv run pytest tests/test_pipeline.py tests/test_limbu_runtime.py tests/test_capture.py tests/test_cli.py::test_cli_prepare_limbu_capture_passes_run_context tests/test_cli.py::test_cli_apply_limbu_review_corrections tests/test_cli.py::test_cli_audit_limbu_output_exit_code tests/test_cli.py::test_cli_create_script_select_model_card tests/test_cli.py::test_cli_export_and_package_recognizer tests/test_training_benchmark.py::test_package_paddleocr_model_writes_auditable_model_card -q` passed,
`uv run pytest tests/test_models_candidate.py::test_candidate_engine_supports_script_select_composite_limbu_routing tests/test_models_candidate.py::test_script_select_composite_can_route_from_secondary_limbu_signal tests/test_models_candidate.py::test_script_select_candidate_audit_rejects_invalid_routing_signal_engine -q` passed,
`uv run python -m py_compile src/ocrtech/limbu_runtime.py src/ocrtech/capture.py src/ocrtech/pipeline.py src/ocrtech/engines.py src/ocrtech/models.py src/ocrtech/cli.py`
passed, CLI help for `parse-limbu-doc`, `prepare-limbu-capture`, and
`preflight-limbu-pipeline` loads, `ocrtech apply-limbu-review-corrections --help`
and `ocrtech audit-limbu-output --help` load, and `git diff --check` passed. The
claim-readiness validators remain structurally valid but non-ready:
`recipes=12 valid=True claim_ready=0` and
`valid=True coverage_ready=False target_sets=3 gaps=50`.
Next real work: improve full-page Sirijonga quality, find or create a real
Devanagari-written Limbu page reference beyond the synthetic smoke pack, make
Devanagari/Sirijonga routing claim-safe with frozen page references, package local
hashed model cards for any custom Devanagari recognizer if it becomes available,
improve page-corner detection for still captures when needed, and only then score
page-level outputs against frozen Limbu page references.
The Devanagari page-source gate is now green for real-source availability. The next
step is to build a non-claim Devanagari page-eval draft from those CDC PDFs and
curriculum line lists, then add line/reference alignment and review evidence before
treating the second script as claim covered.
## Current eval-harness status
The latest page-eval harness change adds an explicit OCR-output mode to page scoring:
`tools/run_page_ocr_eval.py` now records `ocr_output_kind` in `page-eval-run.json`,
`metrics.json`, `metrics.md`, `claim-safety.md`, and each `page-metrics.csv` row.
Claim-safe page scoring hard-warns/fails if the kind is `unknown`; accepted values are
`live_detector`, `oracle_line_boxes`, `sidecar_fixture`, and `unknown` at the scorer CLI
level, while recipe validation only permits claim-ready page recipes to declare
`live_detector`, `oracle_line_boxes`, or `sidecar_fixture`. `tools/run_eval_recipe.py`
passes `eval_pack.ocr_output_kind` through to the page scorer, defaulting recipe-driven
page runs to `live_detector`; `tools/audit_eval_run.py` audits the new run/row identity
field; and `tools/compare_page_ocr_eval_runs.py` surfaces candidate/baseline
`ocr_output_kind` so the required Limbu diagnostic can honestly compare
`oracle_line_boxes` against `live_detector` on the same frozen page pack without hiding
that this is a detector-vs-recognizer experiment, not a normal model-vs-model result.
Docs updated: `docs/EVAL_HARNESS_RUNBOOK.md` and
`docs/LIMBU_FIRST_OCR_PIPELINE.md`.
The latest page-eval harness also reports matched-line text and reading-order metrics
separately from full-page text metrics. `tools/run_page_ocr_eval.py` now writes
matched-line codepoint and grapheme edit/ref counts, micro CER, exact-match accuracy,
and aligned-line reading-order pair accuracy into `metrics.json`, `metrics.md`,
`claim-safety.md`, `page-metrics.csv`, and `line-alignment.csv`. This prevents a high
page CER from being misread as one single failure: the same eval run can now separate
line geometry, recognition on matched lines, aligned-line order, and page text assembly.
Regression coverage includes the case where line boxes and line text are perfect but
block reading order is wrong; page CER rises while matched-line CER remains zero and
matched-line reading-order accuracy falls to zero.
A new draft-pack builder was added at
`tools/build_limbu_sikkim_page_eval_draft.py`. It uses the existing Sikkim Herald
Limbu legacy-font converter to build a non-claim page eval draft with machine-converted
Sirijonga text and PDF-derived line bboxes scaled to a full-page image. The first real
draft pack is
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1`: 86 reference
lines for `limboo_2020-12-03.pdf` page 1, scored against the guarded live-detector OCR
document as `ocr_output_kind=live_detector`. The scorer reports codepoint CER
`0.891560`, grapheme CER `0.897412`, line detection F1 `0.018519`, and 1 matched line
out of 86 references. `audit_eval_run.py` passes the score bundle (`audit_passed=True`)
but `claim_safe=False` because the pack is not frozen/claim-eligible, has only one
page, uses `gt_status=machine_converted_unreviewed`, and has draft conversion caveats
(`U+0041`, `U+201C` missing from the dictionary). Treat this as strong diagnostic
evidence that current full-page Sirijonga failure is detector/layout dominated, not as
accuracy evidence.
A new oracle-line-box materializer was added at
`tools/materialize_page_oracle_line_boxes.py` with tests in
`tests/test_materialize_page_oracle_line_boxes.py`. It reads page eval manifest rows,
uses reference line bboxes only to crop the page image, runs a configured OCR engine on
each crop, and emits page-score-compatible `document.json` files plus a crop manifest.
For PaddleOCR model cards it can force `--recognition-mode line`, which is required for
the Sikkim diagnostic because `auto` can route large title boxes through the full-page
Paddle wrapper. The Sikkim oracle materialization lives at
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/oracle-line-box-documents-v4`
and used the v4 Sirijonga component model card with `recognition_mode_override=line`;
it produced 86 lines and 0 empty predictions.
The oracle-line-box score is
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/score-oracle-line-boxes-v4`.
It audits cleanly (`audit_passed=True`, `claim_safe=False`) and reports codepoint CER
`0.274289`, grapheme CER `0.548983`, line detection F1 `1.0`, and 86/86 matched lines.
The paired comparison at
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/compare-oracle-vs-live-v4`
shows a non-claim-safe codepoint-CER improvement of `0.617272` over the live detector.
Interpretation: fix detection/reading order, but do not pretend oracle boxes solve the
page. The recognizer is still weak on this Sikkim Herald register and the draft
machine-converted reference needs human verification or at least spot-check calibration.
A second materializer was added at `tools/materialize_page_image_line_boxes.py` with
tests in `tests/test_materialize_page_image_line_boxes.py`. It is not an oracle: it
detects line-like components from page pixels using Otsu thresholding and morphology,
can compute final boxes from the original ink pixels, then crops and recognizes each
detected line with the configured engine. The current Sikkim diagnostic uses
`bbox_source=ink`, horizontal kernel `41`, vertical kernel `11`, one dilation
iteration, detector padding `2`, crop padding `12`, and v4 Sirijonga
`recognition_mode=line`. Outputs:
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/image-line-box-documents-v4`,
score:
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/score-image-line-detector-v4`,
comparisons:
`compare-image-line-vs-paddle-live-v4` and `compare-oracle-vs-image-line-v4`.
The image detector improves over Paddle live detection by 0.431305 absolute codepoint
CER, but oracle boxes still improve over it by 0.185967 absolute codepoint CER. Do not
promote this detector blindly; test it on more pages and fix reading order/overprediction.
The detector-only reading-order comparison for this Sikkim image-line detector is
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/compare-detector-reading-order-column-major-v1`;
it reports no `column_major` gain over `top_to_bottom` under the same detector settings.
The materializer now supports `--reading-order auto_layout`, records requested/selected
ordering plus layout features, and the detector-only comparator accepts that mode. The
auto heuristic uses wide-page plus regular line-width evidence to select
`wide_column_right_to_left`. It improves Gorkhapatra matched-line reading-order pair
accuracy from 0.606543 to 0.763985 while keeping Sikkim unchanged at 0.992402; outputs
are `compare-detector-reading-order-auto-layout-v1` under each draft page directory.
This is layout-selection evidence only, not a text-quality claim.
The detector parameter sweep at
`outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/detector-param-sweep-v1`
also reports no morphology-family improvement over the existing `41x11` detector.
The latest Codex pass extended the immutable eval-pack digest from recognizer
oracle-crop runs to page-level OCR runs. Page eval-pack summaries now must carry
`pack_content_sha256` just like recognizer summaries. The page scorer recomputes the
digest from the rows it actually scores, using the canonical page row identity, input
image hash, reference artifact hash, reference text hash, reference fields, provenance,
language, script, domain, GT status, split, and claim eligibility. Claim-safe page
scoring fails when the summary digest is missing or stale. The page score now records
that digest in both `metrics.json` and `page-eval-run.json`.
The audit path also recomputes page `pack_content_sha256` from current source
artifacts and checks the value recorded in `page-eval-run.json`, `metrics.json`, and
the page summary/datasheet. Page comparison now treats `pack_content_sha256` as part
of eval-pack identity, so a candidate and baseline page run cannot be claim-safe
compared if they were scored against different or drifted page-pack contents. The
relevant files touched in this pass are `tools/run_page_ocr_eval.py`,
`tools/audit_eval_run.py`, `tools/compare_page_ocr_eval_runs.py`,
`tools/validate_eval_harness_recipes.py`,
`tests/test_run_page_ocr_eval.py`, `tests/test_audit_eval_run.py`,
`tests/test_compare_page_ocr_eval_runs.py`, `docs/EVAL_HARNESS_GOAL.md`, and
`docs/EVAL_HARNESS_RUNBOOK.md`.
The page recipe fixture now computes the page pack digest before claim-safe recipe
scoring, and the recipe validator blocks digestless page summaries before scoring
instead of only blocking recognizer summaries.
Verification after the digest work: focused recipe/page/audit/validator tests passed;
the full targeted harness suite passed with
`uv run pytest tests/test_apply_native_spotcheck_reviews.py tests/test_audit_eval_run.py tests/test_audit_recognizer_eval_pack.py tests/test_build_recognizer_leakage_manifest.py tests/test_compare_page_ocr_eval_runs.py tests/test_compare_recognizer_eval_runs.py tests/test_eval_harness_runbook.py tests/test_native_spotcheck_bundle.py tests/test_normalize_recognizer_eval_pack.py tests/test_predict_recognizer_eval.py tests/test_run_eval_output_command.py tests/test_run_eval_recipe.py tests/test_run_page_ocr_eval.py tests/test_run_recognizer_eval.py tests/test_run_recognizer_pack_normalization.py tests/test_validate_eval_harness_recipes.py tests/test_validate_eval_target_coverage.py -q`;
targeted `py_compile` passed; `git diff --check` passed. Registry validators are
structurally valid but not claim-ready:
`uv run python tools/validate_eval_harness_recipes.py --recipes configs/eval-harness-recipes-v1.json --root .`
reports `recipes=12 valid=True claim_ready=0`, and
`uv run python tools/validate_eval_target_coverage.py --coverage-config configs/eval-target-coverage-v1.json --eval-recipes configs/eval-harness-recipes-v1.json --root .`
reports `valid=True coverage_ready=False target_sets=3 gaps=50`. Do not mark the
overall eval-harness goal complete until those claim-readiness and coverage gaps are
actually closed.
The latest audit hardening closes a stale-output-audit gap for live-detector page
scores. `tools/audit_eval_run.py` now requires `page-eval-run.json` to carry
`require_output_audit`, `output_audit_filename`, `output_audit_passed_field`, and
`page_output_audit`, then replays the expected per-page audit summary from
`page-metrics.csv` and the current audit JSON beside each scored `document.json`. If a
recorded page score says all Limbu output audits passed but a later reviewer deletes,
edits, or breaks one of those per-page audit files, audit now fails instead of trusting
the stale `metrics.json`/`page-eval-run.json` summaries. Regression coverage:
`tests/test_audit_eval_run.py::test_audit_eval_run_replays_page_output_audit_artifacts`.
Recipe-level page output-audit provenance is also now bound by a command-template
sidecar. `tools/run_eval_recipe.py` writes `output-audit/command-template.txt` beside
`output-audit/output-audit-run.json`, and `tools/audit_eval_run.py` fails if that
sidecar is missing or does not match `recipe-run.json output_audit.command_template`.
The same root audit now requires `output-audit/output-audit-run.md` and checks that the
Markdown report still contains the recorded output-audit row count and every per-page
output-audit line, so the reviewer-facing page audit report cannot go missing or drift
while JSON remains green.
`tools/run_eval_recipe.py` now also honors `runner.output_audit.result_filename` when
recording the per-page audit JSON artifact instead of hard-coding
`limbu-output-audit.json`. `tools/audit_eval_run.py` replays those recorded per-page
audit JSON paths and SHA-256 hashes, and the Markdown summary must expose the same
audit artifact hashes. A page output audit can no longer pass recipe-root audit by
return code alone after the per-page audit file drifts. The audit now also replays
each recorded per-page output-audit command from `output_audit.command_template`, so a
coordinated edit to `recipe-run.json` and `output-audit/output-audit-run.json` cannot
hide that a different audit command was used. The same records now include
stdout/stderr SHA-256 values for each page audit command, and the root audit
recomputes those log hashes and requires the Markdown summary to expose them.
Materialization command records now have the same treatment:
`tools/run_eval_output_command.py` records stdout/stderr SHA-256 values, the Markdown
summary exposes those log hashes, and `tools/audit_eval_run.py` recomputes them so
materialization logs cannot drift after a recipe run. The root audit also replays
materialization record commands from the recorded materialization command template, so
editing both `recipe-run.json` and `materialize/output-command-run.json` with a stale
command no longer passes. Missing recognizer/page fields needed for that replay now
fail audit instead of silently skipping the check.
Regression coverage lives in
`tests/test_run_eval_recipe.py::test_run_eval_recipe_page_level_end_to_end`.
Recipe validation now also requires Limbu `live_detector` page recipes to include
`--require-no-dropped-image-lines` and the `--min-capture-prepared-*` quality gates in
`runner.output_audit`, in addition to `--require-capture-prep`,
`--require-no-pending-review`, and the script-specific `--require-script`. This
prevents a future claim-ready recipe from accepting a precision-first filtered
image-line output that silently dropped detected page lines or a capture-prep bundle
below the declared quality floor.
The two current Limbu page recipes in `configs/eval-harness-recipes-v1.json` were
updated with those flags. Regression coverage:
`tests/test_validate_eval_harness_recipes.py::test_validate_eval_harness_recipes_blocks_page_output_audit_without_strict_flags`.
The operational image-line OCR bundle audit now also replays `image-line-crops/manifest.jsonl`
against `image-line-ocr-run.json` and the current crop files. It checks the recorded
crop-manifest path/hash, detected/kept/removed row counts, document kept-line count, and
per-crop SHA-256 values. It also checks kept crop rows against `document.json` line ids,
bboxes, prediction text, confidence, and embedded image-line crop provenance, so count
consistency is not enough to pass after stale row edits. The same audit now verifies
the image-line run `source_path`/`source_sha256`, model-config availability, and the
duplicated `image_line_ocr` metadata in `document.json` and page metadata, and it
binds image-line `model_config` metadata to `limbu-output-manifest.json`
`provenance.model_config` path/hash. This closes a stale-review-evidence gap where
`image-line-ocr-run.json` could keep pointing at an old crop manifest or where the
bundle could preserve a valid but different recognizer config provenance entry after
crop rows, crop PNGs, or document image-line metadata were edited.
Regression coverage:
`tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_crop_manifest` and
`tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_manifest_against_document`,
plus `tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_run_metadata`,
`tests/test_pipeline.py::test_audit_limbu_output_requires_image_line_model_config_provenance`,
and `tests/test_pipeline.py::test_audit_limbu_output_rejects_image_line_model_config_provenance_mismatch`.
## Current measured Limbu page baseline
The active goal has shifted from more pipeline hardening to measured real-page OCR
quality. The current diagnostic baseline is recorded at
`outputs/limbu-measured-page-baseline-2026-06-22/README.md`, with fresh audit snapshots
beside it. A fixed non-claim comparison set is now recorded at
`outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-page-set-v1.json`
and documented in
`outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-page-set-v1.md`.
It pins 5 real pages: 3 Devanagari-written Limbu CDC curriculum pages and 2 Sirijanga
Limbu pages. It is not a claim-safe result: all real-page references are still
unreviewed or machine-converted, the frozen set is diagnostic rather than
claim-eligible, and the page count is below the 30-unit claim threshold.
The current fixed-set rollup is
`outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-current-baseline-rollup-v1.md`
with structured JSON beside it. It combines the current selected diagnostic systems,
not a single deployed model: CDC Devanagari image-line + Tesseract, Gorkhapatra
Sirijanga operational `conf070`, and Sikkim Sirijanga same-row merge. The five-page
diagnostic aggregate is codepoint CER `0.706734`, grapheme CER `0.748005`, true micro
WER `0.908270`, line F1 `0.801444`, and matched-line CER `0.124367`. Use this only as
a before/after target on fixed page identities.
Current measured signals:
- Devanagari CDC linebox draft with image-line + Tesseract: `0.865400` codepoint CER,
`1.109798` WER over 3 pages. The new Devanagari line-failure analysis in
`outputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-devanagari-cdc-image-line-tesseract/`
says the dominant codepoint error source is `matched_good_geometry_bad_text`
(`45.58%` of attributed error mass), not missed lines. This is not competitive;
wait for the active PP-OCRv5 Devanagari recognizer to finish, then re-score these
pages.
- Gorkhapatra Sirijanga operational live detector: `0.739361` page codepoint CER, but
only `0.046585` matched-line CER. Oracle line boxes are `0.029268` page CER. The
refreshed failure analysis at
`outputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-gorkhapatra-operational-conf070/`
has `236` matched lines, `67` missed references, `2` extra predictions, and
`83.86%` of attributed codepoint error mass from missed reference text. Detector/layout
recall is the next lever.
- Sikkim Herald Sirijanga same-row merge: `0.418057` codepoint CER versus `0.891560`
for the guarded live detector, while oracle boxes are `0.274289`. The refreshed
failure analysis at
`outputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-sikkim-same-row-merge/`
has `77` matched lines, `9` missed references, `13` extra predictions, and a mixed
error profile: missed reference text is largest at `31.61%`, but extra prediction
text is `28.12%` and matched-line text errors are still material. This needs both
layout tuning and recognizer/domain adaptation.
Do not drift back into general provenance work as the next default task. The next useful
block is: finish/admit the Devanagari recognizer when training completes, re-score the
frozen diagnostic pages with it, and tune Sirijanga detector geometry on the same fixed
page identities and failure buckets.
## Current Limbu book OCR work
The active objective is usable Limbu book OCR, not detector tuning by itself. The
system needs to take scanned or photographed Limbu book pages, preserve page/line
structure, support Sirijanga Limbu and Devanagari-written Limbu, emit editable Unicode
book text, and flag unresolved regions for review. The current implementation now has
`tools/parse_limbu_book_pages.py`, which runs the existing page parser per page-manifest
row and aggregates `book.md`, `book.body.md`, `book-review-queue.tsv`,
`book-run.json`, and scorer-compatible `score-documents/`.
The current CDC h20 structural real-page replay is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay`.
It is a replay through the book wrapper, not fresh OCR inference. It scores at page
codepoint CER `0.446556`, page grapheme CER `0.626728`, line precision/recall/F1
`0.628571` / `0.709677` / `0.666667`, missed/extra/blank line rates `0.290323` /
`0.371429` / `0.000000`, matched-line CER `0.175130`, and reading-order pair accuracy
`0.600000`. Replay prep drops two empty selected-output science lines, improving
extra/blank line accounting without changing page CER. The body export bug for
structural `page_marker` lines is fixed in `src/ocrtech/structure.py`: full book output
keeps the social-page `२`, while `book.body.md` omits it.
The confidence-pruning sweep is documented in
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md`. Results:
`confidence >= 0.20` removes one low-confidence science false positive and improves
line F1 to `0.676923` with unchanged page CER; `0.30` and `0.40` improve precision/F1
but worsen page CER; `0.50` starts dropping matched lines and hurts recall. Do not
promote confidence pruning as the default OCR path. Treat `0.20` only as possible
review-triage cleanup. The next quality work is missed/split line recovery on page
regions.
One more same-row merge diagnostic was run after that: h20 page markers plus capped
same-row merge `gap160`, `width500`, `y_tolerance=25`. It produced 34 lines with 2
row merges and improved micro line F1 to `0.707692`, but page codepoint CER regressed
to `0.460808`, grapheme CER to `0.643625`, and matched-line CER to `0.185259`
(score SHA-256 `61129eaa91445c0d8a86f66c5a5827ffa342b331bffe032d12f60d9476ad0b32`).
Reject it. Same-row merge thresholds are not the next useful default; merged crops are
hurting recognition even when geometry improves.
The better direction is now post-recognition line assembly. New tool:
`tools/assemble_page_ocr_lines.py`. It merges adjacent same-row OCR line records after
recognition, records source line ids/texts/bboxes/confidences in metadata, and reads no
reference text. On the CDC h20 structural book replay, `gap160 width500` merged exactly
two top-fragment pairs: maths `img-line-0001|img-line-0002` and science
`img-line-0002|img-line-0003`. It improves micro line precision/recall/F1 to
`0.727273` / `0.774194` / `0.750000`, drops missed references `9 -> 7`, drops extra
predictions `13 -> 9`. The source-break variant preserves original page text exactly:
page codepoint CER stays `0.446556`, grapheme CER stays `0.626728`, and matched-line
CER is `0.177052`. The space-join variant is closer to editable one-line output but
nudges page CER to `0.447348`; assembly plus `confidence >= 0.20` is worse at
`0.451306`. Treat source-break assembly as the current best line-structure score
projection, and treat space-join assembly as an export/review-stage question. Next
useful work: make the review packet/body export assembly-aware without silently
hallucinating missing connector text. The source-break assembly output is
replay-auditable with `tools/audit_page_ocr_line_assembly.py`; current audit passed
with 0 issues, SHA-256
`69eb0db004f0bbabdead620bf63dfc138fb02deb8e3003fb9ec668a88d26afb1`.
`tools/parse_limbu_book_pages.py` now exposes this path through `--line-assembly`.
Important implementation boundary: it does not rewrite audited page OCR directories.
It builds a book-level `line-assembly-source-documents/` projection, writes assembled
documents under `line-assembled-documents/`, audits them by replay, and points
`book.md`, `book.body.md`, and `score-documents/` at those assembled documents. It
also writes `book-line-review-queue.tsv`, which groups source review rows by editable
book line and preserves source line ids. The defaults are the measured source-break
settings (`y=25`, `gap=160`, merged width `500`, component width `250`, overlap
`0.30`, newline joiner).
The integrated CDC replay artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay-line-assembly-integrated`
(`book-run.json` SHA-256
`f2591d47ace1ceb5bc676d3d8c645f6905ab250e1e04fcfbe083ac4cac9f1e25`). It has 27
source review rows and 26 book-line review rows because the two reviewed maths top
fragments collapse into one assembled book line. Its score is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-multi-page-cdc-h20-structural-book-replay-line-assembly-integrated/metrics.json`
with SHA-256 `184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244`.
There is now a book-level artifact audit: `tools/audit_limbu_book_output.py`. It checks
book-run hashes, aggregate markdown, both review queues, score projection, per-page
output audits, and the line-assembly replay audit. The integrated CDC replay audit
passes with 0 issues; `limbu-book-output-audit.json` SHA-256 is
`f5470c9ca04d7a3004f8ed1e3ea3fa280da768f29b65e1fd60fa7f8d42f2f6f5`. This is an
artifact-consistency audit, not an OCR-quality claim.
Book-line review corrections are now actionable with
`tools/apply_limbu_book_line_reviews.py`. It reads accepted rows in
`book-line-review-queue.tsv`, writes corrected book markdown/documents and
score-documents, and emits `book-line-correction-audit.json` plus
`book-line-correction-pairs.jsonl`. It applies corrections to the assembled book-line
projection while recording source line ids; do not use it as evidence that source OCR
fragments were individually corrected.
There is also an explicit book OCR candidate gate:
`tools/admit_limbu_book_ocr_candidate.py`. It requires the book artifact audit,
required page/line metrics, blank/extra/missed-line rates, reading-order accuracy, and
improvement over the `0.458432` CDC baseline. The integrated CDC line-assembly run
passes operational admission
(`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-multi-page-cdc-h20-structural-book-replay-line-assembly-integrated/limbu-book-ocr-candidate-admission.json`,
SHA-256 `8e0110ad320cbe09f37893a30725c25db0cd3d40a2b1c77cc8f0b0ab982c6b9d`) with
absolute codepoint-CER improvement `0.011876`, but remains `claim_safe=false` because
`claim_eligible_rows=0` and only 3 pages are scored. The next improvement should lower
missed-line rate / improve real frozen coverage, not just repackage this candidate.
The admission gate now supports explicit grapheme-CER and WER non-regression baselines
(`--baseline-grapheme-cer`, `--baseline-wer`; zero regression allowed when supplied).
This matters because the recognizer-probe auto-application is a mixed result, not a new
clean best. Selecting the rank-1 refined-crop machine prediction improves page
codepoint CER to `0.442597`, line recall/F1 to `0.806452` / `0.769231`, missed
references to `6`, and reading order to `0.634409`, but it regresses grapheme CER to
`0.639017`, WER to `0.866565`, and matched-line CER to `0.186047`. The paired
comparison against current best is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-detected-tall-recognizer-probe-vs-current-best/comparison.json`
(SHA-256 `5dff9bbce7158c56fd0489452f60ddd30f8b3cd04ddf31301852665ef4e5fbc8`) and is
not claim-safe: codepoint improvement is `0.003959` over 3 pages with CI low
`0.000000`, while grapheme CER regresses by `0.012289`. The strict multi-metric
admission at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-detected-tall-recognizer-probe-application-conf045-delta005-strict-multi-metric/limbu-book-ocr-candidate-admission.json`
(SHA-256 `812fb6e5d19553441e97eada4154879c369e4fcd3924d5717e537f77ef066cdb`) correctly
rejects it for grapheme and WER regressions. Current best remains h20 source-break
line assembly. Do not promote the probe row without reviewed text or a recognizer path
that removes the WER/grapheme regression.
The candidate-only missing-line evidence is now also bound into a book-level review
artifact, so a reviewer does not have to know about the separate detected-tall packet.
New tool: `tools/prepare_limbu_book_missing_line_review_queue.py`. It reads the current
book output and a candidate-only prioritized review TSV/JSONL, verifies every candidate
sample id belongs to a book page, hashes each crop, and writes a separate pending queue
without changing `book.md` or `score-documents/`. Current code also writes
`page_context_path` and `page_context_sha256` when the source page image is available,
with the candidate bbox highlighted on the full page; the audit replays those context
hashes. This keeps candidate insertions page-aware instead of forcing reviewers to judge
from isolated crops. Newly generated queues also write
`book-missing-line-review-dashboard.html`, a static reviewer dashboard with the isolated
crop beside the highlighted full-page context, and bind its SHA-256/card count in the
queue report. This is an operator review surface, not another eval harness. The real
historical artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-detected-tall-prioritized/`:
4 rows, 2 high-priority body-line candidates and 2 structural page-marker candidates.
Report SHA-256 `081cabed352868c26ad5d35fbc77bafc6d95b8ff27281de4d39484e15eadfc20`; TSV
SHA-256 `4ee99c552a00282e3f1bee0b4511d23b90b072d51adac15e829f9054d7237bf5`; Markdown
SHA-256 `fd318583ec70ba178fce876cb6f622ca9e7e35ac2aee77a4cf4af754ad66d01a`. It is now
audited by `tools/audit_limbu_book_missing_line_review_queue.py`; the real audit passes
with 4 pending rows, 0 accepted rows, 4 verified crop hashes, no issues, and SHA-256
`13c9ba67c47ac3bc9159606c72d3e77ce3d4fac1657ea563102d6b7078b49a58`. This advances
the review-packet side of the book OCR objective, but it is not a text-quality
promotion. The rows must still be reviewer-accepted/corrected or solved by a better
non-regressing recognizer/layout path before they can enter editable output.
The application lane now consumes this book-level missing-line TSV directly:
`tools/apply_candidate_line_reviews.py` maps `candidate_row_id`/`candidate_sample_id`
to the older `row_id`/`sample_id` fields. A real pending smoke lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-application-pending/`:
all 4 rows stayed pending, `applied_count=0`, `skipped_count=4`, insertion pairs are
empty (`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`), and the
application audit passes (SHA-256
`cd824e7d7b1b7f7a4a439fe398716c9849acc2bcd3694a3f73124b6466268488`). `book.md` and
`book.body.md` are byte-identical to the current best h20 assembly
(`6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3` and
`7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff`). This is the
right no-op behavior until someone accepts/corrects rows.
The current reviewer-facing unresolved packet is now unified:
`tools/prepare_limbu_book_unresolved_review_packet.py` combines the book-line review
queue and detected-tall missing-line queue, renders crops for existing low-confidence
book lines, copies candidate-only missing-line crops, and writes one TSV/HTML packet.
Real artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/`.
It has 30 rows and 30 verified crop files: 26 existing low-confidence book lines and 4
candidate-only missing-line candidates. Report SHA-256
`c49c5d69105b24d36556b414f187626516353d2fc15de53fd5f098e7ad0d3adb`; TSV SHA-256
`d03c811db6fbac072491dd63bd48b36329eb30f873fac0f31495a6f907486a58`; HTML SHA-256
`f7f1f514fe77c92d763d8a59d19ce14c39019123b14e0aeecf6794caa077b45f`. Direct integrity
check: 0 missing crops, 0 crop-hash mismatches. It now has an independent audit via
`tools/audit_limbu_book_unresolved_review_packet.py`; real audit SHA-256
`3254beec46d99f9c5b6af9c72b39feefca8d97948e7e60f126088ee949886289`, with
`audit_passed=true`, `operational_review_packet_safe=true`, 30 pending rows, 30
verified crops, no issues, and no warnings. This is review evidence only, not a metric
improvement.
The packet-to-application bridge is now wired. `tools/prepare_limbu_book_unresolved_review_packet.py`
preserves `candidate_structural_role`, so candidate page markers remain distinguishable
from body text after review. It also carries explicit `split_corrected_texts` and
`split_bboxes` columns for reviewer-approved split-line repairs. The regenerated
packet has report SHA-256
`3af6d2b767aca7d566529a01f909e850231fca33cba5636c9b042008b07ed5c4`, TSV SHA-256
`9ecd72b9f804124357cef566e58cc655cac676d0e4c81421093aa6a98d416fed`, and audit
SHA-256 `396dc11cdb95ed3d632698fa9bf777e65f0ba260abbcd3debe1be2fec390e91c`
(`audit_passed=true`, 30 pending rows, 30 verified crops, no issues/warnings). New
tool `tools/split_limbu_book_unresolved_review_packet.py` splits the unified packet
into the two applicator-specific TSVs under
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/split-for-application/`:
book-line TSV SHA-256 `83a40bdd0863d2cb38ed347c100493e24dcbf71b48e01b2d2fb373f6d2fbd890`,
candidate-line TSV SHA-256 `d6b5426520cccb95be66b9f0ef8c7daf220aa5fd0d9f4896b3682678fa6bca39`,
split report SHA-256 `586c688f704b1cc09bd93c096d51dc8ab1c7b252cc1cf0b6ae0eb436cf323050`.
`tools/apply_limbu_book_line_reviews.py` now writes a chainable corrected
`book-run.json`, so the candidate-line inserter can run after book-line corrections
instead of requiring the original source run. The pending chain source run SHA-256 is
`738e38a500450f7bd2fb1a516e1a42d08b0e688af9b6bb7b89ab87dfe71e07ef`; pending
application audit SHA-256 values are
`d06a569829174db6aa680e4a9716b9587485fcc6487860cd147fc5897b3e9411` for book-line
correction and `09dfec36a41bff5ee6c1b0287099734da7de264379f5d544da3cce16169c3eaf`
for the chained candidate-line application. Pending-row smoke applications with
`tools/apply_limbu_book_line_reviews.py` and `tools/apply_candidate_line_reviews.py`
skipped all rows and kept `book.md` / `book.body.md` byte-identical to current best
(`6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3` /
`7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff`). This closes
the mechanical gap from one reviewer-facing packet to a corrected-and-inserted book
projection, but it still does not improve OCR until rows are actually reviewed.
The review workflow is now one command via
`tools/apply_limbu_book_unresolved_review_packet.py`: split unified packet, apply
existing book-line corrections, then apply candidate missing-line insertions against
the corrected projection. `tools/apply_candidate_line_reviews.py` now writes a final
chainable `book-run.json` with updated page/document paths and line counts. Real
pending application output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-application-pending-unified/`.
`tools/apply_limbu_book_line_reviews.py` can now deliberately split one accepted
existing book-line row into multiple `TextLine` records via `split_corrected_texts`,
using reviewer-supplied `split_bboxes` when available. Wrapper report SHA-256
`d557307628c2b2c39a9df05d1dba056608003f10999021990457be163c70fbb6`;
final `book-run.json` SHA-256 `b9b7696471e28ca702385987a00ae05fa8a9df3b6f2018d894d75c1d83f421aa`;
final independent candidate-application audit SHA-256
`245b56e4be9e9f1c89f777b45d1f7583f113c9e13d113dbaf412fb4ac2cd0f34`. The wrapper
itself is now audited by `tools/audit_limbu_book_unresolved_review_packet_application.py`;
real audit SHA-256 `cb6e3b9748c20282ca045df766c8220293ef274bbf7cc3f665717be6a1fc2913`
with `audit_passed=true`, `operational_application_safe=true`, 30 packet rows, 0
accepted rows, 0 book-line applications, 0 candidate-line applications, 0 issues, and
0 warnings. The real run split 30 rows cleanly (26 existing book-line, 4
candidate-missing-line), applied 0 because every row is still pending, and kept final
`book.md` / `book.body.md` byte-identical to current best. This removes operator glue
from the review loop; it does not promote text until rows are reviewed.
The missed-reference review coverage audit now exists:
`tools/audit_limbu_book_missed_line_review_coverage.py`. It compares frozen page-eval
`reference_unmatched` rows from `line-alignment.csv` against the unresolved review
packet by bbox overlap, so it is diagnostic-only and must not leak reference text into
OCR output. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-current-best-plus-unresolved-packet/`;
report SHA-256 `743d5c446797e0df0c360a78cea516bbcbb42cfb8de70843504ee0ac784ded07`, TSV
SHA-256 `ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37`.
Result: all 7 frozen unmatched reference rows have review-packet coverage at the
region level: 2 `covered_by_candidate_missing_line`, 2
`covered_by_existing_review_region`, and 3 `swallowed_by_existing_review_region`.
This changes the next engineering move: stop chasing broad detector recall for this
candidate. The remaining missed-line failures are mostly geometry/segmentation inside
already surfaced regions, especially oversized maths rows swallowing short right-side
fragments. Next work should target bbox tightening / row-split handling for existing
book lines, then rescore on the frozen pages.
There is now a bounded split-suggestion aid for that reviewer work:
`tools/prepare_limbu_book_line_split_suggestions.py`. It reads the unified unresolved
packet, verifies crop hashes, and writes visual column-projection subspans for
oversized pending `existing_book_line` rows. It also writes an HTML review page with
original crops beside overlay PNGs. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-suggestions-current-unresolved-packet/`;
report SHA-256 `428eb35644115487aa48530db7b27d3854fcdb3c2603c49f1b3376c5fdba2f6d`,
TSV SHA-256 `b5890b32a355718666530c23085197e7a2404524bdbb78f04bc8da6921681c43`,
HTML SHA-256 `a9b416efb8ea5045e6bdd8af9723916274914b95f30621b05c616b84e0c129d1`.
It now has an independent audit via `tools/audit_limbu_book_line_split_suggestions.py`;
audit SHA-256 `9b7bdf8ae83ed79b2ee36b3d8031ebe26ce930a6321f996761bbd8dab9f81671`,
`audit_passed=true`, `operational_review_aid_safe=true`, 19 suggestion rows, 109 total
visual subspans, 19 overlay PNGs, 0 issues, and 0 warnings. It covers swallowed-row packet IDs
`existing-0005-limbu-devanagari-cdc-maths-p0008-img-line-0007` and
`existing-0011-limbu-devanagari-cdc-maths-p0008-img-line-0013`; the narrow
right-fragment packet row
`existing-0007-limbu-devanagari-cdc-maths-p0008-img-line-0008` did not produce a
multi-segment suggestion under the current gates. Do not overclaim this artifact:
several suggestions are visual word/span segments or non-text visual spans, not
guaranteed logical book-line splits. They are reviewer hints for
`split_corrected_texts` / `split_bboxes`, not automatic OCR recovery and not a metric
improvement.
There is now also a split-prefilled unresolved packet:
`tools/prefill_limbu_book_split_suggestions.py` copies audited split suggestions into
`split_bboxes` for matching pending `existing_book_line` rows while leaving
`review_status`, `corrected_text`, and `split_corrected_texts` unchanged. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled/`;
report SHA-256 `9bc4d3f245133bc572c218536425ce3935b0839a5f0983176dce9853865437e7`,
TSV SHA-256 `b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4`.
It has 30 packet rows, all still pending, with 19 prefilled split-box rows. The
prefilled packet was run through the one-command wrapper at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled-application-pending/`;
wrapper report SHA-256 `507bc6d51b6d1a1c299ec62e9f3ddaeb12773ecdbcda827caaaa7168f5a26598`,
wrapper audit SHA-256 `13763cd2fdb437e7addb6a830a808a957a756b1ed4eb88aa3fafb7e6891d1be1`,
final `book-run.json` SHA-256 `abb937646cfa736da056dc46d5edfba57392435b6be642388a16f6cd783a88ca`.
The audit passes with `operational_application_safe=true`, `accepted_counts={}`, 0
book-line applications, 0 candidate-line applications, 0 issues, and 0 warnings; final
`book.md` / `book.body.md` are byte-identical to current best. Next useful work is
reviewed split text for these rows, then applying and rescoring the frozen pages.
There is now a segment-level transcription worksheet for that review:
`tools/prepare_limbu_book_split_transcription_worksheet.py`. It explodes the
split-prefilled packet into one row per suggested visual segment, crops each segment
from the source page image, records crop hashes, and writes TSV/HTML grouped by packet
row. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-current/`;
report SHA-256 `5eaa08ad0b2535d21989f2ece15dc65d8d36a46695670b5c326776b72f8cb01b`,
worksheet TSV SHA-256 `0b6e75a24515d404a7d8e33a2097b882f99af22732aca6eee1fdf3f1d84f2c0e`,
worksheet HTML SHA-256 `8da8216f287924df90cd206153e64cb9f75bdf35fe8f5cd6ccafc5c274732ebd`.
It has 19 source packet rows, 109 segment rows, and 109 segment crops. Some visual
segments are non-text spans, so reviewers should mark those in `segment_status` rather
than transcribing them as text.
There is now a recognizer candidate-text layer for that worksheet:
`tools/prefill_limbu_book_split_transcription_worksheet.py`. It verifies the worksheet
report and segment crop hashes, writes a segment-crop label file, and merges recognizer
predictions into candidate-only columns while leaving `segment_text` empty and every
`segment_status` pending. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-recognizer-prefill-current/`;
prefill report SHA-256 `60796a0265abff4117701d489a3982dfdfeaf66f9050e2efff4302c8caf608f9`,
label file SHA-256 `1076d642bce34ddb2b4ff9059c42ced0728ba359bd19e1f2625b33ece7390b84`,
prediction TSV SHA-256 `ecf15827b3f8ce58338984421f93889f7150baf3cd2ac82c1824b209278b4ce7`,
prediction summary SHA-256 `c03e881417fbf26e9862b1d9316065cdaa904d18ab111c17f79e03546700b4af`,
augmented TSV SHA-256 `8a7cfe05fa8434178e6ee2464326e25e7ced465ee5da7594bf954d336c6c75e0`,
and HTML SHA-256 `fe8012dce16587551fc548ef6a870c58cf401692a66909f1127e754546355948`.
It has an independent audit via
`tools/audit_limbu_book_split_transcription_worksheet_prefill.py`; audit SHA-256
`2fa090dbdad4d6a36078c254828eec38799dfef3d50687a67af2b53315b3f32c`, passing with
109 label rows, 109 prediction rows, 0 changed review fields, 0 issues, and 0
warnings.
The Devanagari CDC PP-OCRv5 recognizer ran over all 109 segment crops with 0 failures:
100 suggestions are above `min_confidence=0.65`, 8 are low-confidence, and 1 is empty.
This is review acceleration only. The compiler accepts this prefill report as an
alternate worksheet source after review edits, and the unchanged real prefill compiles
to 0 rows with output TSV byte-identical to the split-prefilled packet (pending compile
report SHA-256 `f8cd51be1baaa288182a8ad30cf5966e2a1639fb6b0f9dbaa0e2b631e7119b4b`).
That bridge now exists:
`tools/apply_limbu_book_split_transcription_worksheet.py`. It compiles completed
worksheet groups into `split_corrected_texts` plus matching pruned `split_bboxes`,
excluding segment rows marked non-text/drop/rejected, and can mark completed packet
rows corrected with `--mark-status corrected`. Current real pending output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-application-pending/`;
report SHA-256 `2f148e4b870be32648d5ee545c71b79626464f4457f3772f30ad100c7e168ec1`,
output TSV SHA-256 `b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4`.
Since the worksheet is still unreviewed, `compiled_count=0`, all 19 groups are skipped
as `pending_segments`, and the output TSV is byte-identical to the split-prefilled
packet. Tests prove the completed path flows into
`tools/split_limbu_book_unresolved_review_packet.py`; next real work is actual segment
transcription, then running this compiler, the one-command applicator, and frozen-page
rescoring.
There is now a one-command reviewed worksheet application wrapper:
`tools/apply_limbu_book_split_worksheet_review.py`. It compiles the worksheet, applies
the resulting packet through `tools/apply_limbu_book_unresolved_review_packet.py`, and
runs `tools/audit_limbu_book_unresolved_review_packet_application.py`. Real pending
run:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-worksheet-review-application-pending/`;
wrapper report SHA-256 `96a5974c02d72c59d200895027a89a6069b701e9ee4d291ed5732c2bf8eadec3`,
compiled-packet report SHA-256 `137b53deaad0a0716fecc3d043382a88a0d9de0a716f47acf008111de50cf0ad`,
application report SHA-256 `fc3fe47f0badda0651689e92c2c45560e288840921ada0d8738bc4e14c4f75c3`,
application audit SHA-256 `d82737206bf8238dbb949470f58df7d55006f4a6ce532ae459360c89db8fa7a7`.
Since the worksheet is unreviewed, `compiled_count=0`, `accepted_counts={}`, 0
book-line applications, 0 candidate-line applications, and final markdown is
byte-identical to current best. Fixture tests prove the reviewed path replaces a merged
line with split text lines and the pending path is a no-op. Next actual OCR movement
requires filling worksheet segment text, running this wrapper, then scoring/admitting
`application/final-book/`.
Two automatic recognizer-prefill diagnostics were run after that. First,
`tools/promote_limbu_book_split_transcription_prefill.py` promoted complete
high-confidence segment groups directly into split worksheet text. This is rejected:
the real run promoted 13 groups into 89 text segments, and strict admission failed
because line F1 collapsed to `0.171429`, reading order to `0.421053`, missed-line rate
to `0.612903`, and extra-line rate to `0.889908` even though WER improved to
`0.711653`. Visual segments are not logical OCR lines. Second,
`tools/promote_limbu_book_split_prefill_as_line_text.py` joined complete
high-confidence segment suggestions into the original existing book-line text while
clearing split fields, preserving page/line layout. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups/`;
promotion report SHA-256 `abebf54f05c80c78d9c9d60459a132d4274e0dfb0d454a38c40bdaad782091bf`,
promoted packet TSV SHA-256 `72dee88f38b8d3f217b0cc706f3b8d3b5d2bdcb376c8331cf29a6c1ceddea8dc`.
Application output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-application/`;
application report SHA-256 `1332aba3eeca1d83e400b523eca36fa972f195674c31b66c7b2d5461ca72e1a3`,
application audit SHA-256 `c482d60733a43e0981b38c8a84d8a8b4ed69167494d1117f064edd291f67c968`,
final `book-run.json` SHA-256 `930f0b0475a8b9459e104a145ca5f7e9634e68cb739d9df60b0e4829bc8509a0`.
Strict score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups/`;
wrapper report SHA-256 `9c8b2383c63c56c7dc1e21ac35c775462b026e2557afa61edc9fc5ebb79ba13e`,
metrics SHA-256 `d277042a488bcc2956d2492150504614a88bde077e30f34a0a417afb7fc620c1`,
admission SHA-256 `a0279029f9c346bf51cbbdb5a3c89edfc9db68f0d37dfed49dd3d73e4c21a598`.
This line-text diagnostic is admitted operationally with `claim_safe=false`: codepoint
CER `0.399842`, grapheme CER `0.511521`, WER `0.711653`, matched-line codepoint CER
`0.123640`, line precision/recall/F1 `0.727273` / `0.774194` / `0.750000`,
missed/extra/blank rates `0.225806` / `0.272727` / `0.000000`, and reading-order
accuracy `0.611765`. Treat this as the current best operational diagnostic candidate,
not as claim-safe OCR, because the text was auto-applied rather than human reviewed.
The reviewer-safe version of that same path now exists. The line-text promotion tool
now writes an HTML evidence packet and can run with `--review-status pending`, leaving
all rows unapplied while storing the 13 joined recognizer suggestions in
`corrected_text`. New audit tool:
`tools/audit_limbu_book_line_text_review_packet.py`. Real pending review artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending/`;
promotion report SHA-256 `ce389aa01e11f95e1815cef01f2acbe6ff63debb7f74f58045c234c12ee4fada`,
TSV SHA-256 `ea2f56874371b0b146217d7fa83e883f5a2793ce5672b0038066a5837314d13e`,
HTML SHA-256 `08679a25155a34ac6578a8f0e55d0e153d770f8e22d6930e22282ea755d7f215`,
audit SHA-256 `acac7ff3b890c9cf888a1ca24ed1e3142d3ae55d097651c5ec65c3fcb2d4c8f3`.
The audit reports 13 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. It also checks every TSV `packet_row_id` appears in the HTML,
so the four `candidate_missing_line` rows are now visible under "Other Pending Packet
Rows" instead of being hidden by the line-text view. For post-review edits, rerun the
same audit with `--allow-reviewed`; it permits accepted/corrected rows only when they
have explicit `corrected_text` and a non-machine reviewer identity, permits dropped
existing-book-line rows only with a non-machine reviewer identity, and records edited
TSV hash drift as a warning while still validating row/crop/document references.
No-op application proof:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending-application-pending/`;
application report SHA-256 `f5b143765f8c4cfdbc3d00f7c1100d2f2adbadac4d4a305e62fecd3f6b16d393`,
application audit SHA-256 `187d1e7653d97e869ba72128879f7dfaf5017dcd5539edfbddc3eb075901d4b2`.
The normal unresolved-packet applicator applies 0 rows from the pending packet, and
final `book.md` / `book.body.md` stay byte-identical to the current best. Next operator
step: review the HTML/TSV, mark only verified rows `corrected` or `accepted` with a
real reviewer identity, then use the guarded wrapper
`tools/apply_limbu_book_line_text_review_packet.py` instead of applying the TSV
directly. That wrapper runs the line-text audit with `--allow-reviewed`, rejects
pending-only packets, rejects accepted rows with missing/machine reviewers or blank
`corrected_text`, rejects dropped rows with missing/machine reviewers, applies the
unified unresolved packet, audits the application, and can run frozen-page
score/admission with `--score-out`. Real smoke: invoking it on both
current pending line-text packets correctly fails with `line-text reviewed packet has
no audited accepted/corrected/drop rows ready for application`, because no human-reviewed
rows exist yet.
Low-confidence calibration is now explicit. `tools/promote_limbu_book_split_prefill_as_line_text.py`
has `--allow-low-confidence-above`, leaving strict defaults unchanged. At threshold
`0.60`, it admits three borderline low-confidence segments (`0.637542`, `0.609537`,
`0.600665`) and promotes 16 rows. The auto diagnostic passes operational admission but
is mixed: codepoint CER worsens to `0.406176` from the strict `0.399842`, while
grapheme CER improves to `0.508449`, WER to `0.706189`, and matched-line codepoint CER
to `0.114738`; line F1 and reading order stay `0.750000` and `0.611765`. Evidence:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-threshold060-complete-groups/`;
metrics SHA-256 `f4e1aa3c32e93ec94c845aab7e4f335a9876c22e69cdd6f41ee57797abdd0f81`,
admission SHA-256 `f6c095b40839f3657aa507af2eecf1fc849b1eaa351f39c4bf4f88787afb5731`.
Do not replace the strict 13-row codepoint-best diagnostic with this automatically.
Instead, use the threshold-0.60 pending review packet when a reviewer can inspect
borderline segments:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-review-pending/`;
report / TSV / HTML / audit SHA-256
`26cdf80ca5280fdc07440ab7bbaa73ab82d6b6aa633801e26de54312e24cc64c` /
`6f86cd9966d219e98f8182ef8a090adff41715a8492a573eb1316092142bb3b3` /
`4c907d94c090cc79d83af760c8876d8584968a8ff25b5f4678a9b1fd24a59893` /
`7ea916b354cc3e74761261f5adb7e6c8ceabc992956bf66463c2af189eff8405`.
It has 16 pending suggestions, 30 pending rows, 0 accepted statuses, 0 issues, and 0
warnings. Pending application applies 0 rows and has application audit SHA-256
`688820c7d9ce6b552b9dc53db68eeb65d809ef9206f11a76c4cfdec4055f85f5`.
Preferred review packet for the next human pass:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending/`.
It was generated with `--allow-low-confidence-above 0.60 --suggest-candidate-missing-text`,
so it has 20 pending suggestions: 16 existing-line suggestions plus 4
candidate-missing-line `corrected_text` suggestions. Report / TSV / HTML / audit
SHA-256:
`4ae24b061e5d2c36bc467bafe31134c1c1d1e51de405ab21ebf25c8690f1f07c` /
`d4f4438160031a799cde0eecfb7cea4f8e85b662b3c4b356363131880379e364` /
`3dd21d3dd677b68cc415d7df6da8892c1527bd626600e17f9282666c120b722b` /
`42fb1372dc06c7db3d6484044dfc2fc752a50573c9363880257b1206c43ba76e`.
The pending application at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending-application-pending/`
applies 0 rows and audits cleanly (application report / audit SHA-256
`aa2a106ef8719c5e8f3094dced4ced1883b987a9db13fe6c45e0416acfaa49cc` /
`3ec74b70dec0233d8569b3dcc784aa4e1ca859673288fd76b666ca8cd1f7d51d`). Use this packet
when reviewing both existing-line text corrections and missing-line insertions.
Missed-reference coverage audit for that preferred packet:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-threshold060-line-text-suggestions/`.
Report / TSV SHA-256:
`68565c47c276d53a00af08002b0f734d1cc4094da2ebb82ec7b88a4d2ee401c4` /
`ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37`.
The audit uses the frozen threshold-0.60 line-alignment CSV and the preferred
line-text TSV. It finds 7 missed references total: 2 covered by `candidate_missing_line`
rows, 2 covered by existing review regions, and 3 swallowed by larger existing review
regions, with `uncovered_or_weak_count=0`. This means the current missed-line failures
are review-visible, not solved automatically. Do not call the pending packet claim-safe
or use its machine `corrected_text` as final OCR until human-reviewed rows are applied
and rescored.
Important correction after checking extra predictions: the previous preferred packet
covered all missed references but left one high-confidence extra prediction uncovered,
`limbu-devanagari-cdc-social-p0009/img-line-0001` (`थिक्सिङ् - थिक्`). The packet
generator now has `--line-alignment-extra-predictions`, adding `prediction_unmatched`
book lines as `extra_prediction_line` review rows unless they are already in the
confidence-driven review queue. The superseding packet for human review is:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-pending/`.
It has 31 pending rows, 20 pending suggestions, 0 accepted statuses, 0 issues, and 0
warnings. Promotion report / TSV / HTML / audit SHA-256:
`d5650dc1b67ae9881e6b4ccad5a5c1cacf8a744f02fc6af11bce3ce587bcda44` /
`5e0e618adc8229fea734b8356be7aa3fe6f2c4f09737f9a5685ed15c284ac468` /
`978fb40593d192b23c78f4f240e75c754cf29f404957ad104e6c4683d885ea50` /
`e1053e353a4408c33008926a4923d3d25c93decbd943441d6e8fcc892f398b4d`.
The raw unresolved packet also audits cleanly now. Report / TSV / audit SHA-256:
`55ed8e0c6cfceeb609aeea3cf85862819488c6a59df422b935e817d36350d155` /
`866e40ae4c5d037a347a3740c57610601f54071aa3deeefeed994f1a48d3a601` /
`92d66a8333c387478b4d5bcb5d0fdbf51b231671aa5e1ce60c1ac8b0e26db5a2`.
It has 31 pending rows, 31 crops, 4 candidate-missing rows, 27 existing-book-line
rows, exactly 1 `extra_prediction_line`, 0 issues, and 0 warnings. The raw-packet audit
now accounts for `extra_prediction_line_row_count` separately and verifies the
line-alignment CSV used by `--line-alignment-extra-predictions`.
Structural coverage artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-review-coverage-threshold060-line-text-missing-extra-suggestions/`.
Report / missed TSV / extra TSV SHA-256:
`ae9b7903a719110e77cfbe8089fc0cc0f025bd6f93804cdec4ac1d5a7b24b775` /
`ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37` /
`ad610c8019489fc03c041dfa1f0eb35e31bf76a304f8cf5731b3469752a21d51`.
This coverage audit finds all 7 missed references review-visible and all 9 extra
predictions review-visible (`extra_uncovered_or_weak_count=0`). The guarded application
wrapper correctly rejects the packet while all rows are pending with
`line-text reviewed packet has no audited accepted/corrected/drop rows ready for application`.
The application path now supports actual reviewed extra-line removal. Existing
book-line rows with human `drop` / `reject` / `delete` statuses are removed from the
document, their line ids are removed from blocks, emptied text blocks are dropped, and
drop counts are propagated as `dropped_counts` and `book_line_dropped_count` through
the split, book-line application, unified application audit, and line-text wrapper.
Real smoke:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra/`
marks only `limbu-devanagari-cdc-social-p0009/img-line-0001` as rejected by
`cdjk-review-smoke`; all 20 machine suggestions remain pending. Smoke packet report /
TSV SHA-256:
`1a8fc27bfc3fd9a3a3c47abc766df7abfb97301855c7f4b3725e2ef2722cb1b8` /
`91287f19aaecf89c7735fad7221b71789c605d59bfc71296d7002ac959167120`.
Application:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra-application/`;
application report / audit SHA-256:
`1806662aa1318a340df70067990e345a0e79e363a04ea8891c1858da837a5a01` /
`bcfd4ef72fc93ed7c0a08d26a67df0f2953e44450249747f5a0c4f06f1326f9b`.
It applies 0 accepted rows and 1 dropped existing line, audits cleanly, and scores via
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-missing-extra-smoke-reject-extra/`.
Score/admission report / metrics SHA-256:
`7501fe80695cd1adb87428a94ef3f9d7c6b416200279f3a6d0b16d57e37af08b` /
`7bccf3236644d2d73d2ddc7864f85de64dbe7681b91d41ad3fab7d48d8a45e89`.
Operational admission passes, `claim_safe=false`; codepoint CER `0.436263`, grapheme
CER `0.615975`, WER `0.823739`, line F1 `0.761905`, extra-line rate `0.250000`,
missed-line rate `0.225806`, reading-order accuracy `0.611765`. Treat this only as a
pipeline smoke, because the reviewer identity is synthetic.
Mixed review smoke:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke/`
copies the superseding pending packet and applies one existing-line correction, one
accepted `candidate_missing_line` insertion with text `१`, and one rejected extra
prediction, all under reviewer `cdjk-review-smoke`. Packet report / TSV SHA-256:
`14ca2c76ea261c79730d96480d6e7a4bb901a807d24a306cc4aabb490d50983d` /
`7641025877f89c519d371bc3df613b4a98fa9a4874bf06c9b0ff2993a8602fff`.
Application:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-application/`;
application report / audit SHA-256:
`8b1b983b34c65dba327788959a3200e176dc86639bb50a15f22b792cb5637c31` /
`58095326d648202df10aecdda244b470fba2601c9bf5d82e2c9edeb563dd9617`.
Application audit passes with `accepted_status_count=2`, `dropped_status_count=1`,
`accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1}`,
`dropped_counts={"existing_book_line": 1}`, `book_line_applied_count=2`,
`candidate_line_applied_count=1`, and `book_line_dropped_count=1`. Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke/`;
admission report / metrics SHA-256:
`03adcbacac5f4f1fac68d9f73bf9c85c33f024931c6cff33ab3106f5878bfff1` /
`e3ddf0d31b7bed313f9e22a05be0583e3afa7187cc8a29785b406f6b84b2cc5c`.
Operational admission fails, `claim_safe=false`; codepoint CER `0.438638`, grapheme
CER `0.614439`, line F1 `0.781250`, extra-line rate `0.242424`, missed-line rate
`0.193548`, matched-line CER `0.174901`, and reading-order accuracy `0.553191`.
Conclusion: the combined correction/insertion/deletion review plumbing works, but this
is not a promotable candidate. The next blocker is accepted missing-line insertion
placement and reading-order calibration.
That blocker has a narrow fix for structural page markers. `tools/apply_candidate_line_reviews.py`
now places reviewed candidate insertions with structural role `page_marker` before
body text, keeps ordinary text insertions geometry-placed, and records
`placement_policy` in the insertion metadata. Rerun artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-application/`;
application report / audit / final book-run SHA-256:
`bac62fc812eaed939ceecf7e36fd7ccb0a2329ae7644d2755df0077487c1b179` /
`fc264875a4876dceadccbbdf221b3771726000a4e50518fba9df60d92b63ba40` /
`879547c4827fee26900f5f7d49a7462df0793fd6d7a689bfecf90618f507b56a`.
It audits cleanly with `accepted_status_count=2`, `dropped_status_count=1`,
`accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1}`,
`dropped_counts={"existing_book_line": 1}`, `book_line_applied_count=2`,
`candidate_line_applied_count=1`, and `book_line_dropped_count=1`. Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter/`;
admission report / metrics SHA-256:
`43705c1d33ce7c5f23d04452a364cf1316efa084b90ea6b67b35f458c754a60a` /
`a3718bbab380b2c44d406d938db00bf3fdaa2021d24d1f683eb944419a76d65d`.
Operational admission passes, `claim_safe=false`; codepoint CER `0.435471`, grapheme
CER `0.608295`, WER `0.812810`, line F1 `0.781250`, extra-line rate `0.242424`,
missed-line rate `0.193548`, matched-line CER `0.174901`, and reading-order accuracy
`0.648936` (`61/94` matched-line pairs). This remains synthetic smoke evidence, not
human-review claim evidence, but the mixed review path now passes the operational gate
without relaxing thresholds.
Structural front-matter ordering is now applied in the real projection paths, not just
candidate insertion: `src/ocrtech/structure.py`, `src/ocrtech/image_lines.py`, and
`tools/apply_limbu_book_line_reviews.py` sort explicit `page_marker` blocks before
body text and treat bottom-margin Devanagari digit-prefixed rows like `२ ...` as
front-matter rows for cases where the page marker is fused with neighboring
title/header text. Rerun artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-row-application/`;
application report / audit / final book-run SHA-256:
`c7ea956cdec75c2aefc9260e0e30c30710f77f5e84bc19b6be53948b3e103408` /
`8efc363d1895ed955e8fc692fb10ae1a6ff312f9f146464065e8eb6696a9a657` /
`d012af48bb7c64d65572cfa70ff135a6a8321aa1658c29ae1c26467fb82fedba`.
Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter-row/`;
admission report / metrics SHA-256:
`47ee4098aaa2fc92c9d053c1fc8cfeb3093528a5556dcf450656af8c6b356837` /
`7201ed8390b0bce13ecc5f7aca6643b1b6d77f7be99c1b7b12548460f3150182`.
Operational admission passes, `claim_safe=false`; codepoint CER `0.455265`, grapheme
CER `0.632873`, WER `0.821451`, line F1 `0.781250`, extra-line rate `0.242424`,
missed-line rate `0.193548`, matched-line CER `0.174901`, and reading-order accuracy
`0.787234` (`74/94` matched-line pairs). This is a layout-order tradeoff, not a final
OCR win: reading order improves, but raw page CER is worse than the page-marker-only
run because pending low-confidence text moved into reference order before human
correction.
The text-first best candidate was rerun through the same structural ordering logic:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-application/`.
It applies 13 existing-book-line corrections, leaves 17 packet rows pending, and
applies no candidate-missing insertions. Application report / audit / final book-run
SHA-256:
`d3821317c37f4c914febdfce29e9e113f383d074d2132bd85b619fa68add7ef3` /
`5ebad12995fdb520dcd98daf02c614f0e3d8d6fbff23db441bead8804d4ce05b` /
`902dce10d52e2b54ee9bf3e6ed2e2e0b5d15ef746ad921a28df5c20610fc06bb`.
Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order/`;
score/admission report / metrics SHA-256:
`f24b4f8a3abd66ac89c475ed9cc98e54a402e2369b587a0780dc19cbf85473a7` /
`979827dc2ccdbc4f6c3ea45ee97c1bcc0151200f8c5eaeafb31a7928c72524d4`.
Operational admission passes, `claim_safe=false`; codepoint CER `0.404592`, grapheme
CER `0.520737`, WER `0.722582`, line F1 `0.750000`, extra-line rate `0.272727`,
missed-line rate `0.225806`, matched-line CER `0.123640`, and reading-order accuracy
`0.682353` (`58/85` matched-line pairs). This is now the better text-first diagnostic:
it keeps most of the old best-text CER while improving reading order, but it does not
recover missed lines.
The page-marker selector has also been repaired for the newer unified TSV packet shape:
`tools/select_candidate_line_reviews.py` now accepts TSV as well as JSONL, derives page
size from each candidate row's `book_document`, parses `max_selected_iou` from
`review_reasons`, and checks duplicate markers against the post-review text implied by
accepted existing-line corrections from the same TSV. On the complete-groups packet it
selects maths `१` and science `२`, because accepted row `existing-0019` removes the
leading digit from selected `img-line-0014`; the two body candidates remain rejected.
Selection report / selected JSONL / rejected JSONL SHA-256:
`f52f61ab09ba6f2101ae8049815a01257dd5b57e93c9aab492f2125fefbf2058` /
`7e7b37359d5b4dc8972220054b33f2ffeb388ff7d02a621972035a19a573baa9` /
`40a9ae5a0fb174694cdc1e04d108adb52900b2864bce43aabf4557878f21feaf`.
The applied candidate
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application/`
has applicator audit / independent application audit / book-run SHA-256
`26dc4049979618330eb8016fad699af82465719d7e88749540309adcc40261cc` /
`e135fcb2faa7b1a125c2f706fae41e1fb448dfe918390f434f5075895fc540d4` /
`61d6dfc09e1b7e09f0fd1cf57b5aac988b6c8e65f567f2eda6c19e33928ff775`.
Score/admission
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2/`
passes operational admission, `claim_safe=false`; score/admission report / metrics /
admission SHA-256 are
`9a4f3bca208d515b1e7538536a1abb54000749d4694e2317a981ca3588064ba3` /
`fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4` /
`1bbd1d216d3c3b23def126332aecf1052abf254a0f15891dd58048c103cc2205`. Metrics:
codepoint CER `0.402217`, grapheme CER `0.516129`, WER `0.711653`, line F1
`0.757576`, extra/missed `0.285714` / `0.193548`, matched-line CER `0.123518`,
reading-order accuracy `0.712766` (`67/94`), 3 structural page-marker predictions, 2
matched structural lines, and 2 exact structural-text matches. This is now the best
operational layout candidate, but not the actual endpoint: body-line candidate text is
still noisy, and the next gain needs reviewed body-line recovery or a real line
split/remove path.
The body-line recognizer-probe lane was refreshed on top of that page-marker-v2
candidate, and it should **not** be promoted. Refreshed crop probe:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-text-crop-probe-from-complete-groups-page-marker-v2/`
(probe / JSONL SHA-256
`9e64044df9ec4070dcc28b78a2e0f48c2fd6cb8009fdab1de41a36eb2591a61c` /
`31e8ecbf17ab207c67c3f46510f66771e4f2802315cf052ee52aaaa858ae68d4`). It uses the
two body rows rejected by the TSV-backed page-marker selector; refined crop hashes are
byte-identical to the older probe, so the existing PP-OCRv5 prediction TSVs are valid.
Refreshed comparison:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-crop-recognizer-probe-comparison-from-complete-groups-page-marker-v2/`
(comparison / JSONL SHA-256
`12fd030c336f3d1cdef882de8512f56a5f58e7bd513d301b4dab0fd95418b329` /
`f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d`).
The non-oracle selector
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-recognizer-probe-selection-conf045-delta005-from-complete-groups-page-marker-v2/`
selects one body row (`confidence=0.469282`, delta `0.218220`) and rejects the other
for low confidence/delta (selection / selected JSONL SHA-256
`41655846f1845d1a966a72e7def29b38e58bee53bf5472493a6c3bda536e1dc1` /
`205415b49c984145cbf573aa1e7a65547a931a202bc8789daa58ebe10d64c2b4`). Applying it
required fixing `tools/audit_candidate_line_review_application.py` so chained
applications count only insertions from the current review file and report inherited
candidate insertions separately. The real body-probe application has `applied_count=1`,
`inserted_line_count=1`, `total_inserted_line_count=3`, and
`inherited_inserted_line_count=2`; application audit / book-run SHA-256 are
`d89b9c8966c5c1a01e74bb5edbe3ab19c08054b9f915d9aa5fad1863199f6d99` /
`abe8f59007aa77fb13bf690070230346aa14dc88040fa2b18b8d80ef2fd332de`. Score/admission
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe/`
passes against the old CDC baseline (`claim_safe=false`): codepoint CER `0.441013`,
grapheme CER `0.562212`, WER `0.760834`, line F1 `0.776119`, missed rate `0.161290`,
reading order `0.656863` (score / metrics / admission SHA-256
`b52a853cf05196c618a416fc981cf747b05cc178a6435fc35a0d497380611992` /
`e2307c02624fd601f2ad5674b3405e62620c2565993d3dcbf2425e5b8fd74f79` /
`2b55cb5073c523c2aa69c69f52251452a80ab89e762554074b29d21361db8b6c`). However,
paired comparison against page-marker-v2 current best at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-body-recognizer-probe-vs-page-marker-v2-current-best/`
shows codepoint CER regression `-0.038797` and grapheme CER regression `-0.046083`
(comparison / paired TSV / markdown SHA-256
`1bafa22b69296c98f9186ade50cce68d509491e0b77de8be80c45bb9a4d3a209` /
`cd89984c2dfc9a8e6792b15b1b193348716d9a5c0c140d048278843fbead732f` /
`38104641a4c8744233fa90669395f994047121e21aba25c7af7602741dccf3a8`). Interpretation:
this is useful review evidence and an audit-chainability fix; keep page-marker-v2 as
the selected operational layout candidate.
Admission has now been hardened against this exact false promotion. The old gate could
admit a candidate that beat the historical `0.458432` CDC baseline while regressing
against the actual current best. `tools/admit_limbu_book_ocr_candidate.py` now accepts
`--champion-metrics`, and `tools/score_limbu_book_ocr_candidate.py` forwards it; when
set, codepoint CER, grapheme CER, and WER must not regress beyond the configured
`--max-champion-*` tolerances. Real check: the body-probe candidate fails at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe-champion-gated/`
with champion deltas `-0.038797` codepoint, `-0.046083` grapheme, and `-0.049180` WER
(wrapper/admission SHA-256
`4126d1ca9c4e644190d862301f199f3f46335fb1c7b48191209c6e70e3f46214` /
`ff1425910714bacd54f96640d8f2281c08d78c93f07b0d43c7be3117d9a1c4e1`). The
page-marker-v2 self-check passes at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-gated-self/`
with zero champion regression (wrapper/admission SHA-256
`6103f40ec19bdde710fd1438ff8547382118fbbdadc9f96ce65425767a5ce75d` /
`3b22736a064654783b9a8cee5a455936d7e86cc1d759d1e7536bb5ea69eb727c`). Next sessions
should use champion-gated admission before promoting any new book OCR candidate.
The unresolved reviewer packet has also been refreshed against page-marker-v2 current
best. `tools/prepare_limbu_book_unresolved_review_packet.py` now accepts JSONL missing
line queues, so the selector's rejected candidate rows can be used directly, and it
filters non-pending rows out of unresolved packets. This fixed a real stale-queue
problem: the current book run still points at a queue containing 13 already-corrected
rows. Current packet:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-extra-predictions/`.
It has 18 pending rows and crops: 13 existing low-confidence book lines, 3 extra
prediction lines from page-marker-v2 alignment, and 2 rejected body-line candidates.
Report / TSV / HTML / audit SHA-256:
`1e004bb5c6b206a2ad3faf147e1f54cea3c7d442300b71be295a3c2d9743c5ab` /
`fe8908da4050e90f17d4a6ccbf111ba04b5752491bccb19a01d11b5853bdf84a` /
`17694e6aa1224dec5e3edc12cb742e07d0be9989aa66f9718399d48e3fef23c7` /
`10e95db8204c47bb4fec39264e2914510fed82544221f541f486bebf3f3d26be`. The packet audit
passes with 0 issues. Pending application:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-application-pending/`
applies 0 rows, audits cleanly, and leaves final `book.md` / `book.body.md`
byte-identical to page-marker-v2 current best. Use this packet for the next reviewer
pass; do not go back to the stale h20/current-best-plus-detected-tall packet unless
you are intentionally reproducing old evidence.
Coverage for that packet has been refreshed too:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/`.
Against page-marker-v2 `line-alignment.csv`, 6 unmatched reference lines are all
review-covered: 1 by candidate missing-line row, 2 by existing review regions, and 3
swallowed by larger existing review regions. `uncovered_or_weak_count=0`. All 10
unmatched predictions are covered by existing review regions, with
`extra_uncovered_or_weak_count=0`. Report / missed TSV / extra TSV SHA-256:
`32026349ec549e142036042cf3b6bc2713a41b0438f4c5220b3afb082d7e7abc` /
`596c77aa6770a28467dafa89a8f7122bd080bb274ef98bacdfba7521566d8ad8` /
`95f41b789b6ae9aadf93b4273d1c96b7a39f2e119f1500e7e8187ec9ee350b91`. The current
review packet is therefore coverage-complete for current missed/extra line surfaces;
the blocker is now reviewed correction or safe automatic recovery, then champion-gated
score/admission.
That coverage evidence is now part of the candidate admission contract. The admission
tool accepts `--review-coverage-report` with default maxima of zero uncovered/weak
missed references and zero uncovered/weak extra predictions; the score/admit wrapper
forwards the same option. Current page-marker-v2 was rerun through both champion and
coverage gates at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/`.
It passes operational admission, `claim_safe=false`, with zero champion regression and
coverage values `{missed_reference_count: 6, uncovered_or_weak_count: 0,
extra_prediction_count: 10, extra_uncovered_or_weak_count: 0}`. Wrapper / metrics /
admission SHA-256:
`0018bfb5ee2f48821c3c9b1101d0921bbad54c41cd63be120dc72437fe9f1344` /
`fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4` /
`b9ece419592b3bfc3dbc892880c206a2cbcb24a8777e0c245d1b7614afa30f43`. From here on,
do not promote a book OCR candidate unless it passes champion-gated and
coverage-gated admission.
The two-script coverage registry now points the Devanagari-written Limbu page target
at this real operational candidate instead of the stale live-Tesseract page recipe.
`configs/eval-harness-recipes-v1.json` adds
`limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated`, and
`configs/eval-target-coverage-v1.json` sets the `limbu-first-two-script-ocr`
Devanagari target to `eval_status=registered_real_pack` with
`page_eval_recipe_id=limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated`.
Validation is clean but not claim-ready:
`uv run python tools/validate_eval_harness_recipes.py --recipes configs/eval-harness-recipes-v1.json --root . --out outputs/eval-harness-recipes-current-validation.json`
reports `recipes=12 valid=True claim_ready=0`, and
`uv run python tools/validate_eval_target_coverage.py --coverage-config configs/eval-target-coverage-v1.json --normalization-config configs/recognizer-eval-pack-normalization-v1.json --eval-recipes configs/eval-harness-recipes-v1.json --root . --out outputs/eval-target-coverage-current.json`
reports `valid=True coverage_ready=False target_sets=3 gaps=50`. The new Devanagari
page recipe itself is valid with zero structural issues, but its claim blockers are
real: only 3 CDC pages, 0/3 claim-ready references, `curriculum_text_unreviewed` GT,
unfrozen/non-claim-eligible page pack rows, missing hash-verifiable checkpoint/config/
dictionary artifacts, `claim_safe=false`, and 18 unresolved review rows. The top-level
`limbu-first-two-script-ocr` set now has 6 remaining gaps: Limbu/Sirijanga
normalization, recognizer recipe, and page recipe are not claim-ready; Devanagari
normalization, recognizer recipe, and page recipe are not claim-ready. This is progress
because the registry now represents the real current Devanagari page candidate, not
because the two-script OCR claim is ready.
The Sirijanga page recipe also had an avoidable harness defect: it was declared as a
`sidecar_fixture` page pack but still had a placeholder `REPLACE_WITH_LIMBU_PAGE_OCR`
command. `tools/copy_page_ocr_document.py` now provides the missing reproducible replay
step by copying a pre-materialized `{sample_id}/document.json` bundle from a source
documents directory into the recipe-run `documents` directory while checking the input
page path and recording hashes. `limbu-sirijonga-gorkhapatra-page-current-router` now
uses that tool to replay
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/operational-image-line-width480-minwidth005-conf070-documents`,
has `requires_operator_resolution=false`, and records concrete checkpoint/config/
dictionary paths. A real recipe smoke at
`outputs/eval-recipe-smoke-limbu-sirijonga-gorkhapatra-page-current-router/` succeeds
with `audit_passed=true`, `claim_safe=false`, `ocr_output_kind=sidecar_fixture`, 1 page,
codepoint CER `0.739361`, grapheme CER `0.768262`, WER `0.918969`, line F1 `0.872458`,
matched-line codepoint CER `0.046585`, and matched-line reading-order accuracy
`0.775478`. This removes the unresolved-command blocker; it does not make the page
claim-ready, because the run is a replay of one machine-converted unreviewed page with
0 claim-eligible rows.
The current selected Sirijanga diagnostics now have textless unresolved-region review
packets, matching the book-OCR goal's requirement to surface missed/extra-line regions
instead of only quoting CER. Rollup:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/README.md`.
Gorkhapatra operational `conf070` has 69 rows/crops: 67 missed references and 2 extra
predictions; audit passes with 0 hidden-reference leaks and 0 issues. Refreshed packet
/ JSONL / HTML / audit SHA-256 are
`a503d752571efedc7ccb1ad43f3826c9f6c9744a7244f444a855136b4d252b2a` /
`af8deaa96f9f76ae34342583fd4166c9045637896a085c1387b5ae8dc25e65f2` /
`54739ca9f38cd72b31b9927424a0afd33719ba8f87ac8cc450d9472d4173523f` /
`78dc40e5d99e5c6e5d377296048e73f21660d2660ecd3617c59beba5629cad6d`.
Its failure surface is center-column missed text: missed reference text is 83.8643%
of attributed codepoint error mass, led by middle/center, bottom/center, and
top/center bands. Sikkim same-row merge has 22 rows/crops: 9 missed references and 13
extra predictions; audit passes with 0 hidden-reference leaks and 0 issues. Refreshed
packet / JSONL / HTML / audit SHA-256 are
`d84ac4662411a59bff76987566f5207f73b343dc1f50f1ab49a18f66914c65b4` /
`03798089ab870985388d72abe7ed545c137c172f333bad4b08f00e73217815cc` /
`470beaf4310aa26e3a1ea272822ae25cf204775ae4afbc31e39bf0808cbca8e5` /
`544cd11e33d617579f5f93256c249975f9bb3a692cd6857356a51cbdcf4388e3`.
Sikkim is a mixed failure: missed reference text is 31.6140% of attributed error mass
and extra prediction text is 28.1198%, with top header/large-line misses prominent.
`tools/apply_unresolved_page_review_packet.py` now applies reviewed page-packet rows to
scoreable page-document projections. Pending rows are no-ops; accepted/corrected
missed-reference rows require `corrected_text`; accepted/corrected unmatched
predictions update only when `corrected_text` is supplied; false positives are deleted
only by explicit `drop`/`delete` statuses. Pending application smokes exist for both
diagnostics and score identically to the selected source outputs: Gorkhapatra
`0.739361` codepoint CER and Sikkim `0.418057` codepoint CER. Neither packet nor
pending application is claim evidence; they are review/recovery substrate. Coverage is
now independently audited with `tools/audit_unresolved_page_review_coverage.py`:
Gorkhapatra covers 67/67 missed references and 2/2 extra predictions, and Sikkim
covers 9/9 missed references and 13/13 extra predictions, with zero duplicate or extra
packet keys. Coverage report SHA-256 values are
`bf36f850104d2b993349ebc8d53a22bd1835a8cfbd8fc6860ec45a0fdf9b8045` for Gorkhapatra
and `6de438a4cf7ae08ed0fff29f834622d6024f0e53dccb6923c830841c48d67a7b` for Sikkim.
The latest Sirijanga Gorkhapatra candidate-line diagnostic proves the next lever is
reviewed/corrected missed-line recovery, not blind promotion of high-recall detector
branches. Selected `conf070` page documents compared against the rejected
auto-split-tall candidate set produced 100 candidate-only rows; the packet audit
passes with 0 issues (`candidate-line-review-packet.json` SHA-256
`68f678a77e0b2056c25ecbc1595cf62e63c4ef72aa923d79355a4a9886998ff4`, audit SHA-256
`ac645bb4755bab878c4e65749735d86856c0dcb9e473399bf064db869e8df56c`). An
oracle/reference-overlap selector chose 49 rows against the 67 currently unmatched
references (selection SHA-256
`45fa5a781d62736a2aa3ef5c97762755d0b65ad99f1c7984856e585ab30e5128`), but this is
diagnostic-only and not deployable policy. Applying those rows with candidate OCR text
through `tools/apply_candidate_page_line_reviews.py` inserted 48 lines, skipped 1
empty-text diagnostic row, and reached excellent geometry metrics: line F1 `0.964346`,
precision `0.993007`, recall `0.937294`, 19 missed references, and 2 extra
predictions. The text output is worse than selected `conf070`: page codepoint CER
`0.766779` and grapheme CER `0.798907` (metrics SHA-256
`0467192299d469d02e3ad1b14c9be669a6561e08263098da858d337ea3267da8`). Do not
auto-promote auto-split-tall candidate text. The safe non-oracle artifact is
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-prioritized-candidate-line-review/`:
100 body-line candidates, 99 high priority, 1 medium, audit passed with
`operational_review_queue_safe=true` (audit SHA-256
`0c9cde56dbec68f4a146cf417cee033271814ce7c5887058841028798f2a6985`). Use it as a
review/transcription queue or as input to a better crop recognizer.
The first non-oracle crop-probe bundle for that queue is also prepared at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-text-crop-probe/`.
It processed 100 body-line candidates, refined 44 by trimming top bleed from multi-row
crops, and copied 56 as full crops. Report / JSONL / label file / HTML SHA-256:
`dc5de12cda741533cc31e343147b4e12488b4ee1611e1e8cdb81a038206725c3` /
`9ad687ca53e229565132dc579a79efd6adb6572aadad54fc5c61294fceca3fc5` /
`2c5ae66ea3468f97f456ebcfa4f979703e65ded921f3850054b357cbeb015f2b` /
`46ae06c2412961b02fb9f9f8db7b30fd81afcc702cb8fb79b672fe7fe41094c8`. Use this for a
targeted Sirijanga crop recognizer/reviewer pass; it does not alter selected OCR text.
That pass has now been run. `tools/predict_recognizer_eval.py` had to be fixed first:
Paddle sometimes returned multiple text lines for one crop, and the old writer emitted
raw embedded newlines into `predictions.tsv`. The tool now flattens newlines/tabs
before writing a TSV field. With that fix, the packaged v4 Sirijonga recognizer ran on
100 original candidate crops and 100 refined crop-probe images with 0 failures
(original/refined prediction TSV SHA-256
`8f32f8895ded0a4a1c405333206ebfc55df37606ff1b288ef4319c7067060fc0` /
`66c4fa2d25b46bf24c99ad9936000ec0b73a932478cf1e74e613cabe09f79fd3`). The comparison
has 100 rows, 44 refined rows, 40 changed predictions, max confidence `0.992109`, and
`automatic_application_safe=false` (report SHA-256
`8b2133b39c3d139b8a83d10858301bcad0e92cd821a4042bbf4aeb0bdf27064d`). A threshold
sweep gives one near-miss but no clean default: `0.90` confidence / `0.05` delta
accepts 6 rows and improves page codepoint CER to `0.737090`, but line F1 drops to
`0.866545`, missed references remain 66, and extra predictions rise to 7. Do not admit
this as book OCR output; use it as reviewer/crop-recognition evidence.
There is now an explicit effect audit for this class of mistake:
`tools/audit_candidate_page_line_application_effect.py`. It compares a candidate
page-line application to the selected baseline across page CER, line F1, missed/extra
rates, reading order, and inserted-row alignment. The strict `0.90` / `0.05`
recognizer-probe application is rejected with `book_ocr_objective_safe=false`: 6
inserted rows, 1 matched row, 5 extra predictions, and the matched insert itself has
line codepoint CER `0.888889`. Effect audit SHA-256
`08eff93810712cafe46009ac8aa5a1d6dcd00c3206f2cd5b88aff527c7e0cd82`. A page-CER-only
gain must not override this gate.
The latest feature analysis makes the next move sharper.
`tools/analyze_candidate_line_review_oracle_features.py` compared the 100
auto-split-tall candidate-only rows against oracle selected/rejected labels without
writing hidden reference text or OCR text into the safe feature output. Artifact:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-oracle-feature-analysis/`
(report / TSV SHA-256
`4c45a640338b6510b63bced5b903b847e4725ab87f544fb36ba7def7ac2e2bb0` /
`21e31ff240b2a40220b70b9f029237064b83c951509b6bea62ec9fc4855098bf`). The 49
oracle-selected rows are longer and wider than the 51 rejected rows: candidate text
length mean/median `29.306`/`37` versus `4.471`/`1`, bbox width mean/median
`295.531`/`307` versus `127.137`/`74`. They cluster in body-like bands. Refined
recognizer confidence is a bad selector here: selected rows have mean/median refined
confidence `0.382649`/`0.146180`, while rejected rows have `0.601181`/`0.751105`; all
tested confidence/delta selectors top out at `0.500000` oracle precision and
`0.183673` recall. Do not run another confidence-only promotion loop. The next
worthwhile automatic experiment is a non-oracle geometry/text selector plus the page
effect audit.
That experiment has been run. `tools/select_candidate_line_reviews_by_geometry_text_features.py`
selects candidate-only rows from packet-visible geometry/text features only. With
`min_candidate_text_length=30`, `min_bbox_width=80`, and y-band
`middle,bottom_or_right`, it selects 31 rows; the oracle-overlap diagnostic says all 31
are real missing-line candidates. Applying their machine candidate text gives the good
geometry surface we wanted: line F1 `0.933566`, recall `0.881188`, missed references
down to 36, extras still 2, and all 31 inserted rows matched. It is still rejected as
book OCR because page codepoint CER worsens to `0.761733`, grapheme CER to `0.792281`,
matched-line CER to `0.091202`, and reading-order accuracy to `0.604292` (selection /
metrics / effect-audit SHA-256
`9469f67f8eb1afb300e25bc332e9566fd92f583b77fbc054953027b9883dd444` /
`22689ccec2f4cdf7b0f9f0e4679ef13756b3423d81b5c45d65b9b9779de60e6d` /
`0ba763fb14cfd1e63ff6eb49313ea668ed908986c2533319e18aa4d148364988`). A looser
`min_candidate_text_length=10` setting selects 38 rows, 35 oracle-useful and 3 extras,
and also fails: codepoint CER `0.764760`, line F1 `0.936097`, extra predictions 5,
effect audit SHA-256 `5f4aaef0a472f95757342a3da00ca932eab3453b286d9afb40ba61940cb4d6f2`.
Takeaway: we have a good review/crop-targeting selector, not an automatic text-output
policy. Next work should improve/transcribe those 31 rows and repair reading-order
insertion, not tweak selector thresholds.
Candidate insertion ordering now has explicit modes in
`tools/apply_candidate_page_line_reviews.py`. Keep old `geometry` for reproducibility,
but do not use it as the operational default on this wide-column page: it globally
sorts by raw page/y/x and drops reading-order accuracy to `0.604292`. The new
`preserve_source_append` mode preserves the selected source block order and appends
accepted candidates. On the 31-row precision selector, it keeps all 31 inserted rows
matched and improves reading-order accuracy to `0.750472`, with line F1 `0.933566`,
recall `0.881188`, and extras still 2. It remains rejected because candidate text
worsens page codepoint CER to `0.762153` and grapheme CER to `0.798575` (effect audit
SHA-256 `56068accf83105e4bd0f526f2ddac0d68fb9a234d52899e35b3ec4bae5ce2743`). The
`source_reading_order` mode re-sorts with `wide_column_right_to_left` but is worse here
(codepoint CER `0.773003`, reading order `0.722227`, effect audit SHA-256
`b62ae4e0be13247a0a8c6af34656058c9412c3bb4fce1b2afd922189511c658e`). The practical
choice for reviewed candidate insertions is preserve/append until a better slotting
policy is proven.
The 31 precision-first rows now have a pending transcription worklist:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist/`.
It was created with `tools/prepare_candidate_line_transcription_worklist.py` and
contains JSONL, TSV, HTML, and label files. It is now audited by
`tools/audit_candidate_line_transcription_worklist.py`: all rows are `pending`,
`corrected_text` and `reviewer` are blank, no oracle/reference fields are emitted, crop
hashes validate, and 31/31 HTML crop links resolve. Report / JSONL / TSV / HTML /
labels / audit SHA-256:
`7a92ed3d85faee782f22353a778fa36434c6456ca29182a462349ffcc630b180` /
`a98488a08ed6d28944b171fe50577a9958960bd31cb6cc7180a58b3e5b270ea1` /
`f405d1ede74174f0def525bfa3c5bb9819a3c929671cc74d366a5769e48a31b6` /
`3971d08d5bca15286fcceae6326014ad5edbab89a8be1f2656d5e69703cbee72` /
`1bcf9418af1ee056ccf32bfe56fccc7d51d26ffdef52f51c1f24397eadced5c0` /
`86678953c170569196585aaaf25d6d44fb8d031a7c35f705ad791797d1a0d541`. This is the next
human/reviewer or crop-recognition target.
The reviewed-worklist bridge is also in place.
`tools/apply_candidate_line_transcription_worklist.py` compiles filled worklist rows
into candidate-page application rows, requiring non-empty `corrected_text`, a
non-machine reviewer, valid crop hashes, and accepted/corrected status. Pending rows
are no-ops. The current real 31-row worklist is pending-only, and the pending compile at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist-application-pending/`
has `accepted_count=0`, `skipped_count=31`, `issues=[]`, and
`application_ready=false` (report / accepted JSONL / skipped JSONL SHA-256
`a5116e0509ab3612fb83f21298397d57f10a92f32a4f2936f6a2284050577fce` /
`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855` /
`2f2aaf09512a659629a6cd54928261a3971a4fce07ec0ecc51009331ce18c39e`). Applying that
empty accepted JSONL with `preserve_source_append` inserts 0 rows and scores
identically to selected `conf070`: codepoint CER `0.739361`, grapheme CER `0.768262`,
line F1 `0.872458`, 67 misses, 2 extras, reading-order accuracy `0.775478`; metrics
SHA-256 `5e75442ea41ea8600ef6dd06fc2b246ad61615f5cea6a3107b7e0e3cb090e32b`. Once a
reviewer fills accepted rows, compile -> apply -> score/effect-audit is now the path.
Do not assume that filling those 31 rows is enough. A new diagnostic-only oracle
ceiling tool, `tools/select_candidate_line_reviews_by_oracle_subset.py`, joins the
31-row geometry/text worklist back to the oracle/reference-overlap selected rows and
sets `corrected_text` from the reference only in a clearly marked unsafe artifact:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-oracle-reference-ceiling/`
(31 selected, 0 rejected; report SHA-256
`6eff8712c007c9278401ee2f8e9465b8bc31ab1161878802f41da56d0c75d5a3`). That ceiling
still fails page OCR. With `preserve_source_append`, line F1 improves to `0.933566`,
recall to `0.881188`, misses fall to 36, and matched-line codepoint CER improves to
`0.039297`, but page codepoint CER worsens from selected `conf070`'s `0.739361` to
`0.771993`, grapheme CER worsens to `0.802385`, and reading-order accuracy regresses
to `0.750472` (metrics / effect-audit SHA-256
`cb9b60eab8ff64bbf3f277eda29f2353537b308e03e67878e292714767df6ef4` /
`f55a168c8a7cbda06faf87ed25417b6194ad55e4781096697b66998d15cb4d9a`). Raw geometry
ordering is also unsafe: page codepoint CER `0.771825`, grapheme CER `0.794103`,
reading-order accuracy `0.604292` (metrics / effect-audit SHA-256
`034846c66650367f64aeb34808a51ad807a9ecce18fedc7e79e4bcf864397bd7` /
`c821571fc0efdf80f996585d4eac802af978e0d824233efda59568587a219ff6`).
I then added `candidate_document_order` to `tools/apply_candidate_page_line_reviews.py`.
It maps existing selected lines to the candidate document by bbox IoU and orders
inserted rows by their candidate-document `candidate_line_id`; this is non-oracle
slotting evidence. On the 31-row oracle ceiling it is much better: codepoint CER
`0.739024` slightly beats selected `conf070`'s `0.739361`, line F1 is `0.933566`,
recall `0.881188`, misses 36, extras 2, matched-line codepoint CER `0.039297`, and
reading-order accuracy `0.775506`. The effect gate still rejects it because grapheme
CER worsens to `0.778367` from `0.768262` (application / metrics / effect-audit
SHA-256 `383af5e3dffe2cbf697185f70de706fc314f2864dac0d33ae8b7a9442ed90f6c` /
`473781d0b05563b8833e94ff639628712ec52965f9b71c45cc630269bf9d9f1b` /
`0e779661c45c5361025acbfee7afabdfb4996f44bc5920cc3e43c9ac4cc1a673`). The next
serious work is no longer generic slotting; it is candidate-document-order slotting
plus grapheme-level text/normalization repair for the inserted lines.
The same order mode was tested without oracle text on the non-oracle 31-row
geometry/text selection. It fixes the ordering surface but not recognition: line F1
`0.933566`, recall `0.881188`, misses 36, extras 2, and reading-order accuracy
`0.775506`, but page codepoint CER is still `0.763078`, grapheme CER `0.797416`, and
matched-line codepoint CER `0.091202` (application / metrics / effect-audit SHA-256
`2093b60eb1d924b461ba37c69b606963a1c8692e96d75c1e9e78223c99d708b8` /
`92d518739384e624c405efcec00222e7f75546ae72217f4838e1477a8279e10c` /
`c9939d8d3d3d8949ab2bcfe3ee4eac9dfbec95986670fcb253c30663aa9797f7`). So the current
operational rule is strict: candidate-document-order is the right insertion mode for
reviewed rows, but machine candidate text remains rejected.
The 31-row oracle ceiling has now been ablated by reference span group. No subset or
group combination improves/ties both page codepoint and grapheme CER under
`candidate_document_order`: `span0071-0075` is a codepoint win (`0.732380`) but
grapheme regresses (`0.778532`), `span0242-0266` worsens both (`0.754079` /
`0.782011`), `span0268` worsens both, and `span0310` improves grapheme (`0.767765`)
but worsens codepoint (`0.740118`). Single-row page-text diagnostics also found no row
that improves both. So the next policy is not "choose a safer subset of the same 31"
unless the admission gate changes.
A diagnostic-only oracle reference-order ceiling was then built:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry31-oracle-reference-order-ceiling-application/`.
It uses hidden eval alignment to sort existing selected lines and inserted rows by
reference line ID, so it is not deployable. It proves the objective is attainable with
better ordering: codepoint CER `0.102355`, grapheme CER `0.136823`, line F1
`0.933566`, recall `0.881188`, misses 36, extras 2, reading-order accuracy `1.0`, and
effect audit `book_ocr_objective_safe=true` with 0 issues. Application / metrics /
effect-audit SHA-256:
`8a8120c71e37cd7ca849853bfc1799383521bb8ec4572aaf0b7ffd64608d82a9` /
`3dacf8b231d118d3cdc61b4b41b70908b539cc883bf620feb4749ad13eca08b8` /
`1346510c0fdbafa1bd4b2957bf906b8aedf6328a35b23e8ede4c581986928b12`. The actual next
engineering target is now clear: learn a non-oracle page reading-order/layout policy
that approximates that reference order.
To make that target concrete, I added
`tools/analyze_page_reading_order_geometry.py`. It reads a page `line-alignment.csv`,
sorts matched rows by reference line ID, and emits geometry chunks/jumps. Real artifact:
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-oracle-reference-order-geometry-analysis/`
with 267 matched rows, 21 chunks, and 20 jumps. This proves the reference order is
article/block-level, not a simple right-to-left column grid: top chunks include
`span0019-0025` at median `cx=1138`, `span0034-0051` at `cx=1486`,
`span0052-0069` at `cx=1791`; middle body chunks include `span0160-0187`,
`span0188-0215`, `span0216-0241`, `span0242-0267` at median `cx=205/521/838/1155`.
Report / chunks / jumps SHA-256:
`cd52bb8574864f243ad883b71df25afceb59553aa55ee93d1df079b92ce09fe5` /
`d476d35bd727272960ef7c9dd6841c4e4143d596bb9776e54e437a682e7427b6` /
`48e3d079a9780705438ebd6367013266912f08efe1f39cb2d86f70a510a2d041`. Use this as the
target profile for the next non-oracle ordering heuristic.
I then added `tools/apply_page_reading_order_geometry_profile.py`, which applies a
geometry chunk profile to a page-document projection by assigning each line to the
nearest profile chunk and sorting by chunk index then local line position. With the
profile derived from the oracle reference-order analysis above, the tool reproduces the
layout ceiling on the 31-row oracle-text projection: codepoint CER `0.095711`,
grapheme CER `0.129700`, line F1 `0.933566`, recall `0.881188`, misses 36, extras 2,
reading-order accuracy `1.0`, and effect audit safe with 0 issues. Profile application
/ assignments / metrics / effect-audit SHA-256:
`485b5f0132abf71f8830fd8a6fc1ebefc396311407eaacd684efc87144a00f15` /
`4a735308a2458ecf94e25baecb5d08dfca4964b03bd793469fafdf53281715f4` /
`5e50ddc10ce9b7b58469fe8ba0c7ec5eca15ce6faf005adf59a6a327e5d66102` /
`a2fe4a37d755146b0224a88ee8370af86a67a72f0c5c275da4b01d93a31e498b`.
Applying the same oracle-derived profile to the non-oracle machine-text 31-row
candidate projection is also a large diagnostic win: codepoint CER `0.143146`,
grapheme CER `0.198112`, line F1 `0.933566`, precision `0.992565`, recall
`0.881188`, misses 36, extras 2, matched-line codepoint CER `0.091202`, and
reading-order accuracy `1.0`; the objective effect audit is safe with 31 inserted
lines and 0 issues. Profile application / assignments / metrics / effect-audit
SHA-256:
`0819ebbbe70d36a98baf147b02b2c6828520d5173a3276a3b847325dccac4ec9` /
`9d1461b936b30d837528a2d6b04bee7d34d7d5ce4218f584dacd882a1f5b5e28` /
`8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189` /
`0565e1755d190b3855f84d2946194594ce72d74cf116c4c98472a6f330cffd63`.
Do not overclaim this: both profile-order runs use a profile learned from hidden eval
alignment. The operational next step is to build the same kind of profile from reviewed
layout calibration, page templates, or observable article/block cues, then run this
same application/effect-audit path without oracle provenance.
That non-oracle bridge now exists as
`tools/build_page_reading_order_geometry_profile.py`. It reads geometry-only
line-box-review application references, refuses transcription text by default, blocks
pending geometry unless `--allow-pending` is explicit, and refuses mixed multi-source
profiles unless `--allow-multi-source-profile` is explicit. I exercised it on the real
pending Sirijonga line-box packet by first applying the packet as seed geometry:
`outputs/limbu-sirijonga-line-box-review-packet-2026-06-21-geometry-application-pending-seed/`
has 2 samples, 389 kept line boxes, and application SHA-256
`a953ffe6f97e46f84b8c744e11ac2b7f68587370810ce35c5404c842646e0e6e`. Separate
pending seed profiles were then built per layout: Gorkhapatra has 303 lines, 26 chunks,
25 jumps, `profile_claim_safe=false`, profile SHA-256
`fa02502b0d23285979d6ff385e0836e6578676df44b9ef1ad891d0b55eb28f8c`; Sikkim has 86
lines, 83 chunks, `profile_claim_safe=false`, profile SHA-256
`aff45bc2c0b295163176cf8ca420bfa2c13cdcf9956a6a777355e6ceb6183454`. Applying the
Gorkhapatra pending seed profile to the same 31-row machine-text candidate projection
reproduces the strong profile-order diagnostic exactly at the metric-file level:
codepoint CER `0.143146`, grapheme CER `0.198112`, line F1 `0.933566`, recall
`0.881188`, misses 36, extras 2, reading-order accuracy `1.0`, metrics SHA-256
`8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189`, and effect-audit
SHA-256 `184a2eb9ed9c32cd37b50a85f45dce21ffedbf5b1bca20ea3c6e116cad2a08c4`. The
claim boundary is still strict: this is seed geometry from a pending review packet,
not reviewed/calibrated operational evidence. The next concrete task is to complete
the Gorkhapatra geometry review rows and rebuild the same profile with
`profile_claim_safe=true`, then score it on held-out pages.
I also added `tools/audit_page_reading_order_geometry_profile.py` and a hard
`--require-profile-claim-safe` gate to `tools/apply_page_reading_order_geometry_profile.py`.
The pending Gorkhapatra seed profile is structurally valid but not operational:
`reading-order-geometry-profile-audit-structural.json` reports `audit_passed=true`,
`operational_profile_safe=false`, 26 chunks, 1 pending source, and 0 issues; the
required-safe audit intentionally fails with issue `profile is not operationally
claim-safe`. Structural audit / required-safe audit SHA-256:
`9aecaa40b5ee509ad12e495777a857a76fea7d4ad68793e9055e721a163b8877` /
`694486b5a3f28bcdb3093c085814fdb76daf835be5599f26319daca80b70b3ce`. The applier now refuses this same profile under
`--require-profile-claim-safe` with `profile_claim_safe is not true`. This is the
gate to use for any promoted book-OCR run; diagnostic runs may omit it but must keep
the non-claim-safe boundary.
Finally, `tools/admit_page_reading_order_profile_application.py` now binds the
profile-provenance audit, the profile application report, and the page objective
effect audit into one promotion decision. Running it on the real pending Gorkhapatra
seed-profile projection correctly rejects admission even though the page-effect audit
is safe: `admitted_operational=false`, assignment count 269, candidate metrics SHA-256
`8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189`, admission
SHA-256 `02bcd20dc1a5598f92b9a251dd5ae4669171ba8d9429df4eb9c06cf33bc354b6`. The three
issues are exactly provenance issues: profile audit does not mark
`operational_profile_safe=true`, profile audit does not mark `profile_claim_safe=true`,
and the application report was not run with a claim-safe profile. After geometry review
is completed, rerun: build profile without `--allow-pending`, audit with
`--require-claim-safe`, apply with `--require-profile-claim-safe`, run page scoring and
effect audit, then run this admission gate.
The profile applier now also supports multi-page profile routing with `--profile-map`.
The current pending seed map lives at
`outputs/limbu-sirijonga-line-box-review-packet-2026-06-21-pending-seed-profile-map.json`
(SHA-256 `7e94909dacbe9be20d18a70ddbac5fcc0421b0c2a35c2055482b1f2859895397`) and maps
Gorkhapatra and Sikkim sample IDs to separate pending seed profiles. Applying that map
to the current Gorkhapatra 31-row machine-text projection selects only the Gorkhapatra
profile, records `profile_usage`, assigns 269 lines, and reproduces the single-profile
metric SHA-256 `8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189`.
Map application / effect-audit / admission SHA-256:
`e71697d996deb8516db22d243136c89b9440b4b4144556420c2534dce354871e` /
`88db8bc35b8b64fd2da33ddb629afbaadcf5b3a06acaad44460b65b1e78793b7` /
`8764a46251c9e0bc1b2415de9089f5eb67032e9bf9eccf11167f54da8c18aaf6`. The map now has
its own audit too: structural audit passes with a warning that Sikkim is not present in
the one-page Gorkhapatra document dir (SHA-256
`1090ec5254bad3e5d0ec492739488de5a65d684673d33cb36d50f1dde8c7d7d7`), while
required-safe audit fails because both mapped profiles are pending (SHA-256
`bb4c07c4fcb7f2031ee085313d3a537eda01650b77f3af31ce23d605af136d06`). Admission still
rejects the map-driven run because the selected profile and the map are
pending/non-claim-safe, which is the correct behavior. This closes the obvious
multi-page footgun: do not run one layout profile across a book unless a profile map or
template selector chooses it per page, and do not promote a map-driven run without a
safe profile-map audit.
The book wrapper now participates in this path. `tools/parse_limbu_book_pages.py`
accepts `--reading-order-profile-map`, applies it after optional line assembly, and
writes aggregate `book.md`, `book.body.md`, review queues, and `score-documents/` from
the profile-ordered projection. The wrapper also exposes
`--require-reading-order-profile-safe` and `--require-reading-order-profile-map-safe`.
Supplied profile-map audits are now hash-bound to the exact map being applied, and
map-safe mode fails if no `--reading-order-profile-map-audit` is supplied.
`tools/audit_limbu_book_output.py` now checks the profile-map hash, application report,
assignments JSONL, optional map audit, and per-page profile source documents.
This matters because the previous standalone profile applier could produce a good
score projection while a book run still emitted stale Markdown; the integration test
now checks that Markdown and score documents agree, and the book audit catches stale
profile-order artifacts.
## 2026-06-23 correction: stop overbuilding eval harness
The user correctly pushed back that the eval harness work is becoming the project.
Do not keep adding generic eval machinery unless it directly unblocks real Limbu book
OCR. The current objective is operational: run real Limbu pages through the book OCR
path, preserve visible line structure, surface missed/low-confidence lines for review,
and export training rows only after explicit human acceptance/correction. Keep metrics
as thin guardrails around that loop: source inventory, artifact audit, page/line counts,
review packet audit, and training-pack export validation are enough for the next step.
Current source state was rechecked locally because SSH to `cdjk@100.110.148.43` failed
with authentication rejection. The mirrored local data is enough to proceed:
`outputs/limbu-page-source-inventory-2026-06-23-realbook-pivot/` reports 36 records,
3 real Devanagari-written Limbu CDC page candidates, 3 Devanagari text-only sources,
1 synthetic smoke pack, and `ready=True`. The real CDC PDFs are present under
`data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/`.
The existing 3-page CDC page draft has real rendered page images and unreviewed
curriculum text references:
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/`.
The latest CDC book-output artifact audit passes structurally, but it is not a quality
claim: `page_count=3`, `line_count=35`, `review_line_count=27`, and
`book_line_review_row_count=26`.
The practical next move is therefore not another harness phase. Pick a small real page
set, run or refresh the book OCR path, inspect the generated unresolved/missing-line
review packet, fix only the operational defect that blocks review or drops visible
text, then apply reviewed rows and export a reviewed-only training pack. If the export
says `no training rows were exported`, training must not start.
## 2026-06-23 current CDC PDF book OCR run
The current best operational CDC PDF-derived run is:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/`
It uses the current v4 router with recognizers held fixed on rendered CDC maths pages
8..10. The wrapper emitted editable book Markdown, page-layout overlays, page triage,
layout-gap review, and a passing standalone book-output audit at:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-output-audit-lowconf-nontext-guidance/`
Current counts:
- `page_count=3`
- `line_count=93`
- `book_line_review_row_count=21`
- `review_line_count=22` in the parse summary
- `page_layout_review_row_count=3`
- `page_triage_review_row_count=3`
- `layout_gap_review_row_count=9`
- `operator_attention_page_count=0`
- `book_audit_passed=True`
The latest code change is intentionally narrow: `tools/run_limbu_book_ocr.py` now
classifies wide low-density box-border/illustration fragments inside large layout gaps
as `page_graphic_or_illustration` instead of `candidate_text_band`. This was based on
visual inspection of `gap-0001-cdc-maths-p0008-003`; the crop and page context showed a
graphic/exercise box, not missed Limbu text. The real gap queue now has 6
`inspect_page_graphic_or_illustration` rows and 3 `likely_rule_or_spacing` rows, with
0 candidate-text gap rows, and all 3 pages are accepted by page triage. This reduced
review noise; it did not prove OCR text quality.
`tools/parse_limbu_book_pages.py` now also separates low-confidence square registration
corner hallucinations from real text spotchecks. The contact sheet showed three OCR
hallucinations (`े?`) over 60x60 page-corner marks; those are now
`inspect_non_text_or_drop_low_confidence_detection` instead of
`spotcheck_low_confidence_text`. The line-review action counts in the latest real run
are: 9 `inspect_non_text_or_drop_empty_detection`, 3
`inspect_non_text_or_drop_low_confidence_detection`, 1
`transcribe_or_drop_empty_detection`, and 8 `spotcheck_low_confidence_text`.
Contact sheet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png`
Next action: work the 21-row book-line review queue from this artifact, apply accepted
line text/non-text decisions, and export only reviewed rows into a training pack. Do not
return to generic eval-harness expansion unless a specific guardrail blocks this review
to training loop.
## 2026-06-23 cleaned CDC projection
There is now a chainable cleaned projection that applies only the obvious non-text
drop proposals from the current CDC packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/`
Input review queue:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv`
This queue marks 13 rows as `drop`: 9
`inspect_non_text_or_drop_empty_detection` rows and 3
`inspect_non_text_or_drop_low_confidence_detection` rows, plus one wavy illustration or
rope segment that remained as a blank transcribe/drop row after the first pass. It
leaves 8 rows pending, all low-confidence text spotchecks. The application audit reports
`applied_count=13`, `dropped_count=13`, `skipped_count=8`, and the standalone
book-output audit passes with 0 issues. The cleaned projection line count is 68 because
the chainable assembled documents had 81 text lines before these 13 drops; the upstream
run's 93-line count is from the wrapper/source count.
The reviewed-only training exporter was run against this cleaned projection and
correctly refused with `no training rows were exported`; these drops are useful for
editable book cleanup but are not accepted text labels. Training still requires a human
reviewer to accept/correct remaining text rows.
The current human review station for the remaining rows is:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/book-unresolved-review-packet.html`
Packet directory:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/`
It audits cleanly with `tools/audit_limbu_book_unresolved_review_packet.py`: 8 packet
rows, 8 crops, 8 page contexts, 13 non-pending source rows skipped, 8 prefilled text
rows, and source guidance preserved as 8 `spotcheck_low_confidence_text` rows.
`tools/prepare_limbu_book_unresolved_review_packet.py`
now maps `review_priority`/`review_recommended_action` from the book-line queue into
the packet's `review_category`/`suggested_review_action`, so the HTML no longer loses
the review guidance. The packet TSV/export header now also preserves
`page_context_path`, `page_context_sha256`, and `neighbor_context`, so a reviewed TSV
keeps the visual evidence needed to audit what the reviewer saw.
After review, use this base and packet:
```bash
BASE_BOOK="outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2"
PACKET_DIR="$BASE_BOOK/pending-text-review-packet"
REVIEWED_PACKET="$PACKET_DIR/book-unresolved-review-packet.reviewed.tsv"
HANDOFF_OUT="$PACKET_DIR/reviewed-text-training-handoff-v1"
uv run python tools/audit_limbu_book_unresolved_review_packet.py \
--packet-dir "$PACKET_DIR" \
--packet-tsv "$REVIEWED_PACKET" \
--allow-reviewed
uv run python tools/apply_limbu_book_reviewed_packet_to_training_pack.py "$BASE_BOOK" \
--packet-dir "$PACKET_DIR" \
--review-packet "$REVIEWED_PACKET" \
--out "$HANDOFF_OUT"
```
## 2026-06-23 larger CDC PDF review batch
The one-command runner now accepts `--pdf` directly, not just `--page-manifest` or
`--image-dir`. If PyMuPDF is unavailable in the Paddle runtime, the manifest builder
falls back to Poppler (`pdfinfo` + `pdftoppm`) and records `pdf_renderer=pdftoppm` in
the run source summary. This matters operationally: real CDC books can now enter the
same PDF -> render -> capture-prep -> OCR -> review packet path without hand-built
intermediate manifests or installing PyMuPDF into `/tmp/ocrtech-paddle311`.
Real run:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/`
Command shape:
```bash
PYTHONPATH=src /tmp/ocrtech-paddle311/bin/python tools/run_limbu_book_ocr.py \
--pdf data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/mero_ganit_grade1_limbu.pdf \
--pdf-start-page 11 \
--pdf-end-page 15 \
--pdf-dpi 220 \
--page-id-prefix cdc-maths-p \
--start-index 11 \
--zero-pad 4 \
--out outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler \
--auto-deskew
```
It rendered 5 PDF pages with Poppler, preserved 189 OCR lines, wrote 58 pending
book-line review rows, produced 15 layout-gap rows, accepted all 5 pages in page
triage, and passed `tools/audit_limbu_book_output.py` with 0 issues. Review actions:
22 `inspect_non_text_or_drop_empty_detection`, 17
`inspect_non_text_or_drop_low_confidence_detection`, 3
`transcribe_or_drop_empty_detection`, 2 `verify_non_limbu_or_drop`, and 14
`spotcheck_low_confidence_text`. Layout-gap actions are 6
`inspect_page_graphic_or_illustration` and 9 `likely_rule_or_spacing`, with no
candidate-text gaps.
Current larger review station:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context-strict-v2/book-unresolved-review-packet.html`
Reviewer context from the same source PDF text layer:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/limbu-pdf-text-context.md`
This was generated by `tools/build_limbu_pdf_text_context.py`, which parses
`pdftotext -bbox-layout` lines, rejoins tiny-gap legacy glyph fragments, converts them
through `src/ocrtech/limbu_legacy.py`, and writes JSONL/TSV/Markdown/page text sidecars.
The real CDC pages 11..15 output has 82 text-layer lines, 81 lines with converted
Limbu codepoints, 0 unmapped-codepoint lines, and rendered bboxes mapped through the
book page manifest. Treat it as review assistance only: it is explicitly marked
`pdf_text_layer_legacy_converted_review_context_not_gold`, `is_gold_text=false`, and
`training_use=forbidden_without_human_alignment_review`.
`tools/prepare_limbu_book_unresolved_review_packet.py` now accepts
`--pdf-text-context-jsonl` and attaches nearest PDF text-layer rows to the review packet
by page ID and rendered bbox geometry. `tools/build_limbu_pdf_text_line_coverage.py`
also compares those PDF text-layer boxes with cleaned OCR line boxes to generate
candidate missing/partial source-text rows, then the existing missing-line review queue
feeds them back into the unified packet. The regenerated current strict packet has 38
rows: 25 existing-line rows, 13 candidate missing/partial rows, 38 PDF-context matches,
0 unmatched rows, 0 pending prefilled corrected-text rows, and still audits cleanly.
This context is display/support metadata only; it must not auto-fill reviewer
decisions. The HTML `Accept text` action can deliberately copy OCR text into
`corrected_text`, but a status-only TSV edit is no longer enough to create a label.
The first unresolved packet had 58 pending rows. Codex then made conservative
visual-drop passes using crop/page-context contact sheets and dropped only obvious
non-text rows: 20 repeated 60px page-corner registration marks/page furniture,
including the top-right `े?` hallucinations, plus 13 arrows, blank answer rectangles,
and standalone illustrations. It did not drop text-shaped labels or ambiguous glyph
rows.
Cleaned projection:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/`
Proposal queue:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv`
The cleaned book-output audit passes with 0 issues. Reviewed-only training export still
correctly refuses with `no training rows were exported`, because these are drops, not
accepted text labels. The strict combined packet audit passes with 0 issues and now
contains 38 pending rows with blank `corrected_text` by default. Existing-line actions
are 13 `spotcheck_low_confidence_text`, 11
`inspect_non_text_or_drop_low_confidence_detection`, and 1 `verify_non_limbu_or_drop`.
Candidate source-text coverage actions add 13
`verify_pdf_text_line_missing_or_already_split` rows: 8
`pdf_text_line_missing_y_coverage` and 5 `pdf_text_line_partial_x_coverage`. This is
the current larger label/layout-production target. Do not treat OCR text or PDF context
as ground truth until a non-machine reviewer exports accepted/corrected rows with
explicit reviewed text.
Bounded detector follow-up on the same CDC pages: the PDF-text detector sweep artifact
is
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/`.
It evaluated 216 configs and found a high-recall setting with 76/81 covered source
lines, but that setting produces 335 detector boxes. The practical candidate is the
balanced OCR run
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/`:
214 lines, 60 review rows, audit passed, PDF-text coverage 73 covered / 8 candidates.
This is a useful real-page probe, not a new default.
## 2026-06-23 Limbu phone page scorer-pack status
The current Limbu phone-page OCR path has a reproducible scorer-pack metadata
layer now. `tools/prepare_page_ocr_eval_pack_metadata.py` takes an existing
page-level eval manifest, fills scorer-visible `source_image`/whole-page
`bbox`/page metadata when missing, and writes:
- rewritten manifest: `reference-draft/scorer-pack/eval-manifest.jsonl`
- scorer summary: `reference-draft/scorer-pack/summary.json`
- script inventory: `reference-draft/scorer-pack/script-inventory.json`
- run report: `reference-draft/scorer-pack/page-ocr-eval-pack-metadata.json`
Current command:
```bash
uv run python tools/prepare_page_ocr_eval_pack_metadata.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack \
--eval-pack-version limbu-phone-page-reference-draft-v1 \
--force
```
Current scorer-pack digest is
`b20abcbbe4c6f38e39be055d9d618d29d0ecdc3751a5eb38d50384d89fabed8e`.
It has 2 draft pages, 79 reference lines, and 0/2 claim-ready references. The
active filtered OCR documents are still under
`ocr-line-filter-sirijonga-conf050-margin-markers/` and have 78 lines because
the page marker on `phone-pxl-raw-1` was filtered out. Scoring the sidecar
manifest against those filtered documents writes:
`outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run/`
Current dry-run metrics: codepoint CER `0.000975`, grapheme CER `0.001976`,
text exact `1/2`, line ref/pred/match `79/78/78`, matched-line text CER `0`,
matched-line reading-order pair accuracy `1.0`, full reading-order pair
accuracy `0.952381`. The sidecar run removes avoidable scorer warnings for
missing summary, pack digest, script inventory, source document, bbox, page
output audits, and source-path mismatch.
Do not treat this as a benchmark result. Remaining blockers are real: draft
references, 2 pages only, no claim eligibility, non-frozen/not-unseen pack,
selected checkpoint on the draft test, no hash-verifiable checkpoint/config/
dictionary passed into the scorer, no layout references, and no train/eval
leakage manifest. The next useful work is either to review/promote the phone
references or to run this same scorer-pack pattern on a larger verified Limbu
page set.
Verification for the page-phone harness slice:
```bash
uv run pytest tests/test_prepare_page_ocr_eval_pack_metadata.py \
tests/test_audit_page_ocr_documents.py \
tests/test_run_page_ocr_eval.py \
tests/test_limbu_phone_page_reference_review.py \
tests/test_prepare_limbu_phone_page_benchmark_reference_draft.py \
tests/test_prepare_limbu_phone_row_composition_review.py \
tests/test_audit_limbu_phone_page_ocr_run.py \
tests/test_materialize_page_image_line_boxes.py \
tests/test_filter_page_ocr_documents.py \
tests/test_run_limbu_phone_page_ocr.py \
tests/test_prepare_limbu_book_captures.py \
tests/test_capture.py \
tests/test_audit_yolo_line_detector_predictions.py \
tests/test_export_limbu_line_detector_dataset.py \
tests/test_preflight_limbu_line_detector_training.py \
tests/test_build_limbu_book_page_manifest.py \
tests/test_build_limbu_pdf_text_context.py \
-q
```
Result: `139 passed`, with only the existing SWIG deprecation warnings from PDF
rendering imports.
## The one durable lesson
Every result this session points the same way: **real same-font/native data is the
lever, not model architecture.** Sunuwar 0.342→0.121 with zero model changes. When a
native expert is weak, the question is "what's wrong with the training text/font?",
not "what hyperparameter?". Hold that line.
## 2026-06-30 Limbu Space page eval
`tools/run_limbu_paddle_space_page_eval.py` now materializes the current
`spaces/limbu-ocr/app.py` Paddle page output into the existing page-level OCR
`Document` schema and runs `tools/run_page_ocr_eval.py`. The first diagnostic run is:
`outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed/score/`
It uses the Gorkhapatra Sirijonga current-schema eval manifest and `--max-lines 0`.
Metrics: 1/1 scored pages, 303 reference lines, 306 predicted lines, 291 matched
lines, line precision/recall/F1 `0.950980/0.960396/0.955665`, matched-line
codepoint CER `0.033035`, matched-line grapheme CER `0.063509`, but full-page
codepoint CER `0.824390` and grapheme CER `0.846778`. Same detections rescored
under simple explicit reading orders still fail as page OCR: Rows LTR codepoint
CER `0.815223`, Columns LTR `0.824390`, Columns RTL `0.796636`.
Conclusion: do not expand this into multilingual page OCR yet. Limbu line
detection/recognition is strong enough to keep, but the current page system fails
on layout/reading-order reconstruction. The Gorkhapatra reference is still
machine-converted and unreviewed, so these are diagnostic numbers, not claim-safe
benchmark evidence.
## 2026-06-30 Limbu expanded page evals
New/updated tools:
- `tools/run_limbu_paddle_space_page_eval.py`: generalized for Sirijonga and
Devanagari scorer metadata and fixed to emit one block per detected line.
- `tools/build_limbu_gorkhapatra_silver_page_eval.py`: builds page-level eval rows
from `outputs/limbu-gorkhapatra-pdf-backed-silver-2026-06-23-v3/manifests/page-text-image-pairs.jsonl`
and `line-text-image-pairs.jsonl`.
- `tools/apply_page_reading_order_strategy.py`: rewrites existing OCR documents under
simple geometry reading-order strategies without rerunning Paddle OCR.
The new Gorkhapatra silver page eval pack is:
`outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/`
It contains 7 full-page images and 2,129 PDF text-layer silver reference lines. The
builder converts rendered PDF line boxes from `xyxy` to the eval harness `xywh` bbox
format. It must be run with `regex` so grapheme inventory is real:
```bash
env PYTHONPATH=src uv run --python 3.11 --with regex \
python tools/build_limbu_gorkhapatra_silver_page_eval.py --force
```
Current diagnostic scores:
```text
CDC Devanagari Auto:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/cdc-devanagari-paddle-seed/score/
pages 3/3, page_cp_CER 0.376880, page_graph_CER 0.479263,
line_F1 0.906250, matched_cp_CER 0.057724, order_acc 0.592308
Gorkhapatra Sirijonga silver7 raw Auto:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed/score/
pages 7/7, page_cp_CER 0.814444, page_graph_CER 0.851347,
line_F1 0.943981, matched_cp_CER 0.027091, order_acc 0.720352
Gorkhapatra same detections, Columns RTL:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-columns-rtl-score/
page_cp_CER 0.771116, page_graph_CER 0.803203, order_acc 0.777787
Gorkhapatra same detections, old one-page geometry profile:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-profile-order-score/
page_cp_CER 0.736334, page_graph_CER 0.774886, order_acc 0.626211
Gorkhapatra same detections, Rows LTR:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-score/
page_cp_CER 0.441968, page_graph_CER 0.475313,
line_F1 0.943981, matched_cp_CER 0.027091, order_acc 0.993085
```
Important interpretation: Rows LTR is the best simple order for the seven-page
Gorkhapatra silver pack, so `spaces/limbu-ocr/app.py` now defaults Paddle Page OCR and
Paddle Book OCR to `Rows LTR` with max lines `0` (no cap). This fixes the bad UI
default, not the full page OCR problem. Matched-only reconstruction from bbox-matched
Gorkhapatra target lines has roughly 2-3.5% codepoint CER per page, so the recognizer
is doing useful work on the target Limbu lines. Full-page CER remains high because the
page scope is wrong: Paddle detects headers/non-target text and the Limbu recognizer
turns those crops into Sirijonga-looking garbage. The next real fix is target-region
or language/script filtering for page OCR, then rerun these evals.
## 2026-07-01 Limbu page filtering baseline
Back on the main Limbu OCR goal, the next bottleneck was verified as page-scope
filtering rather than recognition. `tools/filter_page_ocr_documents.py` and
`tools/audit_page_ocr_line_filter.py` now support relative page-y filters:
`--min-y-ratio` and `--max-y-ratio`. Focused tests pass:
```bash
env PYTHONPATH=src uv run --python 3.11 --with pytest \
pytest tests/test_filter_page_ocr_documents.py -q
```
Result: `8 passed`.
Best diagnostic non-oracle filter so far:
```text
source documents:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-documents
filtered documents:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents
score:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score
```
Filter parameters: `--drop-empty --min-y-ratio 0.12 --max-y-ratio 0.96
--min-confidence 0.70`. The filter audit passes with 0 issues. It keeps 2,096 of
2,191 detected lines and removes 95 lines.
Score progression on the seven-page Gorkhapatra Sirijonga silver pack:
```text
raw Auto: page_cp_CER 0.814444, page_graph_CER 0.851347, line_F1 0.943981
Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, line_F1 0.943981
Rows LTR + y filter: page_cp_CER 0.425080, page_graph_CER 0.449335, line_F1 0.960868
Rows LTR + y + conf70: page_cp_CER 0.422770, page_graph_CER 0.445983, line_F1 0.964734
Rows LTR + y + conf80: page_cp_CER 0.422985, page_graph_CER 0.446007, line_F1 0.964929
```
Interpretation: `conf70` is the current baseline, but the improvement is modest. It
removes obvious page furniture and low-confidence junk, but full-page CER remains far
too high for a book OCR claim. The next serious step is a stronger non-oracle
target-region/language filter or page layout segmenter, then rerun the same evals.
## 2026-07-01 Limbu reading-order and assembly check
Do not loop back into training yet. The recognizer is not the main blocker on the
Gorkhapatra Sirijonga silver pages: matched-line codepoint CER is still about 2.7%.
The current failure is page reconstruction.
Added:
- `tools/apply_page_reading_order_strategy.py` modes `Rows LTR Top` and `Rows RTL Top`
for bbox-top row clustering. This keeps the old center-based `Rows LTR` behavior
available for apples-to-apples comparison.
- `tools/assemble_page_ocr_lines.py --max-horizontal-overlap`, with replay support in
`tools/audit_page_ocr_line_assembly.py`, so adjacent fragments with small detector
box overlap can be merged without reference lookup.
- `tests/test_apply_page_reading_order_strategy.py`, plus an overlap case in
`tests/test_assemble_page_ocr_lines.py`.
Focused test command:
```bash
env PYTHONPATH=src uv run --python 3.11 --with pytest \
pytest tests/test_assemble_page_ocr_lines.py \
tests/test_filter_page_ocr_documents.py \
tests/test_apply_page_reading_order_strategy.py -q
```
Result: `17 passed`.
Diagnostic scores on the same seven-page Gorkhapatra Sirijonga PDF-text-layer silver
pack:
```text
Rows LTR + y012/y096 + conf70:
page_cp_CER 0.4227699, page_graph_CER 0.4459828,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9930784
Rows LTR Top + same filter:
page_cp_CER 0.4289698, page_graph_CER 0.4496008,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9924359
Rows LTR + same filter + conservative assembly:
docs: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score
page_cp_CER 0.4226502, page_graph_CER 0.4457416,
line_F1 0.9652050, matched_cp_CER 0.0277639, order_acc 0.9931425
Rows LTR Top + same filter + conservative assembly:
page_cp_CER 0.4288860, page_graph_CER 0.4493596,
line_F1 0.9654350, matched_cp_CER 0.0276650, order_acc 0.9924574
```
Interpretation: top-anchored row order is worse on the held seven pages, and
conservative line assembly only moves CP CER from `0.4227699` to `0.4226502`.
Assembly is useful for readability and auditability, but not the missing system
component. The next block of work should be a non-oracle layout/target-region
calibration stage for the Limbu article region, with held-out scoring. The old
geometry-profile run is not a usable answer; it was non-claim-safe and scored
`page_cp_CER 0.736334`.
## 2026-07-03 expanded Limbu Gorkhapatra pack
Full provenance and commands are in
`docs/LIMBU_GORKHAPATRA_EXPANDED_2026_07_03.md`.
New live crawl:
```text
outputs/gorkhapatra-language-pages-expanded-2026-07-03-rendered-pymupdf/
publication_items=100 aligned=53 language_page_hits=103 warnings=0
```
It adds a PDF-backed Limbu issue:
```text
date_key: 2083-03-13
article: https://gorkhapatraonline.com/news/211870
pdf: https://epaper.gorkhapatraonline.com/uploads/file/2026/6/gorkhapatra/2026-06-26-11-39-45-2083-03-13.pdf
```
Expanded corpus:
```text
outputs/limbu-gorkhapatra-pdf-backed-silver-2026-07-03-v1/
discovered_limbu_rows=9 unique_dates=9 usable_pages=8 total_lines=2441 usable_silver_labels=2430
failed_dates=[{"date_key": "2082-11-12", "error": "no_namdhinggo_page_found"}]
```
Expanded page eval pack:
```text
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-07-03-v1/
sample_count=8 line_count=2441
eval_pack_version=gorkhapatra-limbu-sirijonga-pdf-text-layer-silver-pages-v2
```
Expanded 8-page diagnostic scores:
```text
Rows LTR raw:
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr/score
page_cp_CER 0.4349285, page_graph_CER 0.4692741,
line_F1 0.9418182, matched_cp_CER 0.0261180, order_acc 0.9931821
Rows LTR + y012/y096 + conf70:
docs: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score
filter kept 2398/2509 lines; replay audit passed with 0 issues
page_cp_CER 0.4143301, page_graph_CER 0.4380524,
line_F1 0.9630089, matched_cp_CER 0.0260386, order_acc 0.9931762
Rows LTR + same filter + conservative assembly:
docs: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score
assembly merged 26 groups; replay audit passed with 0 issues
page_cp_CER 0.4142255, page_graph_CER 0.4378626,
line_F1 0.9633943, matched_cp_CER 0.0268101, order_acc 0.9932274
```
Interpretation: the new page is useful and the 8-page pack is stable. Matched-line
CER is still about 2.6%, so the next serious work is target-region/page-layout
calibration, not another recognizer epoch or more line-merging tweaks.
# Codex Handoff: Limbu Book OCR
Updated: 2026-06-29
Repo: `Ampixa/ocr-tech` · Branch: `main` · Visibility: private
This is for Codex acting as critical reviewer + next implementer. The prior agent
(Claude) consults you at milestones specifically to catch blind spots it
rationalizes past. **Be skeptical of every number below and every claim of "fixed."**
The living source of truth is `docs/RECOGNIZER_GOAL_PROGRESS.md`; this handoff is the
fast path into the current state.
2026-06-29 Limbu end-to-end OCR update: the immediate goal is a Limbu-first OCR
system for real page images/PDFs, not a generic multilingual plan and not a YOLO-first
page demo. The Space payload at `spaces/limbu-ocr/` now has `Line OCR`,
`Paddle Page OCR`, `Paddle Book OCR`, image-line fallback tabs, `YOLO Probe`, and
runtime tabs. The production page/book path uses PaddleOCR full-page text detection,
orders detected text lines, routes each crop through the bundled Sirijonga and
Devanagari-written Limbu Paddle recognizers, and returns Unicode text plus
crop/overlay evidence. `Paddle Book OCR` accepts ordered page image files or PDFs and
returns a ZIP with `book.txt`, `book.json`, annotated pages, and line crops. YOLO
remains only an optional probe. Verification on this turn:
`python3 -m py_compile spaces/limbu-ocr/app.py`, a Python 3.11 import/artifact +
synthetic line-segmentation smoke, PaddleOCR API introspection against
`paddleocr==3.7.0`, one real Gorkhapatra page Paddle smoke with 306 detections, a
capped `recognize_page_paddle(max_lines=5)` run, and focused repo tests
`tests/test_image_lines.py tests/test_parse_limbu_book_pages.py tests/test_run_limbu_book_ocr.py`
with result `49 passed, 1 skipped`. During testing, the book runner/capture-prep
interface was fixed so newer capture-audit fields are forwarded and older
programmatic callers do not crash on missing optional namespace fields. Next step is
testing the uploaded Space revision on real Limbu pages, not expanding eval harness
abstractions. Uploaded Space commit:
`https://huggingface.co/spaces/ampixa/limbu-ocr/commit/bda8e8b5ebaf476e7a52cd56996f525fd7687d56`.
The Space is private; authenticated direct-domain access returned HTTP 200 while
unauthenticated access returned HTTP 404.
2026-06-25 OCR Space update: the Limbu-only OCR Hugging Face Space payload is staged
at `spaces/limbu-ocr/` and documented in `docs/LIMBU_OCR_SPACE_HANDOFF.md`. It bundles
the current Sirijonga line recognizer, Devanagari-written Limbu recognizer, and YOLO
line detector behind a self-contained Gradio app. The app passed `py_compile` and a
Python 3.11 lightweight import/artifact health check. It has been uploaded to
`https://huggingface.co/spaces/ampixa/limbu-ocr` at commit
`76f54073ed75d1a4df6ea3d63fb06cd871383b95`, and the runtime URL
`https://ampixa-limbu-ocr.hf.space/` returned HTTP 200. Current HF CPU quota permits
three active CPU Spaces; the intended active set is `ampixa/limbu-ocr`,
`ampixa/limbu-translator`, and `ampixa/limbu-tts`. `ampixa/real-nepali-tts`,
`ampixa/gurung-tts`, and `voidash/bantawa-tts-demo` are paused. Do not resume broad
language inventory from here; the immediate path is OCR deployment/testing and then
real-page validation.
Recognizer training correction: also read
`docs/LIMBU_RECOGNIZER_TRAINING_HANDOFF.md` before discussing Limbu training data,
CER, or the active G6 run. The `1,866` Gorkhapatra train rows are not the whole
Limbu corpus; the active combined pack has `48,507` train rows plus `255` held-out
Gorkhapatra eval rows and is training with checkpoint selection by `norm_edit_dis`.
2026-06-24 update: the next recognizer experiment is now a targeted-fabrication
ablation, not more generic synthetic volume. The new generator is
`tools/build_limbu_targeted_fabrication_pack.py`; it renders verified Limbu text,
oversamples current failure lines/confusion characters, matches real Gorkhapatra line
widths, applies phone-photo degradation, rejects non-rendering fonts, rejects blank
crops, and writes training-only manifests. The first component is
`outputs/limbu-targeted-fabrication-v1-2026-06-24/` with `9,996` fabricated line crops
and `4` render rejects from the ink guard. Important correction: an earlier pass under
this same output name used extracted Gorkhapatra PDF subset fonts and generated blank
images; the corrected pack uses upstream renderable Namdhinggo fonts under
`data/fonts/namdhinggo/Namdhinggo-3.100/`. The combined train/eval pack that includes
it is `outputs/limbu-combined-synth-real-gorkhapatra-targeted-v1-2026-06-24/` with
`58,467` train rows and `255` held-out Gorkhapatra eval rows after `49` exact eval-text
overlap drops; both manifests passed `ocrtech audit-manifest`. Treat this as an
ablation input only. Do not claim OCR quality from synthetic rows.
2026-06-24 Unicode/diacritic coverage update: read
`docs/LIMBU_UNICODE_DIACRITIC_STATS.md` before generating more Sirijonga text. The
Limbu block has `68` assigned codepoints, but the targeted combined pack above covered
only `61/68`; missing were `U+1909`, `U+190A`, `U+191A`, `U+191D`, `U+191E`,
`U+1932`, and `U+1940`. A HarfBuzz/Pillow shaping probe over `21,891` Limbu component
cases rendered cleanly with Namdhinggo, but that only proves the font is permissive;
it does not validate linguistic/order sanity. The new corpus-stat tool is
`tools/analyze_limbu_unicode_corpus_stats.py`; the new coverage text generator is
`tools/build_limbu_unicode_coverage_text.py`. It was corrected to emit
`base+subjoined+vowel`, `base+vowel+KEMPHRENG+final`, and to include `U+1932` in
finals instead of orphaning it. The coverage text pack
`data/synthesis-text/limbu-sirijonga-unicode-coverage-v1-2026-06-24/` has `1,936`
source rows and zero generated order warnings; the rendered training-only component
`outputs/limbu-unicode-coverage-fabrication-v1-2026-06-24/` has `3,872` rendered rows,
zero render rejects, and nonblank pixel stats. The combined pack with both targeted and
Unicode coverage is
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-2026-06-24/` with
`62,339` train rows and `255` eval rows; it covers `68/68` assigned Limbu codepoints
and has no codepoint below `1,000` occurrences after dedupe. Train/eval manifest audits
passed. Treat `unicode-coverage-fabrication-v1` as a separate ablation component, not
claim evidence.
2026-06-24 active training update: the targeted+Unicode combined pack is running on
the AWS G6 L4 instance `i-0fa19dff7b233a353` at `3.219.242.144`, workspace
`/opt/ocrtech`, PID `11133`. Run directory:
`runs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-24/`.
It uses existing pack + targeted fabrication + Unicode coverage, base checkpoint
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams`,
dictionary `data/dictionaries/limbu-sirijonga-v3.txt`, batch `64`, LR `0.00001`,