Download ocr-tech/docs/CODEX_HANDOFF.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 320 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/CODEX_HANDOFF.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/CODEX_HANDOFF.md
-
curl -L -o CODEX_HANDOFF.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/CODEX_HANDOFF.md
Codex Handoff: Limbu Book OCR
Updated: 2026-06-29
Repo: Ampixa/ocr-tech · Branch: main · Visibility: private
This is for Codex acting as critical reviewer + next implementer. The prior agent
(Claude) consults you at milestones specifically to catch blind spots it
rationalizes past. Be skeptical of every number below and every claim of "fixed."
The living source of truth is docs/RECOGNIZER_GOAL_PROGRESS.md; this handoff is the
fast path into the current state.
2026-06-29 Limbu end-to-end OCR update: the immediate goal is a Limbu-first OCR
system for real page images/PDFs, not a generic multilingual plan and not a YOLO-first
page demo. The Space payload at spaces/limbu-ocr/ now has Line OCR,
Paddle Page OCR, Paddle Book OCR, image-line fallback tabs, YOLO Probe, and
runtime tabs. The production page/book path uses PaddleOCR full-page text detection,
orders detected text lines, routes each crop through the bundled Sirijonga and
Devanagari-written Limbu Paddle recognizers, and returns Unicode text plus
crop/overlay evidence. Paddle Book OCR accepts ordered page image files or PDFs and
returns a ZIP with book.txt, book.json, annotated pages, and line crops. YOLO
remains only an optional probe. Verification on this turn:
python3 -m py_compile spaces/limbu-ocr/app.py, a Python 3.11 import/artifact +
synthetic line-segmentation smoke, PaddleOCR API introspection against
paddleocr==3.7.0, one real Gorkhapatra page Paddle smoke with 306 detections, a
capped recognize_page_paddle(max_lines=5) run, and focused repo tests
tests/test_image_lines.py tests/test_parse_limbu_book_pages.py tests/test_run_limbu_book_ocr.py
with result 49 passed, 1 skipped. During testing, the book runner/capture-prep
interface was fixed so newer capture-audit fields are forwarded and older
programmatic callers do not crash on missing optional namespace fields. Next step is
testing the uploaded Space revision on real Limbu pages, not expanding eval harness
abstractions. Uploaded Space commit:
https://huggingface.co/spaces/ampixa/limbu-ocr/commit/bda8e8b5ebaf476e7a52cd56996f525fd7687d56.
The Space is private; authenticated direct-domain access returned HTTP 200 while
unauthenticated access returned HTTP 404.
2026-06-25 OCR Space update: the Limbu-only OCR Hugging Face Space payload is staged
at spaces/limbu-ocr/ and documented in docs/LIMBU_OCR_SPACE_HANDOFF.md. It bundles
the current Sirijonga line recognizer, Devanagari-written Limbu recognizer, and YOLO
line detector behind a self-contained Gradio app. The app passed py_compile and a
Python 3.11 lightweight import/artifact health check. It has been uploaded to
https://huggingface.co/spaces/ampixa/limbu-ocr at commit
76f54073ed75d1a4df6ea3d63fb06cd871383b95, and the runtime URL
https://ampixa-limbu-ocr.hf.space/ returned HTTP 200. Current HF CPU quota permits
three active CPU Spaces; the intended active set is ampixa/limbu-ocr,
ampixa/limbu-translator, and ampixa/limbu-tts. ampixa/real-nepali-tts,
ampixa/gurung-tts, and voidash/bantawa-tts-demo are paused. Do not resume broad
language inventory from here; the immediate path is OCR deployment/testing and then
real-page validation.
Recognizer training correction: also read
docs/LIMBU_RECOGNIZER_TRAINING_HANDOFF.md before discussing Limbu training data,
CER, or the active G6 run. The 1,866 Gorkhapatra train rows are not the whole
Limbu corpus; the active combined pack has 48,507 train rows plus 255 held-out
Gorkhapatra eval rows and is training with checkpoint selection by norm_edit_dis.
2026-06-24 update: the next recognizer experiment is now a targeted-fabrication
ablation, not more generic synthetic volume. The new generator is
tools/build_limbu_targeted_fabrication_pack.py; it renders verified Limbu text,
oversamples current failure lines/confusion characters, matches real Gorkhapatra line
widths, applies phone-photo degradation, rejects non-rendering fonts, rejects blank
crops, and writes training-only manifests. The first component is
outputs/limbu-targeted-fabrication-v1-2026-06-24/ with 9,996 fabricated line crops
and 4 render rejects from the ink guard. Important correction: an earlier pass under
this same output name used extracted Gorkhapatra PDF subset fonts and generated blank
images; the corrected pack uses upstream renderable Namdhinggo fonts under
data/fonts/namdhinggo/Namdhinggo-3.100/. The combined train/eval pack that includes
it is outputs/limbu-combined-synth-real-gorkhapatra-targeted-v1-2026-06-24/ with
58,467 train rows and 255 held-out Gorkhapatra eval rows after 49 exact eval-text
overlap drops; both manifests passed ocrtech audit-manifest. Treat this as an
ablation input only. Do not claim OCR quality from synthetic rows.
2026-06-24 Unicode/diacritic coverage update: read
docs/LIMBU_UNICODE_DIACRITIC_STATS.md before generating more Sirijonga text. The
Limbu block has 68 assigned codepoints, but the targeted combined pack above covered
only 61/68; missing were U+1909, U+190A, U+191A, U+191D, U+191E,
U+1932, and U+1940. A HarfBuzz/Pillow shaping probe over 21,891 Limbu component
cases rendered cleanly with Namdhinggo, but that only proves the font is permissive;
it does not validate linguistic/order sanity. The new corpus-stat tool is
tools/analyze_limbu_unicode_corpus_stats.py; the new coverage text generator is
tools/build_limbu_unicode_coverage_text.py. It was corrected to emit
base+subjoined+vowel, base+vowel+KEMPHRENG+final, and to include U+1932 in
finals instead of orphaning it. The coverage text pack
data/synthesis-text/limbu-sirijonga-unicode-coverage-v1-2026-06-24/ has 1,936
source rows and zero generated order warnings; the rendered training-only component
outputs/limbu-unicode-coverage-fabrication-v1-2026-06-24/ has 3,872 rendered rows,
zero render rejects, and nonblank pixel stats. The combined pack with both targeted and
Unicode coverage is
outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-2026-06-24/ with
62,339 train rows and 255 eval rows; it covers 68/68 assigned Limbu codepoints
and has no codepoint below 1,000 occurrences after dedupe. Train/eval manifest audits
passed. Treat unicode-coverage-fabrication-v1 as a separate ablation component, not
claim evidence.
2026-06-24 active training update: the targeted+Unicode combined pack is running on
the AWS G6 L4 instance i-0fa19dff7b233a353 at 3.219.242.144, workspace
/opt/ocrtech, PID 11133. Run directory:
runs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-24/.
It uses existing pack + targeted fabrication + Unicode coverage, base checkpoint
runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams,
dictionary data/dictionaries/limbu-sirijonga-v3.txt, batch 64, LR 0.00001,
15 epochs, and main_indicator=norm_edit_dis. First eval at step 200 saved a
warmup best with norm_edit_dis=0.04608837027816626 and acc=0.0; do not interpret
that as CER. The dictionary v3 update is append-only over v2 because the new pack
requires seven additional Limbu chars: U+1909, U+190A, U+191A, U+191D,
U+191E, U+1932, and U+1940. After completion, score the exported best checkpoint
on the frozen Gorkhapatra recognizer eval before claiming any improvement.
2026-06-24 completed training result: reject this targeted+Unicode checkpoint. The run
completed on the G6 L4 instance, the completion audit passed with issue_count=0 and
one regex warning, and Paddle selected epoch 14 with
norm_edit_dis=0.8278816454641351, acc=0.1450980335255673. The exported/package-best
score is
outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-best-eval-2026-06-24/score/:
codepoint CER 0.024696356275303644 (244/9880), grapheme CER
0.053276955602537 (252/4730), exact-line accuracy 0.4666666666666667 on 255
matched lines. It is worse than the real-heavy champion and worse than the older
combined baseline; see paired comparisons under
outputs/limbu-targeted-unicode-v1-best-vs-real-heavy-champion-comparison-2026-06-24/comparison/
and
outputs/limbu-targeted-unicode-v1-best-vs-combined-baseline-comparison-2026-06-24/comparison/.
Do not promote this package into book OCR. Keep the real-heavy model as the current
Sirijonga recognizer champion and treat this run as evidence that the current targeted
synthetic/Unicode supplement is hurting real-print performance.
2026-06-24 controlled ablation result: the regression has now been isolated enough to
make a decision. The grid is
runs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/; scores and paired
comparisons are under outputs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/.
All cells were exported, packaged, predicted, scored against the champion's exact
frozen Gorkhapatra eval files, and paired against the real-heavy champion. Results:
control v2 replay LR 1e-6 scored codepoint CER 0.011234817813765181, grapheme CER
0.02452431289640592, exact 0.7333333333333333; this is one edit better than the
champion and not statistically meaningful. v2 plus 256 capped targeted rows scored
worse (0.01194331983805668 codepoint, 0.026004228329809725 grapheme). v2 LR
1e-5 tied codepoint CER but worsened grapheme CER (0.026004228329809725) despite
exact 0.7372549019607844. Both v3 dictionary cells failed catastrophically around
0.98 codepoint CER and 0.0 exact-line accuracy because the dictionary expansion
resizes the recognition head. Do not promote any ablation checkpoint. The next serious
recognizer move is either stay on v2 and improve real/book-domain line data, or create
an explicit full-Unicode training/rehead plan; do not mix dictionary expansion into a
synthetic-data ablation again.
Goal
Build a usable Limbu book OCR system: scanned or photographed real Limbu book pages in, editable Unicode book text out, with page and line structure preserved. It must support both Sirijonga Limbu and Devanagari-written Limbu. The current bottleneck is page layout, reading order, missed-line recovery, source-page triage, and review burden with the current recognizers held fixed. Keep evaluation minimal and tied to acceptance checks on frozen real-book pages: page CER/WER, line precision/recall/F1, reading-order accuracy, blank/extra/missed rates, and review-packet coverage.
Do not expand the eval harness as its own project. Use metrics only when they prove whether the book OCR loop got better. The next useful work should make one of these true: cleaner input page selection, stronger capture prep, better line preservation, fewer unresolved rows per real Limbu page, safer correction application, or a measured improvement over the current diagnostic baselines.
Critical Gorkhapatra Source Rule
For all Naya Nepal/Gorkhapatra language data, the Naya Nepal publication page
(https://gorkhapatraonline.com/publications/naya-nepal) is discovery only. Use it
to identify the language/date/post, but do not use the blog/card image as OCR,
training, or eval evidence. The authoritative source is the full Gorkhapatra epaper
PDF from the date-range endpoint, for example:
https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-16&pdfdaterange2=2026-06-23&slug=gorkhapatra.
The correct text-image pair is the rendered PDF page plus the text layer extracted
from that same PDF page, with PDF URL/hash/page number, rendered image hash, text
extraction command/version, and legacy-conversion map hash recorded. For every
language, classify the PDF text layer as unicode_native, legacy_font,
devanagari_transcription, latin_notation, or no_text_layer; build/reuse an
explicit converter only when the font/text layer requires it. Manual transcription is
the fallback for no_text_layer or undecoded fonts, not the default path. See
docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md before changing this path.
The browser annotation/review UI already exists. Use the PDF span dashboard:
data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html.
It shows the full rendered PDF page, clickable bounded spans, and the right-side
text/rendered-crop panel. Do not use
data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-review.html;
that local file was identified as the wrong/broken UI and deleted. The newer book
unresolved review station is tools/prepare_limbu_book_unresolved_review_packet.py.
Current Limbu Pipeline Pivot
The active local work has narrowed to a Limbu-first operational OCR pipeline that must
handle both Devanagari-written Limbu and Sirijonga-script Limbu. Do not frame this as
"video to page"; moving-camera material must be reduced to still page inputs before it
enters the OCR path. The target is a script-aware OCR path with capture cleanup,
Devanagari/Sirijonga routing,
recognizer-only handling for existing line crops, review queues, line crops, output
manifests, and audits. The current docs/config are
docs/LIMBU_FIRST_OCR_PIPELINE.md and configs/limbu-first-ocr-pipeline-v1.json.
Count sanity check: do not repeat the mistake of treating 39 as the size of the
Limbu eval/data surface. In the current config, 39 is the unresolved review-row count
from one older operational Gorkhapatra page-eval run, paired with 85 dropped image-line
rows. It is review debt for that run, not the corpus size and not the canonical eval
pack size. The materialized recognizer eval pack
outputs/eval-packs/limbu-gorkhapatra-real-v1/ currently has 1,829 label rows and
1,829 manifest rows; its audit passes structurally but claim_ready=false because it
is machine-converted, not declared unseen for selection, has 0 claim-eligible rows, and
lacks source-document/page/bbox provenance. Separately, the CDC Maths PDF text-layer
silver pack at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/
has 81 rendered-line crops, split into 69 train and 12 eval rows after grouping by
sample_signature. Treat those 81 rows as silver training/bootstrap data, not
claim-safe ground truth.
Current 2026-06-23 correction: the user pushed back that eval-harness work was being
overdone. Treat that as authoritative direction. The freshest real-book operational
smoke is
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/,
run with /tmp/ocrtech-paddle311/bin/python. It processed 2 unique source pages, kept
692 lines, wrote 599 review rows, and passed the book artifact audit. The priority
queue/dashboard are just operator views over the same review work: 304 high-priority
empty/no-script transcription-or-drop rows, 15 mixed-script verification rows, and 280
low-confidence spot checks. The canonical queue remains book-line-review-queue.tsv in
page/reading order; book-line-priority-review-queue.tsv is only a sorted working copy.
This evidence says the next product work is reducing review burden and recovering real
page lines, not building more generalized scoring infrastructure.
Current 2026-06-23 PDF book intake bridge: tools/build_limbu_book_page_manifest.py
now accepts either --image-dir or --pdf. The PDF path renders selected book pages to
PNG, writes the normal OCR page manifest, and records source PDF path/hash, PDF page
number/index, rendered image hashes, and DPI per row. This is an operational intake
feature, not an eval-harness feature: real PDFs can now enter the same
capture/OCR/review loop as photographed pages. Real smoke on the local CDC Maths Limbu
PDF:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-pages.jsonl
renders PDF pages 8..10 at 220 DPI into three 1870x2420 PNG pages with no source
attention flags. Passing the manifest through capture prep produced
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/capture-prep/prepared-page-manifest.jsonl
with 3 pages and 0 failed capture audits. That prepared manifest has already been fed
through the book wrapper below; keep this paragraph as intake provenance, not as the
next action.
That front door has now been run through the full book wrapper using the current v4
router, without changing recognizers:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/.
It processes 3 pages end-to-end, emits editable book.md, review queues, page-layout
overlays, page triage, layout-gap review, and a passing book audit. Current counts:
93 preserved lines, 21 book-line review rows, 3 page-layout rows, 3 page-triage rows,
9 layout-gap rows, operator_attention_page_count=0, and book_audit_passed=True.
The review queue now separates page furniture more honestly: 9 high-priority
inspect_non_text_or_drop_empty_detection rows, 3 medium
inspect_non_text_or_drop_low_confidence_detection rows, 1 high
transcribe_or_drop_empty_detection row, and 8 low-confidence spotchecks. The latest
line-review contact sheet is:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png.
Thirteen obvious non-text rows have since been dropped in the v2 cleaned projection
documented later in this handoff. The remaining real work on this artifact is the
8-row text-shaped low-confidence review packet, followed by audited application and
reviewed-only training export. Do not go back to the 21-row queue unless you are
rebuilding the cleaned projection from scratch.
Current 2026-06-23 triage cleanup: tools/apply_limbu_book_page_triage.py now filters
all page-scoped review queues after explicit page rejection: aggregate book review,
book-line review, priority book-line review, and layout-gap review. It also recomputes
counts/provenance, clears stale dashboard paths, and regenerates filtered canonical,
priority, and focused book-line review dashboards from the kept-page queues.
The real p2-rejected check at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/manual-triage-check/triaged-p1-only/
keeps only gorkhapatra-unique-current-p0001, reduces the review surface from 599 rows
to 230 rows, reports priority counts 6 high / 4 medium / 220 low, and passes
tools/audit_limbu_book_output.py with zero issues. This is source/page triage and
operator-noise reduction, not OCR-quality improvement.
Current 2026-06-23 review guidance cleanup: tools/parse_limbu_book_pages.py now writes
book_line_bbox and book_line_aspect_ratio into book-line-review-queue.tsv and
classifies large square/near-square empty detections as
inspect_non_text_or_drop_empty_detection. This keeps portrait/page-furniture boxes out
of the text-transcription bucket without dropping anything automatically. Real rerun:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/
passes the book audit; the p2-rejected projection
.../manual-triage-check/triaged-p1-only/ passes audit_limbu_book_output and leaves
the kept page with 5 empty visible-text transcription/drop rows, 1 non-text inspect/drop
row, 4 mixed-script verification rows, and 220 low-confidence spot checks. This is
review-packet quality work, not a CER/WER improvement.
Current 2026-06-23 layout review packet: tools/run_limbu_book_ocr.py now writes
page-layout-review/page-layout-review-manifest.tsv,
page-layout-review/page-layout-review-dashboard.html, and one full-page overlay PNG
per OCR page. The overlays draw line boxes and reading-order indices over the prepared
page image, color-coded green accepted text / amber review text / red empty OCR /
purple structural text. Real run:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-layout-overlay/
passes the book audit and has 2 layout rows / 2 overlays. Page 1 is 323 lines, 6 empty
OCR boxes, 230 review rows, reading order wide_column_right_to_left. The first overlay
was visually checked and is nonblank; use it to inspect missed boxes and reading-order
problems before crop-level transcription. This is layout-debug/review evidence, not OCR
quality improvement.
Current 2026-06-23 training-loop bridge: tools/export_limbu_book_training_pack.py
exports a reviewed book-run.json projection into line crops, PaddleOCR-style labels,
a JSONL recognizer manifest, dictionary, and script inventory. By default it exports
only lines that carry explicit book-line review, split-review, or layout-gap insertion
metadata; unreviewed OCR rows are excluded so pending recognizer output does not become
training truth. The exporter now revalidates that reviewed metadata before writing
labels: status must be accepted/corrected, reviewer must be present and non-machine,
and the evidence-specific reviewed text must be present and match the exported line.
Candidate insertions require explicit corrected_text, so replayed machine candidate
text cannot become recognizer GT. The runner README now points from review application to this exporter:
uv run python tools/export_limbu_book_training_pack.py <run>-reviewed/final-book --out <run>-training-pack-reviewed. A real bootstrap check on the latest p1-only triaged
projection used --include-unreviewed at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/training-pack-bootstrap-p1-unreviewed/;
it produced 317 crops/labels, skipped 6 empty rows, and counted 221 Sirijonga, 27
Devanagari, and 69 mixed rows. That artifact is explicitly marked
gt_status_counts={"unreviewed_ocr_projection": 317} and is not ground truth. The
reviewed-export command correctly refused an older final-book projection with no
accepted review rows, which means the next real blocker is completing accepted line/gap
review, not adding more harness.
Current 2026-06-23 training pivot after scope correction: stop building eval harness
surface and use real Limbu text sources to train. tools/export_limbu_pdf_text_training_pack.py
now exports converted PDF text-layer lines as rendered full-line crops plus PaddleOCR
labels, without using current OCR predictions as labels. This is deliberately marked
silver, not human-reviewed gold: gt_status=pdf_text_layer_silver, claim_eligible=false.
Real export from CDC Maths PDF pages 11..15:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/.
It has 81 rows from 82 PDF text-layer rows, skipping 1 no-Limbu row; script buckets are
64 Sirijonga and 17 mixed Sirijonga/Devanagari rows. A leakage-clean split was created
under .../splits/ with 69 train and 12 dev rows, grouped by sample signature after a
sample-id split correctly failed on duplicate crop/text leakage. The generic PP-OCRv5
silver run at
runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/ is a negative result: using
the 55-character pack dictionary reinitialized the recognition head, finished at dev
norm_edit_dis=0.0903799184964279, packaged cleanly, but produced poor runtime
predictions. Do not route that package into book OCR except as evidence that same-head
warm-start matters.
The better candidate is the v4 Sirijonga warm-start run at
runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/, generated with
data/dictionaries/limbu-sirijonga-v2.txt, base checkpoint
outputs/limbu-first-router-v1/source-archives/limbu/checkpoints/best_accuracy.pdparams,
20 CPU epochs, batch size 8, and LR 0.00002. It loaded the base checkpoint without
head-shape mismatch warnings. The best Paddle dev checkpoint is epoch 14:
acc=0.49999958333368055, norm_edit_dis=0.7922192746296381; the final epoch drifted
lower, so package best_accuracy, not latest. Export/package/audit are complete under
export-best/ and package-best/, with the model card at
runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/package-best/model-card.json.
Runtime smoke on held-out crop cdc-maths-p0011-pdf-line-0001.png produced the exact
silver label at confidence 0.9353305101394653. The narrow 12-row silver runtime score
is at .../eval-best/score/: codepoint CER 0.012618296529968454, grapheme CER
0.023529411764705882, exact-line accuracy 0.75, 12/12 matched predictions, no
dictionary coverage gaps. This is not claim-safe: it is PDF-text silver, only 12
rows, not unseen for checkpoint selection, and has known train/dev text overlap. Use it
only as admission evidence that the candidate is worth trying on real book pages.
That frozen-page router comparison has now been run and rejects global promotion. A
router variant at
outputs/limbu-router-v5-ppocr-devanagari-v4warm-sirijonga-silver-2026-06-23/model-card.json
keeps the Devanagari primary and routing policy fixed while replacing only the
Sirijonga secondary with the warm-start package; its model audit passes. On the CDC
Maths pages 11..15, the v5 run
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v5-v4warm-silver-cdc-compare/
passes the book audit and preserves the same 189 lines, but review rows increase from
58 to 68. PDF-text coverage is unchanged: 68 covered lines, 13 missing/partial
candidates, 1 skipped no-Limbu row. A quick geometry-aligned PDF-text check showed only
a tiny aggregate text gain (1730 -> 1708 codepoint edits over 1841 reference
codepoints) with several long-line regressions. On the real-print Gorkhapatra two-page
run
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-router-v5-v4warm-silver-compare/,
the v5 router again passes the book audit and preserves the same 692 lines, but review
rows increase from 599 to 665; the usable Limbu page alone rises from 230 to 296 review
rows. The comparison artifact is
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/router-v5-v4warm-admission-comparison.json.
Decision: keep router v4 as the operational default. Use the v5 warm-start recognizer
as training evidence or a selective probe only; do not replace the router with it.
Current 2026-06-23 layout/missed-line probe: after the user pushed back on overdoing
eval harness work, the next check was deliberately narrow and operational. Added
tools/sweep_limbu_pdf_text_line_detector_params.py with focused coverage in
tests/test_sweep_limbu_pdf_text_line_detector_params.py; it is a detector-only
geometry diagnostic, not an OCR-quality harness. On CDC Maths pages 11..15, the sweep
at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/
shows current defaults at 68/81 covered Limbu source lines, 13 missing/partial
candidates, and 189 boxes. The best recall config reaches 76/81 and 5 candidates but
emits 335 boxes, so do not promote it blindly. A balanced config
(horizontal_kernel=23, vertical_kernel=3, min_width=15, min_height=10,
min_area=50, max_height=100) was run through the real book OCR path with router v4
at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/;
it passes audit, improves PDF-text coverage to 73/81 and 8 candidates, but increases
OCR lines from 189 to 214 and review rows from 58 to 60, with 2 operator-attention
pages. Decision: keep current defaults, but use this balanced detector as the next
real-page probe where missed lines are visibly more costly than a small review-row
increase.
The runner now exposes detector probes directly as operational switches:
--detector-profile current|balanced-pdf-text|current-plus-balanced-rescue. current
remains the default. balanced-pdf-text materializes
<run>/profiled-page-manifest/book-pages.jsonl, merges the balanced detector settings
into page parse_kwargs, records the profile and manifest hashes in
limbu-book-ocr-run.json, and refuses conflicting row-level detector settings instead
of silently overriding them. current-plus-balanced-rescue keeps the current detector
as primary and adds the balanced settings only as an additive rescue pass. Use these
for controlled real page probes, e.g.:
/tmp/ocrtech-paddle311/bin/python tools/run_limbu_book_ocr.py \
--image-dir /path/to/book-page-photos \
--out outputs/limbu-live-book-smoke-YYYY-MM-DD/my-book-balanced-probe \
--skip-duplicate-images \
--detector-profile balanced-pdf-text
The switch has also been checked on the real-print Gorkhapatra two-unique-page smoke
with stable page IDs by reusing the current baseline raw manifest. Artifact:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-balanced-detector-profile-probe-stable-pageids/.
It passes audit and records source manifest SHA
b8bb7138a12c42c1a919339bbfad5165f5162ab768f4c8c00874f777b364384c plus profiled
manifest SHA b8f1be0a77ba7e86bf34cd15c3bd4592cbdbca252ca4c00ae20c30bf49e42010.
Compared with gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard, totals
move 692 -> 690 lines and 599 -> 596 review rows, but layout-gap warnings rise 1 -> 5.
On the usable Limbu page, lines move 323 -> 325, review rows 230 -> 231, Sirijonga
classifications 286 -> 290, and other rows 6 -> 5. Treat this as safe probe evidence,
not default-promotion evidence.
For visual inspection of that profile delta, use the new packet:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-balanced-detector-line-geometry-delta-review/.
It was generated by tools/compare_limbu_book_line_geometry.py and compares current
default vs stable balanced-pdf-text by line-box IoU. It reports 679 matched lines,
11 candidate-only lines, and 13 baseline-only lines; on the usable Limbu page, 317
matched, 8 candidate-only, and 6 baseline-only. The packet includes
line-geometry-delta-review.tsv, line-geometry-delta-review.html, copied crops, and
full-page context images. It now also has
line-geometry-delta-contact-sheet.png with SHA
b164f9d3e01a64c74081272a6923921442b4a4023fa3142360fa2268e51ffeb6. A spot check of
crops/added_in_candidate-gorkhapatra-unique-current-p0001-0094-img-line-0094.png
shows real visible page text, but this remains fragment-level evidence. Review the
24-row packet before changing defaults. Current visual read: the profile adds small
real Sirijonga fragments on the usable page, but it also removes larger baseline
text/header detections, so keep balanced-pdf-text opt-in only.
There is now a better opt-in profile: --detector-profile current-plus-balanced-rescue.
It keeps current detector defaults as primary and uses the balanced settings only as a
rescue pass, so it can add missed fragments without replacing the larger baseline
boxes. Real stable-page-ID Gorkhapatra run:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-current-plus-balanced-rescue-profile-probe-stable-pageids/.
It passes audit with 703 lines / 606 review rows / 1 layout-gap row. On the usable
Limbu page: 323 -> 331 lines, 230 -> 234 review rows, 286 -> 292 Sirijonga rows, and
the possible_missed_line_gap flag disappears. Source-pass counts are primary 323 +
rescue-1 8 on the usable page, primary 369 + rescue-1 3 on the rejected page. Geometry
delta packet:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-current-plus-balanced-rescue-line-geometry-delta-review/,
with 692 matched default lines, 11 candidate-only rescue lines, and 0 removed lines.
Contact sheet SHA:
be12597dc1f32ce446b0f122af530cd04b512dd9861db1b33ab9643f6d381544. This is the best
line-preservation probe so far, but keep it opt-in until checked on the next actual
book/photo batch.
The same additive rescue profile was checked on CDC Maths pages 11..15 at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-current-plus-balanced-rescue-cdc-compare/.
It passes audit with 219 lines / 65 review rows / 14 layout-gap rows. Compared with
current default, it preserves all 189 baseline lines and adds 30 rescue boxes; compared
with the replacement balanced-pdf-text run, it gets the same PDF-text coverage
improvement, 68/81 -> 73/81 covered source lines, without deleting any baseline boxes.
Geometry delta packet:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/current-plus-balanced-rescue-line-geometry-delta-review/,
with 189 matched default lines, 30 candidate-only rescue lines, and 0 removed lines.
Contact sheet SHA:
838d1868b445f9ea8a11cf6e34db95fadcb9cb092098ff2321f2bec5680af22b. This reinforces
the current rule: additive rescue is the next opt-in line-preservation probe; it is not
a reason to build more generic eval harness or change defaults before the next real
book/photo batch.
The runner README now includes a Detector profile: section for every run. It records
the active profile, whether it is default/additive/replacement, its usage note, and the
profiled manifest path. This is operator workflow guidance so tomorrow's runs do not
mistake additive rescue evidence for default-promotion evidence.
For default current runs, limbu-book-ocr-run.json and the README now also include a
ready-to-run detector_profile_probe_command that reruns the exact source page manifest
with --detector-profile current-plus-balanced-rescue into a sibling output directory.
They also include detector_profile_probe_compare_command, which runs
tools/compare_limbu_book_line_geometry.py against the current run and the sibling
rescue run. This removes copy-paste reconstruction from the missed-line probe workflow
while keeping the default profile unchanged.
The generated run README now also points directly to
docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md from the "After review" section. That is the
reviewed TSV -> audited final book -> reviewed-only training pack handoff. The immediate
human blocker remains accepted/corrected rows from that workflow, not more runner
plumbing or generic metrics.
Fresh runner outputs with nonzero review rows and no page-triage source-reject/reshoot
blockers now include browser_review_station_command in limbu-book-ocr-run.json and
a Browser review station: section in the README. The command builds a pending
unresolved-review HTML/TSV packet from the run's focused review queue using
tools/prepare_limbu_book_unresolved_review_packet.py. The run summary also includes
browser_review_station_apply_command, which expects the exported
book-unresolved-review-packet.reviewed.tsv to be saved in the browser review station
directory and applies it with tools/apply_limbu_book_unresolved_review_packet.py
using --packet-dir provenance. If the reviewed rows should become training input, the
run summary also includes browser_review_station_training_pack_command, exporting
browser-review-station-applied/final-book through
tools/export_limbu_book_training_pack.py. Clean runs with 0 review rows do not emit
these browser-station commands, and pages that should be rejected or reshot must be
handled in page triage before line review. This closes the setup gap between "OCR run
completed" and "reviewer can accept/correct lines"; it still creates no labels until
the exported reviewed TSV is audited and applied. The runner test now executes the
generated command's target packet builder on a synthetic reviewable low-confidence Limbu
run and checks the generated apply/training-pack command paths, so this path is
verified beyond string matching.
The browser review station also carries layout-gap evidence now. If a run has
layout_gap_review_row_count > 0, the generated command passes
layout-gap-review-queue.tsv as --missing-line-review-queue. The unresolved packet
builder treats each pending layout-gap row as a candidate_missing_line, derives a
page-wide bbox from from_y/to_y, copies the gap crop, renders highlighted page
context, and preserves the neighboring OCR-line context. This is the narrow
missed-line recovery handoff the book-OCR goal needs: the reviewer sees suspected
omissions in the same browser station as low-confidence text, without promoting any
candidate text until reviewed TSV application/audit.
The generated layout-gap browser packet now has an end-to-end application test that
uses the generated apply-command paths. A synthetic suspicious-gap runner case builds
the browser packet, edits the exported TSV as a reviewer would
(review_status=accepted, explicit corrected_text, reviewer), saves it at the
browser_review_station_apply_command --review-packet path, then applies it through
tools/apply_limbu_book_unresolved_review_packet.py with --packet-dir provenance.
The final book projection contains the inserted Unicode line with
candidate_line_review_insertion.row_id equal to the original gap_row_id. This proves
the browser station is not a dead-end review surface for missed lines; it feeds the
existing audited final-book path.
Current 2026-06-23 layout-gap ink guidance: tools/run_limbu_book_ocr.py now adds
dark-pixel ink stats to layout-gap-review-queue.tsv: dark-pixel count, density,
ink bbox, row-band classification, candidate-text band count, horizontal-rule count,
hash-bound candidate text-band crop paths/hashes, full-page gap-context image
paths/hashes, priority, and recommended action. The context image highlights the
suspicious y-band on the full prepared page, so reviewers can reject section rules,
headers, and spacing without interpreting a thin crop in isolation. For text-like gaps,
the dashboard now shows the full gap crop, highlighted page context, and focused
candidate-band crops, so the reviewer does not have to transcribe from a full-width
spacing band. This keeps the missed-line queue focused on gap bands that contain likely
text-shaped ink instead of treating every large geometric space equally. The layout-gap
dashboard and README expose the same priority/action counts. Re-running the packet
writer on the latest p1-only triaged projection at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/layout-gap-review-ink-guidance/
produced 1 gap row, priority counts {"low": 1}, action counts
{"likely_rule_or_spacing": 1}, and the row records 5737 dark pixels with density
0.038029, 0 candidate text bands, 3 horizontal-rule bands, and empty
candidate-band crop lists ([]). This corrects the earlier geometry-only warning: the
p1 gap is likely rule/spacing, not an actual missed body line. Page triage now consumes
this ink-guided gap summary before assigning
operator flags: the same p1 check at
.../triaged-p1-only/ink-guided-page-triage-check/ keeps
suspicious_gap_count=1 for auditability but writes
candidate_text_gap_count=0, horizontal_rule_gap_count=1, removes
possible_missed_line_gap, and recommends review_low_confidence_lines instead of
layout-gap inspection.
Current 2026-06-23 layout-gap application hardening: accepted layout-gap insertions now
carry candidate text-band crop paths/hashes into inserted-line metadata and insertion
pairs when those focused crops exist. Both tools/apply_limbu_book_layout_gap_reviews.py
and tools/audit_limbu_book_layout_gap_review_application.py validate those optional
candidate-band crop hashes, in addition to the full gap crop hash. This is not another
eval-harness feature; it protects the review-to-training bridge from detached TSV edits
or stale evidence after a reviewer inserts a missed line. The audit also cross-checks
layout-gap-insertion-pairs.jsonl against the inserted document lines and metadata, so
a self-consistent pair-file hash is not enough if the pair text/crop evidence no longer
describes the actual edited book projection.
Current 2026-06-23 review-burden cleanup: tools/parse_limbu_book_pages.py now emits
book-line-focused-review-queue.tsv and book-focused-review-dashboard.html beside the
canonical full queue and priority queue. The focused queue keeps every critical/high/
medium row and only a deterministic, evenly spaced sample of
spotcheck_low_confidence_text rows per page (default 25 via
--focused-low-confidence-limit-per-page). This does not remove any canonical review
row, and it does not change OCR text; it gives the operator a bounded low-confidence
spotcheck surface while preserving exhaustive flags for later review/application. Page
triage filters the focused queue when pages are rejected, and
tools/audit_limbu_book_output.py hash/card-count audits the focused queue/dashboard.
tools/apply_limbu_book_review_queues.py records the focused queue and summary in its
top-level application report for provenance, but still applies the canonical queue.
The book-output audit verifies the focused queue is a true subset of the canonical
book-line review queue, not merely a smaller TSV with a matching hash.
The combined review wrapper now refuses an explicit focused queue as
--book-line-review-queue; use the canonical queue for application.
tools/merge_limbu_book_focused_review_queue.py now closes the operator-loop gap:
after a reviewer edits the bounded focused TSV, the tool overlays only editable review
fields (review_status, corrected_text, reviewer, notes, and split fields when
present) onto the full canonical queue. It refuses detached focused rows, header
mismatches, duplicate row identities, and any non-review/provenance field difference,
so crop paths/hashes and OCR metadata cannot drift while focused edits are promoted
back into an application-safe full queue.
Real refresh with stable page IDs:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-focused-review-view-stable-pageids/
was run under /tmp/ocrtech-paddle311/bin/python with the v4 router unchanged,
duplicate-source skipping, and --page-id-prefix gorkhapatra-unique-current-p. It
keeps the same operational shape as the prior two-page run: 2 pages, 692 preserved
lines, 599 canonical book-line review rows, and a passing book-output audit. The
focused queue is 369 rows: all 319 must-review rows plus 50 sampled
low-confidence rows, omitting 230 low-confidence spot checks from the bounded working
view. After explicitly rejecting the known non-Limbu page
gorkhapatra-unique-current-p0002, the p1-only projection at
.../manual-triage-check/triaged-p1-only/ passes audit_limbu_book_output with zero
issues and leaves 230 canonical rows but only 35 focused operator rows (10
must-review rows plus 25 low-confidence spot checks). The triaged projection now has
fresh filtered dashboards too: canonical 230 cards, priority 230 cards, and focused
35 cards, with no rejected-page card in the focused dashboard. This is the current
useful review packet for the real Limbu page; it is review-burden reduction, not
OCR-quality improvement.
No-op merge proof on the current pending p1-only focused queue writes
.../manual-triage-check/triaged-p1-only/book-line-review-queue.focused-merged.tsv:
230 output rows, 35 focused rows consumed, 0 changed rows, and a merge report
beside it. After focused review is filled, the same command is the handoff point into
tools/apply_limbu_book_review_queues.py --book-line-review-queue <merged-full-queue>.
Codex then used the focused contact sheets to make only obvious visual drop decisions,
not Limbu text transcriptions: footer/contact metadata img-line-0179,
date/header metadata img-line-0180, portrait detection img-line-0193, masthead
img-line-0001, website/page furniture img-line-0002, and decorative page
ornament/icon img-line-0039.
The edited focused queue is
.../manual-triage-check/triaged-p1-only/book-line-focused-review-queue.codex-obvious-drops-v2.tsv;
the merged full queue is
.../manual-triage-check/triaged-p1-only/book-line-review-queue.codex-obvious-drops-v2-merged.tsv
with 230 output rows and 6 changed rows. Applying that full queue with
tools/apply_limbu_book_review_queues.py --skip-page-triage writes
.../manual-triage-check/triaged-p1-only/reviewed-obvious-drops-v2/final-book/: 6
book-line drops, 317 remaining lines, 0 layout-gap insertions, and
audit_limbu_book_output passes with zero issues. Reviewed training-pack export
correctly refuses this drop-only projection because there are no accepted text
corrections; drops must not become recognizer labels.
The next operational artifact is a pending-only focused review packet, not a larger eval
harness. It was prepared from the edited focused queue at
.../manual-triage-check/triaged-p1-only/pending-focused-review-packet-v2/ using
tools/prepare_limbu_book_unresolved_review_packet.py --book-line-review-queue .../book-line-focused-review-queue.codex-obvious-drops-v2.tsv and audited with
tools/audit_limbu_book_unresolved_review_packet.py: 29 packet rows, 6
non-pending focused rows skipped, item counts {"existing_book_line": 29}, all rows
existing_low_confidence_line, suggested action correct_existing_line_or_accept,
audit_passed=true, issues=0. The packet HTML has been regenerated as an editable
review station at
.../pending-focused-review-packet-v2/book-unresolved-review-packet.html: each crop
has review_status, prefilled corrected_text, optional split fields, reviewer,
notes, a source-page context image with the active line highlighted, neighboring OCR
lines in document order, local browser autosave, filters for all rows / blank OCR /
prefilled rows / pending / reviewed / preflight issues, per-row quick actions, and a
browser-side "Export reviewed TSV" button that emits the same applicator schema as
book-unresolved-review-packet.tsv. Candidate missing-line rows now also get
geometry-based neighboring OCR lines around their proposed insertion point, so
reviewers can judge missed-line crops against local document order instead of the crop
alone.
The quick actions can accept current text, mark corrected text, drop existing lines,
keep pending, and jump to the next pending row; the same active-row actions are also
available from the keyboard: Ctrl/Cmd+Enter marks a nonblank correction,
Ctrl/Cmd+Shift+Enter accepts nonblank text, Alt+D drops an existing line, Alt+P returns
it to pending, and Alt+N jumps to the next pending row. Accept/correct refuses blank
text in the browser UI. For nonblank OCR rows, corrected_text is prefilled as an
editable starting point while review_status remains pending; this is not training
truth until the reviewer explicitly accepts/corrects the row and fills a reviewer name.
Browser autosave only protects in-progress editing and is guarded by the current packet
signature, so stale local drafts from older regenerated packets are ignored; the
exported TSV remains the review handoff artifact. Export is browser-preflighted for
missing reviewer names, blank accepted text, machine-looking reviewer names, invalid
statuses, and invalid drop/reject rows, but the command-line packet audit remains
authoritative.
tools/audit_limbu_book_unresolved_review_packet.py now accepts --packet-tsv, and
tools/apply_limbu_book_unresolved_review_packet.py now accepts --packet-dir, so an
exported book-unresolved-review-packet.reviewed.tsv can be audited/applied against
the original packet provenance without overwriting the source TSV. See
docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md for the exact commands.
tools/apply_limbu_book_reviewed_packet_to_training_pack.py is now the preferred
post-review handoff: it applies the reviewed TSV, audits the final book, and exports
the reviewed-only training pack in one command, failing if no reviewed rows are
exported. This is the next human review surface. Once those rows are accepted/corrected
in the exported/reviewed packet TSV, run that handoff wrapper against the already-cleaned
reviewed-obvious-drops-v2/final-book/ base. Codex also ran a non-claim mechanical
smoke with one copied OCR row marked accepted in /tmp: it exposed stale chainable
document-hash recording in the book-line/candidate applicators, which is now fixed; the
rerun passed reviewed packet audit, application audit, final book-output audit, and
exported training_rows=1. The smoke artifact was deleted and is not training data,
and the same contract is now covered by a strict synthetic wrapper integration test. The
wrapper now writes a README.md in the handoff directory with artifact paths and the
next split-manifest / train-recognizer commands, while keeping the boundary explicit
that this is not a CER/WER quality claim. A
pending-only proof at
.../pending-focused-review-packet-v2/application-pending-on-reviewed-drops-base/
applies 0 rows, keeps final line count 317, records
book_line_review_row_count=29, and passes both
tools/audit_limbu_book_unresolved_review_packet_application.py and
tools/audit_limbu_book_output.py. The packet splitter now carries packet crop
path/hash into source_crop_paths / source_crop_sha256s on the generated book-line
review TSV, so accepted packet corrections retain crop evidence for training
provenance. tools/audit_limbu_book_unresolved_review_packet.py --allow-reviewed is
now the reviewed-packet preflight, and tools/apply_limbu_book_unresolved_review_packet.py
runs it before splitting or applying. Pending-only packets still pass as no-ops, but
accepted/corrected rows require explicit non-machine reviewer text and explicit
corrected_text or split_corrected_texts; drop/reject rows require reviewer identity
and must be existing book lines. The current pending proof records
reviewed_application_ready=false, which is correct because no human text labels exist
yet. The unified application audit now also verifies the packet-audit file record,
review_packet_audit_passed, reviewed_application_ready, accepted/drop status
counts, and requires reviewed_application_ready=true whenever the application changes
book text. The unified applicator now runs that application audit itself after writing
the final report, writes book-unresolved-review-packet-application-audit.json, and
fails the command if the audit does not pass. Do not restart generic eval-harness work
before this text review produces labels.
The reviewed training-pack exporter now runs tools/audit_limbu_book_output.py on the
source book output before it writes labels and records that audit in
source-book-output-audit/. A real retry against
reviewed-obvious-drops-v2/final-book/ wrote a passing source-book audit at
.../reviewed-obvious-drops-v2/training-pack-reviewed-after-source-audit/source-book-output-audit/
(passed=true, issues=[]) and then correctly refused the export with "no training
rows were exported" because the projection still has only drops, not accepted text
corrections. This is the desired state before human review: source consistency is
proven, but no labels are minted.
The same exporter now recognizes candidate_line_review_insertion metadata as
explicit review evidence, in addition to book-line corrections/splits and layout-gap
insertions. This matters for missed-line recovery: accepted candidate/missing lines
inserted through the unified review packet will now become reviewed training rows
instead of being silently skipped after application.
Direct candidate-line review application now also validates crop evidence whenever a
review row supplies crop_path or crop_sha256: both fields must be present, the crop
must exist, and the SHA-256 must match before insertion. The inserted-line metadata and
candidate-line-insertion-pairs.jsonl now preserve crop_path and crop_sha256, so
reviewed missed-line training rows stay tied to the crop evidence that justified them.
Current 2026-06-23 reviewed-projection audit hardening: applying book-line corrections
or layout-gap insertions now clears inherited focused queues and review dashboards from
the chainable book-run.json, because the canonical review rows may have changed and
the old focused/dashboard views can become stale. The chain writers also recompute
source_book_document_json_sha256 whenever they rewrite source_book_document_json.
tools/audit_limbu_book_output.py now checks document.md and document.body.md
beside the recorded document_json, while still using output_dir for the original
OCR page bundle/manifests/crops. This fixed the real reviewed-obvious-drops-v2 audit
failure mode.
Historical 2026-06-21/2026-06-22 state:
Critical 2026-06-22 correction: do not keep working as if the Devanagari PP-OCRv5
training run is still the blocker. It was intentionally stopped at the
best_accuracy checkpoint after the dev curve regressed while the CPU run still had
hours left. The stopped-run completion audit passed with ready_for_export=true, the
model was exported and packaged at
outputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-stopped-2026-06-22/package/model-card.json
(SHA-256 b05ca0e4a74f1818307bb547780b49969a60e93d273dda7a78e342cfdbd0d788), and
the sealed 31-line recognizer recipe scored codepoint CER 0.118219, grapheme CER
0.256822, and exact-line accuracy 0.290323 at
outputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-stopped-2026-06-22/sealed-heldout-eval/recipe-run-paddle-external/score.
This is not claim-safe because the pack and GT are still draft evidence, but it is
enough for engineering direction. The same recognizer has now been run on the three
real CDC Devanagari pages with the existing image-line detector:
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-stopped-epoch11
scores page codepoint CER 0.480602, grapheme CER 0.697389, WER 0.872727, line
F1 0.400301 mean / 0.350877 micro, and matched-line codepoint CER 0.221018.
That replaced the old image-line Tesseract page result (0.865400 codepoint CER,
0.872156 matched-line CER). A directed detector sweep then tested simple morphology
cleanup with the recognizer held fixed. The selected detector
h31/v5/min_width80/max_height80 lives at
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80
and scores page codepoint CER 0.458432, grapheme CER 0.645161, WER 0.824242,
line F1 0.593750, and matched-line codepoint CER 0.177895. It reduces predictions
from 83 to 33, but still matches only 19/31 reference lines. The paired
comparison against the original PP-OCRv5 detector is non-claim-safe but positive:
codepoint CER improvement 0.022169, grapheme CER improvement 0.052227, paired
pages 3. The updated line-failure analysis at
outputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-devanagari-cdc-ppocrv5-detector-h31-v5-minw80-maxh80
shows 19 matched rows, 12 missed references, 14 extra predictions, and dominant
error mass from missed reference text (47.58%). So the next Devanagari work is
targeted missed-line recovery/layout modeling with this recognizer held fixed. More
packaging, another blind recognizer loop, or another broad morphology sweep is the
wrong next move unless the page-level failure analysis changes.
Follow-up rescue-pass tests confirm that warning: a compact long-line rescue improved
mean line F1 (0.605257 -> 0.626903) but worsened page codepoint CER (0.458432 ->
0.464766), while broader rescue passes regressed to 0.537609 and 0.832146 by
reintroducing noisy extra predictions. Evidence and a CDC geometry review packet live at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/. The review packet has 3
page images and 31 line boxes, with browser review page
devanagari-cdc-linebox-geometry-review-packet/line-box-review.html.
The obvious next hypothesis, splitting wide connected components on vertical whitespace,
has also been tested. The option is now wired in the standalone materializer and
operational parse-limbu-doc image-line path, but all CDC split-wide variants were
no-ops: detector source counts stayed {"primary": 33} and page codepoint CER stayed
0.458432 with the same metric SHA-256
b1d993e9fcf09a7cff18d31c3c7e43ee2ff34cd8a57eaae96526e9f977d4c5cb. Do not repeat
simple wide-component splitting; inspect the geometry packet and implement
page-region/row recovery for the missed areas. The concrete target list is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/devanagari-cdc-page-region-recovery-targets.md:
12 misses split into 3 no-overlap rows, 4 overmerged rows, 2 partial detections, 1
low-IoU refinement, and 2 page markers. Existing-row repair has now been tested and
rejected too: same-row merge gap80 regressed page codepoint CER to 0.463975,
detected-row split gap24/min700 regressed to 0.467933, and detected-row split min900
was a no-op. The detected-row split switches are now available in both the standalone
materializer and the operational parse-limbu-doc / book-manifest image-line path, but
the CDC evidence still rejects them as a default. Next work should recover absent rows
and short page markers from page regions. Region-filtered rescue is the first positive
step: rescue-pass parsers now
accept region_min_x_ratio, region_max_x_ratio, region_min_y_ratio, and
region_max_y_ratio, and the tight bottom-long + top-title CDC diagnostic improves
page codepoint CER from 0.458432 to 0.452890, grapheme CER from 0.645161 to
0.637481, mean line F1 from 0.605257 to 0.635724, and matched references from
19 to 21 with 36 detected lines. The score artifact is
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80-region-rescue-bottom-long-top-title-tight/metrics.json
(SHA-256 1c787c10392edf4f8a27dac018b5406b3a960b963a419aa4a64e9ec1c0acc0af).
The conservative page-marker follow-up is a page-CER tie and the current line-structure
best: codepoint CER stays 0.452890, mean line F1 rises to 0.659689, and matched
references rise to 22 with 37 detections by recovering the social-page २ marker
(score SHA-256 19846ebae905411387ccbb88ee9520021387aac9f3debcc4ce81e88a87e1c836).
The runtime image-line path and standalone materializer now tag conservative
bottom-margin digit-only small-box detections as structural_role=page_marker; the
fresh h20 structural rerun marks exactly the recovered social-page २ marker and keeps
the same metric SHA-256, so it is metadata-only for current scoring. Do not treat this
as script-count exclusion. export_document now writes a separate document.body.md
that omits structural blocks while keeping full document.md and document.json
auditable; the one-page operational social artifact
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/operational-parse-social-page-marker-structural
proves the recovered marker remains in full markdown, is excluded from body markdown,
is bound in limbu-output-manifest.json, and passes audit_limbu_output.
The current h20 unresolved failure surface also has a textless review packet at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/:
24 crop rows, split into 9 missed-reference regions and 15 extra-prediction regions.
Reference text is not written to the review JSONL/HTML; 2 exact machine-text echoes of
hidden reference strings are suppressed, and the latest leak check found zero hidden
reference strings in the packet. It is covered by
tools/audit_unresolved_page_review_packet.py; the current audit artifact
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/unresolved-region-review-audit.json
passes with 24 rows, 0 hidden-reference leaks, and 0 issues (SHA-256
33105c0144c5cfa56ee20554a6aee82434f9c4cb6c19a60cc9533882a918db5b). Treat this as
a review handoff artifact, not a solved OCR-quality improvement.
A targeted h20 + same-row merge gap120 follow-up was tested for the remaining
top-title/fragment near misses and rejected. It raises mean line F1 to 0.694815, but
page codepoint CER regresses to 0.465558, grapheme CER to 0.654378, matched-line
CER to 0.190031, and reading-order pair accuracy to 0.571429; the refreshed
failure analysis still has 9 unmatched references and dominant missed-reference error
mass. Do not chase broader same-row merging as the next default path.
The first book-shaped wrapper has also landed as tools/parse_limbu_book_pages.py. It
reads an ordered JSONL page manifest, runs the existing parse_limbu_document path for
each page, and writes per-page OCR bundles plus aggregate book.md, book.body.md,
book-review-queue.tsv, book-run.json, and a scorer-compatible score-documents/
projection. The persistent sidecar smoke artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-book-sidecar-smoke
(2 pages, 4 lines, 1 aggregate review row, 0 page audit failures, book-run.json
SHA-256 7f9da4cec883e0fc9e72e16f62c1ac633046daea4c86091d25129643376f8819). More
importantly, the selected h20 structural CDC diagnostic has now been replayed through
the book wrapper using real CDC page images and selected materialized page documents.
Replay prep lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-replay-inputs
(summary SHA-256 c28d0511bc42b855125c8e6a2063715ff15a0d085869a2107e32e7dfb4f411cf);
book output lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay
(book-run.json SHA-256 9654cebe89ab68618cf5bf36419e5bd3a9513195faf9d4f9eb4ce5481adb3e8d).
It has 3 pages, 35 lines, 27 aggregate review rows, 0 page audit failures, and score
projection metrics of page codepoint CER 0.446556, grapheme CER 0.626728,
matched-line codepoint CER 0.175130, and matched-line reading-order pair accuracy
0.600000 over 3/3 pages with no missing docs/references or source-path mismatches
(metrics SHA-256 1a353b2d06b89caf081d190ad79add2eee57d1d51b70cb6144be40e53d082d75).
The replay prep drops blank selected-output lines by default and records them; in this
CDC replay it dropped 2 science-page blanks (img-line-0006, img-line-0007). The
scorer now exposes the objective's line-rate surface directly: 31 reference lines, 35
predictions, 22 matches, 9 missed references, 13 extra predictions, 0 blank
predictions, micro line precision/recall/F1 0.628571/0.709677/0.666667, and
missed/extra/blank rates 0.290323/0.371429/0.000000. Treat this as real-page
replay/packaging/scoring evidence, not fresh OCR inference.
The replay exposed and fixed a body-export bug: sidecar-replayed structural
page_marker lines could be grouped into normal text blocks and leak into
book.body.md; src/ocrtech/structure.py now emits structural lines as separate
blocks first. This is editable-text progress, but not finished OCR quality: 27 review
rows remain and matched-line CER is unchanged.
The lower-height h10 marker pass recovers more single-digit markers and reaches 23
matched references, but it regresses page codepoint CER to 0.454473, grapheme CER to
0.639017, WER to 0.844707, and reading-order pair accuracy to 0.538462; keep it
as diagnostic evidence, not the default. Page markers need explicit structural/review
semantics rather than being blindly mixed into page text order.
The h10 marker pass composed with source-break post-recognition line assembly is also
rejected as the default. It gets recall to 0.806452 and missed references to 6, but
page codepoint CER is only 0.452890, precision drops to 0.675676, extra predictions
rise to 12, two blank predictions appear, and reading-order pair accuracy is
0.553191. Score:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-cdc-h10-page-markers-line-assembly-gap160-width500/metrics.json
(SHA-256 6ccf1bce5263f0eed1fc966a6d2f9c3c6567534fdd750e7a05549462a12b6c57).
New operational detector parity work: parse-limbu-doc now supports
min_aspect_ratio / max_aspect_ratio in primary and rescue image-line detector
passes, and opt-in image_line_split_detected_tall_* settings. The geometry probe on
the science page split the known over-tall rescue-1 box [319, 1856, 1075, 115]
into [319, 1856, 1052, 37] and [317, 1944, 1051, 29]. The Paddle follow-up has
now been scored and rejected as a default path. It improves line recall/F1 to
0.838710 / 0.776119 and cuts missed references to 5, but page codepoint CER
regresses to 0.478226, grapheme CER to 0.674347, WER to 0.882958, matched-line
CER to 0.191242, and reading-order pair accuracy is only 0.578431. The book audit
passes (ceb9d681e4ee50ac48880f41b5c0dc25e084cd4cfcc43b0774d573366634210e), but the
candidate gate rejects it (admitted_operational=false, admission SHA-256
2d56eaaf0071fa17341d9905aaa7bc46d5bb89fa40c1428e936645e7727665dc) because it
regresses against the 0.458432 baseline and misses the 0.60 reading-order
threshold. Treat tall split as review/recall evidence, not selected OCR text. Probe:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/detected-tall-split-geometry-probe.json
(SHA-256 51b364b4211f40d54714c903b9038fab284fc83e7853efa85ccc3ebb32225c64).
The exact Paddle follow-up command is documented at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/RUN_ON_PADDLE.md
(SHA-256 f3cb56df021fe20982746b1a657a8ab80248bb019694ef5bfcf04c4f5e776a37) and uses
manifest SHA-256 a6657c2f9cb7be1ba7e8c89c91650bf9ab18c3071a7042b89580a3a10f54dfad.
The review-only reuse path is now implemented too:
tools/prepare_candidate_line_review_packet.py compares selected score documents to a
secondary candidate document set and crops candidate-only lines below the selected-line
IoU threshold; tools/audit_candidate_line_review_packet.py verifies the packet. The
detected-tall-vs-current-best packet has 4 rows (maths marker १, two science
bottom-row split candidates, and science marker २) at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-only-review-packet/.
Packet SHA-256 is f20438d05d158377387a05df9197cc3a6e7e9afec573a60668c1d429d1b83128;
audit SHA-256 is 919615e48fe66c24810e6137b5bb65dd64b348ebfee0b5817af38b47f38c2f40
and passes with 0 issues. Use this pattern for rejected high-recall candidates:
surface missing-line evidence for review while keeping selected OCR text stable.
Accepted candidate-only rows can now be inserted with
tools/apply_candidate_line_reviews.py; it writes a separate book projection,
score-documents, and candidate-line-insertion-pairs.jsonl with provenance, never
mutating the selected OCR run. The real detected-tall packet is still pending-only:
the smoke output at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-line-review-application-pending/
has review_rows=4, applied_count=0, skipped_count=4, audit SHA-256
95521bc3c5ad86bd6cdf19350cfa1323a972ce817ee363451ff79349ccc4f69f, and preserves
selected book.md / book.body.md byte-for-byte. tools/audit_candidate_line_review_application.py
independently audits these application outputs; the real pending application audit
passes with 0 issues, SHA-256
0e6addc547c1ef8df3ae9fe54d6599a16341fb2d57168d3b785da4cbe38cf452. This completes
the review loop mechanically. The pending application projection has also been scored
and admitted through the existing book candidate gate using the application audit:
metrics SHA-256
184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244, operational
admission SHA-256 eb0e559908046322343a7507a0143ae2d724876c069eb2528f0d1950085fd5b5,
admitted_operational=true, claim_safe=false. Because no candidate rows were
accepted, the score is byte-identical to the current h20 source-break line-assembly
candidate. Actual missing-line recovery still depends on accepted human review or a
better automatic selector.
One automatic selector has now been tried and rejected as a default:
tools/select_candidate_line_reviews.py implements page_marker_v2, a conservative
bottom-margin digit-only structural page-marker policy with confidence, selected-line
IoU, size, and duplicate-text guards. On the real detected-tall packet it selected only
the maths marker १ and rejected the other 3 rows; selection report SHA-256
928aaecd9a102c5615e1085b3d29a80188d419c35e17bbfc9c743876f7e20872, selected JSONL
SHA-256 8152ba724a67a4ef25296c6dcd47a02c7050262356065d6c0c0979fa4f366867, rejected
JSONL SHA-256 a3a571ad12c8efade971d0b89baa0ef9b1345cb15d97beba3ec284f5089b3e85.
The applied projection passes independent application audit (SHA-256
b0e377bec4f0fca9f4506fa9126a65c57014adf7d419a04e08f98d50bf3b8743) and improves line
recall/F1 to 0.806452 / 0.769231, but it regresses page codepoint CER to
0.448139 and drops reading-order pair accuracy to 0.553191; admission rejects it
with SHA-256 9cd826a51b38aa29facab4ec332a4084421ac25bf26365dc04f954e6f1b5a76f
(admitted_operational=false). Current best remains h20 source-break line assembly.
Do not spend the next cycle blindly inserting page markers; the issue is structural
export/scoring semantics, not lack of a marker selector.
To make that visible, tools/run_page_ocr_eval.py now emits structural-line
diagnostics in addition to the existing all-line metrics. It counts non-text
metadata.structural_role predictions, matched/unmatched structural predictions,
exact structural text matches, and per-role prediction totals. A replay of the current
best with these metrics has one exact matched structural marker (metrics SHA-256
f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f). The v2
auto-marker replay has two exact matched structural markers (metrics SHA-256
2634d0f70f6dac3cac08844ea1530aff08c26271a92a7eb2c725a879415ce5b9) while still
showing page CER 0.448139 and reading order 0.553191. The correct interpretation:
the marker signal is real, but the editable body-text candidate is worse.
There is now a body-line review-ceiling diagnostic too:
tools/select_candidate_line_reviews_by_reference_overlap.py. It is intentionally
oracle-labeled: it selects candidate-only review rows by overlap with currently
unmatched reference lines, not by deployable OCR evidence. On the detected-tall packet
it selected exactly one text row, the science bottom line, with IoU 0.860889 to
limbu-devanagari-cdc-science-p0009-l0009 (selection SHA-256
d11dcdc1cc0b07626e6d85b97356c86cadd670796b9e926708e2be7585344d05). If the selected
row is inserted with reference text as the corrected text, score improves to page
codepoint CER 0.420428, line recall/F1 0.806452 / 0.769231, and reading order
0.634409 (metrics SHA-256
247c4186059047fc9de16019f358172bc46261de497127581f63335f986cb521). If the same row
is inserted with its candidate OCR text, page codepoint CER is only 0.445764 and
matched-line codepoint CER worsens to 0.191413 (metrics SHA-256
21fd79e6a0645c0084cac959743354f52c224b28b15e4e60be7f56e3ac664d75). The application
audit now detects oracle/reference-assisted review rows and sets
operational_review_safe=false; tools/admit_limbu_book_ocr_candidate.py rejects
such applications even when metrics pass. Admission SHA-256 values:
c8f69e7d011c217e74218ffe280e901420ab99ebefa7f329344af776a6e779c7 for reference
text and 2a97b7f01425efdd514887324c8c23c6b4dd46c48376b99c2299fad736d88b45 for
candidate text. Current best remains h20 source-break line assembly; this diagnostic
only proves that a reviewer-correction lane can recover a real missed body line.
For the actual non-oracle review lane, use
tools/prioritize_candidate_line_reviews.py. It ranks candidate-only rows from packet
fields only and keeps every row pending. The real queue at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/
has 4 rows: 2 high-priority body-line candidates followed by 2 structural marker
checks. Hashes: prioritization JSON
ef3fc75efc52597c97ea2e28196164b31ef3ceb18414ef5b47e559cc4343e51a, JSONL
9a53ad3d3ada7e61211d5406ce35c9ed9a10e86a41ea09ea0f4edcb3766ad49e, TSV
753adcd61f70f490b0429d6ff9d25ff88b0bc1ac8ec20a2f9ae21ec1f514fa19, HTML
5fbd7f39801ac7e29b0503314475ef26e6e61345b26330720bf840ad1e2dcdef. The queue now has
an independent audit at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/prioritized-candidate-line-review-audit.json
(SHA-256 d2fe0e33e3bd5f78698632892581651a3e95332936b7e9ddfa926c1ecc3fcd22): it
passes with 0 issues, 4 source/JSONL/TSV rows, no oracle/reference-assisted rows, and
operational_review_queue_safe=true. The first two rows are the science bottom-row
text candidates and should be reviewed with action transcribe_visible_line_or_reject;
do not apply them without corrected text from a reviewer. Rerun
tools/audit_prioritized_candidate_line_reviews.py only after regenerating or manually
touching the pre-review queue; it expects all rows to stay pending. After reviewer
edits, use tools/audit_candidate_line_review_application.py on the application output.
There is also a non-oracle crop probe for the two body-line rows:
tools/prepare_candidate_line_text_crop_probe.py. The real artifact at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-text-crop-probe/
processed 2 body-line candidates and refined both. Rank 1 was top-bleed-trimmed from
1100x85 to 1100x63; rank 2 was trimmed left of a detected horizontal rule from
1099x77 to 466x77, removing the rule/illustration before recognition. Hashes:
report 7f1dfc09d4cf612e7516dbdea07eb6bc19608e637ad11efe41e8a8112393c79a, JSONL
a7d7cced3cdca1a223aa6c6d00ccc70acabee90183107eac6b70ff61b630f011, HTML
10db1161085a49e722ad373b88ff9656aaf87b5423dad2ec88e8f1c48f1f18b6, label file
47b3594dac7a97f516d21f271c1fe1e794f6d621bfc17539d1cf6b8e540ef4e0, refined rank-1
crop 4e97bca5580d613777199395db9e091e248cc79035a1063a284a896ec9632594, refined
rank-2 crop 4f8221c4552b937e8e0182a4ecf9f1062a9b1056ec377bfd4ffe00e99c020d5a. The
Devanagari PP-OCRv5 recognizer was run locally over the refined crops in both
.venv-paddle313 and /tmp/ocrtech-paddle311; both produced the same prediction TSV
SHA-256 734b208860106d84a7dbd7473e18927293a29b51e8b67298e173c5b2c626bbee with 2 rows
and 0 failures. A comparison artifact at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-crop-recognizer-probe-comparison/
shows it is not promotable: max probe confidence 0.469282, rank-1 confidence improves
0.251062 -> 0.469282, rank-2 confidence improves only 0.363574 -> 0.390961,
prediction text changes twice, and
automatic_application_safe=false (report SHA-256
18cdfb099ae6d93ee96ee350faec0305837349d65142ebaaad05db1553c04eb7, JSONL SHA-256
f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d). Do not promote
the crop probe without reviewed text or a better recognizer path that passes
page-level admission.
The TSV workflow is now wired into application: tools/apply_candidate_line_reviews.py
accepts --candidate-review-file pointing at either JSONL or TSV, while keeping the
old --candidate-review-jsonl alias. tools/audit_candidate_line_review_application.py
also reads TSV review files. A real pending TSV application smoke lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review-application-pending/.
It applied 0 rows because all prioritized rows are still pending, preserved selected
book.md / book.body.md byte-for-byte, passed application audit
(e36036c0e3a9177e7c526465cf16c4a066f54ae37fe920bfb97c1a3bb9278c5c), scored with
current-best metrics (f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f),
and passed operational admission
(453226159011a944a606e0c951c5b7dceda3bee73c76f44d195c44e2d61672c1,
claim_safe=false). This means the next real movement is to review/accept ranked
body-line rows, not to build more plumbing.
The split-worksheet review lane now has the same score/admission closure:
tools/score_limbu_book_ocr_candidate.py scores any candidate book projection with
tools/run_page_ocr_eval.py, then runs tools/admit_limbu_book_ocr_candidate.py.
Admission now accepts unified unresolved-review application audits only by verifying
that the audit's final-book identity matches the candidate book-run.json SHA-256;
normal book audits still require output_dir equality. The application audit tool now
records final_book_output_dir and output_dir for application/final-book/. The
real pending split-worksheet application was scored/admitted at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-worksheet-review-application-pending:
wrapper report SHA-256 f546cb6e4cf007c1a32b8e6799ce3203d1060eafd327cc86a72921217a59a832,
metrics SHA-256 f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f,
page-eval-run SHA-256 3356fb45524953c8837b6ab237c701866285adf11661101de9e634de8b4c153b,
and admission SHA-256 7fadfc25c0191fa71abb9d93e6266c0e0078fea015df7a02bd067d02556db966.
It is operationally admitted and claim-unsafe, with the same metric values as current
best: codepoint CER 0.446556, grapheme CER 0.626728, WER 0.839243, line F1
0.750000, missed/extra/blank rates 0.225806 / 0.272727 / 0.000000, and
reading order 0.611765. The hash differs from the older current-best score because
the eval metadata records the reviewed-worksheet candidate path/system name. This
wrapper closes the post-review gate; it is not OCR improvement by itself.
Remaining failure analysis lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-failure-analysis-devanagari-cdc-region-rescue-bottom-long-top-title-tight/;
missed reference text remains dominant, so do not declare book OCR solved.
parse-limbu-docnow accepts project-relativeprepare-limbu-capturemetadata paths; the previous binding logic double-prefixed relative prepared paths.PaddleOcrEnginenow supportsrecognition_mode=auto, using full PaddleOCR for page images and lower-level PaddleOCRTextRecognitionfor wide/short line crops.outputs/limbu-first-router-v1/model-card.jsonopts the Sirijonga secondary into that auto mode and now points at the exported v4 component undercomponents/limbu-sirijonga-paddle-v4/.- The stronger v4 checkpoint archive
outputs/limbu-recognizer-v4-2026-06-18/limbu-v4.tgz(a0ec317d053402f0966e3b87f3d489b64f33810cc99167f85c0c267101a22cae) has been exported with PaddleOCRtools/export_model.pyfrom/tmp/PaddleOCR-v3.7.0and packaged atoutputs/limbu-v4-paddle-package-2026-06-21/package/model-card.json. The stalestronger_unexported_checkpointadvisory was removed from the router. - The router now has a Devanagari primary guard:
primary_guard_script=devanagari,primary_guard_threshold=0.5. This prevents a Sirijonga recognizer hallucination from replacing a line that Tesseract already sees as clearly Devanagari. - Automatic Limbu post-correction profiles now have an audit/admission gate:
ocrtech audit-limbu-post-correction-profile. Profiles with no admission evidence can still run but are marked experimental andaudit-limbu-outputwarns they are not claim-ready. Profiles that claimadmission.status=validatedmust point at a hashed eval artifact, frozen eval-pack identifier, metric, and non-regressing before/after values; otherwiseparse-limbu-docfails. - Accepted Limbu review correction pairs now have a deterministic derivation lane:
ocrtech derive-limbu-post-correction-profile. It reads one or morelimbu-correction-pairs.jsonlfiles, hashes them into a derivation report, drops unchanged rows, enforces--min-support, rejects conflicting noisy-to-clean mappings, and emits exact-match rules inlimbu-post-correction-profile.json. The output is deliberatelyadmission.status=experimental; held-out admission is still required before any automatic correction is claim-ready. - Limbu post-correction profiles now also have a held-out scorer:
ocrtech score-limbu-post-correction-profile. It applies the exact runtime profile rules to held-out noisy/clean correction pairs, writes JSON/JSONL/Markdown eval artifacts, reports codepoint micro CER before/after, fails on aggregate regression or any regressed held-out row, and printseval_run_sha256foradmission.eval_run_sha256. - Passing post-correction evals can now be stamped into a new admitted profile with
ocrtech admit-limbu-post-correction-profile. It refuses failed evals, profile/eval SHA mismatches, missing frozen eval-pack IDs, aggregate regressions, and row regressions, then writeslimbu-post-correction-profile-admitted.json, an admission report, and an audit directory. Do not hand-edit profile admission fields. - Reviewed Limbu correction pairs now have a frozen pack builder:
ocrtech build-limbu-correction-pair-pack. It canonicalizes noisy/clean rows, rejects duplicate sample IDs and empty text, writes deterministic train and heldout JSONL splits, records source file hashes, and emitspack_content_sha256. Use the train split for derivation and the heldout split for profile scoring. - Frozen correction-pair packs now have an independent replay audit:
ocrtech audit-limbu-correction-pair-pack. It checks summary shape, train/heldout file hashes, row counts, split labels, duplicate sample IDs, row text shape, available source file hashes, and recomputedpack_content_sha256. limbu-output-manifest.jsonnow records hashed input provenance for model cards, fallback model cards, capture-prep metadata, post-correction profiles, review source documents, and review TSVs when those inputs are supplied.audit-limbu-outputrecomputes those hashes, so stale model/profile/review inputs fail the bundle audit.- Real Gorkhapatra candidate page run, now treated as Namdhinggo/Sirijonga-script
evidence rather than Devanagari-written Limbu evidence:
outputs/limbu-dual-script-real-run-2026-06-21/gorkhapatra-devanagari-ocr-v4-guarded. Capture/output audits pass; 98 OCR lines, 22 review lines, average line confidence 0.821, quality score 0.896. Routing selected Tesseract for 7 guarded Devanagari lines and PaddleOCR for 91 Sirijonga-classified lines. The directory name is a stale operational label; accepted visual review and PDF target-font evidence point to Namdhinggo/Sirijonga Limbu, not Devanagari-written Limbu. - A current-schema rerun of that real Sirijonga page now exists at
outputs/limbu-dual-script-real-run-current-2026-06-21/gorkhapatra-sirijonga-ocr-v4-current, with fresh capture prep atoutputs/limbu-dual-script-real-run-current-2026-06-21/gorkhapatra-sirijonga-prep. Capture audit passed; auto-deskew skipped for insufficient near-horizontal line evidence. The OCR output has 92 lines, 4 tables, 25 review rows, and script countslimbu_sirijonga=79,devanagari_limbu=11,mixed_limbu_devanagari=1,other=1. The strict capture/Sirijonga audit passes atstrict-audit-capture-sirijonga/limbu-output-audit.json(SHA-25667fe54feab8f636996de78c36cb303e60c5799bdce2fbeca082a9cd2997ce564). The final strict policy audit with--require-no-pending-reviewfails, correctly, becausereview_line_count=25. Do not present this as finished digitization until the review queue is resolved. The current-schema output has also been scored against a derivative current-input copy of the non-claim Gorkhapatra draft reference atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/score-current-router-v4-current. It is still bad full-page OCR: codepoint CER 0.823802, grapheme CER 0.859202, line F1 0.010127, and 2/303 matched reference lines. The audit has no source-path mismatch but remains non-claim-safe, with the only hard audit issue being that the train/dev leakage check was not run; the larger claim blockers are one-page sample size, unreviewed machine-converted GT, non-frozen pack, and non-claim eligibility.compare-current-router-vs-live-v4-currentshows the fresh run is slightly worse than the old guarded live-detector score, not an improvement. Replaying the best image-line route on the same fresh prepared input gives the current best non-oracle diagnostic. The unpruned routecurrent-image-line-box-documents-width480-wide-rtluses Otsu/ink morphology boxes, a 23x3 kernel, 12 px crop padding, the width-480 Sirijonga line recognizer, and wide right-to-left order;auto_layoutselects that same order from image features. It scores codepoint CER 0.761144, grapheme CER 0.804042, line F1 0.833866, 261/303 matched references, and matched-line codepoint CER 0.106076. The pruned precision-first variantcurrent-image-line-box-documents-width480-wide-rtl-prune-conf085keeps 199/323 lines usingdrop_empty, confidence >= 0.85, and Sirijonga/mixed script filtering; it scores codepoint CER 0.730446, grapheme CER 0.763956, precision 1.0, recall 0.656766, and matched-line codepoint CER 0.036114. This is useful for review triage, not a solved full-page path, because it drops too much content.parse-limbu-docnow has--line-detection-mode image-lineto run this class of detector/crop recognizer path operationally. It writesimage-line-ocr-run.json,image-line-crops/manifest.jsonl, hashes the image-line crops in the Limbu output manifest, and still goes through the normal script-aware review queue and audit path. Final audits for image-line outputs should include--require-no-dropped-image-lines; use--image-line-filter-*only for explicit precision-first triage runs. Operational evidence now exists underoutputs/limbu-operational-image-line-current-2026-06-21: the unpruned run keeps 323 lines, drops 0 detected lines, has 96 review rows, passes capture/Sirijonga audit with the no-dropped-lines policy enabled (SHA-256ec5e62a13be875a86f16cb1725f7cbf586153adbc243d1df1adc3ede2a121300), and fails final strict policy on pending review rows; its score is codepoint CER 0.761144 and line F1 0.833866. The prunedconf085run keeps 199/323 lines and has zero review rows, but the earlier strict pass is now superseded by--require-no-dropped-image-lines: the current final strict audit fails withremoved_line_count=124(SHA-256a7527a6fe61c23bc37598ade9f8a8d9429b188a1616398e3f76c4f3639735fa4). This is correctly classified as a precision-first partial output, not complete page OCR; its line recall is only 0.656766 against the draft reference. - Gorkhapatra Limbu Sirijonga draft page eval now exists at
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21. It was built from PDF-nativeNamdhinggoSILLspans with thelimbu-namdhinggoconverter, using the existing accepted review row forgorkhapatra-language-page-0001-01. The draft has 303 reference lines, 36 spans with unmapped conversion issues, and is explicitly non-claim (gt_status=machine_converted_unreviewed,claim_evidence_eligible=false). The existing guarded live-detector OCR score is audited and non-claim-safe: codepoint CER 0.815643, grapheme CER 0.857214, line detection F1 0.009975, and 2/303 matched reference lines. The oracle-line-box diagnostic using the same v4 recognizer over 303 PDF-derived reference boxes is also audited and non-claim-safe: codepoint CER 0.029268, grapheme CER 0.060792, line F1 1.0, matched-line codepoint CER 0.030031, matched-line grapheme CER 0.063993, and 303/303 matched lines. The paired comparison shows a 0.786375 absolute codepoint-CER improvement over live detection. A tuned morphology detector now reaches line F1 0.833866 and mean IoU 0.879406 on the same page, but page text is still bad (codepoint CER 0.763835, grapheme CER 0.828226) and matched-line text is also poor (codepoint CER 0.285785, grapheme CER 0.551916). The new matched-line reading-order metric is 0.606543 over 33,930 aligned-line pairs; the older rawreading_order_pair_accuracyremains 0.0 only because live predicted IDs areimg-line-*while references arespan-*. This is the clearest local evidence that same-font Gorkhapatra Sirijonga recognition is strong when boxes are oracle-quality, while live full-page OCR still needs better live crop/line recognition and better aligned-line ordering; it does not fill the Devanagari-written Limbu requirement. - Real Sikkim Herald Sirijonga crop run:
outputs/limbu-dual-script-real-run-2026-06-21/sikkim-sirijonga-crop-ocr-v4-guarded. Capture/output audits pass and the line is recognized as Sirijonga Unicode with confidence 0.797. Diagnostic codepoint CER against the demo GT is about 0.095, a real improvement over the earlier deployed component's roughly 0.302, but it is still a one-line diagnostic and remains in review. - Real Sikkim Herald Sirijonga page run:
outputs/limbu-dual-script-real-run-2026-06-21/sikkim-sirijonga-page-ocr-v4-guarded. Capture/output audits pass; 22 detected lines, 20 review lines, average confidence 0.635, quality score 0.619. The page still fails the quality gate. This is operational evidence that the page path runs, not evidence that it is good. - Sikkim Herald Sirijonga draft page eval now has both live-detector and
oracle-line-box diagnostics, plus an image-morphology line-detector prototype. The draft pack is
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1and is not claim evidence. Live detector scoring has codepoint CER 0.891560, grapheme CER 0.897412, and 1/86 matched lines. The image-line detector prototype generated bytools/materialize_page_image_line_boxes.pydetects 125 lines, matches 59/86 references, gets line detection F1 0.559242, codepoint CER 0.460255, and grapheme CER 0.576710. Oracle-line-box scoring, generated bytools/materialize_page_oracle_line_boxes.pywith the v4 component forced torecognition_mode=line, has 86/86 matched lines, codepoint CER 0.274289, and grapheme CER 0.548983. The comparison shows detection/layout is a major failure, the morphology detector is a useful but insufficient prototype, and the remaining oracle-box CER is still too high, so Sikkim-register recognizer data and reference verification are also needed. - There is still no verified Devanagari-written Limbu page reference in the local evidence set. The Gorkhapatra accepted Limbu page is Sirijonga/Namdhinggo; the two-script goal is therefore imbalanced until a real Devanagari-written Limbu page is found or transcribed.
- Local Devanagari-written Limbu text exists at
data/synthesis-text/cdjk-rai-limbu-2026-06-16/toolkit-parallel-text.jsonl(language=limbu,script=Devanagari, sourcebible_brain/LIFWBT_Limbu), but it is text-only and license/claim status is pending. A synthetic rendered Devanagari smoke page now exists atoutputs/limbu-page-eval-drafts/limbu-devanagari-smoke-page-2026-06-21; the sidecar-perfect score/audit pass consistency, butclaim_safe=falseby design because it is synthetic, one page, not frozen, and not human verified. tools/inventory_limbu_page_sources.pyis now the source-readiness gate for the Devanagari side. The current report atoutputs/limbu-page-source-inventory-2026-06-21now has 36 records, 3 real Devanagari page candidates, 3 Devanagari text-only sources, 1 synthetic Devanagari smoke pack, andready_for_real_devanagari_page_eval_draft=true. The curriculumMaths,Science, andSocialJSON files contain Devanagari Limbu line text and now resolve to mirrored CDC Limbu PDFs fetched fromlib.moecdc.gov.np: Maths ref 3757 SHA-25625fb14113adeb8ca474de70d26bd53455ae7b424de9e92eb43d9028b11e4d186, Science ref 3756 SHA-256c24178a1789cad5cd209f9dccc8de54c7aa35cf28d7e5966221ccdd2ff2ee97e, and Social ref 3758 SHA-256d32366b3a4f80b44817ba633cf25bc3395d3c3df7a418f7999c902d71e938502. With--require-real-devanagari-page, the tool exits 0. This is source readiness only, not claim-grade OCR evidence.- A real Devanagari-written Limbu CDC curriculum page draft now exists at
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21, built bytools/build_limbu_devanagari_curriculum_page_eval_draft.py. It renders real CDC PDF pages and uses curriculum JSON line text for Maths page 8, Science page 9, and Social page 9. The pack has 3 pages, 31 total text lines, pack content SHA-256d587e9ee5cc2983f761142c18de071c2578218d41341b5b2058409e1294ef333, manifest SHA-2562a708b3dbd444a05329343482452648e1cc3ecc34954860c95e2935737c10522, and script-inventory SHA-256ad1f497fdfc8bb172948db8644c967f55c0559f9a1140a7ee64a5a978d032462. The sidecar-perfect eval atscore-sidecar-perfectscores 3/3 pages with codepoint CER 0, grapheme CER 0, text exact accuracy 1.0, and reading-order pair accuracy 1.0, but the audit intentionally fails claim safety because line-bbox coverage is 0, layout references are absent, the pack is not frozen, and GT iscurriculum_text_unreviewed. This is the first real Devanagari page draft, not a claim-grade Devanagari eval. - The current Devanagari primary path was measured on that CDC draft with
parse-limbu-doc --engine tesseract(nep+eng, PSM 6). Outputs live atoutputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21/live-tesseract-documents; all threeaudit-limbu-outputchecks pass, so the bundles are internally auditable. The score atscore-live-tesseract-primaryis intentionally non-claim-safe but decisive: 3/3 pages scored, codepoint CER 0.897862, grapheme CER 1.390169, exact page accuracy 0, and all 58 detected lines entered review. Per-page codepoint CER is Maths 0.867220, Science 0.908257, Social 0.927536. A quick PSM/upscale sanity check did not produce a meaningful rescue; the best tested setting was still about 0.87 codepoint CER. Do not trust the currentnep+engTesseract primary as the Devanagari-written Limbu solution. - PDF-native line geometry is not enough to turn the CDC pages into line-box references:
PyMuPDF exposes only 2, 4, and 2 text lines on the selected pages while the
curriculum references have 14, 9, and 8 lines. A morphology line detector plus
Tesseract crop-recognition diagnostic now exists at
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-2026-06-21/image-line-box-documents-tesseractandscore-image-line-tesseract. It detected 83 line-like boxes, had 12 empty crop predictions, and scored codepoint CER 0.865400 / grapheme CER 0.937020. The paired comparison against full-page Tesseract atcompare-image-line-vs-fullpage-tesseractshows only a non-claim-safe 0.032462 absolute codepoint-CER improvement. Cropping helps slightly, but the Devanagari recognizer/data problem remains unsolved. - Correction to the previous bullet: plain PDF text extraction exposes too few lines,
but PDF word groups are usable as unreviewed machine-aligned line boxes for the three
CDC pages. The line-box pack at
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21has pack content SHA-256e696a92dee7ac49b4e81659aee7fb344af4c8fa0d4741c6cd43ef074cd052200andline_box_source=pdf_words_machine_aligned. Science drops two private-use marker PDF lines and Social drops two extra trailing PDF lines during alignment. The sidecar-perfect line-box score audits cleanly (audit_passed=True,claim_safe=False) with line detection F1 1.0 and matched-line CER 0. Full-page Tesseract on the same pack remains terrible: page codepoint CER 0.897862, grapheme CER 1.390169, line detection F1 0.385568, and matched-line codepoint CER 0.869087. The image-line Tesseract diagnostic finds 83 boxes with 12 empty crop predictions and scores page codepoint CER 0.865400, grapheme CER 0.937020, line detection F1 0.400301, and matched-line codepoint CER 0.872156. The paired text comparison again shows only a non-claim-safe 0.032462 absolute codepoint-CER improvement. Treat these CDC pages as a legacy visual-register OCR problem;nep+engTesseract is not a viable Devanagari-written Limbu primary. - A recognizer-level CDC Devanagari crop pack now exists at
outputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crops-2026-06-21, built bytools/build_limbu_devanagari_line_crop_recognizer_pack.py. It converts the line-box page references into 31 real image line crops with Unicode Devanagari labels and raw legacy PDF strings in row metadata. Pack content SHA-256 is6d76ab654a0c210a1291f397f2274878d116713a91010ccc91a62cf892ea09b0; manifest SHA-256 is97b2a148345ee8cd7a9f8322a16a8f3339776cbe1b1e8b75ceeed86932333d5f. The sidecar-perfect recognizer score has 31/31 matched lines, codepoint CER 0, grapheme CER 0, and exact line accuracy 1.0. The diagnosticnep+engTesseract crop score has 31/31 matched predictions, codepoint CER0.852632, grapheme CER0.857143, and exact line accuracy0.0. This isolates recognizer failure from page detection: even oracle/machine-aligned crops are not readable by generic Tesseract.tools/analyze_limbu_devanagari_legacy_target.pythen compared those same predictions against the raw legacy PDF strings stored in the row metadata. The analysis atlegacy-target-analysis-tesseract-nep-engreports Unicode codepoint CER0.852632, raw legacy codepoint CER0.880213, andlegacy_string_ocr_shortcut_supported=false. Do not lead with a "recognize legacy Latin then transduce" workaround; Tesseract is not even closer to the raw legacy string. The practical next step is a same-register visual recognizer/data path. - The same-register data path is now concrete.
tools/build_limbu_devanagari_cdc_line_crop_corpus.pynow builds the current training corpus atoutputs/limbu-recognizer-eval-packs/limbu-devanagari-cdc-line-crop-corpus-devclean-2026-06-21. All 281 local CDC text pages align with PDF word groups, yielding 6,446 raw line crops before leakage filtering. The earlier 501-row dev split leaked duplicate image/sample hashes against train, so do not tune onlimbu-devanagari-cdc-line-crop-corpus-2026-06-21. The dev-clean corpus first removes train/dev rows that overlap the held-out eval split by image hash, sample hash, or text hash, then removes dev rows that overlap train by the same keys. It leaves 5,551 train, 258 dev, and 31 sealed eval rows across 254/24/3 pages. Corpus pack SHA-256:36f10b8b123b792f643e206ca59e3bfd9ce03d5357b5de7ed1cdaba2b92145d1; eval split SHA-256 is unchanged:6d76ab654a0c210a1291f397f2274878d116713a91010ccc91a62cf892ea09b0.audit-train-dev-limbu-specificpasses with leakage checkpassand real train/dev rows;audit-train-dev-vs-heldout-evalalso passes for the combined train+dev manifest against the 31-row eval split. The held-out eval Tesseract baseline from the identical eval split remains codepoint CER0.852632, grapheme CER0.857143, exact line accuracy0.0, and the corpus eval legacy-target analysis again sayslegacy_string_ocr_shortcut_supported=false. - A portable PP-OCRv5-mobile Devanagari-written Limbu training bundle now exists at
runs/limbu-devanagari-cdc-ppocrv5-mobile-training-bundle-2026-06-21. It packages the dev-clean corpus train/dev manifests and images, the PP-OCRv5 mobile recipe, and a passing Limbu-specific corpus audit. Archive:limbu-devanagari-cdc-ppocrv5-mobile-devclean-training-bundle.tar.gz, SHA-25601b3689603309c13cb31ef31ff7dc0def65b556a92269d33483ee11da6e8c14d. Bundle manifest SHA-256:95322aa852b14662e22c53ddbc1ae8346f37a51a7d1ae8348ec4f7a5fbf8da9f. The archived YAML was verified to use portable paths undertraining-recipes/ppocrv5-mobile-devclean/andbundle_assets/images/; run command after extraction ispython PaddleOCR/tools/train.py -c training-recipes/ppocrv5-mobile-devclean/paddle_recognizer.yml. - A train/dev-only CPU smoke for that bundle now exists at
outputs/limbu-devanagari-cdc-ppocrv5-training-smoke-2026-06-21. It was prepared bytools/prepare_paddleocr_training_smoke.pywith 32 train rows and 8 dev rows,sealed_eval_used=false,epoch_num=1,batch_size=2,num_workers=0, and emptypretrained_modelto avoid a network/cache-dependent smoke. The PaddleOCR run completed on CPU under/tmp/ocrtech-paddle311/bin/pythonplus/tmp/PaddleOCR-v3.7.0, saw 16 train iterations and 4 validation iterations, and savedbest_accuracy,latest, anditer_epoch_1checkpoints.tools/audit_paddleocr_training_smoke.pyreportsaudit_passed=true,issues=0, andsealed_eval_used=false; smoke summary SHA-256 is998ef85358e1858c22bac5966ad00290c2e1809b5d1497d61bea38b94ebc73e7, smoke audit SHA-256 is811950bacfb19bcc936fc84ed89cb617a6732aaa6ef6529cef19acfac37284a6, and best checkpoint SHA-256 is3b41eab11423fa1ac3d53bcffd1c8108f211818a9adb7257bf5de25745115535. This proves the bundle can train from train/dev without touching the sealed eval split; it is not quality evidence and should not be admitted. - The eval recipe registry now has a sealed held-out admission slot for this branch:
limbu-devanagari-cdc-ppocrv5-devclean-heldoutinconfigs/eval-harness-recipes-v1.json. It points at the 31-row eval split from the dev-clean corpus, usestrain-manifest.jsonl(train+dev) only as the leakage source, and validates structurally with zero issues. It is intentionally not claim-ready: prediction command and checkpoint are unresolved, GT iscurriculum_text_unreviewed, the pack is not frozen, and row-level claim eligibility is false. After training, fill in the checkpoint/model-card/prediction command and runtools/run_eval_recipe.pyagainst this recipe; do not tune on this eval split. - The Devanagari branch now has an executable training/admission guardrail:
configs/limbu-devanagari-ppocrv5-admission-plan-v1.json, validated bytools/validate_recognizer_training_admission_plan.py. The current validation output atoutputs/limbu-devanagari-ppocrv5-admission-plan-validation-2026-06-21reportsplans=1 valid=True training_ready=True heldout_admission_ready=0, JSON SHA-2566ecc4001c7a278aeaa1a3bf6cb3415fbc5c57b66a897ae29b38a37b0c7d1aa19. It checks the training bundle archive and manifest hashes, exact train/dev/eval counts, no train/dev/eval overlap by sample/image/text hashes, train+dev leakage-source identity, and that checkpoint selection is restricted to train/dev while eval is explicitly forbidden. Run this validator before any GPU training run and again before filling in the held-out admission recipe. - A concrete full-training launcher for that guardrail now exists at
outputs/limbu-devanagari-cdc-ppocrv5-training-job-2026-06-21, prepared bytools/prepare_recognizer_training_job.py. It writesrecognizer-training-job.json(SHA-256b241f1f0bb7a9011429598fd3c92e5aaffced5a1af6bcb796a81bf5927b611d8) and executablerun-training.sh(SHA-25606a2787a34a1da197c78242c4d05c9b318322789de453af08c54f3b03be56831). The launcher extracts the portable bundle intoworkspace/and runs PaddleOCR with/tmp/ocrtech-paddle311/bin/pythonplus/tmp/PaddleOCR-v3.7.0. The job manifest recordstraining_ready=true,heldout_admission_ready=false, 5,551 train rows, 258 dev rows, and the 31-line sealed eval split withused_for_training=falseandarchive_member_hits=[]. One warning is expected: the training bundle manifest records the passing corpus audit, but the audit JSON is not itself packaged inside the tarball. This launcher is not a trained checkpoint and is not admission evidence. - The full-training launcher now has a runtime pre-training preflight:
tools/preflight_recognizer_training_job.py. The real run atoutputs/limbu-devanagari-cdc-ppocrv5-training-runtime-preflight-2026-06-21extracted the bundle into an isolated audit workspace, parsed the packaged PaddleOCR YAML, verified the dictionary and checkpoint directory path, checked all train/dev label rows against extracted crop files, rechecked sealed eval hashes and archive exclusion, and imported Paddle/PaddleOCR with the recorded runtime. It reportsready_to_start_training=true,issues=0,warnings=0, train rows5551, dev rows258, missing images0, malformed label rows0, Python3.11.13, Paddle3.3.1, and PaddleOCR3.7.0. JSON SHA-256:417b8fcb177edd038fe6739a50311656ef2e2930ef8b0ec910361b8b75594289; Markdown SHA-256:fdd24527d29f5576709427622f3c8b092461bb7ce2c77cc9533e971889392686. This is a launch-readiness gate only; it does not replace the post-training completion audit or held-out admission. - A current-host Devanagari training launch is now running. The launcher was prepared
by
tools/prepare_recognizer_training_runtime_launch.pyatoutputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21. Because this machine is CPU-only (compiled_with_cuda=false), it records explicit overrides:Global.use_gpu=False,Global.save_model_dir=training-recipes/ppocrv5-mobile-devclean/checkpoints-current-host-cpu,Train.loader.num_workers=0, andEval.loader.num_workers=0. Launch manifest SHA-256:c56e9211058ff0df87e595fd926b607057339bd96b466468779c13c3eb90d27b; launcher SHA-256:1c58b3815ae1fc3e66026c7c51d6c81948749243d93a4d64afa3607dec77e4fd. It is running in tmux sessionlimbu_devanagari_ppocrv5_training_20260621; verified progress reachedepoch: [11/20], global_step: 7000with CPU ETA around7:33:52. Usetmux attach -t limbu_devanagari_ppocrv5_training_20260621to inspect it, or tailoutputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21/training.log. This is still not a completed checkpoint or admission result. - A structured training status monitor now exists:
tools/monitor_recognizer_training_run.py. The current snapshot atoutputs/limbu-devanagari-cdc-ppocrv5-training-status-2026-06-21reportsstatus=running,ready_for_completion_audit=false,issues=0,warnings=0, latest progressepoch: [11/20], global_step: 7000,acc=0.375,norm_edit_dis=0.778184,ips=2.44833, ETA7:33:52, latest evalacc=0.20542634862688572,norm_edit_dis=0.6081954332798136, best evalacc=0.20542634862688572,norm_edit_dis=0.6081954332798136at epoch 11, andnext_action=continue_monitoring_training. The monitor now records training-log age against a 900-second stale threshold; this snapshot haswarnings=0. Training is still active, so do not run completion/export/admission yet. JSON SHA-256:d0c2a2bebe9a6a9d1bd5938092247e9ff968714b8708d16d49735943930f4a50; Markdown SHA-256:9d991c83d98ad85934e05954a88d9761cfd6773c9c232b343456410f3ce20b8c. Re-run it withuv run python tools/monitor_recognizer_training_run.py --launch-manifest outputs/limbu-devanagari-cdc-ppocrv5-training-launch-current-host-2026-06-21/recognizer-training-runtime-launch.json --root /Users/cdjk/github/llm/ocr-tech --out outputs/limbu-devanagari-cdc-ppocrv5-training-status-2026-06-21 --parse-last-lines 4000. - The post-training completion gate is now explicit:
tools/audit_recognizer_training_completion.pyaudits the completed launcher workspace, checks the selectedbest_accuracycheckpoint, and only then writes the export/package/sealed-eval commands plus a resolvedrunner.type=model_configheld-out recipe. The current real audit atoutputs/limbu-devanagari-cdc-ppocrv5-training-completion-audit-2026-06-21is red:ready_for_export=false,issues=5, JSON SHA-2563fe15749a5c2a83ce9a764c8cc98f9cfe8b107b33d7eb2be728cf831346ef516. The issues are expected before the long training run: no extracted/trained workspace, no config in workspace, no checkpoint, and no dictionary path from config. After runningoutputs/limbu-devanagari-cdc-ppocrv5-training-job-2026-06-21/run-training.sh, run this audit again before exporting, packaging, or touching the sealed eval recipe. - A generic Devanagari
deva-v2checkpoint-export/package/eval record now exists atoutputs/limbu-devanagari-primary-candidate-deva-v2-export-preflight-2026-06-21. It was generated fromoutputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz(archive SHA-2567e81660674468f99b2940224fe5ef3e7c9d062d6a05fd3cf4dabf52ac9b4e1df), with extracted checkpoint SHA-256ddea1db34e89e2c1cdde53d5364eb9156b26dbe8e0a5e67db4c26dd9c97e6676and corrected export recipe SHA-2566e18a8cfcf8ebab247be910f9566b30327f10dcd3aef2270ec3979960f7b86c3./tmp/PaddleOCR-v3.7.0was restored at commitb03f46425e8ff4442b268ce449e3eef758146cd4; export required installing the source tree requirements into/tmp/ocrtech-paddle311and overridingGlobal.character_dict_pathplusGlobal.use_gpu=false. The packaged candidate model-card SHA-256 is0c41a9811484349805cd608410da3626b795c1601a7ba014c0e23bca69c0efd0and model-card audit passes. It predicts the 31 CDC held-out rows with 0 runtime failures but scores codepoint CER0.835628, grapheme CER0.858748, exact-line accuracy0.0, andclaim_safe=false. The score bundle audit passes with zero issues andclaim_safe=falseatoutputs/limbu-devanagari-primary-candidate-deva-v2-export-preflight-2026-06-21/score-cdc-heldout-audit(SHA-2568a538059a0320902196b3a1125edd97b34f21cb9afb4f32826f24c4c08135edf). This is a failed generic baseline, not a Devanagari primary. The same-register CDC PP-OCRv5 training bundle remains the next serious Devanagari path. - Sirijonga page materialization now has an explicit reading-order switch.
tools/materialize_page_image_line_boxes.pyaccepts--reading-order top_to_bottom|column_major; default remains the historical top-to-bottom row-major ordering, whilecolumn_majororders detected line boxes down each detected column before moving right. Unit coverage exists intests/test_materialize_page_image_line_boxes.py::test_detect_line_boxes_can_order_two_column_pages_column_major.tools/compare_limbu_sirijonga_detector_reading_order.pynow compares these modes against reference line boxes without invoking a recognizer. On the current real Sirijonga drafts it is neutral, not a win: Gorkhapatra tuned morphology stays at matched-line reading-order pair accuracy0.606543for both modes, and Sikkim stays at0.992402for both modes. Do not promotecolumn_majoras a default; use the comparison tool to test future detector variants. - Sirijonga detector tuning now has a parameter-sweep tool:
tools/sweep_limbu_sirijonga_detector_params.py, covered bytests/test_sweep_limbu_sirijonga_detector_params.py. It ranks morphology configs without OCR recognition. On Gorkhapatra, the sweepoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/detector-param-sweep-v1evaluated 1,680 configs and improved detector-only line F1 from0.833866to0.844660by usinghorizontal_kernel=23,vertical_kernel=3,max_height=80,min_area=50,bbox_source=ink,threshold=otsu, andtop_to_bottom. It still matches only261/303lines and leaves matched-line reading-order accuracy at0.606543. Because the 315 candidate boxes are a strict subset of the previous 323-line tuned materialization,tools/filter_page_image_line_box_documents.pyreused existing OCR outputs with--max-height 80and scored them atscore-image-line-detector-v4-tuned-maxheight80-filtered; codepoint CER stayed0.763835and grapheme CER moved slightly worse from0.828226to0.828392. Treat this as a geometry cleanup, not a text-quality win. tools/analyze_page_line_failures.pynow joinsline-alignment.csvwith an image-line crop manifest and buckets matched lines by geometry/text quality. On the Gorkhapatra tuned image-line score, the diagnostic outputoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-image-line-detector-v4-tunedreports138/261matched lines as high-IoU (>=0.85) but bad-text, compared with only11low-geometry bad-text matches. The high-IoU bad-text bucket has mean IoU0.893282, mean recognizer confidence0.445911, mean codepoint reference length45.17, mean crop width330.88px, mean crop aspect ratio6.73, and mean grapheme CER0.678634. Width bands confirm the recognizer-width signal: <=256 px matched crops have codepoint CER0.132605and confidence0.805627, while 769+ px crops have codepoint CER0.412653and confidence0.219722.tools/prepare_limbu_sirijonga_width_ablation.pynow prepares the next real Sirijonga measurement instead of another detector tweak. The Gorkhapatra ablation pack isoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/recognizer-width-ablation-v1: 233 high-IoU matched crops, band counts{"000-256": 20, "257-384": 202, "385-512": 2, "513-768": 4, "769-plus": 5}, and model-card variants for[3,48,320],[3,48,480],[3,48,640],[3,48,960], and[3,48,1280]./tmp/ocrtech-paddle311/bin/pythonhas been restored withpaddle3.3.1 andpaddleocr3.7.0, andrun-width-ablation.shhas scored all variants with zero prediction failures. Width 480 is best on this non-claim diagnostic pack: codepoint CER0.100651, grapheme CER0.169567, exact line accuracy0.278970. Width 320 scores codepoint CER0.269910; width 640 is close but worse at0.108741; width 960 is0.212755; width 1280 collapses to0.442740. The width-480 page materialization atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/image-line-box-documents-v4-tuned-width480proves the setting helps line recognition but is not enough for page OCR: matched-line codepoint CER improves from0.285785to0.105977, matched-line grapheme CER improves from0.551916to0.178678, but full-page codepoint CER regresses slightly from0.763835to0.770900and matched-line reading-order accuracy stays0.606543. A layout-specificwide_column_right_to_leftordering diagnostic improves Gorkhapatra matched-line reading-order pair accuracy from0.606543to0.763985with unchanged line F10.833866, but it regresses the Sikkim draft from0.992402to0.753361, so it is a selectable layout mode, not a global Sirijonga default. Gorkhapatra width-480 page materialization with that order scores full-page codepoint CER0.761228and grapheme CER0.803876, a small one-page improvement over width-480 top-to-bottom (0.770900/0.804207) while matched-line codepoint CER remains0.105977.tools/filter_page_ocr_documents.pynow adds non-oracle live-line pruning by empty text, confidence, script, and geometry.tools/sweep_page_ocr_line_pruning.pynow wraps that filter with page eval and paired comparison, so threshold selection is auditable. The latest pass addedtools/audit_page_ocr_line_filter.py; it replays filtered documents from the source OCR documents and checks line reports, source hashes, and kept/removed counts. The pruning sweep now runs this audit for each threshold candidate. On Gorkhapatra width-480 wide-RTL output, the automated sweep atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/ocr-line-pruning-sweep-width480-wide-rtl-v1improves page codepoint CER through threshold0.85: unpruned0.761228, 0.500.746678, 0.600.744996, 0.700.739697, 0.800.738436, 0.850.730362; 0.90 over-prunes and regresses to0.760387. The 0.85 candidate keeps 199/323 lines, removes 124, replay-audits cleanly, and is still one-page non-claim evidence. Sikkim is the counterexample: its automated sweep atoutputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/ocr-line-pruning-sweep-image-line-v4-v1shows 0.85 pruning hurts page codepoint CER from0.460255to0.694308, while mild 0.70 pruning improves it to0.444063with 107/125 lines kept and replay-audits cleanly.tools/admit_page_ocr_line_pruning_threshold.pynow combines these sweeps, admits only thresholds that are common across pages, audit-clean, and non-regressing under both codepoint and grapheme CER, and records the exact per-page documents, score, comparison, and audit hashes for the admitted threshold. The real run atoutputs/limbu-sirijonga-pruning-threshold-admission-2026-06-21selectsconf070: Gorkhapatra improves codepoint CER by0.021531, Sikkim improves by0.016192, andconf085is rejected because it regresses Sikkim by-0.234053codepoint CER. JSON SHA-256:cc8b0b8986c273bafbc73fc74ffc706dd28eeb93343c921aa4572411aae06429; Markdown SHA-256:9e8672e71d58ae501ddd7024acf34a074ead09b12627cf81f1bff6e3b398b6e5.tools/audit_admitted_page_ocr_candidate.pynow audits residual failures for that admitted candidate. The realconf070residual audit atoutputs/limbu-sirijonga-conf070-residual-audit-2026-06-21reports 296 matched lines, 93 unmatched references, 51 unmatched predictions, 13 near-miss unmatched references at IoU >= 0.30, 0 high-IoU unmatched references at IoU >= 0.50, and 26 matched high-text-error rows. Gorkhapatra is now mostly recall-limited: 238 matched, 65 missed references, 2 prediction-only lines, precision0.991667. Sikkim remains both recall- and false-line-limited: 58 matched, 28 missed references, 49 prediction-only lines, precision0.542056. The refreshed audit also replays the admitted per-page score/comparison/filter artifact hashes and reportsartifact_integrity_passed=true. JSON SHA-256:28b0edcb15eaf4a3860e2560872ecd419ad5226f88964bf04feaa58104149f64; Markdown SHA-256:178ce581a9829294da482916880ce9a19df0e103a4f272a4b7b7e11ab44e2516. The refreshed audit also buckets residual geometry: Gorkhapatra misses are mostly short references (51/65) centered across middle/bottom body regions, while Sikkim has medium-width missed references (22/28, bottom-heavy) and many short prediction-only fragments (47/49). Next step: keep width 480 for Sirijonga line recognition, useconf070as the current cross-page operational pruning candidate, then handle Gorkhapatra missed-line recall and Sikkim false-line/layout classification separately. Do not choose 960/1280, split lines, or hard-code a confidence threshold before addressing the page assembly failure. Latest follow-up:tools/filter_page_ocr_documents.py,tools/audit_page_ocr_line_filter.py, andtools/sweep_page_ocr_line_pruning.pynow support page-relative width/height ratio filters. The realmin_width_ratio=0.05sweep pair lives under each Sirijonga draft page directory asocr-line-pruning-sweep-minwidth005-v1. Cross-page admission atoutputs/limbu-sirijonga-pruning-threshold-admission-minwidth005-2026-06-21selectsconf070, improving Gorkhapatra codepoint CER to0.739361and Sikkim to0.423454; JSON SHA-256808a2e08fd853fbc1c612b398e37fe4903808a141a642b82d3338a3f64e632e4. Residual audit atoutputs/limbu-sirijonga-conf070-minwidth005-residual-audit-2026-06-21shows the tradeoff: prediction-only rows fall from 51 to 28 and Sikkim precision rises to0.682927, but missed references rise from 93 to 97 and Sikkim recall drops to0.651163; JSON SHA-256969868c2eac5decc1a2a4a82848f0c051adbd04c5092d7d010b31698b818e2f5. Treat it as precision-oriented review triage, not complete page OCR. Operational replay of that admitted recipe now exists atoutputs/limbu-operational-image-line-current-2026-06-21/gorkhapatra-width480-minwidth005-conf070: 323 detected image lines, 238 kept, 85 removed, and 39 review rows. The capture/Sirijonga audit passes, but the strict final/no-dropped-lines audit fails with39unresolved review rows and85dropped rows, which is the correct behavior for a triage output. Exact page-eval scoring of this operational output is inoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/score-operational-image-line-width480-minwidth005-conf070: codepoint CER0.7393608074011775, grapheme CER0.7682623819778035, metrics SHA-25614d0da1bf80e5afb876adfa6191501f2f18039dc86014123d7cf0d64551c4176, audit SHA-25637c6d0e0efb74d3c61b7a5bd9f3ce5bf4caf0487d47d96ba1f42e7aabfb8b6a9. The eval audit now passes structurally but remains non-claim: the pack is unfrozen, one page, machine-converted/unreviewed, and leakage checking againstrecognizer-width-ablation-v1/manifest.jsonlrecords 233 checked rows withsource_sample_idoverlap ongorkhapatra-limbu-sirijonga-0001-01.tools/analyze_page_line_failures.pynow also reports unmatched-reference and prediction-only geometry plus best unmatched near-miss pairs. It also reports attributed codepoint error mass by source and now joins operational image-line crop manifests that have line ids but nosample_id. The current operational Gorkhapatra width-480 unpruned analysis atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-operational-width480-unpruned-v4has 261 matched rows, 42 unmatched references, 62 unmatched predictions, and only 15 high-IoU bad-text matches. Only 5/42 unmatched references have any unmatched prediction at IoU >= 0.30, and 0/42 have one at IoU >= 0.50. The dominant error source is stillmissed_reference_textat 1,416/2,919 attributed codepoints (0.485). The v4 overlap diagnostic checks each unmatched reference against every predicted box, including already matched predictions: 28/42 have no overlapping prediction at all, 14/42 best-overlap a prediction-only row, 5/42 reach IoU >= 0.30, and 0/42 reach IoU >= 0.50. This rules out matched-line swallowing as the primary cause. The same center-band pattern is present without pruning: 31/106 center references are unmatched with 1,116 missed-reference codepoints, while left is 3/97 and right is 8/100; top combined bands are middle/center (20/42, error mass 810), top/center (6/19, error mass 233), and middle/right (5/38, error mass 163). Result SHA-256 isf540fe59bf23b4f9b81c67b8e21f6827a563336d999781132ca4a2b23732c4a0; report SHA-256:a51ddce081e8d0989ce13bbaa90a575aa4887a5e69a2b70c0745f8d7d0c7b467. The refreshedminwidth005-conf070analysis atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/line-failure-analysis-operational-width480-minwidth005-conf070-v4has 236 matched rows, 67 unmatched references, 2 unmatched predictions, and only 2 high-IoU bad-text matches; the dominant codepoint error source ismissed_reference_textat 2,422/2,888 attributed codepoints (0.839). Its overlap diagnostic is even stricter: 65/67 unmatched references have no overlapping prediction at all, and only 2/67 reach IoU >= 0.30. The analyzer also reports missed-reference spatial bands: the center horizontal band has 42/106 references unmatched and 1,757 missed-reference codepoints, versus left 9/97 and right 16/100; the top combined miss bands are middle/center (20/42, error mass 810), bottom/center (14/45, error mass 664), and top/center (8/19, error mass 283). Result SHA-256:1da537b596b8c6e2b010998fc183289a8e632c75237d509c783b3977ba9e1c65; report SHA-256:d9fef331d97b94877f39b02b8ddf29445d97e96d44cfcaf16e93f6b15ba993ba. Lowering matching thresholds or splitting matched boxes is not the main rescue. A direct pixel check of the top unmatched center-column references showed the text is visible in the prepared image; the failed rows sit inside a rejected Otsu/morphology connected component at[995,401,326,1404], so the failure is over-tall component rejection, not missing ink.- Adaptive thresholding was tested as a bounded detector-only branch and rejected for
now. The best small probe,
adaptive_mean:41:11with the operational 23x3 morphology settings, raised matches from 261 to 263 but produced 320 predictions and F10.844302; as a rescue pass on the previousmax_height=80tuned route it still lowered F1 below the no-rescue0.844660baseline. Do not wire adaptive thresholding as a default without stronger cross-page evidence. - The useful Gorkhapatra recall fix is tall-component splitting.
src/ocrtech/image_lines.py,tools/materialize_page_image_line_boxes.py, andtools/compare_limbu_sirijonga_detector_reading_order.pynow expose opt-insplit_tall_components/--split-tall-componentswith row-projection settingssplit_tall_row_min_ink=2andsplit_tall_max_row_gap=4; split rows carrydetector_source_pass=primary-split-tall. Detector-only comparison atoutputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/compare-detector-split-tall-maxh80-v1uses Otsu, ink boxes, 23x3 morphology,max_height=80, detector padding 2, and tall splitting. It reaches 285/303 matches, 339 predictions, precision0.840708, recall0.940594, F10.887850, mean IoU0.869405, and wide-RTL matched reading-order pair accuracy0.712824. Result SHA-256:06c242545dce5462dcfb25773b71045c1c3cde026843751d62d2c3a7f44a07c3; report SHA-256:f9dd7e679f733937a5b793e45e38eb02394c8b3fb80fc3f8eb039b3ac770e82c; alignment TSV SHA-256:44feb8ba76840f85b008c0a1c773b3a033d1b11637e3f002ceebcdab23ae7414. Full OCR materialization has now been run and rejects split-tall as an admitted page route by itself.image-line-box-documents-v4-split-tall-width480-wide-rtlwas materialized with the v4 width-480 line recognizer,recognition_mode=line, 339 lines, and 0 empty predictions. Run SHA-256:80808cefbc3804b0385a6c2a2fe16522ebe1a7c307b9447210006905d1712c96; crop manifest SHA-256:7e6b62b25ef17687897034ed70eb1bf09c3d50a407d8a786707825cbd4a99cc5; document SHA-256:da1a1664aee6beb1381ed338f884033c44e5c3a17d9bc9b610f86f778b97563b. Scoring atscore-image-line-detector-v4-split-tall-width480-wide-rtlgives page codepoint CER0.791421and grapheme CER0.831042, worse than the non-split wide-RTL score0.761228/0.803876; metrics SHA-256:db52ef81012890dbe77179b9deadb72adb04f90c9228cffd2b68860ae5902b18, run SHA-256:33977edad6ff145a24891ecfda9b990f0895074c5043ef902a4ba46f92c8e990. The paired comparison atcompare-split-tall-width480-wide-rtl-vs-tuned-width480-wide-rtlreports negative absolute improvement: codepoint-0.030193, grapheme-0.027166; comparison SHA-25686906240daae771ba960cbd725b19210bde3eb08d4a2d3c4d95e37ca9610835e. The failure shape changed in the right direction but exposes the next bottleneck: line-failure analysis atline-failure-analysis-split-tall-width480-wide-rtl-v1shows 285 matches, 18 unmatched references, 54 unmatched predictions, andmissed_reference_texterror mass falls to 425, but page assembly is now dominated by matched bad text/extra prediction text and worse reading order (0.712824vs prior wide-RTL0.763985). Analysis SHA-256:b3d8266031538bce0c63f38f6db4e2f485ac2e18fe26f9fcda4bc3408f05524d; report SHA-256eef5b226a0f60f203501a6c0c2ab48166fd5e159bf91f559df70a4f5c79a64e0. Pruning without fixing order does not rescue split-tall enough: the split-tall confidence sweep atocr-line-pruning-sweep-split-tall-width480-wide-rtl-v1has best threshold0.85with codepoint CER0.740875and grapheme CER0.768759(sweep SHA-256085765fe1264ec1b45e879cf2677539eb1436f208a5a2e6a01e9b88cea1d1e5f), still worse than the older non-split Gorkhapatra-only conf085 result0.730362/0.763956. The clean comparison atcompare-split-tall-conf085-vs-tuned-width480-wide-rtl-conf085gives codepoint improvement-0.010513and grapheme-0.004804(comparison SHA-2564cc4b640ca0ab5e1c6beaac2badeae0f1acf1378674b3c4ccc9a8d67eda92068). A source-pass-aware ordering mode was added:wide_column_right_to_left_split_tall_last. It keeps the existing wide-RTL order for primary boxes and places*-split-tallboxes after primary boxes. Detector-only comparison atcompare-detector-split-tall-wide-rtl-last-v1improves split-tall matched-line reading-order accuracy from0.712824to0.765654with the same 285/303 matches and F10.887850; comparison SHA-256:cce99e017e0e3565c50e56d8886588910d567e94685d8f302e703f155347c636. Full OCR atimage-line-box-documents-v4-split-tall-width480-wide-rtl-lastscores codepoint CER0.778638and grapheme CER0.820938, better than plain split-tall but still worse than non-split wide-RTL before pruning. With the same non-oracle conf sweep,ocr-line-pruning-sweep-split-tall-wide-rtl-last-v1reaches best conf085 codepoint CER0.730614and grapheme CER0.763624(sweep SHA-256869381446576317d8258f36557bec4e7da0171e3f613ada59f4c8e44a9b3ca6f). Against the previous non-split conf085 Gorkhapatra result, the clean comparison atcompare-split-tall-wide-rtl-last-conf085-vs-tuned-width480-wide-rtl-conf085is a near-tie: codepoint-0.000252, grapheme+0.000331; comparison SHA-256cfbb25555feab3af1d4863f4f654efd11b94bb18dde047b0b9d2890a330a71f9. Sikkim cross-page detector validation now rejects global admission. Without splitting,compare-detector-split-last-no-split-v1keeps 59/86 Sikkim matches and F10.559242, but drops matched-line reading-order accuracy from0.992402to0.753361(comparison JSON SHA-256014bdc6c9fd08d7f82b8292cdcfe0e0a6a08dd2a2a6a6e3a79024e51d7acc813). With--split-tall-components,compare-detector-split-tall-last-v1adds one prediction without adding matches, lowering F1 to0.556604and preserving the same ordering regression to0.753361(comparison JSON SHA-25677d2bd30cfc2ed0a2827704c0bf21b51ec3698af3aedb1cc3459382530a9b6fc). Full OCR materialization was intentionally skipped because the detector-only Sikkim gate already fails. Keepwide_column_right_to_left_split_tall_lastas a Gorkhapatra/layout-specific diagnostic, not a global Sirijonga route. - A safer gated branch now exists:
auto_layout_split_tall_last. It detects once without tall splitting, uses the existing auto-layout features, and only reruns tall splitting plus split-last ordering if the page qualifies as a wide-column layout. Detector-only Gorkhapatra output atcompare-detector-auto-layout-split-tall-last-v1keeps 285/303 matches, F10.887850, and raises matched-line reading-order accuracy from0.712824to0.765654(comparison JSON SHA-2566364dd87e617b3b76236ea37d71b08fab216d5c5ffb8a3b08d6b03560581237b). Sikkim output at its matchingcompare-detector-auto-layout-split-tall-last-v1selects top-to-bottom, keeps 59/86 matches, avoids the extra split false-positive, improves F1 back to0.559242, and preserves matched-line reading-order accuracy at0.992402(comparison JSON SHA-2565aa325d176af4a85dda65a3e3684ae9c80bc7c9a7485e0cb7e0729f8fddf20e7). Full OCR was also materialized and scored. Gorkhapatraimage-line-box-documents-v4-auto-split-tall-width480emits 339 lines, selectswide_column_right_to_left_split_tall_last, scores codepoint CER0.778638and grapheme CER0.820938, and exactly matches the explicit split-last score (compare-auto-split-tall-vs-split-tall-wide-rtl-lastJSON SHA-2569326ef4b2e33964d985c734b34ba824977658887ee3ee6f61f7ab8cb724c3b61). Sikkimimage-line-box-documents-v4-auto-split-tallemits 125 primary lines, selectstop_to_bottom, scores codepoint CER0.460255and grapheme CER0.576710, and exactly matches the existing image-line baseline (compare-auto-split-tall-vs-image-line-v4JSON SHA-25650b845f80d1212eb2c751ba43070880602c70c506f6682e9f8e1ca0266802e7e). Do not admit this as final OCR policy yet: it is a safer candidate branch, but unpruned Gorkhapatra page CER remains worse than the non-split wide-RTL route, and all current page evidence is still draft, one-page, and not claim safe. - The gated branch has now also been tested through the pruning/admission layer.
Gorkhapatra
ocr-line-pruning-sweep-auto-split-tall-minwidth005-v1has a local best at conf085, codepoint CER0.730614/ grapheme CER0.763624, but Sikkim conf085 regresses to codepoint CER0.702650/ grapheme CER0.731978. Cross-page admission atoutputs/limbu-sirijonga-auto-split-tall-pruning-threshold-admission-2026-06-21selects conf070 instead; admission JSON SHA-2562bc16296a6295a03482ef30a01729c5ae7b37278c6623ca38d6431d86a3665ed. The selected gated conf070 scores Gorkhapatra codepoint CER0.753406/ grapheme CER0.782508and Sikkim codepoint CER0.423454/ grapheme CER0.511091. Direct paired comparison against the already-admitted non-split minwidth005 conf070 route rejects promotion: Sikkim is tied, but Gorkhapatra regresses by0.014045codepoint CER and0.014245grapheme CER. Comparison JSON SHA-256:f26b0911a4ba92bce47ea53bee0af6f88a5ef93f61f6361586daacd74d814b70. Keep auto-split-tall as a recall diagnostic/alternative branch, not the admitted operational route. - Same-row component merge is now implemented as an opt-in detector stage in
src/ocrtech/image_lines.py,src/ocrtech/pipeline.py,src/ocrtech/cli.py,tools/materialize_page_image_line_boxes.py, andtools/compare_limbu_sirijonga_detector_reading_order.py. Defaults are disabled; the tested candidate usesy_tolerance=8,max_gap=120,max_center_delta=300,max_width=420. Detector-only F1 improves Sikkim to0.875000(90preds,77/86matches; JSON SHA-256379960a430a943e6076499eff5a744e084f736134a2e0579ad900e97e9dd7aa8) and Gorkhapatra to0.875000(289preds,259/303matches; current-schema JSON SHA-2561b2fdcdb08375ac5896487b66711787e5c59a7566dc88b60fe6e396ca2903421). Full OCR plus the samedrop_empty/Sirijonga/min_width_ratio=0.05sweep selectsconf070atoutputs/limbu-sirijonga-same-row-merge-pruning-threshold-admission-2026-06-21(JSON SHA-256970a1ad210423d430687d1666f190aa1c4795f516e9998bc4c544a086e45d32f). Do not globally promote it: selected row-mergeconf070improves Sikkim against the admitted minwidth005conf070route (0.412169vs0.423454codepoint CER), but regresses Gorkhapatra (0.744239vs0.739361). Keep it as a conditional fragmented top-to-bottom page repair candidate, not a wide-column operational default. - The conditional same-row gate has now been implemented and measured as
merge_same_row_auto_fragmented_top_to_bottom. It resolves layout on unmerged boxes, only applies to pages selected astop_to_bottom, and requires at least 8 lines and 18% line-count reduction from the proposed merge. Sikkim auto-gated detector output stays at90predictions /77matches / F10.875000with reading-order accuracy0.990089(JSON SHA-256f5a50caaac7ee1eaea2bac4b39917cdb04e5ce8c1c622b620e9dad63e8e5b1f1); Gorkhapatra auto-gated output stays on the non-merge wide-column path with323predictions /261matches / F10.833866(JSON SHA-256da5d57bb77fd1eb2b1f29a69f8b491623631d24e9a8dfccd06a1e7491cbd0d21). Cross-page pruning admission atoutputs/limbu-sirijonga-auto-same-row-merge-pruning-threshold-admission-2026-06-21selectsconf070(JSON SHA-2562784674a4b8ff7382086a9d1c544c25729df5f938a792b21b67d81298c9bfeed). Direct comparison against the previous admitted minwidth005conf070route improves Sikkim (0.412169vs0.423454codepoint CER) and ties Gorkhapatra (0.739361both). Treat this as the current stronger two-page operational candidate, but not a claim-safe final policy. - Operational
parse-limbu-doc --line-detection-mode image-linenow accepts repeatable--image-line-rescue-detector-pass key=value,...plus--image-line-merge-iou-thresholdand the disabled-by-default--image-line-merge-same-row-*geometry gates, carriesdetector_source_passthrough line metadata and crop manifests, records detected/kept/removed source-pass counts inimage-line-ocr-run.json, andaudit-limbu-outputreplays that field and the count summaries againstdocument.jsonand the crop manifest. Regression coverage:tests/test_image_lines.py::test_detect_line_boxes_with_rescue_pass_adds_non_duplicate_components,tests/test_cli.py::test_cli_parse_limbu_doc_passes_post_correction_profile, and existing image-line output audit tests. tools/materialize_page_image_line_boxes.pyandtools/compare_limbu_sirijonga_detector_reading_order.pynow support auditable repeatable--rescue-detector-passmorphology passes. The pass is merged with the primary detector by IoU, recordsdetector_source_passon lines/crop rows, and is exposed to detector-only comparison before spending recognizer time. On the current Gorkhapatra page, the measured rescue variants are rejected. The broadh15,v1,min_width=20pass raises matches from 261 to 281 and recall to 0.927393, but predictions jump to 702 and F1 collapses to 0.559204. The best conservative variant tested,h41,v1,min_width=250,min_area=250,max_height=80,merge_iou=0.30, gives only 264 matches and recall 0.871287 while lowering precision to 0.774194, F1 to 0.819876, and reading-order accuracy to 0.756596; comparison SHA-256088fdca2fe6865f66998bdd10a41908240232a271ce5567a62a4528e7c02c431. Do not promote multi-pass morphology without a better precision gate.- The rescue detector now also supports
min_aspect_ratioandmax_aspect_ratiogates, but the first aspect-gated Gorkhapatra pass is still rejected. The best tested aspect-gated variant,h41,v1,min_width=250,min_aspect_ratio=8,merge_iou=0.30, has 341 predictions, 264 matches, precision 0.774194, recall 0.871287, F1 0.819876, and reading-order accuracy 0.756596, below the no-rescue baseline; comparison SHA-25675934afe974a8189b97b18f61c82dde1d5d87bdbf9a540c93c6ae4aac0f75156. - A geometry-only Sirijanga line-box review packet now exists at
outputs/limbu-sirijonga-line-box-review-packet-2026-06-21. It was built bytools/prepare_line_box_review_packet.pyfrom the current Gorkhapatra and Sikkim page eval manifests and audited bytools/audit_line_box_review_packet.py. It covers 2 real draft pages and 389 seeded line boxes, withline-box-review-audit.jsonreportingaudit_passed=trueand 0 issues. The packet deliberately excludes reference transcription: use it for non-native geometry and reading-order review only.tools/apply_line_box_review_packet.pynow ingests completed geometry reviews into detector/layout references with empty text andclaim_evidence_eligible=false; it blocks pending rows by default. Running it on the current packet correctly blocks because all 389 rows are stillpending. Passing this audit does not make the machine-converted Sirijonga text claim-grade. - The Sikkim detector sweep
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/detector-param-sweep-v1evaluated 900 configs and did not beat the current image-line detector family: best F1 remains0.559242, with125predictions for86references and59matches. Do not spend tomorrow treating a simple morphology parameter tweak orcolumn_majoras the Sikkim fix; the Sikkim path needs better segmentation logic, same-register recognition data, and reference verification. - The previously referenced
/tmp/ocrtech-paddle311/bin/pythonruntime is absent in this shell. A refreshed full-router preflight atoutputs/limbu-first-router-v1/runtime-preflight-2026-06-21-currentfails under the repouvenvironment because Python is 3.14.2 and bothpaddleandpaddleocrare missing. It also now failscomponent-admission:devanagari_primary: the router model card explicitly records the current Tesseract primary as failed on the CDC Devanagari diagnostic. Restore a Paddle-compatible Python runtime before claiming current full Sirijonga router measurements, but do not call the whole router ready until the Devanagari primary is replaced or re-admitted with evidence. - Attempted inspection of the remote 4 TB SSD at
cdjk@100.110.148.43failed with SSH permission denied (publickey,password,keyboard-interactive). Do not assume the remote has or lacks Devanagari-written Limbu pages until access works.
Operating rules (hard-won; do not relitigate)
norm_edit_disis NOT CER. PaddleOCR'snorm_edit_disis a per-sample-normalized macro average; on short-line datasets it runs ~4–8× above true micro CER. Always score withtools/eval_metrics.py(micro codepoint + grapheme-cluster UAX#29) fed byinfer_rec.py. Harness:/home/ubuntu/re_score.pyon the GPU instance. Eyeball the predictions vs GT before trusting any number.- Synthetic eval is not claim evidence. Real-crop CER only. Label every number real vs synthetic.
- Same-font is the lever. Real same-font/same-domain training data beats model tweaks. Different-font real data HURTS. Proven 3× now (Limbu, Devanagari, Sunuwar).
- Labels are machine-converted (legacy→Unicode or transliteration). A low CER can mean the model learned the converter's biases, not that the text is human-correct. Native-reader verification is still owed on all scripts.
- AWS SSO (
aws sso login --profile playaround) must be run in a plain terminal, NOT via this session's command runner — it auto-backgrounds the device-code poll and fails withinvalid_grant.
Verified scorecard (micro CER, this session)
Real newspaper print — STRONG:
Limbu/Sirijonga (native) graph 0.041 cp 0.021 real Gorkhapatra, 303 crops
8 Devanagari (one model) graph 0.075–0.129 real Gorkhapatra
Majhi .075 Magar .078(LEAKED) Sunuwar-Deva .080 Magahi .083
Bhojpuri .101 Thami .120 Wambule .121 Darai .129
Sunuwar (Jenticha, native) graph=cp 0.121 real Herald, 429 crops *** NEW ***
Synthetic / weak:
Tirhuta (native) graph 0.343 cp 0.250 synthetic
Newa/Prachalit translit graph 0.613 cp 0.418 synthetic (transliterated text)
Newa native retrain PENDING re-score (see Open #1)
Blocked (no same-font training data): Lepcha, Kirat Rai (see Open #3).
What this session did
- Metric correction. The whole scorecard had reported
1−norm_edit_disas CER. Re-scored all models with micro CER → Limbu/Devanagari are ~4–8× better than reported; Newa/Tirhuta confirmed weak. Memory of the gotcha saved. - Newa v4 width experiment. Hypothesis: 480px squish was crushing long Prachalit
lines. Widened to 640px, disabled RecConAug, retrained. Verdict: width alone did
NOT rescue Newa (cp 0.628 at epoch 8, undertrained but plateauing, worse than v3).
Confirms YOUR earlier call: the bottleneck is transliteration/data fidelity, not
pixels. Checkpoint preserved (
runs/newa-v4-wide-2026-06-18/). - Sunuwar same-font harvest + retrain. Verified the Sunuwar legacy→Unicode
converter applies uniformly across all 25 Herald editions (87–90% byte-confirmed).
Harvested 3,010 real same-font crops from 22 editions (
tools/harvest_sunuwar_train_crops.py), held the 3 mukhia editions out as the 429-crop eval (clean by-edition split). Fine-tuned the synth model on them → CER 0.342 → 0.121 (−65%), exact 0.7%→18.4%. Data-floor thesis confirmed for a third script. - Newa native corpus. Assembled 4,456 real native Prachalit lines
(
data/real-text/newa-real-native.txt, from nepal-lipi.com + newa-all-clean + nepal-lipi-extra) to replace the 463k transliterated salad, rendered (6,776 imgs, 2 fonts), retrained 30 epochs. Micro CER PENDING. - Devanagari→Gurung Khema transliterator (
tools/devanagari_to_gurungkhema.py). Aksharamukha has NO Khema target; map hand-derived from Unicode 16 names + Pandey L2/22-157 (U+1612F THOLHOMA = virama). Validated 0 dotted-circles on real Gurung text. Caveat: self-built, NOT native-validated.
Open questions — CRITICALLY REVIEW THESE
- Newa native result is uncaptured. The retrain runner logged only
norm_edit_disfor Newa (no micro-CER re-score — a wiring gap), then the instance auto-stopped. Mid-trainingnorm_edit_diswas 0.239 at epoch 11 on a native held-out eval — which looks WEAK, but that eval is harder/cleaner than the transliterated one, so it's not comparable. Need: restart instance, re-scoreruns/newa-native/checkpoints/ best_accuracywith micro CER on held-out native AND on the Zenodo real handwritten pack. Is native text actually helping, or is Newa still broken? Don't assume it helped. - Is Sunuwar 0.121 real or converter-noise-limited? The converter is ~89% byte- confirmed, so GT has ~11% machine-conversion noise. Model+GT may share that bias. Needs a native-reader spot-check of ~50 crops to bound true CER. Also: would more editions / higher byte-coverage push it lower, or is 0.12 the converter's floor?
- Lepcha / Kirat Rai are blocked. Lepcha's shape converter returns 0 on all 26 non-eval editions (different un-decoded font); Kirat Rai's typeface was never cracked (~70/4,343 lines). Same-font training data isn't available for them without new font-decoding work. Is that decoding worth it, or are these languages parked?
magarselection leakage (one of 8 Devanagari packs was the deva-v2 selection set). It sits mid-range so it isn't inflating the headline, but it's not a clean test.- No detector. All numbers are recognition-only on oracle crops. Page-level OCR needs a text detector; detector misses are the largest absent error source.
- Gurung Khema correction (2026-07-15): two official 2022 State Portal Herald editions supply eight real printed pages and 749 mechanically convertible candidates, refuting the old synthetic-only source conclusion. There is still no admitted real eval: independent provenance adjudication, native pixel-label review, crop construction, split/leakage audits, and CER remain absent. The self-transliterated model is still NOT a quality claim.
Pending tasks (priority order)
- Re-score Newa native (Open #1) — restart,
re_score.py, micro CER, report. - Native-reader label verification spot-checks (Sunuwar, Devanagari, Limbu).
- Push Sunuwar lower: harvest is capped at 89% byte-confirm; investigate the unmapped bytes / add more editions; consider mixing synth+same-font.
- Lepcha/Kirat Rai font decoding (deferred by user).
- Build a text detector for page-level eval.
- Newa v5: the real fix is better/native-validated text + real printed-register data, not shape (v4 proved shape insufficient).
Infrastructure
GPU: i-0fa19dff7b233a353 g6.xlarge us-east-1 profile=playaround (SSO)
STATE: stopped (EBS preserved). g6 capacity is intermittent —
start may need retry on InsufficientInstanceCapacity.
SSH: ~/.ssh/ocrtech-g6-training-20260615.pem (public IP changes each start;
last 44.211.88.63 — re-query after start)
On the EBS (preserved):
/home/ubuntu/native-retrain/ sunuwar-samefont + newa-native checkpoints, retrain.log
/home/ubuntu/staged-train/ synth corpora (sunuwar/kiratrai/lepcha), real eval packs,
dicts, run_queue.sh (the prior queue runner)
/home/ubuntu/deva-train/pretrain/devanagari_PP-OCRv5_mobile_rec_pretrained.pdparams (warm-start)
/home/ubuntu/{limbu,newa,tirhuta,deva}-train/ the strong/older models — DO NOT TOUCH
/home/ubuntu/re_score.py, /home/ubuntu/eval_metrics.py (micro-CER harness)
Devanagari shared recognizer + Limbu model are off-limits for the native-expert work —
all retraining is additive in native-retrain/.
Key new files (committed except gitignored data/)
tools/eval_metrics.py micro CER (codepoint + grapheme)
tools/harvest_sunuwar_train_crops.py same-font crop harvester (the winning pattern)
tools/build_sunuwar_real_eval_pack.py 429-crop real eval pack
tools/devanagari_to_gurungkhema.py novel Devanagari→Khema transliterator
src/ocrtech/{sunuwar,lepcha,lepcha_herald,kiratrai}_legacy.py Herald converters + tests
data/ (gitignored): sunuwar-train-pack/, *-real-eval-pack/, real-text/newa-real-native.txt
docs/RECOGNIZER_GOAL_PROGRESS.md living scorecard — READ FIRST
Current eval-harness status
The newer claim-grade harness work lives in docs/EVAL_HARNESS_GOAL.md and
docs/EVAL_HARNESS_RUNBOOK.md. The harness now separates recognizer oracle-crop
scoring from page-level OCR, records artifact hashes and run commands, emits
direct claim-safety.md reports, rejects metadata contradictions, and audits
metrics from row artifacts instead of trusting aggregate JSON. Scorers now require and
hash a script_inventory_file for claim-safe runs, record observed/missing required
codepoint and grapheme coverage, and hard-block claims when required inventory items
are absent. The auditor rejects a
run whose machine metadata lacks concrete platform/Python values or whose
normalization_profile no longer matches the scorer-supported NFC_preserve_whitespace
metric recomputation profile; the recipe validator also requires each eval pack to
declare that same profile, a readable and parseable summary/datasheet, a readable and
parseable script_inventory_file, and a readable
spotcheck_calibration_file whose eligibility/recommendation/source-status fields and
language/script/domain match whenever recipe-default or audited row GT is
machine_converted_spot_checked before a recipe can be claim-ready. Summary/datasheet
language, script, task, dataset, domain, GT, split, pack-version, frozen/unseen, and
default claim-eligibility declarations must also agree with the recipe metadata. Runner
checkpoint/config/dictionary values must also be readable local files, not directories,
so the recipe validator matches the scorer's hash-verifiability gate, and readable
runner dictionaries are checked against recognizer GT or page reference codepoints
before a recipe can be claim-ready. For path-complete
recognizer packs it now runs audit_recognizer_eval_pack.py and turns any pack-audit
issue or warning into a claim-readiness blocker; page recipes get a static
eval-manifest/reference audit for source image hashes, source-document/page/bbox
provenance, row claim eligibility, human review evidence, claim-ready references, and
minimum overall/per-slice scoring-unit counts. Recognizer pack audits apply the same
overall and language/script/domain/GT/split minimum before a recipe can be claim-ready,
leakage_sources are parsed and must be non-empty, page leakage manifests must satisfy
the page manifest schema, external command templates must include their required eval
input and output placeholders and must not contain unsupported placeholders, and recipe-declared checkpoint selection on the test pack
or known leakage is also a claim-readiness blocker, so empty manifests, missing page
evidence, input-disconnected commands, or acknowledged test-set contamination cannot
pass recipe registration as claim-ready.
Rows declared human_verified now also need explicit human review evidence such as
reviewer or review-artifact metadata; a trust label by itself is a claim blocker.
It also checks that checkpoint/config/dictionary run fields still match their hashed
artifact identity entries and that claim-safe runs carry non-empty model/system identity.
Recognizer slices
and crop-image hashes are recomputed from predictions.tsv, unmatched.tsv carries
source-audited missing label/prediction identity, label image hashes, and prediction
scores, recognizer manifest image paths/basenames are rejected when they would make
basename-based label/prediction matching ambiguous, worst-lines.csv carries audited row identity, crop hash, trust metadata,
claim eligibility, and source-document/page/bbox provenance for the worst recognizer
failures, and predictions.tsv rows are reconstructed from recorded label/prediction/manifest sources; page text row
counts, page text totals, line-alignment row contents, aggregate page metric means,
and per-language/script/domain/GT/split page slices are recomputed from scored
documents, references, page-metrics.csv, and line-alignment.csv. Page
page-metrics.csv row identity is reconstructed from the eval manifest, resolved
document.json, embedded document.source_path, scorer metadata defaults, row claim
eligibility, source-document/page/bbox provenance, and parsed reference fields; recognizer predictions.tsv also
carries source-audited row provenance and claim eligibility,
page reference-artifacts.tsv carries source-audited reference path/hash/field/trust
identity, page text-pairs.tsv carries the exact prediction/reference text used for
page text CER, exact-match recomputation, and page confusion recomputation, and page
worst-pages.csv carries audited
input/document hashes, row provenance, claim eligibility, and reference-artifact trust
fields for the worst page failures, run-level language/script/domain/GT/split lists
are checked against canonical row artifacts, undersized per-slice sample counts are
claim blockers so aggregates cannot hide weak languages/scripts, page text exact-match accuracy is carried
in page-metrics.csv and recomputed into aggregate/slice metrics, deterministic bootstrap uncertainty summaries are recomputed from
predictions.tsv and page-metrics.csv, script inventory summaries are recomputed from
recognizer GT rows or page reference text, and source-image and document.json SHA-256
values are rechecked. The auditor
now recomputes train/dev/eval leakage reports from recorded train manifests and
canonical eval rows/manifests, including overlap sets, checked state, and row counts,
and it checks embedded native spot-check calibration summaries, including
language/script/domain/source-GT/recommended-GT binding, against the recorded
calibration artifact. Metadata contradictions are recomputed from manifest/reference
provenance signals and canonical row domain/GT fields; recognizer provenance coverage
and page row source-document/page/bbox/claim-eligibility/language/script/domain/GT
metadata coverage counts are recomputed from source manifests, recognizer
normalizer/defaulted-metadata counts are checked against the recorded summary/datasheet,
hash-valid direct-run summary/datasheet language/script/domain/GT/split/version,
frozen/unseen, default claim-eligibility, and task declarations are checked against
the score run metadata,
dictionary coverage is recomputed from recognizer GT rows or page reference text and
the recorded dictionary file so out-of-vocabulary GT/reference codepoints block
claim-safe status,
zero-length recognizer GT rows and page text references are counted and recomputed so
empty references cannot hide inside CER denominators,
and page
reference coverage is recomputed from page-metrics.csv; page
manifest/scored/missing/claim-eligible counts are recomputed from source artifacts. It
also derives claim-safety blockers from the durable metrics/run metadata, so missing
manifests or summary/datasheets, unrun leakage checks, leakage overlaps, metadata
contradictions, unsafe domains or GT, low match rate, missing provenance, recognizer
row-level claim ineligibility, missing uncertainty evidence, missing script inventory
evidence, missing required script codepoints/graphemes, zero-length GT/reference text,
too-small claim-facing sample size, and incomplete page coverage cannot be hidden by editing claim_safe or warnings.
Direct claim-safety.md reports are checked against the JSON run/metric facts, so
stale human-facing summaries no longer pass just because their sections exist; recipe
root claim-safety.md reports are checked against recipe-run and score JSON facts too,
with recognizer/page-specific expected lines based on eval_mode.
Recipe-root recipe.json must match the selected recipe in the hashed source registry,
recipe-validation.json must match recipe-run.json's embedded validation, and that
embedded validation is recomputed from the current recipe registry plus eval-pack
audit.
Recipe-run timestamp, machine platform/Python, command, and root output pointers for
materialization, audit, and claim-safety artifacts are audited.
Recipe failures now emit recipe-run-error.json with timestamp, machine, command,
recipe registry hash, selected recipe validation, root/output directory, exception
type, message, and traceback.
Root audit.json and recipe-run.json audit summaries are checked for current
issues, warnings, claim-safe status, and score-dir identity.
Recipe-run score summary paths and hashes are checked against score/metrics.json
and the mode-specific score run file: score/eval-run.json for recognizer roots or
score/page-eval-run.json for page roots.
Recipe-run train-manifest path/hash identities are checked against the score eval-run
leakage inputs.
Recipe-run root must be an existing directory, and the hashed recipe registry must
live under that declared root.
Recipe-root materialization JSON, timestamp, machine platform/Python and invocation
metadata, per-record commands/log paths, command-template.txt, output-command-run.md,
recognizer outputs.prediction_file, page outputs.documents_dir, and materialization
record hashes are tied back to the scored prediction file path/hash or scored page
document directory, and materialization model/checkpoint/config/dictionary metadata is
checked against the score eval-run.
Materialization failures emit output-command-run-error.json with the same run
identity plus label or eval-manifest path/hash, so failed external-command attempts
remain attributable.
Target coverage validation constrains eval_status to known states and now reports
non-claim-ready status even when concrete pack/recipe blockers already exist. Only
registered_real_pack targets can become coverage-ready, so partial, shared, missing,
or invented status labels cannot be used as silent claim evidence. The registry also
has a dedicated limbu-first-two-script-ocr target set requiring both
limbu/Limbu and limbu/Devanagari before a two-script Limbu OCR claim, and that
target set now requires page_eval_recipe_id links so recognizer-only readiness
cannot satisfy the full page-capture goal.
Score-level metrics.md reports are checked against metrics.json as well.
Recognizer unmatched.tsv plus label-line, matched-line, unmatched-label,
unmatched-prediction, and match-rate metrics are recomputed from source
label/prediction files; recognizer confusions.json is recomputed from canonical
scored rows, declared recognizer manifest crop hashes and image-key uniqueness are
checked against current crop bytes, page confusions.json is recomputed from canonical text-pairs.tsv, and page
missing.tsv is recomputed from manifest, document, and reference availability.
Worst-example CSVs, reviewer CSV order/fields, and reviewer HTML samples are audited
against canonical scored rows so the visible failure samples cannot silently go stale.
Completed score artifacts are also checked for duplicate sample IDs, image paths,
document paths, and missing-row keys, so duplicate-ID problems cannot be hidden after
normalization. Page scoring also rejects duplicate eval-manifest sample_id and
image_path values and stale declared page image hashes before scoring, and the audit
reader reports duplicate page manifest image paths or source-image hash mismatches if
an older or hand-edited score directory points at one.
Recognizer-vs-recognizer claims now have a paired comparison artifact:
tools/compare_recognizer_eval_runs.py pairs two completed recognizer score
directories by sample_id, requires same eval-pack and row GT identity, reports
candidate/baseline micro CER deltas for codepoints and graphemes, paired
wins/losses/ties, deterministic paired-row bootstrap confidence intervals, and an exact
two-sided sign-test p-value. It writes comparison.json, comparison.md, and
paired-rows.tsv, and its --audit-only mode recomputes those artifacts so stale
comparison summaries fail.
Limbu-first operational OCR slice
User redirected the next production push to Limbu rather than Sunuwar because upcoming digitization work needs a pipeline for imperfect page captures that may contain both Devanagari-written Limbu and Sirijonga-script Limbu. A first operational slice now exists, but it is not a page-level CER claim. The infrastructure is two-script shaped, while the measured local page evidence is currently Sirijonga-heavy; a real Devanagari-written Limbu page reference is still missing.
src/ocrtech/engines.pyscript routing now supportsscript=limbu/sirijonga/limbu-sirijongaby measuring U+1900..U+194F codepoint ratio. Thescript_select_compositebackend also supportsrouting_signal_engine=secondaryorboth, which is important for Limbu because a Devanagari primary may not emit Sirijonga codepoints needed to trigger a switch.src/ocrtech/pipeline.pyexposesparse_limbu_document, which wraps the normal parser and annotates every OCR line with Sirijonga, Devanagari, and Latin ratios, a script class, review reasons, and an explicitpost_correction=not_configuredmarker.--capture-prep-metadatabindslimbu-capture-prep.jsonintodocument.jsonandlimbu-pipeline-audit.json, including the metadata file hash plus raw/prepared image hashes. It now also rerunsaudit-limbu-captureinto the OCR output directory undercapture-prep-audit/, fails stale or incomplete capture bundles, and fails if the OCR input does not match the auditedprepared_image_path.src/ocrtech/pipeline.pyalso exposesapply_limbu_review_corrections, andocrtech apply-limbu-review-correctionsapplies accepted rows fromlimbu-review-queue.tsvback into a correcteddocument.json/document.md. It writeslimbu-correction-audit.jsonpluslimbu-correction-pairs.jsonlfor future Limbu-specific corrector training, and fails on duplicate accepted targets or empty accepted corrections. Corrected-row script-class inference now carries forward the source OCR document'sscript_ratio_thresholdand records it inlimbu-correction-audit.json, so correction pairs cannot silently use a different script policy than the OCR run that produced the review queue.- Both Limbu OCR and review-correction stages now write
limbu-output-manifest.jsonwith stage, run context, source hash when available, document counts, expected artifact paths, SHA-256 hashes, sizes, and missing-artifact list.audit-limbu-outputnow replays manifestsource_path/source_sha256and bindsdocument_source_pathplussource_pathtodocument.json.source_path. For OCR bundles, it also binds manifestprovenance.capture_prep_metadataandprovenance.post_correction_profileto the exact paths and hashes embedded inlimbu-pipeline-audit.json/limbu-post-correction-audit.json. For review-correction bundles, it also binds manifestprovenance.input_documentandprovenance.review_queueto the exact paths and hashes recorded inlimbu-correction-audit.json, so a manifest cannot point at different valid files while the stage replay uses the real inputs. parse-limbu-docnow writes line-crop provenance underline-crops/: a JSONL row per OCR line with bbox, script class, review flags, crop path/hash when the source is croppable, and explicit warnings when crops cannot be generated. Valid crop PNGs are included inlimbu-output-manifest.json.parse-limbu-docalso writeslimbu-review-dashboard.html, a static review contact sheet for queued lines that shows the line crop when available, page/line id, script class, confidence, review reasons, OCR text, and the TSV row shape needed for accepted corrections. The dashboard is included inlimbu-output-manifest.json.- Review queue confidence triage is now script-aware.
parse-limbu-docstill supports the global--low-confidence-threshold, but can override it with--sirijonga-low-confidence-threshold,--devanagari-low-confidence-threshold,--mixed-script-low-confidence-threshold, and--other-script-low-confidence-threshold.limbu-pipeline-audit.json,limbu-line-audit.jsonl,limbu-review-queue.tsv, and the dashboard record the effective threshold so later review/audit can replay why a Sirijonga or Devanagari line entered the queue. Review threshold andscript_ratio_thresholdinputs are now validated as finite ratios in[0, 1].audit-limbu-outputapplies the same validation to embeddedlimbu-pipeline-audit.jsonthresholds before replay, so a refreshed manifest hash cannot hide invalid script-classification policy. Regressions:tests/test_pipeline.py::test_parse_limbu_doc_rejects_non_finite_or_boolean_review_thresholdsandtests/test_pipeline.py::test_parse_limbu_doc_rejects_invalid_script_ratio_threshold, plustests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_embedded_script_ratio_threshold_after_manifest_refresh. audit-limbu-outputnow also replays queued confidence values againstlimbu-line-audit.jsonl. A tampered review queue that keeps the same line id/text but changes confidence now fails audit; regression:tests/test_pipeline.py::test_audit_limbu_output_rejects_review_queue_confidence_mismatch.audit-limbu-outputnow replays the human-facinglimbu-review-dashboard.htmlcards againstlimbu-review-queue.tsvandline-crops/manifest.jsonl, including displayed confidence, text, TSV copy row, and crop link. This catches dashboard tampering even after the output manifest hash is refreshed; regression:tests/test_pipeline.py::test_audit_limbu_output_rejects_review_dashboard_confidence_mismatch_after_manifest_refresh.parse-limbu-docnow has a guarded deterministic Limbu post-correction lane via--post-correction-profile. The JSON profile must declareprofile_id,language=limbu,script_scope, and exact/token rules whose replacements contain Limbu or Devanagari text.script_scopeand rule-levelscript_classvalues are constrained tolimbu_sirijonga,devanagari_limbu,mixed_limbu_devanagari, orother; regressions:tests/test_pipeline.py::test_audit_limbu_post_correction_profile_rejects_unknown_script_scopeandtests/test_pipeline.py::test_audit_limbu_post_correction_profile_rejects_unknown_rule_script_class. Every OCR run writeslimbu-post-correction-audit.jsonandlimbu-post-correction-lines.jsonl; without a profile these artifacts explicitly saystatus=not_configured, and with a profile they record the profile SHA-256, rule ids, original/corrected text, and changed count.build-limbu-correction-pair-packshould be the first step after accumulating reviewed correction pairs. It writestrain-limbu-correction-pairs.jsonl,heldout-limbu-correction-pairs.jsonl, andlimbu-correction-pair-pack.jsonwith frozen-pack status and a pack-content digest. The pack remains calibration evidence, not page OCR ground truth. Accepted review rows, pack build, pack audit, derivation, and held-out scoring now constrain correction-pairscript_classvalues to the same vocabulary as post-correction profiles:limbu_sirijonga,devanagari_limbu,mixed_limbu_devanagari, orother. Missing row values are inferred from clean text, but explicit typos such assirijongafail instead of being silently hidden. The audit replay path uses the same validation, so legacy or hand-written review queues with unsupported accepted-row script labels cannot pass replay just because their hashes match. Regressions:tests/test_pipeline.py::test_apply_limbu_review_corrections_rejects_unknown_script_class,tests/test_pipeline.py::test_limbu_review_correction_replay_rejects_unknown_script_class,tests/test_pipeline.py::test_build_limbu_correction_pair_pack_rejects_unknown_script_class,tests/test_pipeline.py::test_audit_limbu_correction_pair_pack_rejects_unknown_script_class,tests/test_pipeline.py::test_derive_limbu_post_correction_profile_rejects_unknown_script_scope,tests/test_pipeline.py::test_derive_limbu_post_correction_profile_rejects_unknown_pair_script_class, andtests/test_pipeline.py::test_score_limbu_post_correction_profile_rejects_unknown_pair_script_class.audit-limbu-correction-pair-packshould pass before using the pack for derivation or held-out post-correction scoring. It catches stale train/heldout files and digest mismatches after pack creation.derive-limbu-post-correction-profilebridges reviewed correction bundles into that profile format. It refuses ambiguous mappings and writes both JSON and Markdown derivation reports, but it does not produce a validated profile; runaudit-limbu-post-correction-profileand attach held-out evidence before treating an automatic profile as more than triage.score-limbu-post-correction-profileis the missing evidence generator for that admission step. It produceslimbu-post-correction-profile-eval.jsonplus per-row JSONL, and the CLI prints the stable eval JSON SHA-256. Copy that hash, the frozen eval-pack identifier, metrictext_cer_codepoint_micro, and before/after values into profileadmissiononly when the scorer passes.admit-limbu-post-correction-profilenow does that copy safely. It writes a new admitted profile rather than mutating the source file, and immediately audits the result withaudit-limbu-post-correction-profile.ocrtech audit-limbu-outputaudits a Limbu OCR or review-correction output directory by recomputing manifest artifact hashes/sizes, checking missing artifacts, validating document counts and source-path binding, checking OCR line/post-correction/review queue/dashboard/crop counts, verifying capture-prep audit status, verifying post-correction profile hashes and crop hashes when they exist, binding OCR manifest provenance to the embedded capture/profile audit records, rerunning the capture-prep audit from the embedded metadata path intocapture-prep-replay-audit/, replaying each OCR line's embeddeddocument.jsonmetadata.limbu_pipelinescript ratios, script class, review flag, review reasons, and effective threshold from the current line text/confidence after validating the embedded replay thresholds, replaying OCR-stage embeddedpost_correctionmetadata fromlimbu-post-correction-lines.jsonlandlimbu-post-correction-audit.json, and replaying review-correction bundles from the originaldocument_pathplusreview_queue_pathrecorded inlimbu-correction-audit.json. The replay verifies source hashes, manifest provenance bindings, the source script-ratio threshold, applied/changed/skipped counts, correcteddocument.jsontext,limbu-correction-pairs.jsonlrows, and the corrected line's embeddedmetadata.limbu_pipelinereview flags pluspost_correctionreviewer/status/original/corrected-text metadata, so corrected text, training pairs, and the embedded correction story are not accepted on count checks alone. A refreshed manifest hash no longer hides tampered embedded line metadata; regressions:tests/test_pipeline.py::test_audit_limbu_output_replays_document_line_limbu_metadata_after_manifest_refreshtests/test_pipeline.py::test_audit_limbu_output_replays_post_correction_line_metadata_after_manifest_refresh, andtests/test_pipeline.py::test_audit_limbu_output_replays_review_correction_line_metadata_after_manifest_refresh. It writeslimbu-output-audit.jsonand.md, and exits nonzero on stale or incomplete bundles.ocrtech audit-limbu-output --require-reviewer-for-correctionsis now available for strict corrected-bundle audits. It fails accepted correction rows whosereviewerfield is blank; regression:tests/test_pipeline.py::test_audit_limbu_output_strict_policy_requires_reviewer_for_corrections.ocrtech audit-limbu-outputnow has strict operational policy flags for real page digitization:--require-capture-prep, prepared-image quality thresholds (--min-capture-prepared-width,--min-capture-prepared-height,--min-capture-prepared-entropy,--min-capture-prepared-luminance-stddev, and--min-capture-prepared-edge-stddev),--require-no-pending-review, repeated--require-scriptvalues such aslimbu_sirijongaanddevanagari_limbu, and repeated--require-script-count SCRIPT=COUNTgates for minimum per-script line counts. Capture-threshold policy inputs are validated as finite non-negative numbers before audit work starts; regression:tests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_capture_quality_policy. Required script policy inputs are normalized/deduplicated, boolean count values are rejected instead of being treated as1, and invalid script names fail before audit work starts; regressions:tests/test_pipeline.py::test_audit_limbu_output_rejects_invalid_required_script_policyandtests/test_pipeline.py::test_audit_limbu_output_normalizes_required_script_policy. It also has--min-line-count,--min-average-line-confidence, and--min-quality-score, which gate strict acceptance on auditeddocument.jsonquality metadata rather than only provenance;--min-line-countalso rejects boolean API values instead of accepting Python'sTrue == 1behavior. These are opt-in so diagnostics can still be artifact-audited, but real Limbu page outputs should use them to require replayed capture provenance, prepared-image quality, enough OCR text, minimum output quality, no unresolved review rows, and expected script coverage.ocrtech audit-limbu-capture --require-metadata-path-selfis now available for strict capture-prep audits. It fails whenmetadata_pathinsidelimbu-capture-prep.jsondoes not resolve to the exact metadata file being audited; without the flag this remains a portability warning. Regression:tests/test_capture.py::test_audit_limbu_capture_can_require_metadata_path_self_binding.ocrtech audit-limbu-output --require-capture-metadata-path-selfnow forwards that same metadata self-binding requirement into the OCR output's capture-prep replay. Regression:tests/test_pipeline.py::test_audit_limbu_output_can_require_capture_metadata_path_self_binding. A fresh two-script smoke atoutputs/limbu-strict-output-policy-smoke-2026-06-21passes that strict policy;strict-audit/limbu-output-audit.jsonSHA-256 is2fc71f191ba581d58f59480b54695a9297778f3c54a2a63c8a7da339d618142e.src/ocrtech/cli.pyaddsocrtech parse-limbu-doc, emitting normal parser outputs pluslimbu-pipeline-audit.json,limbu-line-audit.jsonl,limbu-post-correction-audit.json,limbu-review-queue.tsv, andlimbu-review-dashboard.html; it also addsocrtech apply-limbu-review-corrections.src/ocrtech/capture.pyandocrtech prepare-limbu-capturepreserve the raw capture, apply EXIF transpose, optional manual crop, optional four-corner perspective rectification via--perspective-quad, optional automatic page-corner detection via--auto-detect-page, optional automatic deskew via--auto-deskew/--max-auto-deskew-degrees, optional manual rotation, grayscale, and autocontrast, and writelimbu-capture-prep.jsonwith source/prepared hashes, source/prepared image-quality metrics (size, luminance spread, edge spread, entropy), timestamp, operation list, and CLI run context when available. Auto-deskew now tries near-horizontal Hough line evidence first and falls back to row-projection scoring over text ink for fragmented glyph rows; it records either the detected/applied angle or a skipped status when evidence is insufficient.ocrtech audit-limbu-capturenow audits a prepared capture bundle orlimbu-capture-prep.json: it verifies raw-copy and prepared-image hashes, validates capture operation metadata, replays image-quality metrics for available images, writeslimbu-capture-audit.json/.md, fails stale or missing prepared artifacts, and warns rather than fails if the original source path has moved after the raw copy was preserved. It also supports opt-in prepared-image quality gates:--min-prepared-width,--min-prepared-height,--min-prepared-entropy,--min-prepared-luminance-stddev, and--min-prepared-edge-stddev; chosen thresholds are recorded asquality_policy.docs/LIMBU_FIRST_OCR_PIPELINE.mddocuments the command, output contract, model shape, capture cleanup requirements, and claim boundary.configs/limbu-first-ocr-pipeline-v1.jsonrecords the intended pipeline profile and current local Limbu/Devanagari asset references.src/ocrtech/limbu_runtime.pyandocrtech preflight-limbu-pipelinenow provide a machine-readiness gate for the packaged router: router model-card audit, nested component model-card audits, Pillow/OpenCV/NumPy/Paddle/PaddleOCR imports, Python executable/version, Tesseract binary discovery, and requirednep/englanguage data. The report is written tolimbu-runtime-preflight.jsonand.md. Use--require-validated-components --required-component-role devanagari_primary --required-component-role sirijonga_secondaryfor the strict two-script gate: this fails warning-onlyexperimental/pendingcomponents and missing component roles, so a dependency-ready runtime cannot be mistaken for an admitted Limbu router. The preflight report now records the enforced policy and fails malformedcomponent_admissionlist entries instead of silently filtering them out.outputs/limbu-first-router-v1/now contains an auditable router package:model-card.jsonis ascript_select_compositecandidate with Tesseractnep+engas the Devanagari primary and packaged PaddleOCR Sirijonga as the secondary. The Sirijonga component now uses the exported v4 package copied intocomponents/limbu-sirijonga-paddle-v4/; both the component and router model-card audits pass. The router keepsrouting_signal_engine=secondaryfor Sirijonga discovery but addsprimary_guard_script=devanagarito stop secondary hallucinations from replacing clearly Devanagari primary lines. The default project.venvstill uses Python 3.14 and fails Paddle / PaddleOCR imports, but a Python 3.11 runtime at/tmp/ocrtech-paddle311importspaddle==3.3.1andpaddleocr==3.7.0; with Homebrew Tesseract plus manually addednep.traineddata, the Python dependency checks pass under/tmp/ocrtech-paddle311. The full router preflight should still fail until the Devanagari primary is replaced or admitted with evidence; the current router recordsdevanagari_primaryas failed. A minimal candidate-engine smoke parse also completed underoutputs/limbu-first-router-v1/runtime-smoke-py311/out, andaudit-limbu-outputpassed for that smoke bundle.tools/build_limbu_devanagari_smoke_page_eval_draft.pybuilds a non-claim synthetic Devanagari-written Limbu page-eval draft from the local real text corpus. The current pack atoutputs/limbu-page-eval-drafts/limbu-devanagari-smoke-page-2026-06-21renders 14 lines from 8 source rows with Noto Sans Devanagari, records source/font/ image/reference/manifest/script-inventory hashes, and writes a sidecar-perfect document for harness regression. The score atscore-sidecar-perfecthas codepoint CER 0, grapheme CER 0, line F1 1.0, matched-line reading-order pair accuracy 1.0 over 91 pairs, no missing required graphemes, andaudit_passed=true; it remainsclaim_safe=falseand is not real-page evidence.tools/inventory_limbu_page_sources.pyinspects known Limbu page/text candidates: curriculum JSON/PDF references, Sikkim Herald Limbu PDFs, Gorkhapatra Namdhinggo listings, local text-only Devanagari sources, and the synthetic smoke pack. It writeslimbu-page-source-inventory.jsonand.mdand can hard-fail with--require-real-devanagari-pagewhen no real Devanagari page source is present.
Verification for this slice:
uv run pytest tests/test_pipeline.py tests/test_limbu_runtime.py tests/test_capture.py tests/test_cli.py::test_cli_prepare_limbu_capture_passes_run_context tests/test_cli.py::test_cli_apply_limbu_review_corrections tests/test_cli.py::test_cli_audit_limbu_output_exit_code tests/test_cli.py::test_cli_create_script_select_model_card tests/test_cli.py::test_cli_export_and_package_recognizer tests/test_training_benchmark.py::test_package_paddleocr_model_writes_auditable_model_card -q passed,
uv run pytest tests/test_models_candidate.py::test_candidate_engine_supports_script_select_composite_limbu_routing tests/test_models_candidate.py::test_script_select_composite_can_route_from_secondary_limbu_signal tests/test_models_candidate.py::test_script_select_candidate_audit_rejects_invalid_routing_signal_engine -q passed,
uv run python -m py_compile src/ocrtech/limbu_runtime.py src/ocrtech/capture.py src/ocrtech/pipeline.py src/ocrtech/engines.py src/ocrtech/models.py src/ocrtech/cli.py
passed, CLI help for parse-limbu-doc, prepare-limbu-capture, and
preflight-limbu-pipeline loads, ocrtech apply-limbu-review-corrections --help
and ocrtech audit-limbu-output --help load, and git diff --check passed. The
claim-readiness validators remain structurally valid but non-ready:
recipes=12 valid=True claim_ready=0 and
valid=True coverage_ready=False target_sets=3 gaps=50.
Next real work: improve full-page Sirijonga quality, find or create a real Devanagari-written Limbu page reference beyond the synthetic smoke pack, make Devanagari/Sirijonga routing claim-safe with frozen page references, package local hashed model cards for any custom Devanagari recognizer if it becomes available, improve page-corner detection for still captures when needed, and only then score page-level outputs against frozen Limbu page references.
The Devanagari page-source gate is now green for real-source availability. The next step is to build a non-claim Devanagari page-eval draft from those CDC PDFs and curriculum line lists, then add line/reference alignment and review evidence before treating the second script as claim covered.
Current eval-harness status
The latest page-eval harness change adds an explicit OCR-output mode to page scoring:
tools/run_page_ocr_eval.py now records ocr_output_kind in page-eval-run.json,
metrics.json, metrics.md, claim-safety.md, and each page-metrics.csv row.
Claim-safe page scoring hard-warns/fails if the kind is unknown; accepted values are
live_detector, oracle_line_boxes, sidecar_fixture, and unknown at the scorer CLI
level, while recipe validation only permits claim-ready page recipes to declare
live_detector, oracle_line_boxes, or sidecar_fixture. tools/run_eval_recipe.py
passes eval_pack.ocr_output_kind through to the page scorer, defaulting recipe-driven
page runs to live_detector; tools/audit_eval_run.py audits the new run/row identity
field; and tools/compare_page_ocr_eval_runs.py surfaces candidate/baseline
ocr_output_kind so the required Limbu diagnostic can honestly compare
oracle_line_boxes against live_detector on the same frozen page pack without hiding
that this is a detector-vs-recognizer experiment, not a normal model-vs-model result.
Docs updated: docs/EVAL_HARNESS_RUNBOOK.md and
docs/LIMBU_FIRST_OCR_PIPELINE.md.
The latest page-eval harness also reports matched-line text and reading-order metrics
separately from full-page text metrics. tools/run_page_ocr_eval.py now writes
matched-line codepoint and grapheme edit/ref counts, micro CER, exact-match accuracy,
and aligned-line reading-order pair accuracy into metrics.json, metrics.md,
claim-safety.md, page-metrics.csv, and line-alignment.csv. This prevents a high
page CER from being misread as one single failure: the same eval run can now separate
line geometry, recognition on matched lines, aligned-line order, and page text assembly.
Regression coverage includes the case where line boxes and line text are perfect but
block reading order is wrong; page CER rises while matched-line CER remains zero and
matched-line reading-order accuracy falls to zero.
A new draft-pack builder was added at
tools/build_limbu_sikkim_page_eval_draft.py. It uses the existing Sikkim Herald
Limbu legacy-font converter to build a non-claim page eval draft with machine-converted
Sirijonga text and PDF-derived line bboxes scaled to a full-page image. The first real
draft pack is
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1: 86 reference
lines for limboo_2020-12-03.pdf page 1, scored against the guarded live-detector OCR
document as ocr_output_kind=live_detector. The scorer reports codepoint CER
0.891560, grapheme CER 0.897412, line detection F1 0.018519, and 1 matched line
out of 86 references. audit_eval_run.py passes the score bundle (audit_passed=True)
but claim_safe=False because the pack is not frozen/claim-eligible, has only one
page, uses gt_status=machine_converted_unreviewed, and has draft conversion caveats
(U+0041, U+201C missing from the dictionary). Treat this as strong diagnostic
evidence that current full-page Sirijonga failure is detector/layout dominated, not as
accuracy evidence.
A new oracle-line-box materializer was added at
tools/materialize_page_oracle_line_boxes.py with tests in
tests/test_materialize_page_oracle_line_boxes.py. It reads page eval manifest rows,
uses reference line bboxes only to crop the page image, runs a configured OCR engine on
each crop, and emits page-score-compatible document.json files plus a crop manifest.
For PaddleOCR model cards it can force --recognition-mode line, which is required for
the Sikkim diagnostic because auto can route large title boxes through the full-page
Paddle wrapper. The Sikkim oracle materialization lives at
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/oracle-line-box-documents-v4
and used the v4 Sirijonga component model card with recognition_mode_override=line;
it produced 86 lines and 0 empty predictions.
The oracle-line-box score is
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/score-oracle-line-boxes-v4.
It audits cleanly (audit_passed=True, claim_safe=False) and reports codepoint CER
0.274289, grapheme CER 0.548983, line detection F1 1.0, and 86/86 matched lines.
The paired comparison at
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/compare-oracle-vs-live-v4
shows a non-claim-safe codepoint-CER improvement of 0.617272 over the live detector.
Interpretation: fix detection/reading order, but do not pretend oracle boxes solve the
page. The recognizer is still weak on this Sikkim Herald register and the draft
machine-converted reference needs human verification or at least spot-check calibration.
A second materializer was added at tools/materialize_page_image_line_boxes.py with
tests in tests/test_materialize_page_image_line_boxes.py. It is not an oracle: it
detects line-like components from page pixels using Otsu thresholding and morphology,
can compute final boxes from the original ink pixels, then crops and recognizes each
detected line with the configured engine. The current Sikkim diagnostic uses
bbox_source=ink, horizontal kernel 41, vertical kernel 11, one dilation
iteration, detector padding 2, crop padding 12, and v4 Sirijonga
recognition_mode=line. Outputs:
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/image-line-box-documents-v4,
score:
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/score-image-line-detector-v4,
comparisons:
compare-image-line-vs-paddle-live-v4 and compare-oracle-vs-image-line-v4.
The image detector improves over Paddle live detection by 0.431305 absolute codepoint
CER, but oracle boxes still improve over it by 0.185967 absolute codepoint CER. Do not
promote this detector blindly; test it on more pages and fix reading order/overprediction.
The detector-only reading-order comparison for this Sikkim image-line detector is
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/compare-detector-reading-order-column-major-v1;
it reports no column_major gain over top_to_bottom under the same detector settings.
The materializer now supports --reading-order auto_layout, records requested/selected
ordering plus layout features, and the detector-only comparator accepts that mode. The
auto heuristic uses wide-page plus regular line-width evidence to select
wide_column_right_to_left. It improves Gorkhapatra matched-line reading-order pair
accuracy from 0.606543 to 0.763985 while keeping Sikkim unchanged at 0.992402; outputs
are compare-detector-reading-order-auto-layout-v1 under each draft page directory.
This is layout-selection evidence only, not a text-quality claim.
The detector parameter sweep at
outputs/limbu-page-eval-drafts/sikkim-sirijonga-2020-12-03-page1-v1/detector-param-sweep-v1
also reports no morphology-family improvement over the existing 41x11 detector.
The latest Codex pass extended the immutable eval-pack digest from recognizer
oracle-crop runs to page-level OCR runs. Page eval-pack summaries now must carry
pack_content_sha256 just like recognizer summaries. The page scorer recomputes the
digest from the rows it actually scores, using the canonical page row identity, input
image hash, reference artifact hash, reference text hash, reference fields, provenance,
language, script, domain, GT status, split, and claim eligibility. Claim-safe page
scoring fails when the summary digest is missing or stale. The page score now records
that digest in both metrics.json and page-eval-run.json.
The audit path also recomputes page pack_content_sha256 from current source
artifacts and checks the value recorded in page-eval-run.json, metrics.json, and
the page summary/datasheet. Page comparison now treats pack_content_sha256 as part
of eval-pack identity, so a candidate and baseline page run cannot be claim-safe
compared if they were scored against different or drifted page-pack contents. The
relevant files touched in this pass are tools/run_page_ocr_eval.py,
tools/audit_eval_run.py, tools/compare_page_ocr_eval_runs.py,
tools/validate_eval_harness_recipes.py,
tests/test_run_page_ocr_eval.py, tests/test_audit_eval_run.py,
tests/test_compare_page_ocr_eval_runs.py, docs/EVAL_HARNESS_GOAL.md, and
docs/EVAL_HARNESS_RUNBOOK.md.
The page recipe fixture now computes the page pack digest before claim-safe recipe
scoring, and the recipe validator blocks digestless page summaries before scoring
instead of only blocking recognizer summaries.
Verification after the digest work: focused recipe/page/audit/validator tests passed;
the full targeted harness suite passed with
uv run pytest tests/test_apply_native_spotcheck_reviews.py tests/test_audit_eval_run.py tests/test_audit_recognizer_eval_pack.py tests/test_build_recognizer_leakage_manifest.py tests/test_compare_page_ocr_eval_runs.py tests/test_compare_recognizer_eval_runs.py tests/test_eval_harness_runbook.py tests/test_native_spotcheck_bundle.py tests/test_normalize_recognizer_eval_pack.py tests/test_predict_recognizer_eval.py tests/test_run_eval_output_command.py tests/test_run_eval_recipe.py tests/test_run_page_ocr_eval.py tests/test_run_recognizer_eval.py tests/test_run_recognizer_pack_normalization.py tests/test_validate_eval_harness_recipes.py tests/test_validate_eval_target_coverage.py -q;
targeted py_compile passed; git diff --check passed. Registry validators are
structurally valid but not claim-ready:
uv run python tools/validate_eval_harness_recipes.py --recipes configs/eval-harness-recipes-v1.json --root .
reports recipes=12 valid=True claim_ready=0, and
uv run python tools/validate_eval_target_coverage.py --coverage-config configs/eval-target-coverage-v1.json --eval-recipes configs/eval-harness-recipes-v1.json --root .
reports valid=True coverage_ready=False target_sets=3 gaps=50. Do not mark the
overall eval-harness goal complete until those claim-readiness and coverage gaps are
actually closed.
The latest audit hardening closes a stale-output-audit gap for live-detector page
scores. tools/audit_eval_run.py now requires page-eval-run.json to carry
require_output_audit, output_audit_filename, output_audit_passed_field, and
page_output_audit, then replays the expected per-page audit summary from
page-metrics.csv and the current audit JSON beside each scored document.json. If a
recorded page score says all Limbu output audits passed but a later reviewer deletes,
edits, or breaks one of those per-page audit files, audit now fails instead of trusting
the stale metrics.json/page-eval-run.json summaries. Regression coverage:
tests/test_audit_eval_run.py::test_audit_eval_run_replays_page_output_audit_artifacts.
Recipe-level page output-audit provenance is also now bound by a command-template
sidecar. tools/run_eval_recipe.py writes output-audit/command-template.txt beside
output-audit/output-audit-run.json, and tools/audit_eval_run.py fails if that
sidecar is missing or does not match recipe-run.json output_audit.command_template.
The same root audit now requires output-audit/output-audit-run.md and checks that the
Markdown report still contains the recorded output-audit row count and every per-page
output-audit line, so the reviewer-facing page audit report cannot go missing or drift
while JSON remains green.
tools/run_eval_recipe.py now also honors runner.output_audit.result_filename when
recording the per-page audit JSON artifact instead of hard-coding
limbu-output-audit.json. tools/audit_eval_run.py replays those recorded per-page
audit JSON paths and SHA-256 hashes, and the Markdown summary must expose the same
audit artifact hashes. A page output audit can no longer pass recipe-root audit by
return code alone after the per-page audit file drifts. The audit now also replays
each recorded per-page output-audit command from output_audit.command_template, so a
coordinated edit to recipe-run.json and output-audit/output-audit-run.json cannot
hide that a different audit command was used. The same records now include
stdout/stderr SHA-256 values for each page audit command, and the root audit
recomputes those log hashes and requires the Markdown summary to expose them.
Materialization command records now have the same treatment:
tools/run_eval_output_command.py records stdout/stderr SHA-256 values, the Markdown
summary exposes those log hashes, and tools/audit_eval_run.py recomputes them so
materialization logs cannot drift after a recipe run. The root audit also replays
materialization record commands from the recorded materialization command template, so
editing both recipe-run.json and materialize/output-command-run.json with a stale
command no longer passes. Missing recognizer/page fields needed for that replay now
fail audit instead of silently skipping the check.
Regression coverage lives in
tests/test_run_eval_recipe.py::test_run_eval_recipe_page_level_end_to_end.
Recipe validation now also requires Limbu live_detector page recipes to include
--require-no-dropped-image-lines and the --min-capture-prepared-* quality gates in
runner.output_audit, in addition to --require-capture-prep,
--require-no-pending-review, and the script-specific --require-script. This
prevents a future claim-ready recipe from accepting a precision-first filtered
image-line output that silently dropped detected page lines or a capture-prep bundle
below the declared quality floor.
The two current Limbu page recipes in configs/eval-harness-recipes-v1.json were
updated with those flags. Regression coverage:
tests/test_validate_eval_harness_recipes.py::test_validate_eval_harness_recipes_blocks_page_output_audit_without_strict_flags.
The operational image-line OCR bundle audit now also replays image-line-crops/manifest.jsonl
against image-line-ocr-run.json and the current crop files. It checks the recorded
crop-manifest path/hash, detected/kept/removed row counts, document kept-line count, and
per-crop SHA-256 values. It also checks kept crop rows against document.json line ids,
bboxes, prediction text, confidence, and embedded image-line crop provenance, so count
consistency is not enough to pass after stale row edits. The same audit now verifies
the image-line run source_path/source_sha256, model-config availability, and the
duplicated image_line_ocr metadata in document.json and page metadata, and it
binds image-line model_config metadata to limbu-output-manifest.json
provenance.model_config path/hash. This closes a stale-review-evidence gap where
image-line-ocr-run.json could keep pointing at an old crop manifest or where the
bundle could preserve a valid but different recognizer config provenance entry after
crop rows, crop PNGs, or document image-line metadata were edited.
Regression coverage:
tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_crop_manifest and
tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_manifest_against_document,
plus tests/test_pipeline.py::test_audit_limbu_output_replays_image_line_run_metadata,
tests/test_pipeline.py::test_audit_limbu_output_requires_image_line_model_config_provenance,
and tests/test_pipeline.py::test_audit_limbu_output_rejects_image_line_model_config_provenance_mismatch.
Current measured Limbu page baseline
The active goal has shifted from more pipeline hardening to measured real-page OCR
quality. The current diagnostic baseline is recorded at
outputs/limbu-measured-page-baseline-2026-06-22/README.md, with fresh audit snapshots
beside it. A fixed non-claim comparison set is now recorded at
outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-page-set-v1.json
and documented in
outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-page-set-v1.md.
It pins 5 real pages: 3 Devanagari-written Limbu CDC curriculum pages and 2 Sirijanga
Limbu pages. It is not a claim-safe result: all real-page references are still
unreviewed or machine-converted, the frozen set is diagnostic rather than
claim-eligible, and the page count is below the 30-unit claim threshold.
The current fixed-set rollup is
outputs/limbu-measured-page-baseline-2026-06-22/frozen-diagnostic-current-baseline-rollup-v1.md
with structured JSON beside it. It combines the current selected diagnostic systems,
not a single deployed model: CDC Devanagari image-line + Tesseract, Gorkhapatra
Sirijanga operational conf070, and Sikkim Sirijanga same-row merge. The five-page
diagnostic aggregate is codepoint CER 0.706734, grapheme CER 0.748005, true micro
WER 0.908270, line F1 0.801444, and matched-line CER 0.124367. Use this only as
a before/after target on fixed page identities.
Current measured signals:
- Devanagari CDC linebox draft with image-line + Tesseract:
0.865400codepoint CER,1.109798WER over 3 pages. The new Devanagari line-failure analysis inoutputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-devanagari-cdc-image-line-tesseract/says the dominant codepoint error source ismatched_good_geometry_bad_text(45.58%of attributed error mass), not missed lines. This is not competitive; wait for the active PP-OCRv5 Devanagari recognizer to finish, then re-score these pages. - Gorkhapatra Sirijanga operational live detector:
0.739361page codepoint CER, but only0.046585matched-line CER. Oracle line boxes are0.029268page CER. The refreshed failure analysis atoutputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-gorkhapatra-operational-conf070/has236matched lines,67missed references,2extra predictions, and83.86%of attributed codepoint error mass from missed reference text. Detector/layout recall is the next lever. - Sikkim Herald Sirijanga same-row merge:
0.418057codepoint CER versus0.891560for the guarded live detector, while oracle boxes are0.274289. The refreshed failure analysis atoutputs/limbu-measured-page-baseline-2026-06-22/line-failure-analysis-sikkim-same-row-merge/has77matched lines,9missed references,13extra predictions, and a mixed error profile: missed reference text is largest at31.61%, but extra prediction text is28.12%and matched-line text errors are still material. This needs both layout tuning and recognizer/domain adaptation.
Do not drift back into general provenance work as the next default task. The next useful block is: finish/admit the Devanagari recognizer when training completes, re-score the frozen diagnostic pages with it, and tune Sirijanga detector geometry on the same fixed page identities and failure buckets.
Current Limbu book OCR work
The active objective is usable Limbu book OCR, not detector tuning by itself. The
system needs to take scanned or photographed Limbu book pages, preserve page/line
structure, support Sirijanga Limbu and Devanagari-written Limbu, emit editable Unicode
book text, and flag unresolved regions for review. The current implementation now has
tools/parse_limbu_book_pages.py, which runs the existing page parser per page-manifest
row and aggregates book.md, book.body.md, book-review-queue.tsv,
book-run.json, and scorer-compatible score-documents/.
The current CDC h20 structural real-page replay is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay.
It is a replay through the book wrapper, not fresh OCR inference. It scores at page
codepoint CER 0.446556, page grapheme CER 0.626728, line precision/recall/F1
0.628571 / 0.709677 / 0.666667, missed/extra/blank line rates 0.290323 /
0.371429 / 0.000000, matched-line CER 0.175130, and reading-order pair accuracy
0.600000. Replay prep drops two empty selected-output science lines, improving
extra/blank line accounting without changing page CER. The body export bug for
structural page_marker lines is fixed in src/ocrtech/structure.py: full book output
keeps the social-page २, while book.body.md omits it.
The confidence-pruning sweep is documented in
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md. Results:
confidence >= 0.20 removes one low-confidence science false positive and improves
line F1 to 0.676923 with unchanged page CER; 0.30 and 0.40 improve precision/F1
but worsen page CER; 0.50 starts dropping matched lines and hurts recall. Do not
promote confidence pruning as the default OCR path. Treat 0.20 only as possible
review-triage cleanup. The next quality work is missed/split line recovery on page
regions.
One more same-row merge diagnostic was run after that: h20 page markers plus capped
same-row merge gap160, width500, y_tolerance=25. It produced 34 lines with 2
row merges and improved micro line F1 to 0.707692, but page codepoint CER regressed
to 0.460808, grapheme CER to 0.643625, and matched-line CER to 0.185259
(score SHA-256 61129eaa91445c0d8a86f66c5a5827ffa342b331bffe032d12f60d9476ad0b32).
Reject it. Same-row merge thresholds are not the next useful default; merged crops are
hurting recognition even when geometry improves.
The better direction is now post-recognition line assembly. New tool:
tools/assemble_page_ocr_lines.py. It merges adjacent same-row OCR line records after
recognition, records source line ids/texts/bboxes/confidences in metadata, and reads no
reference text. On the CDC h20 structural book replay, gap160 width500 merged exactly
two top-fragment pairs: maths img-line-0001|img-line-0002 and science
img-line-0002|img-line-0003. It improves micro line precision/recall/F1 to
0.727273 / 0.774194 / 0.750000, drops missed references 9 -> 7, drops extra
predictions 13 -> 9. The source-break variant preserves original page text exactly:
page codepoint CER stays 0.446556, grapheme CER stays 0.626728, and matched-line
CER is 0.177052. The space-join variant is closer to editable one-line output but
nudges page CER to 0.447348; assembly plus confidence >= 0.20 is worse at
0.451306. Treat source-break assembly as the current best line-structure score
projection, and treat space-join assembly as an export/review-stage question. Next
useful work: make the review packet/body export assembly-aware without silently
hallucinating missing connector text. The source-break assembly output is
replay-auditable with tools/audit_page_ocr_line_assembly.py; current audit passed
with 0 issues, SHA-256
69eb0db004f0bbabdead620bf63dfc138fb02deb8e3003fb9ec668a88d26afb1.
tools/parse_limbu_book_pages.py now exposes this path through --line-assembly.
Important implementation boundary: it does not rewrite audited page OCR directories.
It builds a book-level line-assembly-source-documents/ projection, writes assembled
documents under line-assembled-documents/, audits them by replay, and points
book.md, book.body.md, and score-documents/ at those assembled documents. It
also writes book-line-review-queue.tsv, which groups source review rows by editable
book line and preserves source line ids. The defaults are the measured source-break
settings (y=25, gap=160, merged width 500, component width 250, overlap
0.30, newline joiner).
The integrated CDC replay artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay-line-assembly-integrated
(book-run.json SHA-256
f2591d47ace1ceb5bc676d3d8c645f6905ab250e1e04fcfbe083ac4cac9f1e25). It has 27
source review rows and 26 book-line review rows because the two reviewed maths top
fragments collapse into one assembled book line. Its score is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-multi-page-cdc-h20-structural-book-replay-line-assembly-integrated/metrics.json
with SHA-256 184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244.
There is now a book-level artifact audit: tools/audit_limbu_book_output.py. It checks
book-run hashes, aggregate markdown, both review queues, score projection, per-page
output audits, and the line-assembly replay audit. The integrated CDC replay audit
passes with 0 issues; limbu-book-output-audit.json SHA-256 is
f5470c9ca04d7a3004f8ed1e3ea3fa280da768f29b65e1fd60fa7f8d42f2f6f5. This is an
artifact-consistency audit, not an OCR-quality claim.
Book-line review corrections are now actionable with
tools/apply_limbu_book_line_reviews.py. It reads accepted rows in
book-line-review-queue.tsv, writes corrected book markdown/documents and
score-documents, and emits book-line-correction-audit.json plus
book-line-correction-pairs.jsonl. It applies corrections to the assembled book-line
projection while recording source line ids; do not use it as evidence that source OCR
fragments were individually corrected.
There is also an explicit book OCR candidate gate:
tools/admit_limbu_book_ocr_candidate.py. It requires the book artifact audit,
required page/line metrics, blank/extra/missed-line rates, reading-order accuracy, and
improvement over the 0.458432 CDC baseline. The integrated CDC line-assembly run
passes operational admission
(outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-multi-page-cdc-h20-structural-book-replay-line-assembly-integrated/limbu-book-ocr-candidate-admission.json,
SHA-256 8e0110ad320cbe09f37893a30725c25db0cd3d40a2b1c77cc8f0b0ab982c6b9d) with
absolute codepoint-CER improvement 0.011876, but remains claim_safe=false because
claim_eligible_rows=0 and only 3 pages are scored. The next improvement should lower
missed-line rate / improve real frozen coverage, not just repackage this candidate.
The admission gate now supports explicit grapheme-CER and WER non-regression baselines
(--baseline-grapheme-cer, --baseline-wer; zero regression allowed when supplied).
This matters because the recognizer-probe auto-application is a mixed result, not a new
clean best. Selecting the rank-1 refined-crop machine prediction improves page
codepoint CER to 0.442597, line recall/F1 to 0.806452 / 0.769231, missed
references to 6, and reading order to 0.634409, but it regresses grapheme CER to
0.639017, WER to 0.866565, and matched-line CER to 0.186047. The paired
comparison against current best is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-detected-tall-recognizer-probe-vs-current-best/comparison.json
(SHA-256 5dff9bbce7158c56fd0489452f60ddd30f8b3cd04ddf31301852665ef4e5fbc8) and is
not claim-safe: codepoint improvement is 0.003959 over 3 pages with CI low
0.000000, while grapheme CER regresses by 0.012289. The strict multi-metric
admission at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-detected-tall-recognizer-probe-application-conf045-delta005-strict-multi-metric/limbu-book-ocr-candidate-admission.json
(SHA-256 812fb6e5d19553441e97eada4154879c369e4fcd3924d5717e537f77ef066cdb) correctly
rejects it for grapheme and WER regressions. Current best remains h20 source-break
line assembly. Do not promote the probe row without reviewed text or a recognizer path
that removes the WER/grapheme regression.
The candidate-only missing-line evidence is now also bound into a book-level review
artifact, so a reviewer does not have to know about the separate detected-tall packet.
New tool: tools/prepare_limbu_book_missing_line_review_queue.py. It reads the current
book output and a candidate-only prioritized review TSV/JSONL, verifies every candidate
sample id belongs to a book page, hashes each crop, and writes a separate pending queue
without changing book.md or score-documents/. Current code also writes
page_context_path and page_context_sha256 when the source page image is available,
with the candidate bbox highlighted on the full page; the audit replays those context
hashes. This keeps candidate insertions page-aware instead of forcing reviewers to judge
from isolated crops. Newly generated queues also write
book-missing-line-review-dashboard.html, a static reviewer dashboard with the isolated
crop beside the highlighted full-page context, and bind its SHA-256/card count in the
queue report. This is an operator review surface, not another eval harness. The real
historical artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-detected-tall-prioritized/:
4 rows, 2 high-priority body-line candidates and 2 structural page-marker candidates.
Report SHA-256 081cabed352868c26ad5d35fbc77bafc6d95b8ff27281de4d39484e15eadfc20; TSV
SHA-256 4ee99c552a00282e3f1bee0b4511d23b90b072d51adac15e829f9054d7237bf5; Markdown
SHA-256 fd318583ec70ba178fce876cb6f622ca9e7e35ac2aee77a4cf4af754ad66d01a. It is now
audited by tools/audit_limbu_book_missing_line_review_queue.py; the real audit passes
with 4 pending rows, 0 accepted rows, 4 verified crop hashes, no issues, and SHA-256
13c9ba67c47ac3bc9159606c72d3e77ce3d4fac1657ea563102d6b7078b49a58. This advances
the review-packet side of the book OCR objective, but it is not a text-quality
promotion. The rows must still be reviewer-accepted/corrected or solved by a better
non-regressing recognizer/layout path before they can enter editable output.
The application lane now consumes this book-level missing-line TSV directly:
tools/apply_candidate_line_reviews.py maps candidate_row_id/candidate_sample_id
to the older row_id/sample_id fields. A real pending smoke lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-application-pending/:
all 4 rows stayed pending, applied_count=0, skipped_count=4, insertion pairs are
empty (e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855), and the
application audit passes (SHA-256
cd824e7d7b1b7f7a4a439fe398716c9849acc2bcd3694a3f73124b6466268488). book.md and
book.body.md are byte-identical to the current best h20 assembly
(6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3 and
7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff). This is the
right no-op behavior until someone accepts/corrects rows.
The current reviewer-facing unresolved packet is now unified:
tools/prepare_limbu_book_unresolved_review_packet.py combines the book-line review
queue and detected-tall missing-line queue, renders crops for existing low-confidence
book lines, copies candidate-only missing-line crops, and writes one TSV/HTML packet.
Real artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/.
It has 30 rows and 30 verified crop files: 26 existing low-confidence book lines and 4
candidate-only missing-line candidates. Report SHA-256
c49c5d69105b24d36556b414f187626516353d2fc15de53fd5f098e7ad0d3adb; TSV SHA-256
d03c811db6fbac072491dd63bd48b36329eb30f873fac0f31495a6f907486a58; HTML SHA-256
f7f1f514fe77c92d763d8a59d19ce14c39019123b14e0aeecf6794caa077b45f. Direct integrity
check: 0 missing crops, 0 crop-hash mismatches. It now has an independent audit via
tools/audit_limbu_book_unresolved_review_packet.py; real audit SHA-256
3254beec46d99f9c5b6af9c72b39feefca8d97948e7e60f126088ee949886289, with
audit_passed=true, operational_review_packet_safe=true, 30 pending rows, 30
verified crops, no issues, and no warnings. This is review evidence only, not a metric
improvement.
The packet-to-application bridge is now wired. tools/prepare_limbu_book_unresolved_review_packet.py
preserves candidate_structural_role, so candidate page markers remain distinguishable
from body text after review. It also carries explicit split_corrected_texts and
split_bboxes columns for reviewer-approved split-line repairs. The regenerated
packet has report SHA-256
3af6d2b767aca7d566529a01f909e850231fca33cba5636c9b042008b07ed5c4, TSV SHA-256
9ecd72b9f804124357cef566e58cc655cac676d0e4c81421093aa6a98d416fed, and audit
SHA-256 396dc11cdb95ed3d632698fa9bf777e65f0ba260abbcd3debe1be2fec390e91c
(audit_passed=true, 30 pending rows, 30 verified crops, no issues/warnings). New
tool tools/split_limbu_book_unresolved_review_packet.py splits the unified packet
into the two applicator-specific TSVs under
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/split-for-application/:
book-line TSV SHA-256 83a40bdd0863d2cb38ed347c100493e24dcbf71b48e01b2d2fb373f6d2fbd890,
candidate-line TSV SHA-256 d6b5426520cccb95be66b9f0ef8c7daf220aa5fd0d9f4896b3682678fa6bca39,
split report SHA-256 586c688f704b1cc09bd93c096d51dc8ab1c7b252cc1cf0b6ae0eb436cf323050.
tools/apply_limbu_book_line_reviews.py now writes a chainable corrected
book-run.json, so the candidate-line inserter can run after book-line corrections
instead of requiring the original source run. The pending chain source run SHA-256 is
738e38a500450f7bd2fb1a516e1a42d08b0e688af9b6bb7b89ab87dfe71e07ef; pending
application audit SHA-256 values are
d06a569829174db6aa680e4a9716b9587485fcc6487860cd147fc5897b3e9411 for book-line
correction and 09dfec36a41bff5ee6c1b0287099734da7de264379f5d544da3cce16169c3eaf
for the chained candidate-line application. Pending-row smoke applications with
tools/apply_limbu_book_line_reviews.py and tools/apply_candidate_line_reviews.py
skipped all rows and kept book.md / book.body.md byte-identical to current best
(6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3 /
7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff). This closes
the mechanical gap from one reviewer-facing packet to a corrected-and-inserted book
projection, but it still does not improve OCR until rows are actually reviewed.
The review workflow is now one command via
tools/apply_limbu_book_unresolved_review_packet.py: split unified packet, apply
existing book-line corrections, then apply candidate missing-line insertions against
the corrected projection. tools/apply_candidate_line_reviews.py now writes a final
chainable book-run.json with updated page/document paths and line counts. Real
pending application output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-application-pending-unified/.
tools/apply_limbu_book_line_reviews.py can now deliberately split one accepted
existing book-line row into multiple TextLine records via split_corrected_texts,
using reviewer-supplied split_bboxes when available. Wrapper report SHA-256
d557307628c2b2c39a9df05d1dba056608003f10999021990457be163c70fbb6;
final book-run.json SHA-256 b9b7696471e28ca702385987a00ae05fa8a9df3b6f2018d894d75c1d83f421aa;
final independent candidate-application audit SHA-256
245b56e4be9e9f1c89f777b45d1f7583f113c9e13d113dbaf412fb4ac2cd0f34. The wrapper
itself is now audited by tools/audit_limbu_book_unresolved_review_packet_application.py;
real audit SHA-256 cb6e3b9748c20282ca045df766c8220293ef274bbf7cc3f665717be6a1fc2913
with audit_passed=true, operational_application_safe=true, 30 packet rows, 0
accepted rows, 0 book-line applications, 0 candidate-line applications, 0 issues, and
0 warnings. The real run split 30 rows cleanly (26 existing book-line, 4
candidate-missing-line), applied 0 because every row is still pending, and kept final
book.md / book.body.md byte-identical to current best. This removes operator glue
from the review loop; it does not promote text until rows are reviewed.
The missed-reference review coverage audit now exists:
tools/audit_limbu_book_missed_line_review_coverage.py. It compares frozen page-eval
reference_unmatched rows from line-alignment.csv against the unresolved review
packet by bbox overlap, so it is diagnostic-only and must not leak reference text into
OCR output. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-current-best-plus-unresolved-packet/;
report SHA-256 743d5c446797e0df0c360a78cea516bbcbb42cfb8de70843504ee0ac784ded07, TSV
SHA-256 ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37.
Result: all 7 frozen unmatched reference rows have review-packet coverage at the
region level: 2 covered_by_candidate_missing_line, 2
covered_by_existing_review_region, and 3 swallowed_by_existing_review_region.
This changes the next engineering move: stop chasing broad detector recall for this
candidate. The remaining missed-line failures are mostly geometry/segmentation inside
already surfaced regions, especially oversized maths rows swallowing short right-side
fragments. Next work should target bbox tightening / row-split handling for existing
book lines, then rescore on the frozen pages.
There is now a bounded split-suggestion aid for that reviewer work:
tools/prepare_limbu_book_line_split_suggestions.py. It reads the unified unresolved
packet, verifies crop hashes, and writes visual column-projection subspans for
oversized pending existing_book_line rows. It also writes an HTML review page with
original crops beside overlay PNGs. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-suggestions-current-unresolved-packet/;
report SHA-256 428eb35644115487aa48530db7b27d3854fcdb3c2603c49f1b3376c5fdba2f6d,
TSV SHA-256 b5890b32a355718666530c23085197e7a2404524bdbb78f04bc8da6921681c43,
HTML SHA-256 a9b416efb8ea5045e6bdd8af9723916274914b95f30621b05c616b84e0c129d1.
It now has an independent audit via tools/audit_limbu_book_line_split_suggestions.py;
audit SHA-256 9b7bdf8ae83ed79b2ee36b3d8031ebe26ce930a6321f996761bbd8dab9f81671,
audit_passed=true, operational_review_aid_safe=true, 19 suggestion rows, 109 total
visual subspans, 19 overlay PNGs, 0 issues, and 0 warnings. It covers swallowed-row packet IDs
existing-0005-limbu-devanagari-cdc-maths-p0008-img-line-0007 and
existing-0011-limbu-devanagari-cdc-maths-p0008-img-line-0013; the narrow
right-fragment packet row
existing-0007-limbu-devanagari-cdc-maths-p0008-img-line-0008 did not produce a
multi-segment suggestion under the current gates. Do not overclaim this artifact:
several suggestions are visual word/span segments or non-text visual spans, not
guaranteed logical book-line splits. They are reviewer hints for
split_corrected_texts / split_bboxes, not automatic OCR recovery and not a metric
improvement.
There is now also a split-prefilled unresolved packet:
tools/prefill_limbu_book_split_suggestions.py copies audited split suggestions into
split_bboxes for matching pending existing_book_line rows while leaving
review_status, corrected_text, and split_corrected_texts unchanged. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled/;
report SHA-256 9bc4d3f245133bc572c218536425ce3935b0839a5f0983176dce9853865437e7,
TSV SHA-256 b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4.
It has 30 packet rows, all still pending, with 19 prefilled split-box rows. The
prefilled packet was run through the one-command wrapper at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled-application-pending/;
wrapper report SHA-256 507bc6d51b6d1a1c299ec62e9f3ddaeb12773ecdbcda827caaaa7168f5a26598,
wrapper audit SHA-256 13763cd2fdb437e7addb6a830a808a957a756b1ed4eb88aa3fafb7e6891d1be1,
final book-run.json SHA-256 abb937646cfa736da056dc46d5edfba57392435b6be642388a16f6cd783a88ca.
The audit passes with operational_application_safe=true, accepted_counts={}, 0
book-line applications, 0 candidate-line applications, 0 issues, and 0 warnings; final
book.md / book.body.md are byte-identical to current best. Next useful work is
reviewed split text for these rows, then applying and rescoring the frozen pages.
There is now a segment-level transcription worksheet for that review:
tools/prepare_limbu_book_split_transcription_worksheet.py. It explodes the
split-prefilled packet into one row per suggested visual segment, crops each segment
from the source page image, records crop hashes, and writes TSV/HTML grouped by packet
row. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-current/;
report SHA-256 5eaa08ad0b2535d21989f2ece15dc65d8d36a46695670b5c326776b72f8cb01b,
worksheet TSV SHA-256 0b6e75a24515d404a7d8e33a2097b882f99af22732aca6eee1fdf3f1d84f2c0e,
worksheet HTML SHA-256 8da8216f287924df90cd206153e64cb9f75bdf35fe8f5cd6ccafc5c274732ebd.
It has 19 source packet rows, 109 segment rows, and 109 segment crops. Some visual
segments are non-text spans, so reviewers should mark those in segment_status rather
than transcribing them as text.
There is now a recognizer candidate-text layer for that worksheet:
tools/prefill_limbu_book_split_transcription_worksheet.py. It verifies the worksheet
report and segment crop hashes, writes a segment-crop label file, and merges recognizer
predictions into candidate-only columns while leaving segment_text empty and every
segment_status pending. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-recognizer-prefill-current/;
prefill report SHA-256 60796a0265abff4117701d489a3982dfdfeaf66f9050e2efff4302c8caf608f9,
label file SHA-256 1076d642bce34ddb2b4ff9059c42ced0728ba359bd19e1f2625b33ece7390b84,
prediction TSV SHA-256 ecf15827b3f8ce58338984421f93889f7150baf3cd2ac82c1824b209278b4ce7,
prediction summary SHA-256 c03e881417fbf26e9862b1d9316065cdaa904d18ab111c17f79e03546700b4af,
augmented TSV SHA-256 8a7cfe05fa8434178e6ee2464326e25e7ced465ee5da7594bf954d336c6c75e0,
and HTML SHA-256 fe8012dce16587551fc548ef6a870c58cf401692a66909f1127e754546355948.
It has an independent audit via
tools/audit_limbu_book_split_transcription_worksheet_prefill.py; audit SHA-256
2fa090dbdad4d6a36078c254828eec38799dfef3d50687a67af2b53315b3f32c, passing with
109 label rows, 109 prediction rows, 0 changed review fields, 0 issues, and 0
warnings.
The Devanagari CDC PP-OCRv5 recognizer ran over all 109 segment crops with 0 failures:
100 suggestions are above min_confidence=0.65, 8 are low-confidence, and 1 is empty.
This is review acceleration only. The compiler accepts this prefill report as an
alternate worksheet source after review edits, and the unchanged real prefill compiles
to 0 rows with output TSV byte-identical to the split-prefilled packet (pending compile
report SHA-256 f8cd51be1baaa288182a8ad30cf5966e2a1639fb6b0f9dbaa0e2b631e7119b4b).
That bridge now exists:
tools/apply_limbu_book_split_transcription_worksheet.py. It compiles completed
worksheet groups into split_corrected_texts plus matching pruned split_bboxes,
excluding segment rows marked non-text/drop/rejected, and can mark completed packet
rows corrected with --mark-status corrected. Current real pending output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-application-pending/;
report SHA-256 2f148e4b870be32648d5ee545c71b79626464f4457f3772f30ad100c7e168ec1,
output TSV SHA-256 b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4.
Since the worksheet is still unreviewed, compiled_count=0, all 19 groups are skipped
as pending_segments, and the output TSV is byte-identical to the split-prefilled
packet. Tests prove the completed path flows into
tools/split_limbu_book_unresolved_review_packet.py; next real work is actual segment
transcription, then running this compiler, the one-command applicator, and frozen-page
rescoring.
There is now a one-command reviewed worksheet application wrapper:
tools/apply_limbu_book_split_worksheet_review.py. It compiles the worksheet, applies
the resulting packet through tools/apply_limbu_book_unresolved_review_packet.py, and
runs tools/audit_limbu_book_unresolved_review_packet_application.py. Real pending
run:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-worksheet-review-application-pending/;
wrapper report SHA-256 96a5974c02d72c59d200895027a89a6069b701e9ee4d291ed5732c2bf8eadec3,
compiled-packet report SHA-256 137b53deaad0a0716fecc3d043382a88a0d9de0a716f47acf008111de50cf0ad,
application report SHA-256 fc3fe47f0badda0651689e92c2c45560e288840921ada0d8738bc4e14c4f75c3,
application audit SHA-256 d82737206bf8238dbb949470f58df7d55006f4a6ce532ae459360c89db8fa7a7.
Since the worksheet is unreviewed, compiled_count=0, accepted_counts={}, 0
book-line applications, 0 candidate-line applications, and final markdown is
byte-identical to current best. Fixture tests prove the reviewed path replaces a merged
line with split text lines and the pending path is a no-op. Next actual OCR movement
requires filling worksheet segment text, running this wrapper, then scoring/admitting
application/final-book/.
Two automatic recognizer-prefill diagnostics were run after that. First,
tools/promote_limbu_book_split_transcription_prefill.py promoted complete
high-confidence segment groups directly into split worksheet text. This is rejected:
the real run promoted 13 groups into 89 text segments, and strict admission failed
because line F1 collapsed to 0.171429, reading order to 0.421053, missed-line rate
to 0.612903, and extra-line rate to 0.889908 even though WER improved to
0.711653. Visual segments are not logical OCR lines. Second,
tools/promote_limbu_book_split_prefill_as_line_text.py joined complete
high-confidence segment suggestions into the original existing book-line text while
clearing split fields, preserving page/line layout. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups/;
promotion report SHA-256 abebf54f05c80c78d9c9d60459a132d4274e0dfb0d454a38c40bdaad782091bf,
promoted packet TSV SHA-256 72dee88f38b8d3f217b0cc706f3b8d3b5d2bdcb376c8331cf29a6c1ceddea8dc.
Application output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-application/;
application report SHA-256 1332aba3eeca1d83e400b523eca36fa972f195674c31b66c7b2d5461ca72e1a3,
application audit SHA-256 c482d60733a43e0981b38c8a84d8a8b4ed69167494d1117f064edd291f67c968,
final book-run.json SHA-256 930f0b0475a8b9459e104a145ca5f7e9634e68cb739d9df60b0e4829bc8509a0.
Strict score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups/;
wrapper report SHA-256 9c8b2383c63c56c7dc1e21ac35c775462b026e2557afa61edc9fc5ebb79ba13e,
metrics SHA-256 d277042a488bcc2956d2492150504614a88bde077e30f34a0a417afb7fc620c1,
admission SHA-256 a0279029f9c346bf51cbbdb5a3c89edfc9db68f0d37dfed49dd3d73e4c21a598.
This line-text diagnostic is admitted operationally with claim_safe=false: codepoint
CER 0.399842, grapheme CER 0.511521, WER 0.711653, matched-line codepoint CER
0.123640, line precision/recall/F1 0.727273 / 0.774194 / 0.750000,
missed/extra/blank rates 0.225806 / 0.272727 / 0.000000, and reading-order
accuracy 0.611765. Treat this as the current best operational diagnostic candidate,
not as claim-safe OCR, because the text was auto-applied rather than human reviewed.
The reviewer-safe version of that same path now exists. The line-text promotion tool
now writes an HTML evidence packet and can run with --review-status pending, leaving
all rows unapplied while storing the 13 joined recognizer suggestions in
corrected_text. New audit tool:
tools/audit_limbu_book_line_text_review_packet.py. Real pending review artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending/;
promotion report SHA-256 ce389aa01e11f95e1815cef01f2acbe6ff63debb7f74f58045c234c12ee4fada,
TSV SHA-256 ea2f56874371b0b146217d7fa83e883f5a2793ce5672b0038066a5837314d13e,
HTML SHA-256 08679a25155a34ac6578a8f0e55d0e153d770f8e22d6930e22282ea755d7f215,
audit SHA-256 acac7ff3b890c9cf888a1ca24ed1e3142d3ae55d097651c5ec65c3fcb2d4c8f3.
The audit reports 13 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. It also checks every TSV packet_row_id appears in the HTML,
so the four candidate_missing_line rows are now visible under "Other Pending Packet
Rows" instead of being hidden by the line-text view. For post-review edits, rerun the
same audit with --allow-reviewed; it permits accepted/corrected rows only when they
have explicit corrected_text and a non-machine reviewer identity, permits dropped
existing-book-line rows only with a non-machine reviewer identity, and records edited
TSV hash drift as a warning while still validating row/crop/document references.
No-op application proof:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending-application-pending/;
application report SHA-256 f5b143765f8c4cfdbc3d00f7c1100d2f2adbadac4d4a305e62fecd3f6b16d393,
application audit SHA-256 187d1e7653d97e869ba72128879f7dfaf5017dcd5539edfbddc3eb075901d4b2.
The normal unresolved-packet applicator applies 0 rows from the pending packet, and
final book.md / book.body.md stay byte-identical to the current best. Next operator
step: review the HTML/TSV, mark only verified rows corrected or accepted with a
real reviewer identity, then use the guarded wrapper
tools/apply_limbu_book_line_text_review_packet.py instead of applying the TSV
directly. That wrapper runs the line-text audit with --allow-reviewed, rejects
pending-only packets, rejects accepted rows with missing/machine reviewers or blank
corrected_text, rejects dropped rows with missing/machine reviewers, applies the
unified unresolved packet, audits the application, and can run frozen-page
score/admission with --score-out. Real smoke: invoking it on both
current pending line-text packets correctly fails with line-text reviewed packet has no audited accepted/corrected/drop rows ready for application, because no human-reviewed
rows exist yet.
Low-confidence calibration is now explicit. tools/promote_limbu_book_split_prefill_as_line_text.py
has --allow-low-confidence-above, leaving strict defaults unchanged. At threshold
0.60, it admits three borderline low-confidence segments (0.637542, 0.609537,
0.600665) and promotes 16 rows. The auto diagnostic passes operational admission but
is mixed: codepoint CER worsens to 0.406176 from the strict 0.399842, while
grapheme CER improves to 0.508449, WER to 0.706189, and matched-line codepoint CER
to 0.114738; line F1 and reading order stay 0.750000 and 0.611765. Evidence:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-threshold060-complete-groups/;
metrics SHA-256 f4e1aa3c32e93ec94c845aab7e4f335a9876c22e69cdd6f41ee57797abdd0f81,
admission SHA-256 f6c095b40839f3657aa507af2eecf1fc849b1eaa351f39c4bf4f88787afb5731.
Do not replace the strict 13-row codepoint-best diagnostic with this automatically.
Instead, use the threshold-0.60 pending review packet when a reviewer can inspect
borderline segments:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-review-pending/;
report / TSV / HTML / audit SHA-256
26cdf80ca5280fdc07440ab7bbaa73ab82d6b6aa633801e26de54312e24cc64c /
6f86cd9966d219e98f8182ef8a090adff41715a8492a573eb1316092142bb3b3 /
4c907d94c090cc79d83af760c8876d8584968a8ff25b5f4678a9b1fd24a59893 /
7ea916b354cc3e74761261f5adb7e6c8ceabc992956bf66463c2af189eff8405.
It has 16 pending suggestions, 30 pending rows, 0 accepted statuses, 0 issues, and 0
warnings. Pending application applies 0 rows and has application audit SHA-256
688820c7d9ce6b552b9dc53db68eeb65d809ef9206f11a76c4cfdec4055f85f5.
Preferred review packet for the next human pass:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending/.
It was generated with --allow-low-confidence-above 0.60 --suggest-candidate-missing-text,
so it has 20 pending suggestions: 16 existing-line suggestions plus 4
candidate-missing-line corrected_text suggestions. Report / TSV / HTML / audit
SHA-256:
4ae24b061e5d2c36bc467bafe31134c1c1d1e51de405ab21ebf25c8690f1f07c /
d4f4438160031a799cde0eecfb7cea4f8e85b662b3c4b356363131880379e364 /
3dd21d3dd677b68cc415d7df6da8892c1527bd626600e17f9282666c120b722b /
42fb1372dc06c7db3d6484044dfc2fc752a50573c9363880257b1206c43ba76e.
The pending application at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending-application-pending/
applies 0 rows and audits cleanly (application report / audit SHA-256
aa2a106ef8719c5e8f3094dced4ced1883b987a9db13fe6c45e0416acfaa49cc /
3ec74b70dec0233d8569b3dcc784aa4e1ca859673288fd76b666ca8cd1f7d51d). Use this packet
when reviewing both existing-line text corrections and missing-line insertions.
Missed-reference coverage audit for that preferred packet:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-threshold060-line-text-suggestions/.
Report / TSV SHA-256:
68565c47c276d53a00af08002b0f734d1cc4094da2ebb82ec7b88a4d2ee401c4 /
ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37.
The audit uses the frozen threshold-0.60 line-alignment CSV and the preferred
line-text TSV. It finds 7 missed references total: 2 covered by candidate_missing_line
rows, 2 covered by existing review regions, and 3 swallowed by larger existing review
regions, with uncovered_or_weak_count=0. This means the current missed-line failures
are review-visible, not solved automatically. Do not call the pending packet claim-safe
or use its machine corrected_text as final OCR until human-reviewed rows are applied
and rescored.
Important correction after checking extra predictions: the previous preferred packet
covered all missed references but left one high-confidence extra prediction uncovered,
limbu-devanagari-cdc-social-p0009/img-line-0001 (थिक्सिङ् - थिक्). The packet
generator now has --line-alignment-extra-predictions, adding prediction_unmatched
book lines as extra_prediction_line review rows unless they are already in the
confidence-driven review queue. The superseding packet for human review is:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-pending/.
It has 31 pending rows, 20 pending suggestions, 0 accepted statuses, 0 issues, and 0
warnings. Promotion report / TSV / HTML / audit SHA-256:
d5650dc1b67ae9881e6b4ccad5a5c1cacf8a744f02fc6af11bce3ce587bcda44 /
5e0e618adc8229fea734b8356be7aa3fe6f2c4f09737f9a5685ed15c284ac468 /
978fb40593d192b23c78f4f240e75c754cf29f404957ad104e6c4683d885ea50 /
e1053e353a4408c33008926a4923d3d25c93decbd943441d6e8fcc892f398b4d.
The raw unresolved packet also audits cleanly now. Report / TSV / audit SHA-256:
55ed8e0c6cfceeb609aeea3cf85862819488c6a59df422b935e817d36350d155 /
866e40ae4c5d037a347a3740c57610601f54071aa3deeefeed994f1a48d3a601 /
92d66a8333c387478b4d5bcb5d0fdbf51b231671aa5e1ce60c1ac8b0e26db5a2.
It has 31 pending rows, 31 crops, 4 candidate-missing rows, 27 existing-book-line
rows, exactly 1 extra_prediction_line, 0 issues, and 0 warnings. The raw-packet audit
now accounts for extra_prediction_line_row_count separately and verifies the
line-alignment CSV used by --line-alignment-extra-predictions.
Structural coverage artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-review-coverage-threshold060-line-text-missing-extra-suggestions/.
Report / missed TSV / extra TSV SHA-256:
ae9b7903a719110e77cfbe8089fc0cc0f025bd6f93804cdec4ac1d5a7b24b775 /
ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37 /
ad610c8019489fc03c041dfa1f0eb35e31bf76a304f8cf5731b3469752a21d51.
This coverage audit finds all 7 missed references review-visible and all 9 extra
predictions review-visible (extra_uncovered_or_weak_count=0). The guarded application
wrapper correctly rejects the packet while all rows are pending with
line-text reviewed packet has no audited accepted/corrected/drop rows ready for application.
The application path now supports actual reviewed extra-line removal. Existing
book-line rows with human drop / reject / delete statuses are removed from the
document, their line ids are removed from blocks, emptied text blocks are dropped, and
drop counts are propagated as dropped_counts and book_line_dropped_count through
the split, book-line application, unified application audit, and line-text wrapper.
Real smoke:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra/
marks only limbu-devanagari-cdc-social-p0009/img-line-0001 as rejected by
cdjk-review-smoke; all 20 machine suggestions remain pending. Smoke packet report /
TSV SHA-256:
1a8fc27bfc3fd9a3a3c47abc766df7abfb97301855c7f4b3725e2ef2722cb1b8 /
91287f19aaecf89c7735fad7221b71789c605d59bfc71296d7002ac959167120.
Application:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra-application/;
application report / audit SHA-256:
1806662aa1318a340df70067990e345a0e79e363a04ea8891c1858da837a5a01 /
bcfd4ef72fc93ed7c0a08d26a67df0f2953e44450249747f5a0c4f06f1326f9b.
It applies 0 accepted rows and 1 dropped existing line, audits cleanly, and scores via
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-missing-extra-smoke-reject-extra/.
Score/admission report / metrics SHA-256:
7501fe80695cd1adb87428a94ef3f9d7c6b416200279f3a6d0b16d57e37af08b /
7bccf3236644d2d73d2ddc7864f85de64dbe7681b91d41ad3fab7d48d8a45e89.
Operational admission passes, claim_safe=false; codepoint CER 0.436263, grapheme
CER 0.615975, WER 0.823739, line F1 0.761905, extra-line rate 0.250000,
missed-line rate 0.225806, reading-order accuracy 0.611765. Treat this only as a
pipeline smoke, because the reviewer identity is synthetic.
Mixed review smoke:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke/
copies the superseding pending packet and applies one existing-line correction, one
accepted candidate_missing_line insertion with text १, and one rejected extra
prediction, all under reviewer cdjk-review-smoke. Packet report / TSV SHA-256:
14ca2c76ea261c79730d96480d6e7a4bb901a807d24a306cc4aabb490d50983d /
7641025877f89c519d371bc3df613b4a98fa9a4874bf06c9b0ff2993a8602fff.
Application:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-application/;
application report / audit SHA-256:
8b1b983b34c65dba327788959a3200e176dc86639bb50a15f22b792cb5637c31 /
58095326d648202df10aecdda244b470fba2601c9bf5d82e2c9edeb563dd9617.
Application audit passes with accepted_status_count=2, dropped_status_count=1,
accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1},
dropped_counts={"existing_book_line": 1}, book_line_applied_count=2,
candidate_line_applied_count=1, and book_line_dropped_count=1. Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke/;
admission report / metrics SHA-256:
03adcbacac5f4f1fac68d9f73bf9c85c33f024931c6cff33ab3106f5878bfff1 /
e3ddf0d31b7bed313f9e22a05be0583e3afa7187cc8a29785b406f6b84b2cc5c.
Operational admission fails, claim_safe=false; codepoint CER 0.438638, grapheme
CER 0.614439, line F1 0.781250, extra-line rate 0.242424, missed-line rate
0.193548, matched-line CER 0.174901, and reading-order accuracy 0.553191.
Conclusion: the combined correction/insertion/deletion review plumbing works, but this
is not a promotable candidate. The next blocker is accepted missing-line insertion
placement and reading-order calibration.
That blocker has a narrow fix for structural page markers. tools/apply_candidate_line_reviews.py
now places reviewed candidate insertions with structural role page_marker before
body text, keeps ordinary text insertions geometry-placed, and records
placement_policy in the insertion metadata. Rerun artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-application/;
application report / audit / final book-run SHA-256:
bac62fc812eaed939ceecf7e36fd7ccb0a2329ae7644d2755df0077487c1b179 /
fc264875a4876dceadccbbdf221b3771726000a4e50518fba9df60d92b63ba40 /
879547c4827fee26900f5f7d49a7462df0793fd6d7a689bfecf90618f507b56a.
It audits cleanly with accepted_status_count=2, dropped_status_count=1,
accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1},
dropped_counts={"existing_book_line": 1}, book_line_applied_count=2,
candidate_line_applied_count=1, and book_line_dropped_count=1. Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter/;
admission report / metrics SHA-256:
43705c1d33ce7c5f23d04452a364cf1316efa084b90ea6b67b35f458c754a60a /
a3718bbab380b2c44d406d938db00bf3fdaa2021d24d1f683eb944419a76d65d.
Operational admission passes, claim_safe=false; codepoint CER 0.435471, grapheme
CER 0.608295, WER 0.812810, line F1 0.781250, extra-line rate 0.242424,
missed-line rate 0.193548, matched-line CER 0.174901, and reading-order accuracy
0.648936 (61/94 matched-line pairs). This remains synthetic smoke evidence, not
human-review claim evidence, but the mixed review path now passes the operational gate
without relaxing thresholds.
Structural front-matter ordering is now applied in the real projection paths, not just
candidate insertion: src/ocrtech/structure.py, src/ocrtech/image_lines.py, and
tools/apply_limbu_book_line_reviews.py sort explicit page_marker blocks before
body text and treat bottom-margin Devanagari digit-prefixed rows like २ ... as
front-matter rows for cases where the page marker is fused with neighboring
title/header text. Rerun artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-row-application/;
application report / audit / final book-run SHA-256:
c7ea956cdec75c2aefc9260e0e30c30710f77f5e84bc19b6be53948b3e103408 /
8efc363d1895ed955e8fc692fb10ae1a6ff312f9f146464065e8eb6696a9a657 /
d012af48bb7c64d65572cfa70ff135a6a8321aa1658c29ae1c26467fb82fedba.
Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter-row/;
admission report / metrics SHA-256:
47ee4098aaa2fc92c9d053c1fc8cfeb3093528a5556dcf450656af8c6b356837 /
7201ed8390b0bce13ecc5f7aca6643b1b6d77f7be99c1b7b12548460f3150182.
Operational admission passes, claim_safe=false; codepoint CER 0.455265, grapheme
CER 0.632873, WER 0.821451, line F1 0.781250, extra-line rate 0.242424,
missed-line rate 0.193548, matched-line CER 0.174901, and reading-order accuracy
0.787234 (74/94 matched-line pairs). This is a layout-order tradeoff, not a final
OCR win: reading order improves, but raw page CER is worse than the page-marker-only
run because pending low-confidence text moved into reference order before human
correction.
The text-first best candidate was rerun through the same structural ordering logic:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-application/.
It applies 13 existing-book-line corrections, leaves 17 packet rows pending, and
applies no candidate-missing insertions. Application report / audit / final book-run
SHA-256:
d3821317c37f4c914febdfce29e9e113f383d074d2132bd85b619fa68add7ef3 /
5ebad12995fdb520dcd98daf02c614f0e3d8d6fbff23db441bead8804d4ce05b /
902dce10d52e2b54ee9bf3e6ed2e2e0b5d15ef746ad921a28df5c20610fc06bb.
Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order/;
score/admission report / metrics SHA-256:
f24b4f8a3abd66ac89c475ed9cc98e54a402e2369b587a0780dc19cbf85473a7 /
979827dc2ccdbc4f6c3ea45ee97c1bcc0151200f8c5eaeafb31a7928c72524d4.
Operational admission passes, claim_safe=false; codepoint CER 0.404592, grapheme
CER 0.520737, WER 0.722582, line F1 0.750000, extra-line rate 0.272727,
missed-line rate 0.225806, matched-line CER 0.123640, and reading-order accuracy
0.682353 (58/85 matched-line pairs). This is now the better text-first diagnostic:
it keeps most of the old best-text CER while improving reading order, but it does not
recover missed lines.
The page-marker selector has also been repaired for the newer unified TSV packet shape:
tools/select_candidate_line_reviews.py now accepts TSV as well as JSONL, derives page
size from each candidate row's book_document, parses max_selected_iou from
review_reasons, and checks duplicate markers against the post-review text implied by
accepted existing-line corrections from the same TSV. On the complete-groups packet it
selects maths १ and science २, because accepted row existing-0019 removes the
leading digit from selected img-line-0014; the two body candidates remain rejected.
Selection report / selected JSONL / rejected JSONL SHA-256:
f52f61ab09ba6f2101ae8049815a01257dd5b57e93c9aab492f2125fefbf2058 /
7e7b37359d5b4dc8972220054b33f2ffeb388ff7d02a621972035a19a573baa9 /
40a9ae5a0fb174694cdc1e04d108adb52900b2864bce43aabf4557878f21feaf.
The applied candidate
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application/
has applicator audit / independent application audit / book-run SHA-256
26dc4049979618330eb8016fad699af82465719d7e88749540309adcc40261cc /
e135fcb2faa7b1a125c2f706fae41e1fb448dfe918390f434f5075895fc540d4 /
61d6dfc09e1b7e09f0fd1cf57b5aac988b6c8e65f567f2eda6c19e33928ff775.
Score/admission
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2/
passes operational admission, claim_safe=false; score/admission report / metrics /
admission SHA-256 are
9a4f3bca208d515b1e7538536a1abb54000749d4694e2317a981ca3588064ba3 /
fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4 /
1bbd1d216d3c3b23def126332aecf1052abf254a0f15891dd58048c103cc2205. Metrics:
codepoint CER 0.402217, grapheme CER 0.516129, WER 0.711653, line F1
0.757576, extra/missed 0.285714 / 0.193548, matched-line CER 0.123518,
reading-order accuracy 0.712766 (67/94), 3 structural page-marker predictions, 2
matched structural lines, and 2 exact structural-text matches. This is now the best
operational layout candidate, but not the actual endpoint: body-line candidate text is
still noisy, and the next gain needs reviewed body-line recovery or a real line
split/remove path.
The body-line recognizer-probe lane was refreshed on top of that page-marker-v2
candidate, and it should not be promoted. Refreshed crop probe:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-text-crop-probe-from-complete-groups-page-marker-v2/
(probe / JSONL SHA-256
9e64044df9ec4070dcc28b78a2e0f48c2fd6cb8009fdab1de41a36eb2591a61c /
31e8ecbf17ab207c67c3f46510f66771e4f2802315cf052ee52aaaa858ae68d4). It uses the
two body rows rejected by the TSV-backed page-marker selector; refined crop hashes are
byte-identical to the older probe, so the existing PP-OCRv5 prediction TSVs are valid.
Refreshed comparison:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-crop-recognizer-probe-comparison-from-complete-groups-page-marker-v2/
(comparison / JSONL SHA-256
12fd030c336f3d1cdef882de8512f56a5f58e7bd513d301b4dab0fd95418b329 /
f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d).
The non-oracle selector
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-recognizer-probe-selection-conf045-delta005-from-complete-groups-page-marker-v2/
selects one body row (confidence=0.469282, delta 0.218220) and rejects the other
for low confidence/delta (selection / selected JSONL SHA-256
41655846f1845d1a966a72e7def29b38e58bee53bf5472493a6c3bda536e1dc1 /
205415b49c984145cbf573aa1e7a65547a931a202bc8789daa58ebe10d64c2b4). Applying it
required fixing tools/audit_candidate_line_review_application.py so chained
applications count only insertions from the current review file and report inherited
candidate insertions separately. The real body-probe application has applied_count=1,
inserted_line_count=1, total_inserted_line_count=3, and
inherited_inserted_line_count=2; application audit / book-run SHA-256 are
d89b9c8966c5c1a01e74bb5edbe3ab19c08054b9f915d9aa5fad1863199f6d99 /
abe8f59007aa77fb13bf690070230346aa14dc88040fa2b18b8d80ef2fd332de. Score/admission
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe/
passes against the old CDC baseline (claim_safe=false): codepoint CER 0.441013,
grapheme CER 0.562212, WER 0.760834, line F1 0.776119, missed rate 0.161290,
reading order 0.656863 (score / metrics / admission SHA-256
b52a853cf05196c618a416fc981cf747b05cc178a6435fc35a0d497380611992 /
e2307c02624fd601f2ad5674b3405e62620c2565993d3dcbf2425e5b8fd74f79 /
2b55cb5073c523c2aa69c69f52251452a80ab89e762554074b29d21361db8b6c). However,
paired comparison against page-marker-v2 current best at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-body-recognizer-probe-vs-page-marker-v2-current-best/
shows codepoint CER regression -0.038797 and grapheme CER regression -0.046083
(comparison / paired TSV / markdown SHA-256
1bafa22b69296c98f9186ade50cce68d509491e0b77de8be80c45bb9a4d3a209 /
cd89984c2dfc9a8e6792b15b1b193348716d9a5c0c140d048278843fbead732f /
38104641a4c8744233fa90669395f994047121e21aba25c7af7602741dccf3a8). Interpretation:
this is useful review evidence and an audit-chainability fix; keep page-marker-v2 as
the selected operational layout candidate.
Admission has now been hardened against this exact false promotion. The old gate could
admit a candidate that beat the historical 0.458432 CDC baseline while regressing
against the actual current best. tools/admit_limbu_book_ocr_candidate.py now accepts
--champion-metrics, and tools/score_limbu_book_ocr_candidate.py forwards it; when
set, codepoint CER, grapheme CER, and WER must not regress beyond the configured
--max-champion-* tolerances. Real check: the body-probe candidate fails at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe-champion-gated/
with champion deltas -0.038797 codepoint, -0.046083 grapheme, and -0.049180 WER
(wrapper/admission SHA-256
4126d1ca9c4e644190d862301f199f3f46335fb1c7b48191209c6e70e3f46214 /
ff1425910714bacd54f96640d8f2281c08d78c93f07b0d43c7be3117d9a1c4e1). The
page-marker-v2 self-check passes at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-gated-self/
with zero champion regression (wrapper/admission SHA-256
6103f40ec19bdde710fd1438ff8547382118fbbdadc9f96ce65425767a5ce75d /
3b22736a064654783b9a8cee5a455936d7e86cc1d759d1e7536bb5ea69eb727c). Next sessions
should use champion-gated admission before promoting any new book OCR candidate.
The unresolved reviewer packet has also been refreshed against page-marker-v2 current
best. tools/prepare_limbu_book_unresolved_review_packet.py now accepts JSONL missing
line queues, so the selector's rejected candidate rows can be used directly, and it
filters non-pending rows out of unresolved packets. This fixed a real stale-queue
problem: the current book run still points at a queue containing 13 already-corrected
rows. Current packet:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-extra-predictions/.
It has 18 pending rows and crops: 13 existing low-confidence book lines, 3 extra
prediction lines from page-marker-v2 alignment, and 2 rejected body-line candidates.
Report / TSV / HTML / audit SHA-256:
1e004bb5c6b206a2ad3faf147e1f54cea3c7d442300b71be295a3c2d9743c5ab /
fe8908da4050e90f17d4a6ccbf111ba04b5752491bccb19a01d11b5853bdf84a /
17694e6aa1224dec5e3edc12cb742e07d0be9989aa66f9718399d48e3fef23c7 /
10e95db8204c47bb4fec39264e2914510fed82544221f541f486bebf3f3d26be. The packet audit
passes with 0 issues. Pending application:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-application-pending/
applies 0 rows, audits cleanly, and leaves final book.md / book.body.md
byte-identical to page-marker-v2 current best. Use this packet for the next reviewer
pass; do not go back to the stale h20/current-best-plus-detected-tall packet unless
you are intentionally reproducing old evidence.
Coverage for that packet has been refreshed too:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/.
Against page-marker-v2 line-alignment.csv, 6 unmatched reference lines are all
review-covered: 1 by candidate missing-line row, 2 by existing review regions, and 3
swallowed by larger existing review regions. uncovered_or_weak_count=0. All 10
unmatched predictions are covered by existing review regions, with
extra_uncovered_or_weak_count=0. Report / missed TSV / extra TSV SHA-256:
32026349ec549e142036042cf3b6bc2713a41b0438f4c5220b3afb082d7e7abc /
596c77aa6770a28467dafa89a8f7122bd080bb274ef98bacdfba7521566d8ad8 /
95f41b789b6ae9aadf93b4273d1c96b7a39f2e119f1500e7e8187ec9ee350b91. The current
review packet is therefore coverage-complete for current missed/extra line surfaces;
the blocker is now reviewed correction or safe automatic recovery, then champion-gated
score/admission.
That coverage evidence is now part of the candidate admission contract. The admission
tool accepts --review-coverage-report with default maxima of zero uncovered/weak
missed references and zero uncovered/weak extra predictions; the score/admit wrapper
forwards the same option. Current page-marker-v2 was rerun through both champion and
coverage gates at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/.
It passes operational admission, claim_safe=false, with zero champion regression and
coverage values {missed_reference_count: 6, uncovered_or_weak_count: 0, extra_prediction_count: 10, extra_uncovered_or_weak_count: 0}. Wrapper / metrics /
admission SHA-256:
0018bfb5ee2f48821c3c9b1101d0921bbad54c41cd63be120dc72437fe9f1344 /
fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4 /
b9ece419592b3bfc3dbc892880c206a2cbcb24a8777e0c245d1b7614afa30f43. From here on,
do not promote a book OCR candidate unless it passes champion-gated and
coverage-gated admission.
The two-script coverage registry now points the Devanagari-written Limbu page target
at this real operational candidate instead of the stale live-Tesseract page recipe.
configs/eval-harness-recipes-v1.json adds
limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated, and
configs/eval-target-coverage-v1.json sets the limbu-first-two-script-ocr
Devanagari target to eval_status=registered_real_pack with
page_eval_recipe_id=limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated.
Validation is clean but not claim-ready:
uv run python tools/validate_eval_harness_recipes.py --recipes configs/eval-harness-recipes-v1.json --root . --out outputs/eval-harness-recipes-current-validation.json
reports recipes=12 valid=True claim_ready=0, and
uv run python tools/validate_eval_target_coverage.py --coverage-config configs/eval-target-coverage-v1.json --normalization-config configs/recognizer-eval-pack-normalization-v1.json --eval-recipes configs/eval-harness-recipes-v1.json --root . --out outputs/eval-target-coverage-current.json
reports valid=True coverage_ready=False target_sets=3 gaps=50. The new Devanagari
page recipe itself is valid with zero structural issues, but its claim blockers are
real: only 3 CDC pages, 0/3 claim-ready references, curriculum_text_unreviewed GT,
unfrozen/non-claim-eligible page pack rows, missing hash-verifiable checkpoint/config/
dictionary artifacts, claim_safe=false, and 18 unresolved review rows. The top-level
limbu-first-two-script-ocr set now has 6 remaining gaps: Limbu/Sirijanga
normalization, recognizer recipe, and page recipe are not claim-ready; Devanagari
normalization, recognizer recipe, and page recipe are not claim-ready. This is progress
because the registry now represents the real current Devanagari page candidate, not
because the two-script OCR claim is ready.
The Sirijanga page recipe also had an avoidable harness defect: it was declared as a
sidecar_fixture page pack but still had a placeholder REPLACE_WITH_LIMBU_PAGE_OCR
command. tools/copy_page_ocr_document.py now provides the missing reproducible replay
step by copying a pre-materialized {sample_id}/document.json bundle from a source
documents directory into the recipe-run documents directory while checking the input
page path and recording hashes. limbu-sirijonga-gorkhapatra-page-current-router now
uses that tool to replay
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-2026-06-21/operational-image-line-width480-minwidth005-conf070-documents,
has requires_operator_resolution=false, and records concrete checkpoint/config/
dictionary paths. A real recipe smoke at
outputs/eval-recipe-smoke-limbu-sirijonga-gorkhapatra-page-current-router/ succeeds
with audit_passed=true, claim_safe=false, ocr_output_kind=sidecar_fixture, 1 page,
codepoint CER 0.739361, grapheme CER 0.768262, WER 0.918969, line F1 0.872458,
matched-line codepoint CER 0.046585, and matched-line reading-order accuracy
0.775478. This removes the unresolved-command blocker; it does not make the page
claim-ready, because the run is a replay of one machine-converted unreviewed page with
0 claim-eligible rows.
The current selected Sirijanga diagnostics now have textless unresolved-region review
packets, matching the book-OCR goal's requirement to surface missed/extra-line regions
instead of only quoting CER. Rollup:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/README.md.
Gorkhapatra operational conf070 has 69 rows/crops: 67 missed references and 2 extra
predictions; audit passes with 0 hidden-reference leaks and 0 issues. Refreshed packet
/ JSONL / HTML / audit SHA-256 are
a503d752571efedc7ccb1ad43f3826c9f6c9744a7244f444a855136b4d252b2a /
af8deaa96f9f76ae34342583fd4166c9045637896a085c1387b5ae8dc25e65f2 /
54739ca9f38cd72b31b9927424a0afd33719ba8f87ac8cc450d9472d4173523f /
78dc40e5d99e5c6e5d377296048e73f21660d2660ecd3617c59beba5629cad6d.
Its failure surface is center-column missed text: missed reference text is 83.8643%
of attributed codepoint error mass, led by middle/center, bottom/center, and
top/center bands. Sikkim same-row merge has 22 rows/crops: 9 missed references and 13
extra predictions; audit passes with 0 hidden-reference leaks and 0 issues. Refreshed
packet / JSONL / HTML / audit SHA-256 are
d84ac4662411a59bff76987566f5207f73b343dc1f50f1ab49a18f66914c65b4 /
03798089ab870985388d72abe7ed545c137c172f333bad4b08f00e73217815cc /
470beaf4310aa26e3a1ea272822ae25cf204775ae4afbc31e39bf0808cbca8e5 /
544cd11e33d617579f5f93256c249975f9bb3a692cd6857356a51cbdcf4388e3.
Sikkim is a mixed failure: missed reference text is 31.6140% of attributed error mass
and extra prediction text is 28.1198%, with top header/large-line misses prominent.
tools/apply_unresolved_page_review_packet.py now applies reviewed page-packet rows to
scoreable page-document projections. Pending rows are no-ops; accepted/corrected
missed-reference rows require corrected_text; accepted/corrected unmatched
predictions update only when corrected_text is supplied; false positives are deleted
only by explicit drop/delete statuses. Pending application smokes exist for both
diagnostics and score identically to the selected source outputs: Gorkhapatra
0.739361 codepoint CER and Sikkim 0.418057 codepoint CER. Neither packet nor
pending application is claim evidence; they are review/recovery substrate. Coverage is
now independently audited with tools/audit_unresolved_page_review_coverage.py:
Gorkhapatra covers 67/67 missed references and 2/2 extra predictions, and Sikkim
covers 9/9 missed references and 13/13 extra predictions, with zero duplicate or extra
packet keys. Coverage report SHA-256 values are
bf36f850104d2b993349ebc8d53a22bd1835a8cfbd8fc6860ec45a0fdf9b8045 for Gorkhapatra
and 6de438a4cf7ae08ed0fff29f834622d6024f0e53dccb6923c830841c48d67a7b for Sikkim.
The latest Sirijanga Gorkhapatra candidate-line diagnostic proves the next lever is
reviewed/corrected missed-line recovery, not blind promotion of high-recall detector
branches. Selected conf070 page documents compared against the rejected
auto-split-tall candidate set produced 100 candidate-only rows; the packet audit
passes with 0 issues (candidate-line-review-packet.json SHA-256
68f678a77e0b2056c25ecbc1595cf62e63c4ef72aa923d79355a4a9886998ff4, audit SHA-256
ac645bb4755bab878c4e65749735d86856c0dcb9e473399bf064db869e8df56c). An
oracle/reference-overlap selector chose 49 rows against the 67 currently unmatched
references (selection SHA-256
45fa5a781d62736a2aa3ef5c97762755d0b65ad99f1c7984856e585ab30e5128), but this is
diagnostic-only and not deployable policy. Applying those rows with candidate OCR text
through tools/apply_candidate_page_line_reviews.py inserted 48 lines, skipped 1
empty-text diagnostic row, and reached excellent geometry metrics: line F1 0.964346,
precision 0.993007, recall 0.937294, 19 missed references, and 2 extra
predictions. The text output is worse than selected conf070: page codepoint CER
0.766779 and grapheme CER 0.798907 (metrics SHA-256
0467192299d469d02e3ad1b14c9be669a6561e08263098da858d337ea3267da8). Do not
auto-promote auto-split-tall candidate text. The safe non-oracle artifact is
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-prioritized-candidate-line-review/:
100 body-line candidates, 99 high priority, 1 medium, audit passed with
operational_review_queue_safe=true (audit SHA-256
0c9cde56dbec68f4a146cf417cee033271814ce7c5887058841028798f2a6985). Use it as a
review/transcription queue or as input to a better crop recognizer.
The first non-oracle crop-probe bundle for that queue is also prepared at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-text-crop-probe/.
It processed 100 body-line candidates, refined 44 by trimming top bleed from multi-row
crops, and copied 56 as full crops. Report / JSONL / label file / HTML SHA-256:
dc5de12cda741533cc31e343147b4e12488b4ee1611e1e8cdb81a038206725c3 /
9ad687ca53e229565132dc579a79efd6adb6572aadad54fc5c61294fceca3fc5 /
2c5ae66ea3468f97f456ebcfa4f979703e65ded921f3850054b357cbeb015f2b /
46ae06c2412961b02fb9f9f8db7b30fd81afcc702cb8fb79b672fe7fe41094c8. Use this for a
targeted Sirijanga crop recognizer/reviewer pass; it does not alter selected OCR text.
That pass has now been run. tools/predict_recognizer_eval.py had to be fixed first:
Paddle sometimes returned multiple text lines for one crop, and the old writer emitted
raw embedded newlines into predictions.tsv. The tool now flattens newlines/tabs
before writing a TSV field. With that fix, the packaged v4 Sirijonga recognizer ran on
100 original candidate crops and 100 refined crop-probe images with 0 failures
(original/refined prediction TSV SHA-256
8f32f8895ded0a4a1c405333206ebfc55df37606ff1b288ef4319c7067060fc0 /
66c4fa2d25b46bf24c99ad9936000ec0b73a932478cf1e74e613cabe09f79fd3). The comparison
has 100 rows, 44 refined rows, 40 changed predictions, max confidence 0.992109, and
automatic_application_safe=false (report SHA-256
8b2133b39c3d139b8a83d10858301bcad0e92cd821a4042bbf4aeb0bdf27064d). A threshold
sweep gives one near-miss but no clean default: 0.90 confidence / 0.05 delta
accepts 6 rows and improves page codepoint CER to 0.737090, but line F1 drops to
0.866545, missed references remain 66, and extra predictions rise to 7. Do not admit
this as book OCR output; use it as reviewer/crop-recognition evidence.
There is now an explicit effect audit for this class of mistake:
tools/audit_candidate_page_line_application_effect.py. It compares a candidate
page-line application to the selected baseline across page CER, line F1, missed/extra
rates, reading order, and inserted-row alignment. The strict 0.90 / 0.05
recognizer-probe application is rejected with book_ocr_objective_safe=false: 6
inserted rows, 1 matched row, 5 extra predictions, and the matched insert itself has
line codepoint CER 0.888889. Effect audit SHA-256
08eff93810712cafe46009ac8aa5a1d6dcd00c3206f2cd5b88aff527c7e0cd82. A page-CER-only
gain must not override this gate.
The latest feature analysis makes the next move sharper.
tools/analyze_candidate_line_review_oracle_features.py compared the 100
auto-split-tall candidate-only rows against oracle selected/rejected labels without
writing hidden reference text or OCR text into the safe feature output. Artifact:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-oracle-feature-analysis/
(report / TSV SHA-256
4c45a640338b6510b63bced5b903b847e4725ab87f544fb36ba7def7ac2e2bb0 /
21e31ff240b2a40220b70b9f029237064b83c951509b6bea62ec9fc4855098bf). The 49
oracle-selected rows are longer and wider than the 51 rejected rows: candidate text
length mean/median 29.306/37 versus 4.471/1, bbox width mean/median
295.531/307 versus 127.137/74. They cluster in body-like bands. Refined
recognizer confidence is a bad selector here: selected rows have mean/median refined
confidence 0.382649/0.146180, while rejected rows have 0.601181/0.751105; all
tested confidence/delta selectors top out at 0.500000 oracle precision and
0.183673 recall. Do not run another confidence-only promotion loop. The next
worthwhile automatic experiment is a non-oracle geometry/text selector plus the page
effect audit.
That experiment has been run. tools/select_candidate_line_reviews_by_geometry_text_features.py
selects candidate-only rows from packet-visible geometry/text features only. With
min_candidate_text_length=30, min_bbox_width=80, and y-band
middle,bottom_or_right, it selects 31 rows; the oracle-overlap diagnostic says all 31
are real missing-line candidates. Applying their machine candidate text gives the good
geometry surface we wanted: line F1 0.933566, recall 0.881188, missed references
down to 36, extras still 2, and all 31 inserted rows matched. It is still rejected as
book OCR because page codepoint CER worsens to 0.761733, grapheme CER to 0.792281,
matched-line CER to 0.091202, and reading-order accuracy to 0.604292 (selection /
metrics / effect-audit SHA-256
9469f67f8eb1afb300e25bc332e9566fd92f583b77fbc054953027b9883dd444 /
22689ccec2f4cdf7b0f9f0e4679ef13756b3423d81b5c45d65b9b9779de60e6d /
0ba763fb14cfd1e63ff6eb49313ea668ed908986c2533319e18aa4d148364988). A looser
min_candidate_text_length=10 setting selects 38 rows, 35 oracle-useful and 3 extras,
and also fails: codepoint CER 0.764760, line F1 0.936097, extra predictions 5,
effect audit SHA-256 5f4aaef0a472f95757342a3da00ca932eab3453b286d9afb40ba61940cb4d6f2.
Takeaway: we have a good review/crop-targeting selector, not an automatic text-output
policy. Next work should improve/transcribe those 31 rows and repair reading-order
insertion, not tweak selector thresholds.
Candidate insertion ordering now has explicit modes in
tools/apply_candidate_page_line_reviews.py. Keep old geometry for reproducibility,
but do not use it as the operational default on this wide-column page: it globally
sorts by raw page/y/x and drops reading-order accuracy to 0.604292. The new
preserve_source_append mode preserves the selected source block order and appends
accepted candidates. On the 31-row precision selector, it keeps all 31 inserted rows
matched and improves reading-order accuracy to 0.750472, with line F1 0.933566,
recall 0.881188, and extras still 2. It remains rejected because candidate text
worsens page codepoint CER to 0.762153 and grapheme CER to 0.798575 (effect audit
SHA-256 56068accf83105e4bd0f526f2ddac0d68fb9a234d52899e35b3ec4bae5ce2743). The
source_reading_order mode re-sorts with wide_column_right_to_left but is worse here
(codepoint CER 0.773003, reading order 0.722227, effect audit SHA-256
b62ae4e0be13247a0a8c6af34656058c9412c3bb4fce1b2afd922189511c658e). The practical
choice for reviewed candidate insertions is preserve/append until a better slotting
policy is proven.
The 31 precision-first rows now have a pending transcription worklist:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist/.
It was created with tools/prepare_candidate_line_transcription_worklist.py and
contains JSONL, TSV, HTML, and label files. It is now audited by
tools/audit_candidate_line_transcription_worklist.py: all rows are pending,
corrected_text and reviewer are blank, no oracle/reference fields are emitted, crop
hashes validate, and 31/31 HTML crop links resolve. Report / JSONL / TSV / HTML /
labels / audit SHA-256:
7a92ed3d85faee782f22353a778fa36434c6456ca29182a462349ffcc630b180 /
a98488a08ed6d28944b171fe50577a9958960bd31cb6cc7180a58b3e5b270ea1 /
f405d1ede74174f0def525bfa3c5bb9819a3c929671cc74d366a5769e48a31b6 /
3971d08d5bca15286fcceae6326014ad5edbab89a8be1f2656d5e69703cbee72 /
1bcf9418af1ee056ccf32bfe56fccc7d51d26ffdef52f51c1f24397eadced5c0 /
86678953c170569196585aaaf25d6d44fb8d031a7c35f705ad791797d1a0d541. This is the next
human/reviewer or crop-recognition target.
The reviewed-worklist bridge is also in place.
tools/apply_candidate_line_transcription_worklist.py compiles filled worklist rows
into candidate-page application rows, requiring non-empty corrected_text, a
non-machine reviewer, valid crop hashes, and accepted/corrected status. Pending rows
are no-ops. The current real 31-row worklist is pending-only, and the pending compile at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist-application-pending/
has accepted_count=0, skipped_count=31, issues=[], and
application_ready=false (report / accepted JSONL / skipped JSONL SHA-256
a5116e0509ab3612fb83f21298397d57f10a92f32a4f2936f6a2284050577fce /
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /
2f2aaf09512a659629a6cd54928261a3971a4fce07ec0ecc51009331ce18c39e). Applying that
empty accepted JSONL with preserve_source_append inserts 0 rows and scores
identically to selected conf070: codepoint CER 0.739361, grapheme CER 0.768262,
line F1 0.872458, 67 misses, 2 extras, reading-order accuracy 0.775478; metrics
SHA-256 5e75442ea41ea8600ef6dd06fc2b246ad61615f5cea6a3107b7e0e3cb090e32b. Once a
reviewer fills accepted rows, compile -> apply -> score/effect-audit is now the path.
Do not assume that filling those 31 rows is enough. A new diagnostic-only oracle
ceiling tool, tools/select_candidate_line_reviews_by_oracle_subset.py, joins the
31-row geometry/text worklist back to the oracle/reference-overlap selected rows and
sets corrected_text from the reference only in a clearly marked unsafe artifact:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-oracle-reference-ceiling/
(31 selected, 0 rejected; report SHA-256
6eff8712c007c9278401ee2f8e9465b8bc31ab1161878802f41da56d0c75d5a3). That ceiling
still fails page OCR. With preserve_source_append, line F1 improves to 0.933566,
recall to 0.881188, misses fall to 36, and matched-line codepoint CER improves to
0.039297, but page codepoint CER worsens from selected conf070's 0.739361 to
0.771993, grapheme CER worsens to 0.802385, and reading-order accuracy regresses
to 0.750472 (metrics / effect-audit SHA-256
cb9b60eab8ff64bbf3f277eda29f2353537b308e03e67878e292714767df6ef4 /
f55a168c8a7cbda06faf87ed25417b6194ad55e4781096697b66998d15cb4d9a). Raw geometry
ordering is also unsafe: page codepoint CER 0.771825, grapheme CER 0.794103,
reading-order accuracy 0.604292 (metrics / effect-audit SHA-256
034846c66650367f64aeb34808a51ad807a9ecce18fedc7e79e4bcf864397bd7 /
c821571fc0efdf80f996585d4eac802af978e0d824233efda59568587a219ff6).
I then added candidate_document_order to tools/apply_candidate_page_line_reviews.py.
It maps existing selected lines to the candidate document by bbox IoU and orders
inserted rows by their candidate-document candidate_line_id; this is non-oracle
slotting evidence. On the 31-row oracle ceiling it is much better: codepoint CER
0.739024 slightly beats selected conf070's 0.739361, line F1 is 0.933566,
recall 0.881188, misses 36, extras 2, matched-line codepoint CER 0.039297, and
reading-order accuracy 0.775506. The effect gate still rejects it because grapheme
CER worsens to 0.778367 from 0.768262 (application / metrics / effect-audit
SHA-256 383af5e3dffe2cbf697185f70de706fc314f2864dac0d33ae8b7a9442ed90f6c /
473781d0b05563b8833e94ff639628712ec52965f9b71c45cc630269bf9d9f1b /
0e779661c45c5361025acbfee7afabdfb4996f44bc5920cc3e43c9ac4cc1a673). The next
serious work is no longer generic slotting; it is candidate-document-order slotting
plus grapheme-level text/normalization repair for the inserted lines.
The same order mode was tested without oracle text on the non-oracle 31-row
geometry/text selection. It fixes the ordering surface but not recognition: line F1
0.933566, recall 0.881188, misses 36, extras 2, and reading-order accuracy
0.775506, but page codepoint CER is still 0.763078, grapheme CER 0.797416, and
matched-line codepoint CER 0.091202 (application / metrics / effect-audit SHA-256
2093b60eb1d924b461ba37c69b606963a1c8692e96d75c1e9e78223c99d708b8 /
92d518739384e624c405efcec00222e7f75546ae72217f4838e1477a8279e10c /
c9939d8d3d3d8949ab2bcfe3ee4eac9dfbec95986670fcb253c30663aa9797f7). So the current
operational rule is strict: candidate-document-order is the right insertion mode for
reviewed rows, but machine candidate text remains rejected.
The 31-row oracle ceiling has now been ablated by reference span group. No subset or
group combination improves/ties both page codepoint and grapheme CER under
candidate_document_order: span0071-0075 is a codepoint win (0.732380) but
grapheme regresses (0.778532), span0242-0266 worsens both (0.754079 /
0.782011), span0268 worsens both, and span0310 improves grapheme (0.767765)
but worsens codepoint (0.740118). Single-row page-text diagnostics also found no row
that improves both. So the next policy is not "choose a safer subset of the same 31"
unless the admission gate changes.
A diagnostic-only oracle reference-order ceiling was then built:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry31-oracle-reference-order-ceiling-application/.
It uses hidden eval alignment to sort existing selected lines and inserted rows by
reference line ID, so it is not deployable. It proves the objective is attainable with
better ordering: codepoint CER 0.102355, grapheme CER 0.136823, line F1
0.933566, recall 0.881188, misses 36, extras 2, reading-order accuracy 1.0, and
effect audit book_ocr_objective_safe=true with 0 issues. Application / metrics /
effect-audit SHA-256:
8a8120c71e37cd7ca849853bfc1799383521bb8ec4572aaf0b7ffd64608d82a9 /
3dacf8b231d118d3cdc61b4b41b70908b539cc883bf620feb4749ad13eca08b8 /
1346510c0fdbafa1bd4b2957bf906b8aedf6328a35b23e8ede4c581986928b12. The actual next
engineering target is now clear: learn a non-oracle page reading-order/layout policy
that approximates that reference order.
To make that target concrete, I added
tools/analyze_page_reading_order_geometry.py. It reads a page line-alignment.csv,
sorts matched rows by reference line ID, and emits geometry chunks/jumps. Real artifact:
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-oracle-reference-order-geometry-analysis/
with 267 matched rows, 21 chunks, and 20 jumps. This proves the reference order is
article/block-level, not a simple right-to-left column grid: top chunks include
span0019-0025 at median cx=1138, span0034-0051 at cx=1486,
span0052-0069 at cx=1791; middle body chunks include span0160-0187,
span0188-0215, span0216-0241, span0242-0267 at median cx=205/521/838/1155.
Report / chunks / jumps SHA-256:
cd52bb8574864f243ad883b71df25afceb59553aa55ee93d1df079b92ce09fe5 /
d476d35bd727272960ef7c9dd6841c4e4143d596bb9776e54e437a682e7427b6 /
48e3d079a9780705438ebd6367013266912f08efe1f39cb2d86f70a510a2d041. Use this as the
target profile for the next non-oracle ordering heuristic.
I then added tools/apply_page_reading_order_geometry_profile.py, which applies a
geometry chunk profile to a page-document projection by assigning each line to the
nearest profile chunk and sorting by chunk index then local line position. With the
profile derived from the oracle reference-order analysis above, the tool reproduces the
layout ceiling on the 31-row oracle-text projection: codepoint CER 0.095711,
grapheme CER 0.129700, line F1 0.933566, recall 0.881188, misses 36, extras 2,
reading-order accuracy 1.0, and effect audit safe with 0 issues. Profile application
/ assignments / metrics / effect-audit SHA-256:
485b5f0132abf71f8830fd8a6fc1ebefc396311407eaacd684efc87144a00f15 /
4a735308a2458ecf94e25baecb5d08dfca4964b03bd793469fafdf53281715f4 /
5e50ddc10ce9b7b58469fe8ba0c7ec5eca15ce6faf005adf59a6a327e5d66102 /
a2fe4a37d755146b0224a88ee8370af86a67a72f0c5c275da4b01d93a31e498b.
Applying the same oracle-derived profile to the non-oracle machine-text 31-row
candidate projection is also a large diagnostic win: codepoint CER 0.143146,
grapheme CER 0.198112, line F1 0.933566, precision 0.992565, recall
0.881188, misses 36, extras 2, matched-line codepoint CER 0.091202, and
reading-order accuracy 1.0; the objective effect audit is safe with 31 inserted
lines and 0 issues. Profile application / assignments / metrics / effect-audit
SHA-256:
0819ebbbe70d36a98baf147b02b2c6828520d5173a3276a3b847325dccac4ec9 /
9d1461b936b30d837528a2d6b04bee7d34d7d5ce4218f584dacd882a1f5b5e28 /
8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189 /
0565e1755d190b3855f84d2946194594ce72d74cf116c4c98472a6f330cffd63.
Do not overclaim this: both profile-order runs use a profile learned from hidden eval
alignment. The operational next step is to build the same kind of profile from reviewed
layout calibration, page templates, or observable article/block cues, then run this
same application/effect-audit path without oracle provenance.
That non-oracle bridge now exists as
tools/build_page_reading_order_geometry_profile.py. It reads geometry-only
line-box-review application references, refuses transcription text by default, blocks
pending geometry unless --allow-pending is explicit, and refuses mixed multi-source
profiles unless --allow-multi-source-profile is explicit. I exercised it on the real
pending Sirijonga line-box packet by first applying the packet as seed geometry:
outputs/limbu-sirijonga-line-box-review-packet-2026-06-21-geometry-application-pending-seed/
has 2 samples, 389 kept line boxes, and application SHA-256
a953ffe6f97e46f84b8c744e11ac2b7f68587370810ce35c5404c842646e0e6e. Separate
pending seed profiles were then built per layout: Gorkhapatra has 303 lines, 26 chunks,
25 jumps, profile_claim_safe=false, profile SHA-256
fa02502b0d23285979d6ff385e0836e6578676df44b9ef1ad891d0b55eb28f8c; Sikkim has 86
lines, 83 chunks, profile_claim_safe=false, profile SHA-256
aff45bc2c0b295163176cf8ca420bfa2c13cdcf9956a6a777355e6ceb6183454. Applying the
Gorkhapatra pending seed profile to the same 31-row machine-text candidate projection
reproduces the strong profile-order diagnostic exactly at the metric-file level:
codepoint CER 0.143146, grapheme CER 0.198112, line F1 0.933566, recall
0.881188, misses 36, extras 2, reading-order accuracy 1.0, metrics SHA-256
8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189, and effect-audit
SHA-256 184a2eb9ed9c32cd37b50a85f45dce21ffedbf5b1bca20ea3c6e116cad2a08c4. The
claim boundary is still strict: this is seed geometry from a pending review packet,
not reviewed/calibrated operational evidence. The next concrete task is to complete
the Gorkhapatra geometry review rows and rebuild the same profile with
profile_claim_safe=true, then score it on held-out pages.
I also added tools/audit_page_reading_order_geometry_profile.py and a hard
--require-profile-claim-safe gate to tools/apply_page_reading_order_geometry_profile.py.
The pending Gorkhapatra seed profile is structurally valid but not operational:
reading-order-geometry-profile-audit-structural.json reports audit_passed=true,
operational_profile_safe=false, 26 chunks, 1 pending source, and 0 issues; the
required-safe audit intentionally fails with issue profile is not operationally claim-safe. Structural audit / required-safe audit SHA-256:
9aecaa40b5ee509ad12e495777a857a76fea7d4ad68793e9055e721a163b8877 /
694486b5a3f28bcdb3093c085814fdb76daf835be5599f26319daca80b70b3ce. The applier now refuses this same profile under
--require-profile-claim-safe with profile_claim_safe is not true. This is the
gate to use for any promoted book-OCR run; diagnostic runs may omit it but must keep
the non-claim-safe boundary.
Finally, tools/admit_page_reading_order_profile_application.py now binds the
profile-provenance audit, the profile application report, and the page objective
effect audit into one promotion decision. Running it on the real pending Gorkhapatra
seed-profile projection correctly rejects admission even though the page-effect audit
is safe: admitted_operational=false, assignment count 269, candidate metrics SHA-256
8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189, admission
SHA-256 02bcd20dc1a5598f92b9a251dd5ae4669171ba8d9429df4eb9c06cf33bc354b6. The three
issues are exactly provenance issues: profile audit does not mark
operational_profile_safe=true, profile audit does not mark profile_claim_safe=true,
and the application report was not run with a claim-safe profile. After geometry review
is completed, rerun: build profile without --allow-pending, audit with
--require-claim-safe, apply with --require-profile-claim-safe, run page scoring and
effect audit, then run this admission gate.
The profile applier now also supports multi-page profile routing with --profile-map.
The current pending seed map lives at
outputs/limbu-sirijonga-line-box-review-packet-2026-06-21-pending-seed-profile-map.json
(SHA-256 7e94909dacbe9be20d18a70ddbac5fcc0421b0c2a35c2055482b1f2859895397) and maps
Gorkhapatra and Sikkim sample IDs to separate pending seed profiles. Applying that map
to the current Gorkhapatra 31-row machine-text projection selects only the Gorkhapatra
profile, records profile_usage, assigns 269 lines, and reproduces the single-profile
metric SHA-256 8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189.
Map application / effect-audit / admission SHA-256:
e71697d996deb8516db22d243136c89b9440b4b4144556420c2534dce354871e /
88db8bc35b8b64fd2da33ddb629afbaadcf5b3a06acaad44460b65b1e78793b7 /
8764a46251c9e0bc1b2415de9089f5eb67032e9bf9eccf11167f54da8c18aaf6. The map now has
its own audit too: structural audit passes with a warning that Sikkim is not present in
the one-page Gorkhapatra document dir (SHA-256
1090ec5254bad3e5d0ec492739488de5a65d684673d33cb36d50f1dde8c7d7d7), while
required-safe audit fails because both mapped profiles are pending (SHA-256
bb4c07c4fcb7f2031ee085313d3a537eda01650b77f3af31ce23d605af136d06). Admission still
rejects the map-driven run because the selected profile and the map are
pending/non-claim-safe, which is the correct behavior. This closes the obvious
multi-page footgun: do not run one layout profile across a book unless a profile map or
template selector chooses it per page, and do not promote a map-driven run without a
safe profile-map audit.
The book wrapper now participates in this path. tools/parse_limbu_book_pages.py
accepts --reading-order-profile-map, applies it after optional line assembly, and
writes aggregate book.md, book.body.md, review queues, and score-documents/ from
the profile-ordered projection. The wrapper also exposes
--require-reading-order-profile-safe and --require-reading-order-profile-map-safe.
Supplied profile-map audits are now hash-bound to the exact map being applied, and
map-safe mode fails if no --reading-order-profile-map-audit is supplied.
tools/audit_limbu_book_output.py now checks the profile-map hash, application report,
assignments JSONL, optional map audit, and per-page profile source documents.
This matters because the previous standalone profile applier could produce a good
score projection while a book run still emitted stale Markdown; the integration test
now checks that Markdown and score documents agree, and the book audit catches stale
profile-order artifacts.
2026-06-23 correction: stop overbuilding eval harness
The user correctly pushed back that the eval harness work is becoming the project. Do not keep adding generic eval machinery unless it directly unblocks real Limbu book OCR. The current objective is operational: run real Limbu pages through the book OCR path, preserve visible line structure, surface missed/low-confidence lines for review, and export training rows only after explicit human acceptance/correction. Keep metrics as thin guardrails around that loop: source inventory, artifact audit, page/line counts, review packet audit, and training-pack export validation are enough for the next step.
Current source state was rechecked locally because SSH to cdjk@100.110.148.43 failed
with authentication rejection. The mirrored local data is enough to proceed:
outputs/limbu-page-source-inventory-2026-06-23-realbook-pivot/ reports 36 records,
3 real Devanagari-written Limbu CDC page candidates, 3 Devanagari text-only sources,
1 synthetic smoke pack, and ready=True. The real CDC PDFs are present under
data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/.
The existing 3-page CDC page draft has real rendered page images and unreviewed
curriculum text references:
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/.
The latest CDC book-output artifact audit passes structurally, but it is not a quality
claim: page_count=3, line_count=35, review_line_count=27, and
book_line_review_row_count=26.
The practical next move is therefore not another harness phase. Pick a small real page
set, run or refresh the book OCR path, inspect the generated unresolved/missing-line
review packet, fix only the operational defect that blocks review or drops visible
text, then apply reviewed rows and export a reviewed-only training pack. If the export
says no training rows were exported, training must not start.
2026-06-23 current CDC PDF book OCR run
The current best operational CDC PDF-derived run is:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/
It uses the current v4 router with recognizers held fixed on rendered CDC maths pages 8..10. The wrapper emitted editable book Markdown, page-layout overlays, page triage, layout-gap review, and a passing standalone book-output audit at:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-output-audit-lowconf-nontext-guidance/
Current counts:
page_count=3line_count=93book_line_review_row_count=21review_line_count=22in the parse summarypage_layout_review_row_count=3page_triage_review_row_count=3layout_gap_review_row_count=9operator_attention_page_count=0book_audit_passed=True
The latest code change is intentionally narrow: tools/run_limbu_book_ocr.py now
classifies wide low-density box-border/illustration fragments inside large layout gaps
as page_graphic_or_illustration instead of candidate_text_band. This was based on
visual inspection of gap-0001-cdc-maths-p0008-003; the crop and page context showed a
graphic/exercise box, not missed Limbu text. The real gap queue now has 6
inspect_page_graphic_or_illustration rows and 3 likely_rule_or_spacing rows, with
0 candidate-text gap rows, and all 3 pages are accepted by page triage. This reduced
review noise; it did not prove OCR text quality.
tools/parse_limbu_book_pages.py now also separates low-confidence square registration
corner hallucinations from real text spotchecks. The contact sheet showed three OCR
hallucinations (े?) over 60x60 page-corner marks; those are now
inspect_non_text_or_drop_low_confidence_detection instead of
spotcheck_low_confidence_text. The line-review action counts in the latest real run
are: 9 inspect_non_text_or_drop_empty_detection, 3
inspect_non_text_or_drop_low_confidence_detection, 1
transcribe_or_drop_empty_detection, and 8 spotcheck_low_confidence_text.
Contact sheet:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png
Next action: work the 21-row book-line review queue from this artifact, apply accepted line text/non-text decisions, and export only reviewed rows into a training pack. Do not return to generic eval-harness expansion unless a specific guardrail blocks this review to training loop.
2026-06-23 cleaned CDC projection
There is now a chainable cleaned projection that applies only the obvious non-text drop proposals from the current CDC packet:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/
Input review queue:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv
This queue marks 13 rows as drop: 9
inspect_non_text_or_drop_empty_detection rows and 3
inspect_non_text_or_drop_low_confidence_detection rows, plus one wavy illustration or
rope segment that remained as a blank transcribe/drop row after the first pass. It
leaves 8 rows pending, all low-confidence text spotchecks. The application audit reports
applied_count=13, dropped_count=13, skipped_count=8, and the standalone
book-output audit passes with 0 issues. The cleaned projection line count is 68 because
the chainable assembled documents had 81 text lines before these 13 drops; the upstream
run's 93-line count is from the wrapper/source count.
The reviewed-only training exporter was run against this cleaned projection and
correctly refused with no training rows were exported; these drops are useful for
editable book cleanup but are not accepted text labels. Training still requires a human
reviewer to accept/correct remaining text rows.
The current human review station for the remaining rows is:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/book-unresolved-review-packet.html
Packet directory:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/
It audits cleanly with tools/audit_limbu_book_unresolved_review_packet.py: 8 packet
rows, 8 crops, 8 page contexts, 13 non-pending source rows skipped, 8 prefilled text
rows, and source guidance preserved as 8 spotcheck_low_confidence_text rows.
tools/prepare_limbu_book_unresolved_review_packet.py
now maps review_priority/review_recommended_action from the book-line queue into
the packet's review_category/suggested_review_action, so the HTML no longer loses
the review guidance. The packet TSV/export header now also preserves
page_context_path, page_context_sha256, and neighbor_context, so a reviewed TSV
keeps the visual evidence needed to audit what the reviewer saw.
After review, use this base and packet:
BASE_BOOK="outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2"
PACKET_DIR="$BASE_BOOK/pending-text-review-packet"
REVIEWED_PACKET="$PACKET_DIR/book-unresolved-review-packet.reviewed.tsv"
HANDOFF_OUT="$PACKET_DIR/reviewed-text-training-handoff-v1"
uv run python tools/audit_limbu_book_unresolved_review_packet.py \
--packet-dir "$PACKET_DIR" \
--packet-tsv "$REVIEWED_PACKET" \
--allow-reviewed
uv run python tools/apply_limbu_book_reviewed_packet_to_training_pack.py "$BASE_BOOK" \
--packet-dir "$PACKET_DIR" \
--review-packet "$REVIEWED_PACKET" \
--out "$HANDOFF_OUT"
2026-06-23 larger CDC PDF review batch
The one-command runner now accepts --pdf directly, not just --page-manifest or
--image-dir. If PyMuPDF is unavailable in the Paddle runtime, the manifest builder
falls back to Poppler (pdfinfo + pdftoppm) and records pdf_renderer=pdftoppm in
the run source summary. This matters operationally: real CDC books can now enter the
same PDF -> render -> capture-prep -> OCR -> review packet path without hand-built
intermediate manifests or installing PyMuPDF into /tmp/ocrtech-paddle311.
Real run:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/
Command shape:
PYTHONPATH=src /tmp/ocrtech-paddle311/bin/python tools/run_limbu_book_ocr.py \
--pdf data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/mero_ganit_grade1_limbu.pdf \
--pdf-start-page 11 \
--pdf-end-page 15 \
--pdf-dpi 220 \
--page-id-prefix cdc-maths-p \
--start-index 11 \
--zero-pad 4 \
--out outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler \
--auto-deskew
It rendered 5 PDF pages with Poppler, preserved 189 OCR lines, wrote 58 pending
book-line review rows, produced 15 layout-gap rows, accepted all 5 pages in page
triage, and passed tools/audit_limbu_book_output.py with 0 issues. Review actions:
22 inspect_non_text_or_drop_empty_detection, 17
inspect_non_text_or_drop_low_confidence_detection, 3
transcribe_or_drop_empty_detection, 2 verify_non_limbu_or_drop, and 14
spotcheck_low_confidence_text. Layout-gap actions are 6
inspect_page_graphic_or_illustration and 9 likely_rule_or_spacing, with no
candidate-text gaps.
Current larger review station:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context-strict-v2/book-unresolved-review-packet.html
Reviewer context from the same source PDF text layer:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/limbu-pdf-text-context.md
This was generated by tools/build_limbu_pdf_text_context.py, which parses
pdftotext -bbox-layout lines, rejoins tiny-gap legacy glyph fragments, converts them
through src/ocrtech/limbu_legacy.py, and writes JSONL/TSV/Markdown/page text sidecars.
The real CDC pages 11..15 output has 82 text-layer lines, 81 lines with converted
Limbu codepoints, 0 unmapped-codepoint lines, and rendered bboxes mapped through the
book page manifest. Treat it as review assistance only: it is explicitly marked
pdf_text_layer_legacy_converted_review_context_not_gold, is_gold_text=false, and
training_use=forbidden_without_human_alignment_review.
tools/prepare_limbu_book_unresolved_review_packet.py now accepts
--pdf-text-context-jsonl and attaches nearest PDF text-layer rows to the review packet
by page ID and rendered bbox geometry. tools/build_limbu_pdf_text_line_coverage.py
also compares those PDF text-layer boxes with cleaned OCR line boxes to generate
candidate missing/partial source-text rows, then the existing missing-line review queue
feeds them back into the unified packet. The regenerated current strict packet has 38
rows: 25 existing-line rows, 13 candidate missing/partial rows, 38 PDF-context matches,
0 unmatched rows, 0 pending prefilled corrected-text rows, and still audits cleanly.
This context is display/support metadata only; it must not auto-fill reviewer
decisions. The HTML Accept text action can deliberately copy OCR text into
corrected_text, but a status-only TSV edit is no longer enough to create a label.
The first unresolved packet had 58 pending rows. Codex then made conservative
visual-drop passes using crop/page-context contact sheets and dropped only obvious
non-text rows: 20 repeated 60px page-corner registration marks/page furniture,
including the top-right े? hallucinations, plus 13 arrows, blank answer rectangles,
and standalone illustrations. It did not drop text-shaped labels or ambiguous glyph
rows.
Cleaned projection:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/
Proposal queue:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv
The cleaned book-output audit passes with 0 issues. Reviewed-only training export still
correctly refuses with no training rows were exported, because these are drops, not
accepted text labels. The strict combined packet audit passes with 0 issues and now
contains 38 pending rows with blank corrected_text by default. Existing-line actions
are 13 spotcheck_low_confidence_text, 11
inspect_non_text_or_drop_low_confidence_detection, and 1 verify_non_limbu_or_drop.
Candidate source-text coverage actions add 13
verify_pdf_text_line_missing_or_already_split rows: 8
pdf_text_line_missing_y_coverage and 5 pdf_text_line_partial_x_coverage. This is
the current larger label/layout-production target. Do not treat OCR text or PDF context
as ground truth until a non-machine reviewer exports accepted/corrected rows with
explicit reviewed text.
Bounded detector follow-up on the same CDC pages: the PDF-text detector sweep artifact
is
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/.
It evaluated 216 configs and found a high-recall setting with 76/81 covered source
lines, but that setting produces 335 detector boxes. The practical candidate is the
balanced OCR run
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/:
214 lines, 60 review rows, audit passed, PDF-text coverage 73 covered / 8 candidates.
This is a useful real-page probe, not a new default.
2026-06-23 Limbu phone page scorer-pack status
The current Limbu phone-page OCR path has a reproducible scorer-pack metadata
layer now. tools/prepare_page_ocr_eval_pack_metadata.py takes an existing
page-level eval manifest, fills scorer-visible source_image/whole-page
bbox/page metadata when missing, and writes:
- rewritten manifest:
reference-draft/scorer-pack/eval-manifest.jsonl - scorer summary:
reference-draft/scorer-pack/summary.json - script inventory:
reference-draft/scorer-pack/script-inventory.json - run report:
reference-draft/scorer-pack/page-ocr-eval-pack-metadata.json
Current command:
uv run python tools/prepare_page_ocr_eval_pack_metadata.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack \
--eval-pack-version limbu-phone-page-reference-draft-v1 \
--force
Current scorer-pack digest is
b20abcbbe4c6f38e39be055d9d618d29d0ecdc3751a5eb38d50384d89fabed8e.
It has 2 draft pages, 79 reference lines, and 0/2 claim-ready references. The
active filtered OCR documents are still under
ocr-line-filter-sirijonga-conf050-margin-markers/ and have 78 lines because
the page marker on phone-pxl-raw-1 was filtered out. Scoring the sidecar
manifest against those filtered documents writes:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run/
Current dry-run metrics: codepoint CER 0.000975, grapheme CER 0.001976,
text exact 1/2, line ref/pred/match 79/78/78, matched-line text CER 0,
matched-line reading-order pair accuracy 1.0, full reading-order pair
accuracy 0.952381. The sidecar run removes avoidable scorer warnings for
missing summary, pack digest, script inventory, source document, bbox, page
output audits, and source-path mismatch.
Do not treat this as a benchmark result. Remaining blockers are real: draft references, 2 pages only, no claim eligibility, non-frozen/not-unseen pack, selected checkpoint on the draft test, no hash-verifiable checkpoint/config/ dictionary passed into the scorer, no layout references, and no train/eval leakage manifest. The next useful work is either to review/promote the phone references or to run this same scorer-pack pattern on a larger verified Limbu page set.
Verification for the page-phone harness slice:
uv run pytest tests/test_prepare_page_ocr_eval_pack_metadata.py \
tests/test_audit_page_ocr_documents.py \
tests/test_run_page_ocr_eval.py \
tests/test_limbu_phone_page_reference_review.py \
tests/test_prepare_limbu_phone_page_benchmark_reference_draft.py \
tests/test_prepare_limbu_phone_row_composition_review.py \
tests/test_audit_limbu_phone_page_ocr_run.py \
tests/test_materialize_page_image_line_boxes.py \
tests/test_filter_page_ocr_documents.py \
tests/test_run_limbu_phone_page_ocr.py \
tests/test_prepare_limbu_book_captures.py \
tests/test_capture.py \
tests/test_audit_yolo_line_detector_predictions.py \
tests/test_export_limbu_line_detector_dataset.py \
tests/test_preflight_limbu_line_detector_training.py \
tests/test_build_limbu_book_page_manifest.py \
tests/test_build_limbu_pdf_text_context.py \
-q
Result: 139 passed, with only the existing SWIG deprecation warnings from PDF
rendering imports.
The one durable lesson
Every result this session points the same way: real same-font/native data is the lever, not model architecture. Sunuwar 0.342→0.121 with zero model changes. When a native expert is weak, the question is "what's wrong with the training text/font?", not "what hyperparameter?". Hold that line.
2026-06-30 Limbu Space page eval
tools/run_limbu_paddle_space_page_eval.py now materializes the current
spaces/limbu-ocr/app.py Paddle page output into the existing page-level OCR
Document schema and runs tools/run_page_ocr_eval.py. The first diagnostic run is:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed/score/
It uses the Gorkhapatra Sirijonga current-schema eval manifest and --max-lines 0.
Metrics: 1/1 scored pages, 303 reference lines, 306 predicted lines, 291 matched
lines, line precision/recall/F1 0.950980/0.960396/0.955665, matched-line
codepoint CER 0.033035, matched-line grapheme CER 0.063509, but full-page
codepoint CER 0.824390 and grapheme CER 0.846778. Same detections rescored
under simple explicit reading orders still fail as page OCR: Rows LTR codepoint
CER 0.815223, Columns LTR 0.824390, Columns RTL 0.796636.
Conclusion: do not expand this into multilingual page OCR yet. Limbu line detection/recognition is strong enough to keep, but the current page system fails on layout/reading-order reconstruction. The Gorkhapatra reference is still machine-converted and unreviewed, so these are diagnostic numbers, not claim-safe benchmark evidence.
2026-06-30 Limbu expanded page evals
New/updated tools:
tools/run_limbu_paddle_space_page_eval.py: generalized for Sirijonga and Devanagari scorer metadata and fixed to emit one block per detected line.tools/build_limbu_gorkhapatra_silver_page_eval.py: builds page-level eval rows fromoutputs/limbu-gorkhapatra-pdf-backed-silver-2026-06-23-v3/manifests/page-text-image-pairs.jsonlandline-text-image-pairs.jsonl.tools/apply_page_reading_order_strategy.py: rewrites existing OCR documents under simple geometry reading-order strategies without rerunning Paddle OCR.
The new Gorkhapatra silver page eval pack is:
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/
It contains 7 full-page images and 2,129 PDF text-layer silver reference lines. The
builder converts rendered PDF line boxes from xyxy to the eval harness xywh bbox
format. It must be run with regex so grapheme inventory is real:
env PYTHONPATH=src uv run --python 3.11 --with regex \
python tools/build_limbu_gorkhapatra_silver_page_eval.py --force
Current diagnostic scores:
CDC Devanagari Auto:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/cdc-devanagari-paddle-seed/score/
pages 3/3, page_cp_CER 0.376880, page_graph_CER 0.479263,
line_F1 0.906250, matched_cp_CER 0.057724, order_acc 0.592308
Gorkhapatra Sirijonga silver7 raw Auto:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed/score/
pages 7/7, page_cp_CER 0.814444, page_graph_CER 0.851347,
line_F1 0.943981, matched_cp_CER 0.027091, order_acc 0.720352
Gorkhapatra same detections, Columns RTL:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-columns-rtl-score/
page_cp_CER 0.771116, page_graph_CER 0.803203, order_acc 0.777787
Gorkhapatra same detections, old one-page geometry profile:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-profile-order-score/
page_cp_CER 0.736334, page_graph_CER 0.774886, order_acc 0.626211
Gorkhapatra same detections, Rows LTR:
score dir: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-score/
page_cp_CER 0.441968, page_graph_CER 0.475313,
line_F1 0.943981, matched_cp_CER 0.027091, order_acc 0.993085
Important interpretation: Rows LTR is the best simple order for the seven-page
Gorkhapatra silver pack, so spaces/limbu-ocr/app.py now defaults Paddle Page OCR and
Paddle Book OCR to Rows LTR with max lines 0 (no cap). This fixes the bad UI
default, not the full page OCR problem. Matched-only reconstruction from bbox-matched
Gorkhapatra target lines has roughly 2-3.5% codepoint CER per page, so the recognizer
is doing useful work on the target Limbu lines. Full-page CER remains high because the
page scope is wrong: Paddle detects headers/non-target text and the Limbu recognizer
turns those crops into Sirijonga-looking garbage. The next real fix is target-region
or language/script filtering for page OCR, then rerun these evals.
2026-07-01 Limbu page filtering baseline
Back on the main Limbu OCR goal, the next bottleneck was verified as page-scope
filtering rather than recognition. tools/filter_page_ocr_documents.py and
tools/audit_page_ocr_line_filter.py now support relative page-y filters:
--min-y-ratio and --max-y-ratio. Focused tests pass:
env PYTHONPATH=src uv run --python 3.11 --with pytest \
pytest tests/test_filter_page_ocr_documents.py -q
Result: 8 passed.
Best diagnostic non-oracle filter so far:
source documents:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-documents
filtered documents:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents
score:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score
Filter parameters: --drop-empty --min-y-ratio 0.12 --max-y-ratio 0.96 --min-confidence 0.70. The filter audit passes with 0 issues. It keeps 2,096 of
2,191 detected lines and removes 95 lines.
Score progression on the seven-page Gorkhapatra Sirijonga silver pack:
raw Auto: page_cp_CER 0.814444, page_graph_CER 0.851347, line_F1 0.943981
Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, line_F1 0.943981
Rows LTR + y filter: page_cp_CER 0.425080, page_graph_CER 0.449335, line_F1 0.960868
Rows LTR + y + conf70: page_cp_CER 0.422770, page_graph_CER 0.445983, line_F1 0.964734
Rows LTR + y + conf80: page_cp_CER 0.422985, page_graph_CER 0.446007, line_F1 0.964929
Interpretation: conf70 is the current baseline, but the improvement is modest. It
removes obvious page furniture and low-confidence junk, but full-page CER remains far
too high for a book OCR claim. The next serious step is a stronger non-oracle
target-region/language filter or page layout segmenter, then rerun the same evals.
2026-07-01 Limbu reading-order and assembly check
Do not loop back into training yet. The recognizer is not the main blocker on the Gorkhapatra Sirijonga silver pages: matched-line codepoint CER is still about 2.7%. The current failure is page reconstruction.
Added:
tools/apply_page_reading_order_strategy.pymodesRows LTR TopandRows RTL Topfor bbox-top row clustering. This keeps the old center-basedRows LTRbehavior available for apples-to-apples comparison.tools/assemble_page_ocr_lines.py --max-horizontal-overlap, with replay support intools/audit_page_ocr_line_assembly.py, so adjacent fragments with small detector box overlap can be merged without reference lookup.tests/test_apply_page_reading_order_strategy.py, plus an overlap case intests/test_assemble_page_ocr_lines.py.
Focused test command:
env PYTHONPATH=src uv run --python 3.11 --with pytest \
pytest tests/test_assemble_page_ocr_lines.py \
tests/test_filter_page_ocr_documents.py \
tests/test_apply_page_reading_order_strategy.py -q
Result: 17 passed.
Diagnostic scores on the same seven-page Gorkhapatra Sirijonga PDF-text-layer silver pack:
Rows LTR + y012/y096 + conf70:
page_cp_CER 0.4227699, page_graph_CER 0.4459828,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9930784
Rows LTR Top + same filter:
page_cp_CER 0.4289698, page_graph_CER 0.4496008,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9924359
Rows LTR + same filter + conservative assembly:
docs: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score
page_cp_CER 0.4226502, page_graph_CER 0.4457416,
line_F1 0.9652050, matched_cp_CER 0.0277639, order_acc 0.9931425
Rows LTR Top + same filter + conservative assembly:
page_cp_CER 0.4288860, page_graph_CER 0.4493596,
line_F1 0.9654350, matched_cp_CER 0.0276650, order_acc 0.9924574
Interpretation: top-anchored row order is worse on the held seven pages, and
conservative line assembly only moves CP CER from 0.4227699 to 0.4226502.
Assembly is useful for readability and auditability, but not the missing system
component. The next block of work should be a non-oracle layout/target-region
calibration stage for the Limbu article region, with held-out scoring. The old
geometry-profile run is not a usable answer; it was non-claim-safe and scored
page_cp_CER 0.736334.
2026-07-03 expanded Limbu Gorkhapatra pack
Full provenance and commands are in
docs/LIMBU_GORKHAPATRA_EXPANDED_2026_07_03.md.
New live crawl:
outputs/gorkhapatra-language-pages-expanded-2026-07-03-rendered-pymupdf/
publication_items=100 aligned=53 language_page_hits=103 warnings=0
It adds a PDF-backed Limbu issue:
date_key: 2083-03-13
article: https://gorkhapatraonline.com/news/211870
pdf: https://epaper.gorkhapatraonline.com/uploads/file/2026/6/gorkhapatra/2026-06-26-11-39-45-2083-03-13.pdf
Expanded corpus:
outputs/limbu-gorkhapatra-pdf-backed-silver-2026-07-03-v1/
discovered_limbu_rows=9 unique_dates=9 usable_pages=8 total_lines=2441 usable_silver_labels=2430
failed_dates=[{"date_key": "2082-11-12", "error": "no_namdhinggo_page_found"}]
Expanded page eval pack:
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-07-03-v1/
sample_count=8 line_count=2441
eval_pack_version=gorkhapatra-limbu-sirijonga-pdf-text-layer-silver-pages-v2
Expanded 8-page diagnostic scores:
Rows LTR raw:
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr/score
page_cp_CER 0.4349285, page_graph_CER 0.4692741,
line_F1 0.9418182, matched_cp_CER 0.0261180, order_acc 0.9931821
Rows LTR + y012/y096 + conf70:
docs: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score
filter kept 2398/2509 lines; replay audit passed with 0 issues
page_cp_CER 0.4143301, page_graph_CER 0.4380524,
line_F1 0.9630089, matched_cp_CER 0.0260386, order_acc 0.9931762
Rows LTR + same filter + conservative assembly:
docs: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents
score: outputs/limbu-paddle-space-page-eval-2026-07-03/gorkhapatra-sirijonga-silver8-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score
assembly merged 26 groups; replay audit passed with 0 issues
page_cp_CER 0.4142255, page_graph_CER 0.4378626,
line_F1 0.9633943, matched_cp_CER 0.0268101, order_acc 0.9932274
Interpretation: the new page is useful and the 8-page pack is stable. Matched-line CER is still about 2.6%, so the next serious work is target-region/page-layout calibration, not another recognizer epoch or more line-merging tweaks.