Download ocr-tech-mps-integration/docs/RECOGNIZER_GOAL_PROGRESS.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 288 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/docs/RECOGNIZER_GOAL_PROGRESS.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech-mps-integration/docs/RECOGNIZER_GOAL_PROGRESS.md
-
curl -L -o RECOGNIZER_GOAL_PROGRESS.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech-mps-integration/docs/RECOGNIZER_GOAL_PROGRESS.md
Limbu Book OCR Goal Progress
Living checklist for the active goal. Update often as work lands.
Active goal: build a usable Limbu book OCR system that accepts scanned or photographed real Limbu book pages and outputs editable Unicode text with page and line structure preserved. It must support both Sirijonga Limbu and Devanagari-written Limbu. The current bottleneck is page layout, reading order, missed-line recovery, and review burden with the current recognizers held fixed. Evaluation is a guardrail, not the product: keep it to frozen real-book acceptance checks such as page CER/WER, line detection precision/recall/F1, reading-order accuracy, blank/extra/missed rates, and review-packet coverage. Do not expand a general eval harness unless it directly proves one of those book OCR acceptance checks.
Historical context: the earlier recognizer-language program remains useful evidence, but it is not the active thread goal. Any next work should make the book OCR loop more true: cleaner source selection, stronger page preparation, better line preservation, lower manual review burden, safer correction application, or measured improvement over the current diagnostic baselines.
2026-06-24 controlled recognizer ablation verdict: after the targeted+Unicode bulk
run regressed, a five-cell ablation grid was trained on the AWS G6 L4 host and scored
against the current champion's exact frozen Gorkhapatra eval identity. The artifact
root is outputs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/; the launcher
and scoring script are in runs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/.
Results on 255 matched lines: champion reference remains codepoint CER
0.011336032388663968, grapheme CER 0.024735729386892176, exact 0.7294117647058823.
Control v2 replay at LR 1e-6 scored 0.011234817813765181 / 0.02452431289640592
/ 0.7333333333333333, a one-edit non-significant improvement and therefore only a
sanity check. v2 plus 256 capped targeted rows scored worse:
0.01194331983805668 / 0.026004228329809725 / 0.7294117647058823. v2 replay at
LR 1e-5 tied codepoint CER but worsened grapheme CER:
0.011336032388663968 / 0.026004228329809725 / 0.7372549019607844. Both v3
dictionary cells failed catastrophically, around 0.98 codepoint CER and 0.0
exact-line accuracy, because expanding the dictionary from the v2 checkpoint resizes
recognition-head parameters. Do not promote any ablation checkpoint. Keep the
real-heavy v2 dictionary champion as the current recognizer. Next recognizer work
should not add dictionary characters by warm-starting directly from the v2 head, should
not chase Paddle norm_edit_dis alone, and should focus on real/book-page line
harvesting or a separate from-scratch/rehead training plan if full Unicode coverage is
actually required.
Started: 2026-06-16. Owner: Claude session.
LIMBU OPERATIONAL OCR PIPELINE PIVOT (2026-06-21)
2026-06-23 recognizer corpus correction: do not describe the current Limbu
training inventory as only the 1,866 Gorkhapatra train rows. That count is only
the train split of
outputs/limbu-gorkhapatra-pdf-backed-silver-2026-06-23-v3/, a PDF-backed
silver real-domain pack with 255 held-out eval rows. The larger local Sirijonga
training inventory already includes 18,418 rendered Namdhinggo Limbu prose train
rows in outputs/limbu-prose-rendered-train-2026-06-16/, 28,155 augmented
rendered train rows in data/limbu-aug-train/train.txt, and the prior
runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/ checkpoint trained from the
rendered prose pack. The 1,866-row Gorkhapatra G6 run was therefore a narrow
real-domain fine-tune, not a full-corpus Limbu recognizer run. The corrected
training policy is: use synthetic/rendered rows for glyph, word, augmentation,
and font coverage; use Gorkhapatra PDF-backed rows for real-domain adaptation and
checkpoint selection; keep synthetic rows out of headline CER claims. The
dedicated builder tools/build_limbu_combined_recognizer_pack.py materializes
this as an auditable combined pack, rewrites legacy label paths to repo-root
paths, tags source components, and drops exact normalized train-text overlaps
against the held-out Gorkhapatra eval text before training.
2026-06-23 combined corpus G6 launch: the corrected combined training pack is
outputs/limbu-combined-synth-real-gorkhapatra-v1-2026-06-23/. It has 48,507
train rows and 255 eval rows after dropping 13 exact normalized train-text
overlaps with the Gorkhapatra eval text. Train composition is 28,155
augmented rendered rows, 18,418 rendered Namdhinggo prose rows, 1,853
Gorkhapatra PDF-backed silver rows, and 81 CDC Sirijonga PDF-backed silver
rows. Eval is only the held-out Gorkhapatra PDF-backed silver real-print split.
Both manifests passed ocrtech audit-manifest. The generated recipe is
runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-g6-l4-ppocrv5-2026-06-23/
with pack hash
1f77f7a0f947fc475afcef5568f08a1f0649a92dc8ee1a7eb0620956b734085b, base
checkpoint
runs/limbu-gorkhapatra-pdf-backed-silver-v4warm-20ep-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams,
batch size 64, learning rate 0.00001, 15 epochs, warmup 1, GPU enabled,
and train_drop_last=false. The first generated run used Paddle's default
main_indicator=acc; that was wrong for the current CER-oriented goal because
it can overwrite a lower-edit-distance checkpoint with a higher exact-line-acc
checkpoint. src/ocrtech/training.py and the CLI now expose
--main-indicator, and the active corrected recipe is
runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/
with main_indicator=norm_edit_dis. The AWS G6 instance
i-0fa19dff7b233a353 was restarted on 2026-06-23, current public IP
35.175.116.142, remote workspace /opt/ocrtech. An initial exact-acc run was
stopped after proving the selection-rule problem, then a norm-edit run initially
hit OOM because the old parent process was still holding GPU memory. After
clearing the GPU, the active corrected run was launched as remote PID 2404 via
/opt/ocrtech/runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/train-g6.sh.
Startup sanity passed: zero missing label images, Paddle 3.3.1 sees one NVIDIA
L4, training dataloader has 758 iters, eval dataloader has 4 iters, and the
checkpoint loaded successfully from the previous Gorkhapatra best model. First
eval at global step 200 saved by norm-edit with norm_edit_dis=0.8731627801677526
and acc=0.1686274443675512.
2026-06-24 CER improvement path: targeted fabrication, not generic synthetic
volume. The combined recognizer pack already has heavy synthetic coverage:
46,573/48,507 train rows are synthetic/rendered and only 1,934 are
real-or-silver line crops. The norm-edit run selected epoch 1 as best
(norm_edit_dis=0.8731627801677526), while later evals hovered lower, so dumping
more random synthetic rows into the same mixture is the wrong next move. The next
CER work should build a targeted fabrication factory:
- render from real Limbu/Sirijonga text, not random Unicode strings, using verified renderable upstream Namdhinggo fonts and any CDC/book fonts we can verify. Do not use extracted PDF subset fonts unless they pass visible-ink renderability checks;
- match the real line-width distribution, especially long newspaper/book lines that stress the recognizer input width and CTC/attention decoding;
- apply phone/page degradations learned from the June 23 PXL photos and prepared images: slight perspective, blur, JPEG, uneven illumination, stroke thickening/ thinning, paper background, local crop padding variation, and downsampled low-resolution edges;
- oversample hard graphemes and substitutions from
confusions.json,worst-lines.csv, and current phone/cdc/gorkhapatra OCR outputs instead of uniformly sampling the dictionary; - keep source tags and sampling weights explicit so synthetic rows can be downweighted or capped relative to real/silver rows;
- never score claims on synthetic rows. Evaluate every ablation on held-out Gorkhapatra PDF-backed silver and separate CDC/phone-page review packs.
Practical acceptance gate for the next fabrication run: create a new
limbu-targeted-fabrication-v1 component, train as a short warm-start ablation,
and keep it only if it improves held-out real/silver micro CER without degrading
CDC and phone-page matched-line CER. If it only improves synthetic validation or
exact-line acc while norm_edit_dis/CER stagnates, reject it.
2026-06-24 targeted fabrication v1 materialized and corrected: the new generator
tools/build_limbu_targeted_fabrication_pack.py now builds the targeted synthetic
component described above. It reads verified synthesis-text JSONL/plain text, optional
worst-lines.csv, optional confusions.json, verified upstream Namdhinggo fonts, and
an optional real line-crop width profile. It writes Paddle labels, rendered images,
manifests/train.jsonl, dictionary.txt, rejects.jsonl, and
targeted-fabrication-summary.{json,md}. Every row is explicitly tagged
claim_evidence_eligible=false, gt_status=synthetic_text, and
source_training_component=limbu-targeted-fabrication-v1; synthetic rows are
training-only and must not be used for headline CER claims. The first usable pack is
outputs/limbu-targeted-fabrication-v1-2026-06-24/: 9,996 phone-photo degraded
line images from 5,000 selected base text lines, 4 render rejects caught by the ink
guard, 1,866 Gorkhapatra train line widths used as the width profile, four verified
upstream Namdhinggo fonts, current combined-run worst-lines.csv/confusions.json,
and RAQM shaping enabled. Crop dimensions are width min/p50/max 89/608/2460, height
min/p50/max 32/61/147, dark-pixel min/p50/max 28/2082/113592, and min-gray
min/p50/max 0/16/129. Correction: the earlier generated pack under the same name
used extracted PDF subset fonts and was effectively blank; those fonts do not render
visible Unicode Limbu through Pillow/RAQM and are now rejected by an explicit font
renderability test plus a per-crop ink guard. The generator has focused tests in
tests/test_build_limbu_targeted_fabrication_pack.py, including this blank-font
regression.
The combined recognizer builder now accepts this component through
--targeted-fabrication-train-manifest. The first combined pack is
outputs/limbu-combined-synth-real-gorkhapatra-targeted-v1-2026-06-24/: 58,467
train rows and 255 held-out Gorkhapatra eval rows after dropping 49 exact eval-text
overlaps. Train composition is 28,155 augmented rendered rows, 18,418 rendered
Namdhinggo prose rows, 9,960 targeted-fabrication rows, 1,853 Gorkhapatra
PDF-backed silver rows, and 81 CDC Sirijonga PDF-backed silver rows. Both
train.jsonl and eval.jsonl passed ocrtech audit-manifest with hash verification.
Next action is not to trust the new synthetic component; train a short warm-start
ablation from the current best checkpoint and keep it only if held-out real/silver CER
improves without phone/CDC regression.
2026-06-24 Unicode/diacritic coverage hardening: the statistical pass is
documented in docs/LIMBU_UNICODE_DIACRITIC_STATS.md. Unicode 17.0.0 UCD was checked
against the local Python unicodedata view; the Limbu block remains 68 assigned
codepoints in U+1900..U+194F. A HarfBuzz/Pillow Namdhinggo shaping probe rendered
21,891/21,891 assigned/component-order cases cleanly, which proves renderability but
also proves the font is too permissive to be a grammar oracle. The corpus-stat pass on
the targeted combined pack found 61/68 codepoint coverage, with seven missing
codepoints and low counts for U+1919, U+1924, U+1926, U+1928, U+192B,
U+1944, U+194E, and U+194F. It also showed that NFC/NFD do not repair these
labels because most Limbu marks have combining class 0; only U+193B, U+1939, and
U+193A have non-zero combining classes.
The coverage generator tools/build_limbu_unicode_coverage_text.py now uses the
measured corpus order instead of random permutations: base+subjoined+vowel,
base+vowel+KEMPHRENG+final, and no generator-created final+sign context. It also
fixes the earlier omission of U+1932 from finals. The generated source
data/synthesis-text/limbu-sirijonga-unicode-coverage-v1-2026-06-24/coverage-text.jsonl
has 1,936 rows and passes the new corpus-stat audit with warning_counts={}. The
rendered component outputs/limbu-unicode-coverage-fabrication-v1-2026-06-24/ has
3,872 mixed-degradation line crops, 0 render rejects, and dark-pixel min/p50/max
33/1880/49435. The combined pack
outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-2026-06-24/
contains 62,339 train rows and 255 held-out eval rows, passed train/eval manifest
audits, covers 68/68 assigned Limbu codepoints, and has no assigned Limbu codepoint
below 1,000 occurrences after dedupe. Existing inherited corpus warnings remain in
older rows; the new Unicode coverage supplement itself added none. Next training should
treat unicode-coverage-fabrication-v1 as an explicit ablation component.
2026-06-24 targeted+Unicode G6 training launch: training is now running on the
combined existing pack plus targeted fabrication plus Unicode coverage supplement, not
on the coverage supplement alone. The run directory is
runs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-24/.
It warm-starts from
runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams,
uses append-only dictionary data/dictionaries/limbu-sirijonga-v3.txt (98 chars;
v2 prefix plus U+1909, U+190A, U+191A, U+191D, U+191E, U+1932,
U+1940), batch size 64, LR 0.00001, 15 epochs, warmup 1,
main_indicator=norm_edit_dis, and eval every 200 steps. The staged G6 workspace is
/opt/ocrtech on AWS instance i-0fa19dff7b233a353, public IP 3.219.242.144,
training PID 11133. Remote preflight resolved 62,339 train rows and 255 eval
rows with zero missing images and zero malformed rows; Paddle 3.3.1 sees one NVIDIA
L4, with 975 train iters and 4 eval iters. Startup head-shape warnings are
expected because the dictionary grew by seven chars. First eval at global step 200
saved the initial best with norm_edit_dis=0.04608837027816626, acc=0.0, and
fps=243.75362537495823; this is a warmup checkpoint, not the final CER answer. After
completion, pull back the run, audit training completion, export/package
best_accuracy, run frozen Gorkhapatra recognizer eval, and compare real CER against
the real-heavy champion before deciding whether the synthetic additions helped.
2026-06-24 targeted+Unicode G6 training result: reject. The 15-epoch G6 run
completed without crash or OOM and the local completion audit passed with issue_count=0,
warning_count=1 at
outputs/limbu-combined-targeted-unicode-v1-15ep-training-audit-2026-06-24/; the warning
is only that the audit regex did not extract the best validation metric even though the
training log records it. Paddle selected epoch 14 with
norm_edit_dis=0.8278816454641351, acc=0.1450980335255673, and
fps=245.24747110530726. The exported/package-best model scored worse on the frozen
Gorkhapatra recognizer eval:
outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-best-eval-2026-06-24/score/
has 255 matched lines, codepoint CER 0.024696356275303644 (244/9880), grapheme CER
0.053276955602537 (252/4730), and exact-line accuracy 0.4666666666666667. Against
the real-heavy champion at
outputs/limbu-targeted-unicode-v1-best-vs-real-heavy-champion-comparison-2026-06-24/comparison/,
it is worse by 0.013360323886639675 codepoint CER and 0.028541226215644824
grapheme CER, with paired wins/losses/ties 21/102/132 and 22/98/135. Against the
older combined baseline at
outputs/limbu-targeted-unicode-v1-best-vs-combined-baseline-comparison-2026-06-24/comparison/,
it is worse by 0.008198380566801618 codepoint CER and 0.015010570824524318
grapheme CER. Do not promote this package into book OCR. Treat it as a negative
ablation showing that the current targeted synthetic/Unicode supplement diluted the
real-heavy signal instead of improving real print CER. The eval remains non-claim-safe
silver PDF-text ground truth, and the stale row-level split=train metadata warning in
the candidate eval manifest should be fixed before the next serious comparison.
2026-06-22 measured-page correction: the active goal is no longer pipeline
plumbing. The Devanagari-written Limbu PP-OCRv5 run was stopped at best_accuracy
after dev regression, completion-audited, exported, packaged, and scored. The sealed
31-line recognizer eval is non-claim-safe but useful: codepoint CER 0.118219,
grapheme CER 0.256822, exact-line accuracy 0.290323. On the three real CDC
Devanagari pages, the trained recognizer cut image-line Tesseract page codepoint CER
from 0.865400 to 0.480602; a directed detector cleanup
(horizontal_kernel=31, vertical_kernel=5, min_width=80, max_height=80) moved it
to 0.458432 with grapheme CER 0.645161, WER 0.824242, line F1 0.593750, and
matched-line CER 0.177895. The paired detector comparison is positive but
non-claim-safe (3 pages; codepoint CER improvement 0.022169). The selected failure
analysis still shows 12/31 reference lines missed and dominant error mass from
missed reference text (47.58%). The next Devanagari work is targeted missed-line
recovery/layout modeling with the PP-OCRv5 recognizer held fixed, not another broad
pipeline pass or blind recognizer run. Current evidence lives under
outputs/limbu-measured-page-baseline-2026-06-22/ and
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/.
2026-06-22 router full-page smoke: the Devanagari PP-OCRv5 package is now wired
into an operational Limbu router instead of the failed Tesseract primary. The first
artifact was outputs/limbu-router-v2-ppocr-devanagari-sirijonga-2026-06-22/: primary
paddleocr Devanagari CDC PP-OCRv5, secondary Sirijonga v4 PP-OCRv5, line-level
script selection, Devanagari primary guard, and hashed component model cards. It passed
ocrtech audit-model and preflight-limbu-pipeline under the Paddle Python 3.11
runtime, but the guard was too blunt for Sirijonga pages. A fresh two-page v2
Gorkhapatra book smoke at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2p-router-v2-ppocr/ completed
but is not usable OCR: book-run.json reports 692 detected lines and 671 review
rows across 2 pages. Page 1 alone has 323 image-line detections with unfiltered
morphology parameters (horizontal_kernel=23, min_width=35, filter_enabled=false);
page 2 has 369. Page 1 is visually Sirijonga, but v2 classified 317/323 lines as
Devanagari because the Devanagari primary guard blocked the Sirijonga secondary after
low-confidence Devanagari hallucinations. ScriptSelectCompositeEngine now supports
primary_guard_min_confidence, preserving old behavior unless configured. The current
working router is
outputs/limbu-router-v4-ppocr-devanagari-sirijonga-confidence-guard-2026-06-22/,
with primary_guard_min_confidence=0.8; it passes model audit, runtime preflight, and
focused script_select_composite tests. On the first Sirijonga page, the runnable
filtered smoke
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-filter-conf060/
keeps 152 lines and 80 review rows after drop_empty, require_script=sirijonga,
and min_confidence=0.6; kept script counts are 151 Sirijonga, 1 mixed, 0
Devanagari, 0 other. Treat that filtered output as diagnostic only, not selected book
OCR: inspection showed many removed rows are real Sirijonga text with low recognition
confidence, so confidence filtering silently drops page content. The selected operational
baseline for the same page is now the v4 preserve-mode run at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-preserve/.
It keeps all 323 detected lines, writes 230 review rows, reports script counts
286 Sirijonga, 27 Devanagari, 4 mixed, and 6 other, and passes the strict
--require-no-dropped-image-lines --min-line-count 25 audit at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-preserve/no-dropped-audit/.
The filtered run fails that same audit with removed_line_count=171. This proves the
trained recognizers are packaged and callable in the book pipeline and that the prior
router was suppressing Sirijonga. The remaining failure is page quality and review
burden: preserve mode keeps the text but many lines are low-confidence or noisy on this
complex newspaper sheet. For tomorrow's book pages, default to preserve mode plus review
queue; use script/confidence filtering only for diagnostics or optional cleanup views,
not as the digitization source of truth. Do not start another broad recognizer training
pass from this evidence.
2026-06-22 photographed-page prep path: raw multi-page capture preparation is now
wired into the book OCR path via tools/prepare_limbu_book_captures.py. The tool reads
a raw page JSONL manifest, prepares each page with the existing capture pipeline
(auto_deskew, crop, perspective, grayscale/autocontrast options), audits every capture
bundle, and emits prepared-page-manifest.jsonl rows whose image_path points at the
prepared image and whose parse_kwargs.capture_prep_metadata points at the matching
metadata file. This closes the operational gap between photographed/slanted pages and
tools/parse_limbu_book_pages.py; it is not another eval harness. A real one-page
Sirijonga smoke produced
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-capture-prep-preserve-manifest/prepared-page-manifest.jsonl
with capture audit pass and prepared-image quality
1985x3062, entropy 2.894443, luminance stddev 76.480214, edge stddev 74.158261.
Auto-deskew skipped on that already-straight page because there was insufficient
near-horizontal line evidence. Running v4 preserve OCR from that prepared manifest at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-capture-prep-preserve/
kept 323 lines, wrote 230 review rows, dropped 0 image-line rows, and passed the
strict capture-plus-preserve audit at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-capture-prep-preserve/strict-capture-no-dropped-audit/.
Use this capture-prep -> preserve-OCR -> strict audit chain as tomorrow's default for
book page photos. The book capture-prep wrapper now also writes
capture-prep-dashboard.html, a raw/prepared contact sheet with operation summaries,
prepared-image quality, and capture-audit status per page; the dashboard path, hash,
and card count are recorded in book-capture-prep-run.json and surfaced by the
one-command runner. This gives the operator a fast visual gate for crop, deskew, and
perspective mistakes before OCR review.
2026-06-23 fragmented-text deskew fallback: src/ocrtech/capture.py now keeps the
existing Hough near-horizontal line deskew path, but if it cannot find enough long line
segments it falls back to row-projection scoring over the thresholded text ink. This
targets photographed pages where Limbu glyph rows are fragmented into short components
rather than long strokes. Blank/no-evidence pages still skip auto-deskew. The focused
capture test now covers a slanted page made of separated short text fragments and
passes through prepare_limbu_capture(..., auto_deskew=True) with an applied angle in
the expected range.
2026-06-22 current one-page runner verification: the current one-command runner was
rerun on the known one-page Sirijonga folder at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir-current/.
It completed under /tmp/ocrtech-paddle311, generated one raw-page manifest row,
prepared one page, and passed the book artifact audit with zero issues. Top-level
counts: 1 page, 323 preserved lines, 230 book-line review rows, script counts
286 Sirijonga / 27 Devanagari / 4 mixed / 6 other, and
operator_attention_pages=1 with high_review_burden. The capture prep dashboard is
hash-bound in limbu-book-ocr-run.json with one page card and two images (raw and
prepared); the book review dashboard has 230 cards and 230 crop images. The
book-line review queue has 230 rows and includes source_crop_paths and
source_crop_sha256s. This is still not an OCR quality claim, but it proves the current
operator path emits the review/capture surfaces needed for real page triage.
2026-06-22 current multi-page runner verification: the current one-command runner
was also rerun on the full four-image Gorkhapatra source folder at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-4p-runner-v4-preserve-image-dir-current/.
It completed under /tmp/ocrtech-paddle311 and passed the book artifact audit with zero
issues. Counts: 4 pages, 1384 preserved lines, 1198 book-line review rows, script
counts 682 Sirijonga / 64 Devanagari / 30 mixed / 608 other. The top-level
source summary reports 4 discovered images, 4 kept images, 0 skipped images, and
2 duplicate SHA-256 groups; the runner now also records this as the source attention
flag duplicate_source_images_present, so the known source contamination is visible at
the run front door. The capture prep dashboard has 4 page cards and 8 raw/prepared
images; the book review dashboard has 1198 cards and 1198 crop images; the
book-line review queue has 1198 rows with crop paths and crop hashes. Operator
attention flags expose the page pattern: pages 1 and 3 are high_review_burden, while
pages 2 and 4 are all_lines_need_review plus high_other_script_share. This is valid
multi-page plumbing and triage evidence for the current code, not a clean Limbu quality
target.
2026-06-22 refreshed operational run after scope correction: because the default
repo uv run environment is Python 3.14 and cannot import the Paddle runtime, the
current book runner was executed with the documented Paddle runtime
/tmp/ocrtech-paddle311/bin/python. The one-page folder run at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-current-py311/
completed with 1 page, 323 preserved lines, 230 book-line review rows,
1 layout-gap review row, and a passing book artifact audit. The cleaner duplicate
skipping multi-page run at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311/
completed with 2 unique pages, 692 preserved lines, 599 book-line review rows,
1 layout-gap review row, and passed=true in
book-output-audit/limbu-book-output-audit.json. Source selection discovered 4
images, kept 2, skipped 2 duplicate images, and recorded
source_images_skipped,duplicate_source_images_removed. Page 1 is the usable
Sirijonga operational target (323 lines, 230 review rows, one possible missed-line
gap); visual inspection of that gap crop suggests a section/spacing break rather than
an obvious missing line, so it remains review triage, not automatic insertion. Page 2 is
a Nepali newspaper page, not a Limbu OCR target: it has 369/369 review rows,
298 other-script lines, and the operator flags all_lines_need_review plus
high_other_script_share. The immediate product bottleneck is therefore source
selection plus review-burden reduction on real Limbu pages, not another eval harness.
2026-06-22 page-level triage packet: tools/run_limbu_book_ocr.py now writes a
page-level operator packet before line-level review under page-triage-review/:
page-triage-review-queue.tsv and page-triage-review-dashboard.html. The TSV keeps
page image path/hash, operator flags, per-page script counts, review ratio, layout-gap
count, and an advisory recommended_action such as
source_reject_candidate, inspect_layout_gap_then_line_review,
review_low_confidence_lines, or accept_page. It does not delete lines or make a
quality claim; it is meant to stop a bad source page from turning into hundreds of
line-review tasks. A refreshed duplicate-skipping real run at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-page-triage-links/
emits 2 page-triage rows: page 1 is inspect_layout_gap_then_line_review (323
lines, 230 review rows, one layout-gap warning), while page 2 is
source_reject_candidate (369/369 review rows, 298 other-script lines,
other ratio 0.8076). The page-triage dashboard image links were verified to resolve
relative to the dashboard directory (../capture-prep/...).
2026-06-22 page-triage application path: tools/apply_limbu_book_page_triage.py
now turns explicit page-level reject/reshoot/drop decisions into a chainable reviewed
book projection. It copies only kept page documents, rewrites aggregate book.md /
book.body.md, writes new score-documents/, filters book-line-review-queue.tsv
and layout-gap-review-queue.tsv to kept pages, and records rejected pages plus source
hashes in page-triage-application-audit.json. Advisory recommendations alone do not
remove pages; an operator must explicitly set operator_decision or review_status to
a reject-style value. tools/apply_limbu_book_review_queues.py now applies this
page-triage projection first when a page-triage queue is present, then applies book-line
corrections and layout-gap insertions to the filtered projection. A real smoke copied
the two-page page-triage queue, explicitly rejected
gorkhapatra-unique-current-p0002, and ran the combined review command to
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-page-triage-links-reviewed-p1-only/.
The final book contains only gorkhapatra-unique-current-p0001 (323 lines), the page
2 Nepali text is absent from final-book/book.md, the page-triage audit filtered the
line-review queue from 599 rows to 230 rows by removing 369 rejected-page rows,
and the final layout-gap application consistency audit passed with zero issues.
2026-06-22 line-review priority guidance: tools/parse_limbu_book_pages.py now
adds review_priority_rank, review_priority, and review_recommended_action to
book-line-review-queue.tsv, and the book review dashboard displays those values as
operator-facing pills. The goal is not to remove review rows or call them correct; it is
to make the review packet usable by separating empty/no-script detections, mixed-script
risks, and ordinary low-confidence spot checks. A refreshed two-page run at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-review-priority/
still has 599 review rows, but they are now bucketed as 304 high-priority
transcribe_or_drop_empty_detection, 15 medium-priority
verify_mixed_script_text, and 280 low-priority
spotcheck_low_confidence_text. After page-triage rejection of the known non-Limbu page,
the usable Sirijonga page has the review work shape we actually care about: 6
high-priority empty detections, 4 medium mixed-script rows, and 220 low-priority
low-confidence spot checks. This is a practical review-burden reduction mechanism
without silently dropping OCR text.
2026-06-23 scope correction and current priority packet: the user correctly pushed
back that we were overdoing the eval-harness direction. Keep evaluation as acceptance
evidence for the book OCR loop, not as the main product. The current freshest real-book
run is
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/,
executed under /tmp/ocrtech-paddle311/bin/python because the repo uv run Python
3.14 environment cannot import Paddle. It processes 2 unique source pages after
duplicate skipping, preserves 692 OCR lines, writes 599 review rows, and passes the
book artifact audit. The priority queue/dashboard are operator views only:
book-line-review-queue.tsv remains in page/reading order for application, while
book-line-priority-review-queue.tsv is a sorted working copy. Both have 599 rows;
the priority copy starts with rank 020 high-priority
transcribe_or_drop_empty_detection rows. Counts are 304 high
transcribe_or_drop_empty_detection, 15 medium verify_mixed_script_text, and
280 low spotcheck_low_confidence_text. This confirms the next practical work is
reducing review burden and recovering/preserving real page lines, not expanding generic
metrics.
2026-06-23 triaged projection cleanup: page-level triage application now filters
all page-scoped review queues, not only the canonical book-line queue. Explicitly
rejected pages are removed from book-review-queue.tsv,
book-line-review-queue.tsv, book-line-priority-review-queue.tsv, and
layout-gap-review-queue.tsv; aggregate review counts, priority/action counts, script
counts, and source-document provenance hashes are recomputed for the kept pages.
Inherited review-dashboard paths are cleared from the triaged book-run.json so a
filtered projection cannot accidentally point an operator back to rejected-page cards.
The runner README now tells the operator to inspect page triage and set
operator_decision=reject, reshoot, or drop before applying line/gap reviews. A
real check on the 2026-06-23 two-page run with page 2 explicitly rejected writes
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/manual-triage-check/triaged-p1-only/:
1 kept page, 323 lines, 230 review rows, priority counts 6 high / 4 medium /
220 low, all filtered queues contain only gorkhapatra-unique-current-p0001, and
tools/audit_limbu_book_output.py passes with zero issues. This is an operational
source-selection and review-burden improvement, not an OCR quality claim.
2026-06-23 non-text empty-detection guidance: the book-line review queue now records
book_line_bbox and book_line_aspect_ratio, and empty OCR rows whose geometry is a
large square/near-square detection are separated from text transcription as
inspect_non_text_or_drop_empty_detection. This is intentionally conservative: it does
not drop the row or change OCR text; it prevents obvious portraits/page furniture from
being mixed into the "transcribe visible text" bucket. A real rerun at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/
passes the book audit with 2 pages, 692 lines, and 599 review rows. Across both
pages it separates 6 non-text empty detections, leaving 298
transcribe_or_drop_empty_detection rows. After explicitly rejecting the non-Limbu
second page, the p1-only projection at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/
passes tools/audit_limbu_book_output.py with zero issues and has the actionable
review shape: 5 empty visible-text transcription/drop rows, 1 non-text inspect/drop
row, 4 mixed-script verification rows, and 220 low-confidence spot checks. This is
review-packet quality work, not a CER/WER improvement.
2026-06-23 full-page layout overlay review: tools/run_limbu_book_ocr.py now emits
a page-layout review packet under page-layout-review/ before page triage and
line-level transcription. It writes page-layout-review-manifest.tsv, a
page-layout-review-dashboard.html, and one full-page overlay PNG per OCR page with
line boxes and order indices drawn over the prepared page image. Overlay colors are
green for accepted text, amber for review text, red for empty OCR, and purple for
structural text. This directly targets the active layout/missed-line bottleneck: an
operator can now inspect whole-page line coverage and reading order before spending time
on individual crop transcription. A real two-page Paddle run at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-layout-overlay/
passes the book audit, writes 2 layout rows and 2 overlay images, and records page 1
as 323 lines / 6 empty lines / 230 review lines with reading order
wide_column_right_to_left. Visual inspection of
page-layout-review/overlays/layout-0001-gorkhapatra-unique-current-p0001.png confirms
the overlay is nonblank and shows the current failure surface: many low-confidence body
rows plus a small number of empty OCR/page-furniture detections. This is review and
layout-debug evidence, not a text-quality claim.
2026-06-23 reviewed-line training-pack bridge: tools/export_limbu_book_training_pack.py
now turns a reviewed Limbu book projection into line crops, PaddleOCR-style labels,
a JSONL recognizer manifest, dictionary, and script inventory. The default is
deliberately strict: only lines with explicit book-line review correction, split-review,
or layout-gap insertion metadata are exported. Pending/unreviewed OCR rows are skipped
so the training set does not silently become a copy of current recognizer output. The
exporter now revalidates that metadata before writing labels: status must be
accepted/corrected, reviewer must be present and non-machine, and the evidence-specific
reviewed text must be present and match the exported line. Candidate insertions require
explicit corrected_text, so replayed machine candidate text cannot leak into
recognizer training. The OCR runner README now continues the operator workflow after review application:
uv run python tools/export_limbu_book_training_pack.py <run>-reviewed/final-book --out <run>-training-pack-reviewed. A real default export against the older p1-only
final-book correctly refused to write a pack because there were no accepted line/gap
review rows. A bootstrap diagnostic export from the latest p1-only triaged projection
used --include-unreviewed and wrote
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/training-pack-bootstrap-p1-unreviewed/:
317 crops/labels, 6 empty rows skipped, dictionary with 97 characters, and script
sample counts 221 Sirijonga / 27 Devanagari / 69 mixed. That pack is explicitly
marked gt_status_counts={"unreviewed_ocr_projection": 317} and is not training truth.
The practical next blocker is accepted reviewer corrections/insertions, not more eval
harness.
2026-06-23 layout-gap ink guidance: layout-gap rows now include image evidence, not
only geometry. tools/run_limbu_book_ocr.py writes gap_ink_dark_pixel_count,
gap_ink_density, gap_ink_bbox, gap_ink_row_bands,
gap_candidate_text_band_crop_paths, gap_candidate_text_band_crop_sha256s,
gap_candidate_text_band_count, gap_horizontal_rule_count,
page_context_path, page_context_sha256, gap_review_priority, and
gap_recommended_action into
layout-gap-review-queue.tsv, and the dashboard/README surface priority/action counts.
The page context image highlights the suspicious y-band on the full prepared page, so a
reviewer can reject rules, headers, and spacing before treating a crop as a missed text
line. When a gap has a candidate text-like band, the dashboard shows focused
candidate-band crops in addition to the full gap crop so transcription can target the
missed line instead of a wide spacing band.
The applicator remains compatible because these are additional review columns; accepted
insertions still require explicit review_status and corrected_text. Re-running the
packet writer against the latest p1-only triaged projection at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/layout-gap-review-ink-guidance/
kept the one suspicious gap but downgraded it to low-priority
likely_rule_or_spacing: 5737 dark pixels, density 0.038029, 0 candidate text
bands, 3 horizontal-rule bands, and no candidate-band crops. This corrects the
earlier geometry-only warning: the p1 gap is likely section rule/spacing, not an actual
missed body line. Page triage now consumes the gap-review summary before assigning
operator flags. The refreshed
real p1 triage check at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/ink-guided-page-triage-check/
keeps suspicious_gap_count=1 for auditability, but writes
candidate_text_gap_count=0, horizontal_rule_gap_count=1, removes
possible_missed_line_gap, and recommends review_low_confidence_lines instead of
layout-gap inspection.
2026-06-23 layout-gap review evidence hardening: accepted layout-gap insertions now
preserve focused candidate text-band crop paths/hashes in both inserted-line metadata
and layout-gap-insertion-pairs.jsonl when those crops exist. The applicator validates
the full gap crop plus every candidate-band crop before accepting a row; the independent
application audit now revalidates the same candidate-band crop hashes after the fact.
It also checks every insertion-pair row against the actual inserted document line and
insertion metadata: page id, gap row id, inserted text, bbox, full crop path/hash, and
candidate-band crop lists must agree.
This keeps reviewed missing-line insertions tied to the actual image evidence that
prompted them, and prevents stale/tampered review TSV rows from feeding the later
training-pack exporter. It is deliberately scoped to OCR review integrity, not a new
general eval harness.
2026-06-23 focused low-confidence review view: the book parser now writes
book-line-focused-review-queue.tsv and book-focused-review-dashboard.html in
addition to the exhaustive canonical line-review queue and full priority queue. The
focused queue includes every critical/high/medium row and only a deterministic,
evenly spaced sample of spotcheck_low_confidence_text rows per page (default 25,
configurable with --focused-low-confidence-limit-per-page). This is review-burden
control, not line dropping: the canonical queue still preserves every low-confidence
flag and remains available for exhaustive review/application. Page triage filters the
focused queue along with the canonical and priority queues after explicit page
rejection, and the book-output audit now verifies the focused queue hash, focused
summary counts, and focused-dashboard card count. The combined review-application
wrapper records the focused queue and summary for provenance, but it still applies the
canonical review queue; using the sampled focused queue for application would silently
skip rows. The book-output audit now also proves every focused row is present in the
canonical queue, so a detached or manually edited focused TSV cannot pass just because
its hash and card count match. The combined review wrapper refuses an explicit focused
queue as --book-line-review-queue; the canonical queue is the only application
source. To make the focused view usable without hand-copying edits,
tools/merge_limbu_book_focused_review_queue.py overlays reviewed focused rows back
onto the full canonical queue while requiring identical headers and identical
non-review/provenance fields. Only review fields are merged, and detached/tampered
focused rows are refused.
Refreshed real-book evidence:
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-focused-review-view-stable-pageids/
was run under /tmp/ocrtech-paddle311/bin/python with the v4 router held fixed,
duplicate-source skipping, and stable gorkhapatra-unique-current-p000* page IDs. It
passes the book-output audit with 2 pages, 692 preserved lines, and 599 canonical
book-line review rows. The new focused queue contains 369 rows: all 319
must-review rows plus 50 sampled low-confidence spot checks, omitting 230
low-confidence rows from the bounded working view while keeping them in the canonical
queue. After explicit source rejection of the known non-Limbu page
gorkhapatra-unique-current-p0002, the p1-only projection at
.../manual-triage-check/triaged-p1-only/ passes audit_limbu_book_output with zero
issues. The actual Limbu page now has 230 canonical review rows and 35 focused
operator rows (10 must-review rows plus 25 low-confidence spot checks). The
page-triage projection now regenerates filtered review dashboards instead of leaving
the operator without a visual surface after stale dashboards are cleared; the p1-only
artifact has canonical and priority dashboards with 230 cards each and a focused
dashboard with 35 cards, with no rejected-page card. This is the packet to review
next; it does not claim recognition improved. A no-op merge of the current pending
focused queue into the full canonical p1-only queue writes
.../manual-triage-check/triaged-p1-only/book-line-review-queue.focused-merged.tsv
with 230 output rows, 35 focused rows consumed, and 0 changed rows; once focused
review is filled, that merged full queue is the one to pass to review application.
Codex made a conservative first reviewed projection from the focused contact sheets by
marking only obvious non-body/page-furniture rows as drops: img-line-0179
footer/contact metadata, img-line-0180 date/header metadata, img-line-0193
portrait detection, img-line-0001 masthead, img-line-0002 website, and
img-line-0039 decorative page ornament/icon. The edited focused queue is
.../manual-triage-check/triaged-p1-only/book-line-focused-review-queue.codex-obvious-drops-v2.tsv;
the merged full canonical queue is
.../manual-triage-check/triaged-p1-only/book-line-review-queue.codex-obvious-drops-v2-merged.tsv
with 230 output rows and 6 changed rows. Applying that full queue produces
.../manual-triage-check/triaged-p1-only/reviewed-obvious-drops-v2/final-book/: 6
book-line drops, 317 remaining lines, 0 layout-gap insertions, and a passing
audit_limbu_book_output with zero issues. Exporting a reviewed training pack from
this drop-only projection correctly refuses with "no training rows were exported";
that is desired because no accepted text correction exists yet.
After the scope correction, the useful next packet is not another eval harness but a
small pending operator packet. Reusing the existing unresolved-review packet writer on
the edited focused queue writes
.../manual-triage-check/triaged-p1-only/pending-focused-review-packet-v2/. It has
29 pending rows, skips the 6 already-dropped focused rows, contains only existing
book-line checks, and audits cleanly with
tools/audit_limbu_book_unresolved_review_packet.py (audit_passed=true,
issues=0). Every row is currently existing_low_confidence_line with suggested
action correct_existing_line_or_accept. The packet HTML is now a real review station,
not just a static contact sheet:
.../pending-focused-review-packet-v2/book-unresolved-review-packet.html shows each
crop with editable review_status, prefilled corrected_text, optional split fields,
reviewer, notes, local browser autosave, per-row quick actions, and an "Export reviewed
TSV" button that preserves the packet TSV schema for the existing applicator. The quick
actions accept current text, mark corrected text, drop existing lines, keep pending, and
jump to the next pending row; accept/correct refuses blank text in the browser UI.
Candidate missing-line rows now also show neighboring OCR lines around the proposed
geometry insertion point, which gives reviewers local reading-order context before
accepting or correcting a missed-line crop.
Browser autosave is only refresh protection; the exported TSV remains the review
handoff artifact. Export is browser-preflighted for missing reviewer names, blank
accepted text, machine-looking reviewer names, invalid statuses, and invalid drop/reject
rows; the command-line packet audit remains authoritative.
tools/audit_limbu_book_unresolved_review_packet.py --packet-tsv and
tools/apply_limbu_book_unresolved_review_packet.py --packet-dir now support applying
an exported book-unresolved-review-packet.reviewed.tsv against the original packet
provenance without overwriting the source TSV. The exact commands live in
docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md.
tools/apply_limbu_book_reviewed_packet_to_training_pack.py is now the preferred
post-review command: it applies the reviewed TSV to the cleaned book, audits the final
book, exports the reviewed-only training pack, and fails if zero reviewed training rows
are produced. This is the immediate blocker before training: a reviewer must accept or
correct those line texts in the packet HTML/TSV, then run the handoff wrapper against
the already-cleaned reviewed-obvious-drops-v2/final-book/ base. A non-claim
mechanical smoke with one copied OCR row marked accepted exposed stale chainable
document-hash recording in the book-line/candidate applicators; after fixing those
hashes, the smoke passed reviewed packet audit, application audit, final book-output
audit, and exported training_rows=1. The smoke artifact was deleted and is not
training data; the same contract is now covered by a strict synthetic wrapper
integration test. The handoff wrapper now writes a README.md beside the report with
artifact paths and the next split-manifest / train-recognizer commands, while
stating that the handoff is not a CER/WER quality claim. A pending-only
mechanical proof now exists at
.../pending-focused-review-packet-v2/application-pending-on-reviewed-drops-base/:
29 packet rows, status counts {"pending": 29}, 0 book-line rows applied, 0
candidate rows applied, final line count 317, packet-application audit passed, and
book-output audit passed. The packet splitter now also passes the reviewed packet crop
path/hash into the generated book-line review TSV as source_crop_paths and
source_crop_sha256s, so accepted packet corrections keep crop evidence in correction
metadata and later training provenance. The unified packet audit now has a reviewed
mode (--allow-reviewed) and the unified applicator runs that audit before splitting
or applying the TSV. Pending-only packets still pass as no-ops, but accepted/corrected
rows require explicit reviewer text that does not look machine-generated and explicit
corrected_text or split_corrected_texts; drop/reject rows require reviewer identity
and must target existing book lines. The current pending proof records
reviewed_application_ready=false, which is correct because no human text labels exist
yet. The unified application audit now also verifies the packet-audit file record,
review_packet_audit_passed, reviewed_application_ready, accepted/drop status
counts, and requires reviewed_application_ready=true whenever the application changes
book text. The unified applicator now runs that application audit itself after writing
the final report, writes book-unresolved-review-packet-application-audit.json, and
fails the command if the audit does not pass. Do not detour into broader scoring until
those accepted corrections exist.
The reviewed training-pack exporter now runs tools/audit_limbu_book_output.py on the
source book output before it writes labels and records that audit in
source-book-output-audit/. A real retry against
reviewed-obvious-drops-v2/final-book/ wrote a passing source-book audit at
.../reviewed-obvious-drops-v2/training-pack-reviewed-after-source-audit/source-book-output-audit/
(passed=true, issues=[]) and then correctly refused the export with "no training
rows were exported" because the projection still has only drops, not accepted text
corrections. This is the desired state before human review: source consistency is
proven, but no labels are minted.
The same exporter now recognizes candidate_line_review_insertion metadata as
explicit review evidence, in addition to book-line corrections/splits and layout-gap
insertions. This matters for missed-line recovery: accepted candidate/missing lines
inserted through the unified review packet will now become reviewed training rows
instead of being silently skipped after application.
Direct candidate-line review application now also validates crop evidence whenever a
review row supplies crop_path or crop_sha256: both fields must be present, the crop
must exist, and the SHA-256 must match before insertion. The inserted-line metadata and
candidate-line-insertion-pairs.jsonl now preserve crop_path and crop_sha256, so
reviewed missed-line training rows stay tied to the crop evidence that justified them.
The reviewed-application path was hardened after this real run exposed stale metadata.
Book-line and layout-gap chainable outputs now clear inherited focused queues and
review dashboards, because old focused rows cease to be a true subset once canonical
review statuses change. They also recompute source_book_document_json_sha256 when
rewriting the source document pointer. The book-output audit now checks markdown beside
the recorded document_json, while keeping output_dir tied to the original OCR page
bundle. This makes corrected/drop projections auditable without pretending old review
dashboards still describe them.
2026-06-22 crop-bound correction application: tools/apply_limbu_book_line_reviews.py
now validates source_crop_paths and source_crop_sha256s on every actionable
book-line review row before applying accepted/drop corrections. A stale or tampered crop
hash fails the apply step, and successful corrections carry the crop paths/hashes into
line metadata and correction pairs. This keeps the unresolved-line review loop tied to
the actual reviewed crop images, instead of accepting detached TSV edits.
2026-06-22 operational missed-line triage: tools/run_limbu_book_ocr.py now reads
the final per-page document.json geometry and records a layout_gap_summary for each
book page. Pages with unusually large vertical row gaps get the operator flag
possible_missed_line_gap, and pages whose geometry cannot be read get
line_geometry_unavailable. This is intentionally not another eval harness layer; it
is a front-door run-summary warning that a page may have preserved confident text while
still missing a visible line region. The runner now also writes
layout-gap-review/layout-gap-review-queue.tsv and
layout-gap-review/layout-gap-review-dashboard.html, with hash-bound crop bands for
each suspicious vertical gap, so the operator can inspect likely missed-line regions
directly from the run output. tools/apply_limbu_book_layout_gap_reviews.py now applies
accepted rows from that queue as inserted reviewed lines in a new chainable book
projection, requiring non-empty corrected_text and validating the gap crop SHA-256
before it writes book.md, book.body.md, book-documents/, score-documents/,
book-run.json, and layout-gap-review-application-audit.json.
tools/audit_limbu_book_layout_gap_review_application.py verifies that application
bundle: source run hash, review queue hash, accepted crop hashes, corrected document
hashes, score-document projections, insertion-pair count, and current-versus-inherited
inserted-line counts.
tools/apply_limbu_book_review_queues.py now wraps the reviewed-book operator flow:
given a top-level runner output, it applies book-line corrections first, then
layout-gap insertions, audits the final gap application, and writes the final reviewed
book under final-book/ with source/intermediate/final hashes in
limbu-book-review-queues-application.json. The initial OCR runner README now prints
the exact apply_limbu_book_review_queues.py command for the run, and the wrapper
writes its own top-level README pointing at the final editable text, final book-run,
and consistency audit.
2026-06-22 operator runner correction: after the eval-harness work started to
outgrow the immediate product need, the default path was collapsed into one operational
command instead of another scoring layer. tools/run_limbu_book_ocr.py now takes a raw
book-page manifest, prepares captures, runs the selected v4 preserve-mode router over
the prepared manifest, writes book-level editable text and review queues, runs the
existing book artifact audit, and emits limbu-book-ocr-run.json. It does not change
detector or recognizer behavior and it is not an OCR-quality claim. The live Sirijonga
one-page smoke at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve/ produced
the same operational counts as the manual flow: 1 page, 323 preserved lines, 230
book-line review rows, script counts 286 Sirijonga / 27 Devanagari / 4 mixed /
6 other, and a passing book artifact audit with zero issues. Use this runner for
tomorrow's raw page batches unless a page needs manual crop/perspective values in the
manifest. The runner now also emits operator attention flags in the top-level JSON,
README, and CLI output for pages with no detected lines, all/high review burden, high
other script share, or failed page artifact audit. These flags are triage cues for
inspection/reshoot/review, not eval metrics.
2026-06-22 folder ingestion path: tools/build_limbu_book_page_manifest.py now
turns a directory of page images into the raw JSONL page manifest with deterministic
natural filename ordering, generated page IDs, and source image hashes. The book runner
also accepts --image-dir directly, generates raw-page-manifest/book-pages.jsonl
inside the run directory, and then follows the same capture-prep -> preserve-OCR ->
book-audit path. A real one-page folder-input smoke at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir/
matches the manifest-based runner: 1 page, 323 preserved lines, 230 book-line
review rows, script counts 286 Sirijonga / 27 Devanagari / 4 mixed / 6 other,
and a passing book artifact audit with zero issues. This is the practical entry point
for tomorrow's captured page folders; hand-edit the generated manifest only when page
order, crop, or perspective needs per-page correction.
2026-06-22 book review dashboard: multi-page book runs now emit
book-ocr/book-review-dashboard.html, a book-level contact sheet built from
book-line-review-queue.tsv and the existing per-page line-crop manifests. The
dashboard is hash-bound in book-run.json, and tools/audit_limbu_book_output.py
checks both the dashboard hash and card count against the book-line review queue. The
book-line TSV now also carries source crop paths, crop SHA-256s, and crop warnings for
each review row, and the book audit verifies those crop hashes; for line-assembled or
reading-order-profiled documents, these crop references resolve back to the original
per-page OCR bundle instead of the derived document directory. The
real one-page folder-input smoke at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir-dashboard/
has 230 dashboard cards and 230 crop images, matching the 230 book-line review
rows. This closes the immediate review-surface gap for tomorrow's page folders: the
operator no longer has to open one per-page dashboard per page just to review weak
book OCR lines.
2026-06-22 real 4-page folder run: the current folder-based runner has now processed
the full 4-image Gorkhapatra input directory end-to-end at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-4p-runner-v4-preserve-image-dir/.
It generated the raw page manifest, prepared all captures, ran v4 preserve-mode OCR,
wrote book Markdown/body Markdown, score documents, book-line-review-queue.tsv, and
the book dashboard, and passed the book artifact audit with zero issues. Aggregate
counts: 4 pages, 1384 preserved lines, 1198 book-line review rows, script counts
682 Sirijonga / 64 Devanagari / 30 mixed / 608 other. Per-page counts expose
the next real failure pattern: pages 1 and 3 each have 323 lines and 230 review rows
with mostly Sirijonga output, while pages 2 and 4 each have 369 lines and all 369
lines in review, including 298 other rows. Visual inspection and the source-pack
metadata showed this is not a clean four-page Limbu book target: page 2 is a Nepali /
Devanagari नयाँ नेपाल page, the pack records both लिम्बू and माझी candidates, and
the -01/-02 image pairs contain duplicate SHA-256 content across candidates. Treat
this output as a useful end-to-end book plumbing smoke, not an OCR-quality result or a
detector failure diagnosis. tools/run_limbu_book_ocr.py now records per-page line,
review, script, and audit counts in limbu-book-ocr-run.json and the run README so this
kind of bad-page pattern is visible without opening book-run.json.
2026-06-22 source-folder correction: the operational fix is source selection, not a
larger eval harness. tools/build_limbu_book_page_manifest.py and
tools/run_limbu_book_ocr.py now support --include-filename-regex,
--exclude-filename-regex, and --skip-duplicate-images. The manifest summary records
discovered, skipped, duplicate, and kept images, including duplicate SHA-256 groups, so
mixed export folders cannot silently masquerade as clean book batches. The runner also
copies those source-selection counts into limbu-book-ocr-run.json, the run README.md,
and CLI output, so an operator does not have to open the lower-level manifest summary to
notice skipped pages or duplicate captures. On the current Gorkhapatra pack, filtering
to .*-01\.png$ and enabling duplicate skipping produced
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-filtered-01-dedup-manifest/book-pages.jsonl
with exactly one unique kept target image; the later 0002-01 file was skipped as a
duplicate of 0001-01, and the -02 files were skipped by the filename filter. The
practical rule for tomorrow is: first build or let the runner build a hash-bound page
manifest, inspect the summary for skipped/duplicate/mixed-language evidence, then OCR
only the selected page set. Do not spend more time expanding metrics until the input
batch is actually the book pages we intend to digitize.
The manifest builder itself now emits the same source attention flags before OCR:
source_images_skipped, duplicate_source_images_removed, and
duplicate_source_images_present. Use tools/build_limbu_book_page_manifest.py as the
cheap preflight when a folder may be an export pack rather than a clean book sequence.
The current Gorkhapatra source preflight at
outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-current-source-preflight/
returns pages=4 and source_attention_flags=duplicate_source_images_present without
running OCR.
Missed-line rescue follow-up: generic secondary morphology rescue was tested and
rejected. The compact long-line rescue raised mean line F1 from 0.605257 to
0.626903 but worsened page codepoint CER from 0.458432 to 0.464766; broader
normal/small-fragment rescues regressed to 0.537609 and 0.832146 codepoint CER by
reintroducing noisy extra predictions. Evidence and the geometry review packet are in
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/. The next move should split
oversized low-IoU detections and recover missed bottom/right and top/center lines, not
add another undirected rescue pass. A narrower split-wide component option was then
implemented in both the materializer and operational parse-limbu-doc image-line
path, but CDC scoring showed it was a no-op: every tested split-wide variant kept
{"primary": 33} detector source counts and the same page codepoint CER 0.458432
(metric SHA-256 b1d993e9fcf09a7cff18d31c3c7e43ee2ff34cd8a57eaae96526e9f977d4c5cb).
Do not spend the next pass on simple wide-component whitespace splitting; use the
geometry review packet and failure analysis to drive page-region/row recovery. The
concrete CDC target list is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/devanagari-cdc-page-region-recovery-targets.md:
12 misses split into 3 no-overlap rows, 4 overmerged rows, 2 partial detections, 1
low-IoU refinement, and 2 page markers. Follow-up existing-row repair tests were also
negative: same-row merge gap80 regressed page codepoint CER to 0.463975, detected-row
split gap24/min700 regressed to 0.467933, and detected-row split min900 was a no-op.
The detected-row split path is now wired into both the standalone materializer and
operational parse-limbu-doc / book-manifest image-line runs, but it is rejected as a
default by that CDC evidence; next work should target absent-row and short page-marker
recovery from page regions. Region-filtered rescue is now the first
positive result in that direction: the materializer and runtime image-line rescue pass
parsers support region_min_x_ratio, region_max_x_ratio, region_min_y_ratio, and
region_max_y_ratio, and the tight bottom-long + top-title CDC diagnostic improves
page codepoint CER from 0.458432 to 0.452890, grapheme CER from 0.645161 to
0.637481, mean line F1 from 0.605257 to 0.635724, and matched references from
19 to 21 with 36 detected lines. Score:
outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80-region-rescue-bottom-long-top-title-tight/metrics.json
(SHA-256 1c787c10392edf4f8a27dac018b5406b3a960b963a419aa4a64e9ec1c0acc0af). The
conservative page-marker extension is a page-CER tie and the current line-structure
best: it keeps codepoint CER 0.452890, raises mean line F1 to 0.659689, and
matches 22 references with 37 detections by recovering the social-page २ marker
(score SHA-256 19846ebae905411387ccbb88ee9520021387aac9f3debcc4ce81e88a87e1c836).
The runtime image-line path and standalone materializer now tag conservative
bottom-margin digit-only small-box detections as structural_role=page_marker; a fresh
h20 structural rerun marks exactly the recovered social-page २ marker and keeps the
same metric SHA-256, so the change is metadata-only for current scoring. It is not yet
script-count exclusion or final page-order handling. export_document now also writes
document.body.md, which omits structural blocks while preserving the full document.md
and document.json; the one-page operational social run at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/operational-parse-social-page-marker-structural
keeps the marker in full markdown, excludes it from body markdown, binds the body file
in limbu-output-manifest.json, and passes audit_limbu_output.
The remaining h20 unresolved regions now have a textless review packet at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/:
24 crop rows, split into 9 missed-reference regions and 15 extra-prediction regions.
Its JSONL/HTML hide reference text, suppress 2 exact machine-text echoes of hidden
reference strings, and currently have zero hidden-reference text leaks against the
failure analysis. The packet now has an independent audit tool,
tools/audit_unresolved_page_review_packet.py; the current audit artifact
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/unresolved-region-review-audit.json
passes with 24 rows, 0 hidden-reference leaks, and 0 issues (SHA-256
33105c0144c5cfa56ee20554a6aee82434f9c4cb6c19a60cc9533882a918db5b). This
satisfies the review-surface requirement for the current CDC diagnostic, not the
underlying missed-line recovery requirement.
A targeted h20 + same-row merge gap120 follow-up was tested for the remaining
top-title/fragment near misses and rejected. It raises mean line F1 to 0.694815, but
page codepoint CER regresses to 0.465558, grapheme CER to 0.654378, matched-line
CER to 0.190031, and reading-order pair accuracy to 0.571429; the refreshed
failure analysis still has 9 unmatched references and dominant missed-reference error
mass. Do not chase broader same-row merging as the next default path.
A narrower capped same-row merge (gap160, width500, y_tolerance=25) was also
tested to merge only short top fragments. It produced 34 lines with 2 row merges,
improved micro line F1 to 0.707692, and reduced missed references to 8, but page
codepoint CER still regressed to 0.460808, grapheme CER to 0.643625, and
matched-line CER to 0.185259 (score SHA-256
61129eaa91445c0d8a86f66c5a5827ffa342b331bffe032d12f60d9476ad0b32). Reject it as a
default path too; merged crops are damaging recognition even when the geometry looks
better.
The lower-height h10 marker extension recovers more single-digit markers and matches
23 references, but it regresses page codepoint CER to 0.454473, grapheme CER to
0.639017, WER to 0.844707, and reading-order pair accuracy to 0.538462; keep it
diagnostic-only, not the default path. The refreshed failure analysis is in
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-failure-analysis-devanagari-cdc-region-rescue-bottom-long-top-title-tight/;
missed reference text is still dominant (45.78% of codepoint error mass), so this is
progress, not finished book OCR.
Combining h10 markers with source-break post-recognition line assembly was also tested
and rejected as the default: page codepoint CER is 0.452890, line recall rises to
0.806452, but precision falls to 0.675676, extra predictions rise to 12, two
blank predictions appear, and reading-order pair accuracy is only 0.553191. Score:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-cdc-h10-page-markers-line-assembly-gap160-width500/metrics.json
(SHA-256 6ccf1bce5263f0eed1fc966a6d2f9c3c6567534fdd750e7a05549462a12b6c57).
The detected-tall geometry candidate has now been scored and rejected as the default.
Operational parse-limbu-doc supports primary/rescue min_aspect_ratio and
max_aspect_ratio, plus opt-in image_line_split_detected_tall_* settings. The
geometry probe on the science page split the known bad rescue-1 box
[319, 1856, 1075, 115] into [319, 1856, 1052, 37] and [317, 1944, 1051, 29].
Probe:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/detected-tall-split-geometry-probe.json
(SHA-256 51b364b4211f40d54714c903b9038fab284fc83e7853efa85ccc3ebb32225c64).
The Paddle follow-up was run through the book wrapper in a dedicated local Python 3.13
Paddle environment. It produces 3 pages, 38 source lines, 28 review rows, and a passing
book artifact audit (limbu-book-output-audit.json SHA-256
ceb9d681e4ee50ac48880f41b5c0dc25e084cd4cfcc43b0774d573366634210e), but the score
regresses page codepoint CER to 0.478226, grapheme CER to 0.674347, and WER to
0.882958. It improves line recall/F1 to 0.838710 / 0.776119 and cuts missed
references to 5, but matched-line CER worsens to 0.191242 and reading-order pair
accuracy is only 0.578431. The candidate admission gate rejects it
(admitted_operational=false, admission SHA-256
2d56eaaf0071fa17341d9905aaa7bc46d5bb89fa40c1428e936645e7727665dc) because it
regresses against the 0.458432 codepoint-CER baseline and misses the 0.60
reading-order threshold. Keep detected-tall split as recall/review diagnostic evidence,
not default book OCR text.
That reuse path now exists: tools/prepare_candidate_line_review_packet.py compares a
selected document set against a secondary candidate document set and writes
candidate-only line crops whose max selected-line IoU is below a configured threshold;
tools/audit_candidate_line_review_packet.py checks hashes, crops, and row shape. The
detected-tall-vs-current-best packet at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-only-review-packet/
has 4 rows (maths marker १, two science bottom-row split candidates, and science
marker २) and passes audit with 0 issues (packet SHA-256
f20438d05d158377387a05df9197cc3a6e7e9afec573a60668c1d429d1b83128, audit SHA-256
919615e48fe66c24810e6137b5bb65dd64b348ebfee0b5817af38b47f38c2f40). This is now the
preferred way to exploit high-recall rejected geometry: review it as missing-line
evidence without contaminating selected book text.
Accepted candidate-only rows now have an application lane:
tools/apply_candidate_line_reviews.py. It inserts reviewed secondary-candidate lines
into a new book projection with provenance metadata, writes full/body markdown,
score-documents, and insertion pairs, and leaves the measured selected run untouched.
The real detected-tall packet is still all pending; the pending smoke at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-line-review-application-pending/
has review_rows=4, applied_count=0, skipped_count=4, empty insertion pairs
(SHA-256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855), and
preserves selected book.md / book.body.md byte-for-byte. Audit SHA-256:
95521bc3c5ad86bd6cdf19350cfa1323a972ce817ee363451ff79349ccc4f69f. The application
output now also has an independent artifact audit,
tools/audit_candidate_line_review_application.py; the real pending application audit
passes with 0 issues and SHA-256
0e6addc547c1ef8df3ae9fe54d6599a16341fb2d57168d3b785da4cbe38cf452.
The pending application projection has also been scored and admitted through the same
book candidate gate, using that independent application audit instead of a normal book
audit. The score is byte-identical to the current selected h20 source-break
line-assembly metrics (SHA-256
184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244): page codepoint
CER 0.446556, line F1 0.750000, missed/extra/blank counts 7 / 9 / 0, and
reading-order pair accuracy 0.611765. Admission passes operationally with
admitted_operational=true, claim_safe=false, and SHA-256
eb0e559908046322343a7507a0143ae2d724876c069eb2528f0d1950085fd5b5. This proves
reviewed candidate-line application outputs can use the same audit/score/admission
lane; it does not add missing lines until rows are actually accepted.
An automatic candidate-line selector now exists, but it is not a default OCR policy.
tools/select_candidate_line_reviews.py implements page_marker_v2, limited to
bottom-margin digit-only structural page markers with confidence, selected-line IoU,
size, and duplicate-text guards. On the real detected-tall packet, v2 selected only
the maths marker १ and rejected the other 3 rows (selection report SHA-256
928aaecd9a102c5615e1085b3d29a80188d419c35e17bbfc9c743876f7e20872; selected JSONL
SHA-256 8152ba724a67a4ef25296c6dcd47a02c7050262356065d6c0c0979fa4f366867; rejected
JSONL SHA-256 a3a571ad12c8efade971d0b89baa0ef9b1345cb15d97beba3ec284f5089b3e85).
The applied v2 projection passes application audit (SHA-256
b0e377bec4f0fca9f4506fa9126a65c57014adf7d419a04e08f98d50bf3b8743) and improves line
recall/F1 to 0.806452 / 0.769231, but page codepoint CER regresses to 0.448139
and reading-order pair accuracy drops to 0.553191. Admission rejects it
(admitted_operational=false, admission SHA-256
9cd826a51b38aa29facab4ec332a4084421ac25bf26365dc04f954e6f1b5a76f). Treat this as proof that page
markers need separate structural export/scoring semantics; do not insert them blindly
into the selected editable text stream.
tools/run_page_ocr_eval.py now exposes that separation as diagnostics, without
changing the all-line metrics used by admission. It counts non-text
metadata.structural_role predictions, matches, unmatched predictions, exact text
matches, and per-role totals. Structural-metric replays show current h20 assembly has
one exact matched structural page marker (metrics SHA-256
f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f), while the v2
auto-marker application has two exact matched structural page markers (metrics SHA-256
2634d0f70f6dac3cac08844ea1530aff08c26271a92a7eb2c725a879415ce5b9). That added marker
explains the improved line recall/F1 (0.806452 / 0.769231), but it does not justify
promotion because page CER and reading order still regress. Use these structural
metrics as evidence, not as a substitute admission gate.
The candidate-only review lane now has an explicit oracle/reference-overlap diagnostic
for body-line recovery: tools/select_candidate_line_reviews_by_reference_overlap.py
selects rows only when they overlap currently unmatched reference lines, and labels the
output oracle_reference_overlap_v1. On the real detected-tall packet it selected one
text row, the science bottom line, with IoU 0.860889 to
limbu-devanagari-cdc-science-p0009-l0009 (selection SHA-256
d11dcdc1cc0b07626e6d85b97356c86cadd670796b9e926708e2be7585344d05). Applying that row
with reference text as corrected text gives the review-ceiling score: page codepoint CER
0.420428, line recall/F1 0.806452 / 0.769231, missed/extra counts 6 / 9, and
reading order 0.634409 (metrics SHA-256
247c4186059047fc9de16019f358172bc46261de497127581f63335f986cb521). Applying the
same row with candidate OCR text gives page codepoint CER 0.445764 but matched-line
codepoint CER worsens to 0.191413 (metrics SHA-256
21fd79e6a0645c0084cac959743354f52c224b28b15e4e60be7f56e3ac664d75). The audit and
admission path now blocks these oracle/reference-assisted applications from operational
admission: application audits record operational_review_safe=false and admissions
reject them (c8f69e7d011c217e74218ffe280e901420ab99ebefa7f329344af776a6e779c7 for
reference text, 2a97b7f01425efdd514887324c8c23c6b4dd46c48376b99c2299fad736d88b45
for candidate text). This proves the geometry surface contains real missed-line
evidence, but direct candidate text is not enough; the next useful path is reviewer
correction or better recognition on detected-tall body candidates.
A non-oracle reviewer priority queue now exists for that next path:
tools/prioritize_candidate_line_reviews.py. It consumes the candidate-only review
JSONL and ranks rows using only packet-visible evidence: structural role, candidate
text, confidence, selected-line IoU, bbox size/position, and crop links. The real
prioritized queue at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/
has 4 pending rows: 2 high-priority body-line candidates and 2 structural page-marker
checks. Artifact SHA-256 values: prioritization JSON
ef3fc75efc52597c97ea2e28196164b31ef3ceb18414ef5b47e559cc4343e51a, JSONL
9a53ad3d3ada7e61211d5406ce35c9ed9a10e86a41ea09ea0f4edcb3766ad49e, TSV
753adcd61f70f490b0429d6ff9d25ff88b0bc1ac8ec20a2f9ae21ec1f514fa19, HTML
5fbd7f39801ac7e29b0503314475ef26e6e61345b26330720bf840ad1e2dcdef. The independent
queue audit at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/prioritized-candidate-line-review-audit.json
passed with 0 issues (SHA-256
d2fe0e33e3bd5f78698632892581651a3e95332936b7e9ddfa926c1ecc3fcd22), confirming 4
source rows, 4 JSONL rows, 4 TSV rows, no oracle/reference-assisted review rows, and
operational_review_queue_safe=true. Ranks 1-2 are the two science bottom-row text
candidates; both remain pending with suggested action
transcribe_visible_line_or_reject. This is the safe reviewer-facing artifact to use
before applying candidate lines; it is not an automatic selector and does not change
the current selected OCR candidate.
A text-focused crop probe now exists for those body-line rows:
tools/prepare_candidate_line_text_crop_probe.py. It is non-oracle and only prepares
recognizer/reviewer triage crops; it does not alter selected OCR output. On the real
prioritized queue it processed 2 body-line candidates and refined both: rank 1 was
top-bleed-trimmed from 1100x85 to 1100x63, and rank 2 was trimmed left of a
detected horizontal rule from 1099x77 to 466x77, removing the rule and right-side
illustration (probe SHA-256
7f1dfc09d4cf612e7516dbdea07eb6bc19608e637ad11efe41e8a8112393c79a, JSONL SHA-256
a7d7cced3cdca1a223aa6c6d00ccc70acabee90183107eac6b70ff61b630f011, label-file
SHA-256 47b3594dac7a97f516d21f271c1fe1e794f6d621bfc17539d1cf6b8e540ef4e0, HTML SHA-256
10db1161085a49e722ad373b88ff9656aaf87b5423dad2ec88e8f1c48f1f18b6). The Devanagari
PP-OCRv5 line recognizer was then run locally over the refined crops in both
.venv-paddle313 and /tmp/ocrtech-paddle311; both produced identical predictions
(TSV SHA-256 734b208860106d84a7dbd7473e18927293a29b51e8b67298e173c5b2c626bbee, 2 rows,
0 failures). The original-vs-refined comparison at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-crop-recognizer-probe-comparison/
shows this is not promotable: max probe confidence is only 0.469282, rank 1 improves
from 0.251062 to 0.469282, rank 2 improves from 0.363574 to 0.390961, prediction
text changes twice, and
automatic_application_safe=false (comparison SHA-256
18cdfb099ae6d93ee96ee350faec0305837349d65142ebaaad05db1553c04eb7, JSONL SHA-256
f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d). This confirms
the refined crop is useful reviewer/recognizer evidence, not an automatic OCR-quality
improvement.
The application path now supports that TSV workflow directly:
tools/apply_candidate_line_reviews.py --candidate-review-file accepts JSONL or TSV
review files, and tools/audit_candidate_line_review_application.py audits both. A
real pending TSV smoke at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review-application-pending/
left all 4 rows pending (applied_count=0), preserved selected book.md and
book.body.md byte-for-byte, passed application audit with
operational_review_safe=true (audit SHA-256
e36036c0e3a9177e7c526465cf16c4a066f54ae37fe920bfb97c1a3bb9278c5c), scored with
current-best metrics (score SHA-256
f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f), and passed
operational admission (SHA-256
453226159011a944a606e0c951c5b7dceda3bee73c76f44d195c44e2d61672c1,
claim_safe=false). The next non-oracle improvement requires actual accepted reviewer
rows in this TSV, not another automatic promotion.
The first multi-page book wrapper now exists as tools/parse_limbu_book_pages.py. It
reads an ordered JSONL page manifest, runs the existing parse_limbu_document path for
each page, and aggregates per-page bundles into book.md, book.body.md,
book-review-queue.tsv, book-run.json, and scorer-compatible score-documents/
with page hashes, audit status, script counts, total line counts, and review-row
counts. A persistent sidecar smoke run at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-book-sidecar-smoke
has 2 pages, 4 lines, 1 aggregate review row, 0 page audit failures, and script counts
devanagari_limbu=1, limbu_sirijonga=2, mixed_limbu_devanagari=1, other=0
(book-run.json SHA-256
7f9da4cec883e0fc9e72e16f62c1ac633046daea4c86091d25129643376f8819). The selected
h20 structural CDC diagnostic has now been replayed through that book wrapper using
real CDC page images and sidecars derived from the materialized selected page
documents. The prep artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-replay-inputs
(summary SHA-256 c28d0511bc42b855125c8e6a2063715ff15a0d085869a2107e32e7dfb4f411cf);
the book output is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay
(book-run.json SHA-256 9654cebe89ab68618cf5bf36419e5bd3a9513195faf9d4f9eb4ce5481adb3e8d).
It has 3 pages, 35 lines, 27 aggregate review rows, 0 page audit failures, and script
counts devanagari_limbu=35, limbu_sirijonga=0, mixed_limbu_devanagari=0,
other=0. The score projection evaluates cleanly against the replay manifest with
3/3 scored pages, 0 missing documents/references, 0 source-path mismatches, page
codepoint CER 0.446556, page grapheme CER 0.626728, matched-line codepoint CER
0.175130, and matched-line reading-order pair accuracy 0.600000
(score-multi-page-cdc-h20-structural-book-replay/metrics.json SHA-256
1a353b2d06b89caf081d190ad79add2eee57d1d51b70cb6144be40e53d082d75). The replay prep
now drops blank selected-output lines by default and records them; in this CDC replay
it dropped 2 science-page blank lines (img-line-0006, img-line-0007). The scorer
now also exposes the objective's line-count rates directly: 31 references, 35
predictions, 22 matches, 9 missed references, 13 extra predictions, 0 blank
predictions, micro line precision/recall/F1 0.628571/0.709677/0.666667, and
missed/extra/blank rates 0.290323/0.371429/0.000000. This is replay evidence
from existing selected outputs, not fresh OCR inference, but it closes the book-level
packaging/scoring path on real pages.
The replay also exposed a body-export bug: sidecar-replayed structural page markers
could be grouped into body text blocks. src/ocrtech/structure.py now emits
structural lines as their own blocks before normal paragraph grouping, so full
book.md keeps the social-page २ marker and book.body.md omits it. The underlying
OCR quality is still not solved: 27 review rows remain and matched-line CER is
unchanged from the h20 diagnostic.
A confidence-threshold replay sweep confirms that pruning is not the next main lever.
Using tools/filter_page_ocr_documents.py on the CDC h20 structural book replay,
confidence >= 0.20 removes one science-page false positive and improves micro line
F1 to 0.676923 while tying page CER at 0.446556; it is only a possible
review-triage cleanup. confidence >= 0.30 and 0.40 improve precision and F1 but
worsen page CER to 0.447348 and 0.448139, and confidence >= 0.50 drops matched
lines, reducing recall to 0.645161. The sweep is documented in
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md. Do not promote a
confidence threshold as default book OCR output; recover missed/split page lines
instead.
The first post-recognition line assembly diagnostic now exists as
tools/assemble_page_ocr_lines.py. It merges adjacent same-row OCR line records after
recognition, preserving source line ids/texts/bboxes/confidences in metadata, and uses
no reference text. On the CDC h20 structural book replay, gap160 width500 merged only
two top-fragment pairs, reducing 35 OCR lines to 33. It improves micro line
precision/recall/F1 to 0.727273 / 0.774194 / 0.750000, drops missed references
from 9 to 7, and drops extra predictions from 13 to 9. After preserving
original block order, the source-break variant keeps page codepoint CER tied at
0.446556 and page grapheme CER tied at 0.626728, while matched-line CER becomes
0.177052. The space-join variant is better as editable one-line text but nudges page
CER to 0.447348; combining space-join assembly with confidence >= 0.20 worsens page
CER to 0.451306. Treat source-break assembly as the current best score projection
for line structure, and treat space-join assembly as an export/review question. The
source-break artifact has a replay audit from tools/audit_page_ocr_line_assembly.py;
it passes with 0 issues and SHA-256
69eb0db004f0bbabdead620bf63dfc138fb02deb8e3003fb9ec668a88d26afb1. Artifacts are
documented in outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md.
tools/parse_limbu_book_pages.py now exposes this as an opt-in operational book
projection with --line-assembly. The implementation deliberately does not mutate the
audited per-page OCR bundles; it writes line-assembly-source-documents/,
line-assembled-documents/, line-assembly-manifest.jsonl, an assembly replay audit,
and points book.md, book.body.md, and score-documents/ at the assembled
documents. It also writes book-line-review-queue.tsv, an assembly-aware review
surface that groups source review rows by editable book line while preserving source
line ids. The default book-wrapper assembly settings are the measured source-break
projection (y_tolerance=25, max_gap=160, max_merged_width=500,
max_component_width=250, min_vertical_overlap_ratio=0.30, newline joiner).
The integrated CDC replay artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay-line-assembly-integrated
(book-run.json SHA-256
f2591d47ace1ceb5bc676d3d8c645f6905ab250e1e04fcfbe083ac4cac9f1e25). It has 27
source review rows and 26 book-line review rows because the two reviewed maths top
fragments collapse into one assembled book line. Its score has the same metric SHA-256
as the standalone source-break diagnostic,
184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244.
The aggregate book bundle now has an independent artifact audit,
tools/audit_limbu_book_output.py; the integrated CDC replay audit passes with 0
issues (limbu-book-output-audit.json SHA-256
f5470c9ca04d7a3004f8ed1e3ea3fa280da768f29b65e1fd60fa7f8d42f2f6f5). This proves
book-run hashes, aggregate markdown, review queues, score projection, page output
audits, and line-assembly audit consistency. It does not certify OCR quality.
Accepted book-line-review-queue.tsv edits can now be applied with
tools/apply_limbu_book_line_reviews.py, which writes a corrected editable book-line
projection plus book-line-correction-audit.json and
book-line-correction-pairs.jsonl. The correction is intentionally applied to the
assembled book line and records source line ids; it does not pretend that a reviewed
assembled-line edit can always be split back into original OCR fragments.
tools/admit_limbu_book_ocr_candidate.py now gates book OCR candidates against the
objective metric surface: book artifact audit, page CER/WER, line precision/recall/F1,
reading-order accuracy, blank/extra/missed-line rates, and improvement over the
0.458432 CDC baseline. The integrated CDC line-assembly run passes this operational
gate (limbu-book-ocr-candidate-admission.json SHA-256
8e0110ad320cbe09f37893a30725c25db0cd3d40a2b1c77cc8f0b0ab982c6b9d) with page
codepoint CER 0.446556, line F1 0.750000, and absolute codepoint-CER improvement
0.011876. It remains claim_safe=false because the eval has claim_eligible_rows=0
and only 3 scored pages; this is the current operational candidate, not a final claim.
The candidate gate now also supports explicit grapheme-CER and WER non-regression
baselines (--baseline-grapheme-cer, --baseline-wer; zero regression allowed when
supplied). This was added after the recognizer-probe application exposed a real gate
hole. Applying the rank-1 refined-crop machine prediction improves page codepoint CER
to 0.442597, line recall/F1 to 0.806452 / 0.769231, missed references to 6,
and reading order to 0.634409, but it regresses grapheme CER to 0.639017, WER to
0.866565, and matched-line CER to 0.186047. The paired comparison against the
current best is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-detected-tall-recognizer-probe-vs-current-best/comparison.json
(SHA-256 5dff9bbce7158c56fd0489452f60ddd30f8b3cd04ddf31301852665ef4e5fbc8) and is
not claim-safe: codepoint improvement is only 0.003959 over 3 pages with CI low
0.000000, while grapheme CER regresses by 0.012289. The strict multi-metric
admission at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-detected-tall-recognizer-probe-application-conf045-delta005-strict-multi-metric/limbu-book-ocr-candidate-admission.json
(SHA-256 812fb6e5d19553441e97eada4154879c369e4fcd3924d5717e537f77ef066cdb) correctly
rejects it for grapheme and WER regressions. Keep the h20 source-break line-assembly
integrated projection as current best; the recognizer-probe row is review evidence,
not a promotable OCR policy.
The missing-line review evidence is now visible at book level instead of living only
beside the selected OCR artifact. New tool:
tools/prepare_limbu_book_missing_line_review_queue.py. It binds candidate-only rows
from the detected-tall prioritized queue to the current best book pages, validates crop
paths, hashes each crop, and writes a separate pending review queue without altering
book.md or score-documents/. Current code also writes page_context_path and
page_context_sha256 for each candidate when the source page image is available, with
the candidate bbox highlighted on the full page so the reviewer can reject marginalia,
rules, or spacing before accepting an insertion. Newly generated queues also include
book-missing-line-review-dashboard.html, a static reviewer dashboard that shows each
candidate crop next to its highlighted page context and is hash/card-count bound in the
queue report. This is reviewer workflow for book OCR cleanup, not a new eval-harness
track. The real historical artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-detected-tall-prioritized/
with 4 rows: 2 high-priority body-line candidates and 2 structural page-marker
candidates. Report SHA-256:
081cabed352868c26ad5d35fbc77bafc6d95b8ff27281de4d39484e15eadfc20; TSV SHA-256:
4ee99c552a00282e3f1bee0b4511d23b90b072d51adac15e829f9054d7237bf5; Markdown
SHA-256: fd318583ec70ba178fce876cb6f622ca9e7e35ac2aee77a4cf4af754ad66d01a. It now has
an independent audit via tools/audit_limbu_book_missing_line_review_queue.py; the
real audit passes with 4 pending rows, 0 accepted rows, 4 verified crop hashes, no
issues, and SHA-256 13c9ba67c47ac3bc9159606c72d3e77ce3d4fac1657ea563102d6b7078b49a58.
Newly generated queues also audit any recorded page-context image hashes.
This is movement toward the objective's review-packet requirement, not an OCR-quality
promotion. The next real quality step is still reviewer acceptance/correction or a
recognizer/layout path that converts these rows into non-regressing page text.
That reviewer-to-application lane now accepts the book-level missing-line TSV directly:
tools/apply_candidate_line_reviews.py understands the queue aliases
candidate_row_id/candidate_sample_id as row_id/sample_id. A pending smoke from
the audited queue produced
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-application-pending/
with applied_count=0, skipped_count=4, empty insertion pairs
(e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855), and a passing
application audit (SHA-256 cd824e7d7b1b7f7a4a439fe398716c9849acc2bcd3694a3f73124b6466268488).
book.md and book.body.md are byte-identical to the current best h20 assembly
(6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3 and
7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff). This proves
accepted/corrected missing-line reviews can flow into the existing insertion projection
without changing pending OCR output.
There is now one unified unresolved review packet instead of separate queues:
tools/prepare_limbu_book_unresolved_review_packet.py combines the current best
book-line review queue with the detected-tall missing-line queue, renders crops for
existing low-confidence book lines from source page images, copies candidate-only
missing-line crops, and records crop hashes in one TSV/HTML packet. The real packet at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/
has 30 rows and 30 verified crop files: 26 existing low-confidence book lines plus 4
candidate-only missing-line candidates. Report SHA-256
c49c5d69105b24d36556b414f187626516353d2fc15de53fd5f098e7ad0d3adb; TSV SHA-256
d03c811db6fbac072491dd63bd48b36329eb30f873fac0f31495a6f907486a58; HTML SHA-256
f7f1f514fe77c92d763d8a59d19ce14c39019123b14e0aeecf6794caa077b45f. A direct
integrity check over the TSV found 0 missing crops and 0 crop-hash mismatches. This is
now independently audited by tools/audit_limbu_book_unresolved_review_packet.py; the
real audit passes with 30 pending rows, 30 verified crops, no issues, no warnings, and
SHA-256 3254beec46d99f9c5b6af9c72b39feefca8d97948e7e60f126088ee949886289. This is
the current reviewer-facing packet for unresolved lines; it does not use reference text
or change OCR metrics.
The packet-to-application bridge is now wired. tools/prepare_limbu_book_unresolved_review_packet.py
preserves candidate_structural_role, so candidate page markers remain distinguishable
from body text after review. It also carries explicit split_corrected_texts and
split_bboxes columns for reviewer-approved split-line repairs. The regenerated
packet has report SHA-256
3af6d2b767aca7d566529a01f909e850231fca33cba5636c9b042008b07ed5c4, TSV SHA-256
9ecd72b9f804124357cef566e58cc655cac676d0e4c81421093aa6a98d416fed, and audit
SHA-256 396dc11cdb95ed3d632698fa9bf777e65f0ba260abbcd3debe1be2fec390e91c
(audit_passed=true, 30 pending rows, 30 verified crops, no issues/warnings). New
tool tools/split_limbu_book_unresolved_review_packet.py splits the unified packet
into the two applicator-specific TSVs under
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/split-for-application/:
book-line TSV SHA-256 83a40bdd0863d2cb38ed347c100493e24dcbf71b48e01b2d2fb373f6d2fbd890,
candidate-line TSV SHA-256 d6b5426520cccb95be66b9f0ef8c7daf220aa5fd0d9f4896b3682678fa6bca39,
split report SHA-256 586c688f704b1cc09bd93c096d51dc8ab1c7b252cc1cf0b6ae0eb436cf323050.
tools/apply_limbu_book_line_reviews.py now writes a chainable corrected
book-run.json, so the candidate-line inserter can run after book-line corrections
instead of requiring the original source run. The pending chain source run SHA-256 is
738e38a500450f7bd2fb1a516e1a42d08b0e688af9b6bb7b89ab87dfe71e07ef; pending
application audit SHA-256 values are
d06a569829174db6aa680e4a9716b9587485fcc6487860cd147fc5897b3e9411 for book-line
correction and 09dfec36a41bff5ee6c1b0287099734da7de264379f5d544da3cce16169c3eaf
for the chained candidate-line application. Pending-row smoke applications with
tools/apply_limbu_book_line_reviews.py and tools/apply_candidate_line_reviews.py
skipped all rows and kept book.md / book.body.md byte-identical to current best
(6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3 /
7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff). This closes
the mechanical gap from one reviewer-facing packet to a corrected-and-inserted book
projection, but it still does not improve OCR until rows are actually reviewed.
The review workflow is now one command via
tools/apply_limbu_book_unresolved_review_packet.py: split unified packet, apply
existing book-line corrections, then apply candidate missing-line insertions against
the corrected projection. tools/apply_candidate_line_reviews.py now writes a final
chainable book-run.json with updated page/document paths and line counts. Real
pending application output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-application-pending-unified/.
tools/apply_limbu_book_line_reviews.py can now deliberately split one accepted
existing book-line row into multiple TextLine records via split_corrected_texts,
using reviewer-supplied split_bboxes when available. Wrapper report SHA-256
d557307628c2b2c39a9df05d1dba056608003f10999021990457be163c70fbb6;
final book-run.json SHA-256 b9b7696471e28ca702385987a00ae05fa8a9df3b6f2018d894d75c1d83f421aa;
final independent candidate-application audit SHA-256
245b56e4be9e9f1c89f777b45d1f7583f113c9e13d113dbaf412fb4ac2cd0f34. The wrapper
itself is now audited by tools/audit_limbu_book_unresolved_review_packet_application.py;
real audit SHA-256 cb6e3b9748c20282ca045df766c8220293ef274bbf7cc3f665717be6a1fc2913
with audit_passed=true, operational_application_safe=true, 30 packet rows, 0
accepted rows, 0 book-line applications, 0 candidate-line applications, 0 issues, and
0 warnings. The real run split 30 rows cleanly (26 existing book-line, 4
candidate-missing-line), applied 0 because every row is still pending, and kept final
book.md / book.body.md byte-identical to current best. This removes operator glue
from the review loop; it does not promote text until rows are reviewed.
The missed-reference review coverage audit now exists:
tools/audit_limbu_book_missed_line_review_coverage.py. It compares frozen page-eval
reference_unmatched rows from line-alignment.csv against the unresolved review
packet by bbox overlap, so it is diagnostic-only and must not leak reference text into
OCR output. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-current-best-plus-unresolved-packet/;
report SHA-256 743d5c446797e0df0c360a78cea516bbcbb42cfb8de70843504ee0ac784ded07, TSV
SHA-256 ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37.
Result: all 7 frozen unmatched reference rows have review-packet coverage at the
region level: 2 covered_by_candidate_missing_line, 2
covered_by_existing_review_region, and 3 swallowed_by_existing_review_region.
This changes the next engineering move: stop chasing broad detector recall for this
candidate. The remaining missed-line failures are mostly geometry/segmentation inside
already surfaced regions, especially oversized maths rows swallowing short right-side
fragments. Next work should target bbox tightening / row-split handling for existing
book lines, then rescore on the frozen pages.
tools/prepare_limbu_book_line_split_suggestions.py now generates a bounded review
aid for that split work. It reads the unified unresolved packet, verifies crop hashes,
and emits visual column-projection subspans for oversized pending existing_book_line
rows so a reviewer can populate split_corrected_texts / split_bboxes without
starting from a blank crop. It also writes an HTML review page with original crops
beside overlay PNGs. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-suggestions-current-unresolved-packet/;
report SHA-256 428eb35644115487aa48530db7b27d3854fcdb3c2603c49f1b3376c5fdba2f6d,
TSV SHA-256 b5890b32a355718666530c23085197e7a2404524bdbb78f04bc8da6921681c43,
HTML SHA-256 a9b416efb8ea5045e6bdd8af9723916274914b95f30621b05c616b84e0c129d1.
It now has an independent audit via tools/audit_limbu_book_line_split_suggestions.py;
audit SHA-256 9b7bdf8ae83ed79b2ee36b3d8031ebe26ce930a6321f996761bbd8dab9f81671,
audit_passed=true, operational_review_aid_safe=true, 19 suggestion rows, 109 total
visual subspans, 19 overlay PNGs, 0 issues, and 0 warnings. It produced 19 suggestion rows, including the swallowed-row packet IDs
existing-0005-limbu-devanagari-cdc-maths-p0008-img-line-0007 and
existing-0011-limbu-devanagari-cdc-maths-p0008-img-line-0013; the narrow
right-fragment case existing-0007-limbu-devanagari-cdc-maths-p0008-img-line-0008
did not produce a multi-segment suggestion. Boundary: this is not automatic OCR
recovery. Some suggestions are word/span splits or non-text visual spans, not logical
line splits, so reviewers must verify/edit them before applying.
tools/prefill_limbu_book_split_suggestions.py now closes the next operator-glue gap:
it copies those audited visual suggestions into a new unified unresolved packet by
pre-filling split_bboxes for matching pending existing_book_line rows, while
leaving review_status, corrected_text, and split_corrected_texts unchanged. Real
output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled/;
report SHA-256 9bc4d3f245133bc572c218536425ce3935b0839a5f0983176dce9853865437e7,
TSV SHA-256 b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4.
It has 30 packet rows, all still pending, with 19 prefilled split_bboxes rows. The
prefilled packet was run through the existing one-command wrapper at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled-application-pending/;
wrapper report SHA-256 507bc6d51b6d1a1c299ec62e9f3ddaeb12773ecdbcda827caaaa7168f5a26598,
wrapper audit SHA-256 13763cd2fdb437e7addb6a830a808a957a756b1ed4eb88aa3fafb7e6891d1be1,
final book-run.json SHA-256 abb937646cfa736da056dc46d5edfba57392435b6be642388a16f6cd783a88ca.
The wrapper audit passes with 30 packet rows, accepted_counts={}, 0 book-line
applications, 0 candidate-line applications, 0 issues, and 0 warnings; final book.md
and book.body.md remain byte-identical to current best. The next improvement now
requires actual reviewed split text, not another bridge script.
tools/prepare_limbu_book_split_transcription_worksheet.py now turns that prefilled
packet into a segment-level worksheet so reviewers do not have to hand-build JSON
arrays. It crops each suggested segment from the source page image, records crop
hashes, and writes TSV/HTML grouped by source packet row. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-current/;
report SHA-256 5eaa08ad0b2535d21989f2ece15dc65d8d36a46695670b5c326776b72f8cb01b,
worksheet TSV SHA-256 0b6e75a24515d404a7d8e33a2097b882f99af22732aca6eee1fdf3f1d84f2c0e,
worksheet HTML SHA-256 8da8216f287924df90cd206153e64cb9f75bdf35fe8f5cd6ccafc5c274732ebd.
It contains 19 source packet rows, 109 segment worksheet rows, and 109 segment crops.
Some segment crops are non-text visual spans, so reviewers must use segment_status
to reject those instead of turning every visual segment into OCR text.
tools/prefill_limbu_book_split_transcription_worksheet.py now adds a recognizer
candidate-text layer to that worksheet without touching review fields. It verifies the
worksheet report and all segment crop hashes, writes a recognizer label file, and
merges predictions into candidate_segment_text / confidence / status columns while
leaving segment_text empty and every segment_status pending. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-recognizer-prefill-current/;
prefill report SHA-256 60796a0265abff4117701d489a3982dfdfeaf66f9050e2efff4302c8caf608f9,
label file SHA-256 1076d642bce34ddb2b4ff9059c42ced0728ba359bd19e1f2625b33ece7390b84,
prediction TSV SHA-256 ecf15827b3f8ce58338984421f93889f7150baf3cd2ac82c1824b209278b4ce7,
prediction summary SHA-256 c03e881417fbf26e9862b1d9316065cdaa904d18ab111c17f79e03546700b4af,
augmented TSV SHA-256 8a7cfe05fa8434178e6ee2464326e25e7ced465ee5da7594bf954d336c6c75e0,
and HTML SHA-256 fe8012dce16587551fc548ef6a870c58cf401692a66909f1127e754546355948.
It is independently audited by
tools/audit_limbu_book_split_transcription_worksheet_prefill.py; audit SHA-256
2fa090dbdad4d6a36078c254828eec38799dfef3d50687a67af2b53315b3f32c,
audit_passed=true, operational_review_aid_safe=true, 109 label rows, 109
prediction rows, 0 changed review fields, 0 issues, and 0 warnings.
The Devanagari CDC PP-OCRv5 recognizer ran over 109 segment crops with 0 failures:
100 predictions are above min_confidence=0.65, 8 are low-confidence, and 1 is empty
(non-empty confidence min/median/max 0.174888 / 0.8584135 / 0.990676). This is
review acceleration, not auto-application. The compiler now accepts this prefill
report as an alternate worksheet source after review edits, and the unchanged real
prefill report compiles to 0 rows with output TSV byte-identical to the split-prefilled
packet (pending compile report SHA-256
f8cd51be1baaa288182a8ad30cf5966e2a1639fb6b0f9dbaa0e2b631e7119b4b).
tools/apply_limbu_book_split_transcription_worksheet.py now compiles completed
worksheet groups back into review-packet rows. It writes split_corrected_texts and
matching pruned split_bboxes, excluding segment rows marked non-text/drop/rejected,
and can mark completed packet rows corrected via --mark-status corrected. The current
real worksheet is still unreviewed, so the pending compile at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-application-pending/
has report SHA-256 2f148e4b870be32648d5ee545c71b79626464f4457f3772f30ad100c7e168ec1,
output TSV SHA-256 b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4,
compiled_count=0, 19 worksheet packet groups skipped as pending_segments, and a
byte-identical output TSV to the split-prefilled packet. Tests cover the completed
path: a reviewed group with one non-text segment compiles to two text lines/two bboxes
and the existing packet splitter preserves it as an accepted existing-book-line row.
tools/apply_limbu_book_split_worksheet_review.py now provides the one-command path
from reviewed worksheet to corrected book projection: compile worksheet -> apply the
unified review packet -> audit the application. Real pending run:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-worksheet-review-application-pending/;
wrapper report SHA-256 96a5974c02d72c59d200895027a89a6069b701e9ee4d291ed5732c2bf8eadec3,
compiled-packet report SHA-256 137b53deaad0a0716fecc3d043382a88a0d9de0a716f47acf008111de50cf0ad,
application report SHA-256 fc3fe47f0badda0651689e92c2c45560e288840921ada0d8738bc4e14c4f75c3,
application audit SHA-256 d82737206bf8238dbb949470f58df7d55006f4a6ce532ae459360c89db8fa7a7.
Because the real worksheet remains unreviewed, this run has compiled_count=0,
accepted_counts={}, 0 book-line applications, 0 candidate-line applications, and
final markdown byte-identical to current best. Fixture tests prove the reviewed path
does apply a split and the pending path stays no-op. After real review, the remaining
step is now one command: tools/score_limbu_book_ocr_candidate.py scores the
resulting application/final-book/ against the frozen page set and runs the book
candidate admission gate. Admission was tightened to accept unified unresolved-review
application audits only by matching the audit's final-book identity to the candidate
book-run.json SHA-256; the application audit now also records
final_book_output_dir / output_dir for application/final-book/. The real pending
worksheet application was scored/admitted at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-worksheet-review-application-pending/;
wrapper report SHA-256 f546cb6e4cf007c1a32b8e6799ce3203d1060eafd327cc86a72921217a59a832,
metrics SHA-256 f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f,
page-eval-run SHA-256 3356fb45524953c8837b6ab237c701866285adf11661101de9e634de8b4c153b,
and admission SHA-256 7fadfc25c0191fa71abb9d93e6266c0e0078fea015df7a02bd067d02556db966.
It passes operational admission with claim_safe=false and the same metric values as
current best: codepoint CER 0.446556, grapheme CER 0.626728, WER 0.839243,
micro line F1 0.750000, missed/extra/blank rates 0.225806 / 0.272727 /
0.000000, and reading-order accuracy 0.611765. This closes the post-review
scoring gate; it is not quality movement until segment rows are actually reviewed.
The recognizer-prefilled segment worksheet was then tested in two automatic
diagnostic modes. The direct split promotion path
(tools/promote_limbu_book_split_transcription_prefill.py) is rejected: it promoted
13 complete groups into 89 text segment rows, improved WER to 0.711653, but collapsed
line F1 to 0.171429, reading order to 0.421053, missed-line rate to 0.612903,
and extra-line rate to 0.889908; strict admission failed. The useful path is the
new line-text diagnostic, tools/promote_limbu_book_split_prefill_as_line_text.py,
which joins complete high-confidence segment suggestions into the original book-line
text and clears split fields so layout is preserved. Real output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups/;
promotion report SHA-256 abebf54f05c80c78d9c9d60459a132d4274e0dfb0d454a38c40bdaad782091bf,
packet TSV SHA-256 72dee88f38b8d3f217b0cc706f3b8d3b5d2bdcb376c8331cf29a6c1ceddea8dc.
Applied output:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-application/;
application audit SHA-256 c482d60733a43e0981b38c8a84d8a8b4ed69167494d1117f064edd291f67c968,
final book-run.json SHA-256 930f0b0475a8b9459e104a145ca5f7e9634e68cb739d9df60b0e4829bc8509a0.
Strict score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups/;
metrics SHA-256 d277042a488bcc2956d2492150504614a88bde077e30f34a0a417afb7fc620c1,
admission SHA-256 a0279029f9c346bf51cbbdb5a3c89edfc9db68f0d37dfed49dd3d73e4c21a598.
It passes with admitted_operational=true, claim_safe=false, no admission issues,
codepoint CER 0.399842, grapheme CER 0.511521, WER 0.711653, matched-line
codepoint CER 0.123640, line precision/recall/F1 0.727273 / 0.774194 /
0.750000, missed/extra/blank rates 0.225806 / 0.272727 / 0.000000, and
reading-order accuracy 0.611765. This is now the best operational diagnostic
candidate, but the next serious step is not to trust automatic text blindly; it is to
turn these admitted suggestions into review UX and calibrate/verify the six skipped
low-confidence groups plus any pages outside this frozen set.
That review bridge now exists. tools/promote_limbu_book_split_prefill_as_line_text.py
now writes an HTML evidence packet and supports --review-status pending, which keeps
all rows unapplied while placing the 13 joined recognizer suggestions in
corrected_text for reviewer inspection. A new audit tool,
tools/audit_limbu_book_line_text_review_packet.py, verifies the pending packet hashes,
status safety, crop/document references, and that every TSV packet row appears in the
HTML. Real pending review packet:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending/;
promotion report SHA-256 ce389aa01e11f95e1815cef01f2acbe6ff63debb7f74f58045c234c12ee4fada,
TSV SHA-256 ea2f56874371b0b146217d7fa83e883f5a2793ce5672b0038066a5837314d13e,
HTML SHA-256 08679a25155a34ac6578a8f0e55d0e153d770f8e22d6930e22282ea755d7f215,
audit SHA-256 acac7ff3b890c9cf888a1ca24ed1e3142d3ae55d097651c5ec65c3fcb2d4c8f3.
The audit passes with operational_review_packet_safe=true, 13 pending suggestions,
30 pending rows, 0 accepted statuses, 0 issues, and 0 warnings; the four
candidate_missing_line rows are visible in the HTML under "Other Pending Packet
Rows". The audit now has a reviewed mode for the post-review step:
--allow-reviewed permits accepted rows only when they carry explicit corrected_text
and a non-machine reviewer identity; edited TSV hash drift is recorded as a warning
rather than suppressing row/crop/document validation. Applying this pending packet
through the normal unresolved-packet applicator is a verified no-op:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending-application-pending/;
application audit SHA-256 187d1e7653d97e869ba72128879f7dfaf5017dcd5539edfbddc3eb075901d4b2,
0 book-line applications, 0 candidate-line applications, and final markdown hashes
byte-identical to the current best. This is the artifact to hand to a reviewer before
trying to turn the diagnostic CER gain into claimable OCR text.
There is now a guarded post-review application wrapper:
tools/apply_limbu_book_line_text_review_packet.py. It runs the line-text packet audit
with --allow-reviewed, refuses pending-only packets, refuses accepted rows with
missing or machine reviewer identity, refuses accepted rows with blank
corrected_text, refuses dropped rows with missing or machine reviewer identity,
applies the unified unresolved packet, audits the application, and can optionally run
frozen-page score/admission with --score-out. Real smoke:
invoking it on both current pending line-text packets correctly fails with
line-text reviewed packet has no audited accepted/corrected/drop rows ready for application, because no human-reviewed rows exist yet. This closes the mechanical gap
between review and scored application without making pending machine suggestions
look reviewed.
The same tool now has an explicit low-confidence calibration switch,
--allow-low-confidence-above, without changing default behavior. At threshold 0.60
it admits three borderline segments (0.637542, 0.609537, 0.600665) and promotes
16 rows instead of 13. The auto-applied threshold-0.60 diagnostic passes operational
admission but is mixed: codepoint CER regresses to 0.406176 versus the strict
0.399842, while grapheme CER improves to 0.508449, WER to 0.706189, and
matched-line codepoint CER to 0.114738; line metrics stay unchanged. Evidence:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-threshold060-complete-groups/;
metrics SHA-256 f4e1aa3c32e93ec94c845aab7e4f335a9876c22e69cdd6f41ee57797abdd0f81,
admission SHA-256 f6c095b40839f3657aa507af2eecf1fc849b1eaa351f39c4bf4f88787afb5731.
Because of that metric tradeoff, threshold 0.60 should not automatically replace the
stricter 13-row diagnostic, but it should be reviewer-visible. The richer pending
review packet is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-review-pending/;
report / TSV / HTML / audit SHA-256
26cdf80ca5280fdc07440ab7bbaa73ab82d6b6aa633801e26de54312e24cc64c /
6f86cd9966d219e98f8182ef8a090adff41715a8492a573eb1316092142bb3b3 /
4c907d94c090cc79d83af760c8876d8584968a8ff25b5f4678a9b1fd24a59893 /
7ea916b354cc3e74761261f5adb7e6c8ceabc992956bf66463c2af189eff8405.
It audits cleanly with 16 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. Pending application is a verified no-op with application audit
SHA-256 688820c7d9ce6b552b9dc53db68eeb65d809ef9206f11a76c4cfdec4055f85f5 and
byte-identical final markdown. Its HTML also exposes all four missing-line candidate
crops. Use this richer packet when review time allows borderline segment inspection;
otherwise use the stricter 13-suggestion packet.
A still better reviewer packet now exists for the actual missed-line workflow:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending/.
It is generated with --allow-low-confidence-above 0.60 --suggest-candidate-missing-text,
so it keeps the 16 existing-line suggestions and also copies the four pending
candidate_missing_line machine texts into corrected_text as editable suggestions.
All rows remain pending. Report / TSV / HTML / audit SHA-256:
4ae24b061e5d2c36bc467bafe31134c1c1d1e51de405ab21ebf25c8690f1f07c /
d4f4438160031a799cde0eecfb7cea4f8e85b662b3c4b356363131880379e364 /
3dd21d3dd677b68cc415d7df6da8892c1527bd626600e17f9282666c120b722b /
42fb1372dc06c7db3d6484044dfc2fc752a50573c9363880257b1206c43ba76e.
The audit passes with 20 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. Pending application is a verified no-op:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending-application-pending/;
application report / audit SHA-256
aa2a106ef8719c5e8f3094dced4ced1883b987a9db13fe6c45e0416acfaa49cc /
3ec74b70dec0233d8569b3dcc784aa4e1ca859673288fd76b666ca8cd1f7d51d, 0 applied rows,
and final markdown byte-identical to current best. This is the preferred packet when a
reviewer will inspect both existing-line corrections and missing-line insertions.
The preferred packet also now has a missed-reference coverage audit:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-threshold060-line-text-suggestions/.
Report / TSV SHA-256:
68565c47c276d53a00af08002b0f734d1cc4094da2ebb82ec7b88a4d2ee401c4 /
ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37.
Against the threshold-0.60 line-text score, all 7 frozen missed-reference rows are
review-visible: 2 covered by candidate_missing_line, 2 covered by existing review
regions, 3 swallowed by larger existing review regions, and uncovered_or_weak_count=0.
This is review-coverage evidence only; the rows remain pending and are not claim-safe
OCR until a human accepts/corrects them and the final book is rescored.
That audit exposed one remaining packet gap: the previous preferred packet covered
all missed references but left one high-confidence extra prediction uncovered,
limbu-devanagari-cdc-social-p0009/img-line-0001 (थिक्सिङ् - थिक्). The raw
unresolved packet generator now supports --line-alignment-extra-predictions, which
adds prediction_unmatched book lines as extra_prediction_line review rows unless
they are already present in the confidence-driven book-line review queue. Focused
tests cover this path. The superseding reviewer packet is:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-pending/.
It has 31 pending rows, 20 pending suggestions, 0 accepted statuses, 0 audit issues,
and 0 warnings. Promotion report / TSV / HTML / audit SHA-256:
d5650dc1b67ae9881e6b4ccad5a5c1cacf8a744f02fc6af11bce3ce587bcda44 /
5e0e618adc8229fea734b8356be7aa3fe6f2c4f09737f9a5685ed15c284ac468 /
978fb40593d192b23c78f4f240e75c754cf29f404957ad104e6c4683d885ea50 /
e1053e353a4408c33008926a4923d3d25c93decbd943441d6e8fcc892f398b4d.
The raw unresolved packet now audits cleanly too: report / TSV / audit SHA-256
55ed8e0c6cfceeb609aeea3cf85862819488c6a59df422b935e817d36350d155 /
866e40ae4c5d037a347a3740c57610601f54071aa3deeefeed994f1a48d3a601 /
92d66a8333c387478b4d5bcb5d0fdbf51b231671aa5e1ce60c1ac8b0e26db5a2, with 31
pending rows, 4 candidate-missing rows, 27 existing-book-line rows, exactly 1
extra_prediction_line, 0 issues, and 0 warnings. The raw-packet audit now accounts
for extra_prediction_line_row_count separately and verifies the frozen
line-alignment CSV hash used to append extra predictions.
Its structural review coverage artifact is
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-review-coverage-threshold060-line-text-missing-extra-suggestions/;
report / missed TSV / extra TSV SHA-256:
ae9b7903a719110e77cfbe8089fc0cc0f025bd6f93804cdec4ac1d5a7b24b775 /
ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37 /
ad610c8019489fc03c041dfa1f0eb35e31bf76a304f8cf5731b3469752a21d51.
Coverage is now complete for the frozen line-structure errors: 7/7 missed references
review-visible and 9/9 extra predictions review-visible. The guarded application
wrapper correctly rejects the packet while all rows are pending, so this is a human
review packet only.
The application side now supports actual extra-line rejection instead of merely
showing the row. tools/apply_limbu_book_line_reviews.py accepts explicit drop/reject
statuses for existing book lines, removes the line from the document, removes its
line id from blocks, drops emptied text blocks, and records dropped_counts /
book_line_dropped_count through the split, unified application, line-text wrapper,
and independent application audit. A real smoke copied the superseding packet to
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra/,
marked only limbu-devanagari-cdc-social-p0009/img-line-0001 rejected by
cdjk-review-smoke, then applied and scored it. Smoke packet report / TSV SHA-256:
1a8fc27bfc3fd9a3a3c47abc766df7abfb97301855c7f4b3725e2ef2722cb1b8 /
91287f19aaecf89c7735fad7221b71789c605d59bfc71296d7002ac959167120.
Application:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra-application/;
application report / audit SHA-256:
1806662aa1318a340df70067990e345a0e79e363a04ea8891c1858da837a5a01 /
bcfd4ef72fc93ed7c0a08d26a67df0f2953e44450249747f5a0c4f06f1326f9b.
Result: line-text packet audit passed, application audit passed,
accepted_status_count=0, dropped_status_count=1,
dropped_counts={"existing_book_line": 1}, and book_line_dropped_count=1. Smoke
score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-missing-extra-smoke-reject-extra/;
score/admission report / metrics SHA-256:
7501fe80695cd1adb87428a94ef3f9d7c6b416200279f3a6d0b16d57e37af08b /
7bccf3236644d2d73d2ddc7864f85de64dbe7681b91d41ad3fab7d48d8a45e89.
Operational admission passes but claim_safe=false; codepoint CER 0.436263,
grapheme CER 0.615975, WER 0.823739, line F1 0.761905, extra-line rate
0.250000, missed-line rate 0.225806, and reading-order accuracy 0.611765.
This is synthetic review-smoke evidence, not human-review claim evidence.
A second mixed smoke copied the superseding pending packet to
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke/,
then applied one existing-line correction, one accepted candidate_missing_line
insertion with text १, and one rejected extra prediction. Mixed packet report / TSV
SHA-256: 14ca2c76ea261c79730d96480d6e7a4bb901a807d24a306cc4aabb490d50983d /
7641025877f89c519d371bc3df613b4a98fa9a4874bf06c9b0ff2993a8602fff.
Application:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-application/;
application report / audit SHA-256:
8b1b983b34c65dba327788959a3200e176dc86639bb50a15f22b792cb5637c31 /
58095326d648202df10aecdda244b470fba2601c9bf5d82e2c9edeb563dd9617.
The mixed application audits cleanly with accepted_status_count=2,
dropped_status_count=1, accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1}, dropped_counts={"existing_book_line": 1},
book_line_applied_count=2, candidate_line_applied_count=1, and
book_line_dropped_count=1. It then scores via
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke/;
score/admission report / metrics SHA-256:
03adcbacac5f4f1fac68d9f73bf9c85c33f024931c6cff33ab3106f5878bfff1 /
e3ddf0d31b7bed313f9e22a05be0583e3afa7187cc8a29785b406f6b84b2cc5c.
Operational admission fails and claim_safe=false: codepoint CER 0.438638,
grapheme CER 0.614439, line F1 0.781250, extra-line rate 0.242424,
missed-line rate 0.193548, matched-line CER 0.174901, and reading-order accuracy
0.553191. Treat this as pipeline-plumbing evidence only. The next concrete gap is
accepted missing-line insertion placement/reading-order calibration; otherwise a
seemingly correct insertion can improve recall while failing operational admission.
That gap is now patched for structural page markers. tools/apply_candidate_line_reviews.py
places reviewed candidate insertions with structural role page_marker before body
text, leaves ordinary text insertions geometry-placed, and records
placement_policy in insertion metadata. A rerun of the same mixed packet with this
front-matter policy is:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-application/;
application report / audit / final book-run SHA-256:
bac62fc812eaed939ceecf7e36fd7ccb0a2329ae7644d2755df0077487c1b179 /
fc264875a4876dceadccbbdf221b3771726000a4e50518fba9df60d92b63ba40 /
879547c4827fee26900f5f7d49a7462df0793fd6d7a689bfecf90618f507b56a.
It audits cleanly with the same action counts as the failing mixed smoke:
accepted_status_count=2, dropped_status_count=1,
accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1},
dropped_counts={"existing_book_line": 1}, book_line_applied_count=2,
candidate_line_applied_count=1, and book_line_dropped_count=1. Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter/;
admission report / metrics SHA-256:
43705c1d33ce7c5f23d04452a364cf1316efa084b90ea6b67b35f458c754a60a /
a3718bbab380b2c44d406d938db00bf3fdaa2021d24d1f683eb944419a76d65d.
Operational admission now passes, claim_safe=false: codepoint CER 0.435471,
grapheme CER 0.608295, WER 0.812810, line F1 0.781250, extra-line rate
0.242424, missed-line rate 0.193548, matched-line CER 0.174901, and
reading-order accuracy 0.648936 (61/94 matched-line pairs). This is still
synthetic review evidence; it is useful because it validates the mixed review
application path without weakening the admission gate.
The structural ordering policy was then generalized in the actual projection paths:
src/ocrtech/structure.py, src/ocrtech/image_lines.py, and
tools/apply_limbu_book_line_reviews.py now sort explicit page_marker blocks before
body text and also treat bottom-margin Devanagari digit-prefixed rows like २ ... as
front-matter rows. The latter handles page numbers fused into neighboring title/header
text, while avoiding a broad "bottom text first" rule. Rerun artifact:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-row-application/;
application report / audit / final book-run SHA-256:
c7ea956cdec75c2aefc9260e0e30c30710f77f5e84bc19b6be53948b3e103408 /
8efc363d1895ed955e8fc692fb10ae1a6ff312f9f146464065e8eb6696a9a657 /
d012af48bb7c64d65572cfa70ff135a6a8321aa1658c29ae1c26467fb82fedba.
Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter-row/;
admission report / metrics SHA-256:
47ee4098aaa2fc92c9d053c1fc8cfeb3093528a5556dcf450656af8c6b356837 /
7201ed8390b0bce13ecc5f7aca6643b1b6d77f7be99c1b7b12548460f3150182.
Operational admission passes, claim_safe=false: codepoint CER 0.455265,
grapheme CER 0.632873, WER 0.821451, line F1 0.781250, extra-line rate
0.242424, missed-line rate 0.193548, matched-line CER 0.174901, and
reading-order accuracy 0.787234 (74/94 matched-line pairs). The tradeoff is real:
reading order improves sharply, but raw page CER regresses versus page-marker-only
ordering because low-confidence unreviewed text is now placed in the reference order.
Do not promote this as claim evidence; use it as layout-order evidence and keep the
human review boundary intact.
The text-first candidate was also rerun through the structural ordering code:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-application/.
It applies the same 13 existing-book-line corrections as the previous best-text
candidate, leaves 17 rows pending, and applies no candidate-missing insertions.
Application report / audit / final book-run SHA-256:
d3821317c37f4c914febdfce29e9e113f383d074d2132bd85b619fa68add7ef3 /
5ebad12995fdb520dcd98daf02c614f0e3d8d6fbff23db441bead8804d4ce05b /
902dce10d52e2b54ee9bf3e6ed2e2e0b5d15ef746ad921a28df5c20610fc06bb.
Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order/;
score/admission report / metrics SHA-256:
f24b4f8a3abd66ac89c475ed9cc98e54a402e2369b587a0780dc19cbf85473a7 /
979827dc2ccdbc4f6c3ea45ee97c1bcc0151200f8c5eaeafb31a7928c72524d4.
Operational admission passes, claim_safe=false: codepoint CER 0.404592,
grapheme CER 0.520737, WER 0.722582, line F1 0.750000, extra-line rate
0.272727, missed-line rate 0.225806, matched-line CER 0.123640, and
reading-order accuracy 0.682353 (58/85 matched-line pairs). This is the better
text-first operational candidate after structural ordering: it gives up a small amount
of raw CER versus the old text-first run (0.399842 -> 0.404592) but improves
reading order (0.611765 -> 0.682353). It still does not solve the missing-line
problem because no candidate-missing rows are accepted.
The conservative page-marker selector has now been wired to the unified TSV review
packet without weakening its guards. tools/select_candidate_line_reviews.py now reads
candidate-missing TSV rows, derives page size from the row's book_document, parses
max_selected_iou from review_reasons, and rejects a page marker when the same digit
would still be embedded at the start of an overlapping selected line after accepted
existing-line corrections from the same TSV are applied. On the complete-groups packet
this selects the maths १ marker and the science २ marker; the science marker is
allowed because accepted review row existing-0019 removes the leading digit from
img-line-0014. The two body candidates remain rejected. Selection report / selected
JSONL / rejected JSONL SHA-256:
f52f61ab09ba6f2101ae8049815a01257dd5b57e93c9aab492f2125fefbf2058 /
7e7b37359d5b4dc8972220054b33f2ffeb388ff7d02a621972035a19a573baa9 /
40a9ae5a0fb174694cdc1e04d108adb52900b2864bce43aabf4557878f21feaf.
Applied on top of the structural-order best-text book, the new projection lives at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application/.
Applicator audit / independent application audit / book-run SHA-256:
26dc4049979618330eb8016fad699af82465719d7e88749540309adcc40261cc /
e135fcb2faa7b1a125c2f706fae41e1fb448dfe918390f434f5075895fc540d4 /
61d6dfc09e1b7e09f0fd1cf57b5aac988b6c8e65f567f2eda6c19e33928ff775.
Score/admission report / metrics / admission SHA-256:
9a4f3bca208d515b1e7538536a1abb54000749d4694e2317a981ca3588064ba3 /
fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4 /
1bbd1d216d3c3b23def126332aecf1052abf254a0f15891dd58048c103cc2205.
Operational admission passes, claim_safe=false: codepoint CER 0.402217, grapheme
CER 0.516129, WER 0.711653, line precision/recall/F1 0.714286 / 0.806452 /
0.757576, extra-line rate 0.285714, missed-line rate 0.193548, matched-line CER
0.123518, and reading-order accuracy 0.712766 (67/94 matched-line pairs). It has
3 structural page-marker predictions, 2 matched structural lines, and 2 exact
structural-text matches. This is the current best operational layout candidate, but it
is not a full OCR-quality solution: body-line candidate text is still too noisy, and
the next gain has to come from reviewed body-line recovery or a real line split/remove
path, not more blind marker insertion.
The body-line recognizer-probe lane was refreshed against that current best and exposed
one more audit bug. The refreshed crop probe
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-text-crop-probe-from-complete-groups-page-marker-v2/
uses the two body-line rejections from the TSV-backed page-marker selector. It again
refines both body rows (trim_top_bleed and left_of_horizontal_rule), with probe /
JSONL SHA-256 9e64044df9ec4070dcc28b78a2e0f48c2fd6cb8009fdab1de41a36eb2591a61c /
31e8ecbf17ab207c67c3f46510f66771e4f2802315cf052ee52aaaa858ae68d4. Existing
recognizer predictions are valid because the refined crop hashes are byte-identical to
the older probe. The refreshed comparison
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-crop-recognizer-probe-comparison-from-complete-groups-page-marker-v2/
has comparison / JSONL SHA-256
12fd030c336f3d1cdef882de8512f56a5f58e7bd513d301b4dab0fd95418b329 /
f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d. The non-oracle
machine selector
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-recognizer-probe-selection-conf045-delta005-from-complete-groups-page-marker-v2/
selects only the first body row (confidence=0.469282, delta 0.218220) and rejects
the second for low confidence/delta; selection / selected JSONL SHA-256:
41655846f1845d1a966a72e7def29b38e58bee53bf5472493a6c3bda536e1dc1 /
205415b49c984145cbf573aa1e7a65547a931a202bc8789daa58ebe10d64c2b4.
tools/audit_candidate_line_review_application.py now distinguishes current-review
insertions from inherited candidate insertions in chained applications; the real body
probe application has applied_count=1, inserted_line_count=1,
total_inserted_line_count=3, and inherited_inserted_line_count=2.
Application audit / book-run SHA-256:
d89b9c8966c5c1a01e74bb5edbe3ab19c08054b9f915d9aa5fad1863199f6d99 /
abe8f59007aa77fb13bf690070230346aa14dc88040fa2b18b8d80ef2fd332de.
Score/admission:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe/;
score report / metrics / admission SHA-256:
b52a853cf05196c618a416fc981cf747b05cc178a6435fc35a0d497380611992 /
e2307c02624fd601f2ad5674b3405e62620c2565993d3dcbf2425e5b8fd74f79 /
2b55cb5073c523c2aa69c69f52251452a80ab89e762554074b29d21361db8b6c.
Operational admission passes against the old CDC baseline, claim_safe=false, with
codepoint CER 0.441013, grapheme CER 0.562212, WER 0.760834, line F1
0.776119, missed-line rate 0.161290, and reading-order accuracy 0.656863.
Do not promote it over page-marker-v2: paired comparison against the current best
at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-body-recognizer-probe-vs-page-marker-v2-current-best/
shows codepoint CER regresses by 0.038797 and grapheme CER regresses by 0.046083
(comparison / paired TSV / markdown SHA-256:
1bafa22b69296c98f9186ade50cce68d509491e0b77de8be80c45bb9a4d3a209 /
cd89984c2dfc9a8e6792b15b1b193348716d9a5c0c140d048278843fbead732f /
38104641a4c8744233fa90669395f994047121e21aba25c7af7602741dccf3a8). Treat this as
review evidence and an audit-chainability fix, not the selected OCR policy.
That promotion mistake is now guarded in code. tools/admit_limbu_book_ocr_candidate.py
accepts --champion-metrics, and tools/score_limbu_book_ocr_candidate.py passes it
through. When supplied, admission rejects candidates that regress codepoint CER,
grapheme CER, or WER against the champion beyond the configured --max-champion-*
tolerances. Real verification against the page-marker-v2 champion:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe-champion-gated/
fails admission with champion deltas -0.038797 codepoint CER, -0.046083 grapheme
CER, and -0.049180 WER (wrapper/admission SHA-256
4126d1ca9c4e644190d862301f199f3f46335fb1c7b48191209c6e70e3f46214 /
ff1425910714bacd54f96640d8f2281c08d78c93f07b0d43c7be3117d9a1c4e1). The
page-marker-v2 self-check at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-gated-self/
passes with zero champion regression (wrapper/admission SHA-256
6103f40ec19bdde710fd1438ff8547382118fbbdadc9f96ce65425767a5ce75d /
3b22736a064654783b9a8cee5a455936d7e86cc1d759d1e7536bb5ea69eb727c). From now on,
do not call a candidate "current best" unless it passes this champion-gated admission,
not merely the older 0.458432 CDC baseline gate.
The reviewer-facing unresolved packet has been refreshed against the actual
page-marker-v2 current best. tools/prepare_limbu_book_unresolved_review_packet.py
now accepts selector-style JSONL for --missing-line-review-queue and filters
non-pending rows out of unresolved packets, reporting skipped counts; this matters
because the page-marker-v2 book run points at a queue with 13 already-corrected rows.
Real packet:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-extra-predictions/.
It uses the page-marker-v2 book output, its own line-alignment.csv, and the
page-marker-v2 selector's rejected body-candidate JSONL. The packet has 18 pending
rows and 18 verified crops: 13 existing low-confidence book lines, 3 extra prediction
lines from current-best alignment, and 2 candidate missing body lines. It skipped 13
non-pending existing rows. Report / TSV / HTML / audit SHA-256:
1e004bb5c6b206a2ad3faf147e1f54cea3c7d442300b71be295a3c2d9743c5ab /
fe8908da4050e90f17d4a6ccbf111ba04b5752491bccb19a01d11b5853bdf84a /
17694e6aa1224dec5e3edc12cb742e07d0be9989aa66f9718399d48e3fef23c7 /
10e95db8204c47bb4fec39264e2914510fed82544221f541f486bebf3f3d26be. Audit passes
with 0 issues and operational_review_packet_safe=true. The pending application at
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-application-pending/
applies 0 book lines and 0 candidate lines, audits cleanly (application/audit/final
book-run SHA-256
4fd8812729cbd73e5332d0dbf21bcb6959221caa7c515d0e818cbcdf3f2bcb19 /
239610a43b40f765543d168561b5c3e7b378c89e08e129b0e012b67f8d83764a /
cf0b2970cbc3aa56dd72ffe7fc6dc68b43b5ffa2ca4e99e31d1f7f8c52a632bc), and final
book.md / book.body.md are byte-identical to page-marker-v2 current best. This is
now the current packet to hand to a reviewer before further body-line recovery.
Coverage has been refreshed for that packet too:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/.
Against page-marker-v2 line-alignment.csv, there are 6 unmatched reference lines:
1 is covered by a candidate missing-line packet row, 2 are covered by existing review
regions, and 3 are swallowed by larger existing review regions. uncovered_or_weak_count
is 0. All 10 unmatched predictions are covered by existing review regions, with
extra_uncovered_or_weak_count=0. Report / missed coverage TSV / extra coverage TSV
SHA-256:
32026349ec549e142036042cf3b6bc2713a41b0438f4c5220b3afb082d7e7abc /
596c77aa6770a28467dafa89a8f7122bd080bb274ef98bacdfba7521566d8ad8 /
95f41b789b6ae9aadf93b4273d1c96b7a39f2e119f1500e7e8187ec9ee350b91. This proves
the current packet covers the current missed/extra line surfaces for review. It does
not prove OCR completion; the remaining work is correction or safe automatic recovery
of those reviewed regions, followed by champion-gated scoring/admission.
Admission now knows about that requirement directly. tools/admit_limbu_book_ocr_candidate.py
accepts --review-coverage-report plus strict default maxima for uncovered missed and
extra lines, and tools/score_limbu_book_ocr_candidate.py forwards those inputs. The
current page-marker-v2 candidate was rerun with both champion metrics and review
coverage:
outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/.
It passes operational admission with claim_safe=false, zero champion regression, and
review coverage values {missed_reference_count: 6, uncovered_or_weak_count: 0, extra_prediction_count: 10, extra_uncovered_or_weak_count: 0}. Wrapper / metrics /
admission SHA-256:
0018bfb5ee2f48821c3c9b1101d0921bbad54c41cd63be120dc72437fe9f1344 /
fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4 /
b9ece419592b3bfc3dbc892880c206a2cbcb24a8777e0c245d1b7614afa30f43. Future
promotion attempts should supply both --champion-metrics and
--review-coverage-report; passing the old baseline-only gate is no longer enough
evidence for book OCR.
The eval harness registry now reflects current book-candidate evidence for both
Limbu scripts. Page-level recipes
limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated and
limbu-sirijonga-gorkhapatra-book-conf070-coverage-gated are registered in
configs/eval-harness-recipes-v1.json, and the limbu-first-two-script-ocr
targets in configs/eval-target-coverage-v1.json point at them with
eval_status=registered_real_pack. Validator status:
outputs/eval-harness-recipes-current-validation.json reports
recipes=13 valid=True claim_ready=0; outputs/eval-target-coverage-current.json
reports valid=True coverage_ready=False target_sets=3 gaps=52. Both book
candidates are structurally valid and operationally admitted, but neither is
claim-ready. The Devanagari side is still blocked by only 3 CDC pages, 0 claim-ready
references, curriculum_text_unreviewed GT, non-frozen/non-claim-eligible rows,
missing hash-verifiable model artifacts, claim_safe=false, and 18 unresolved review
rows. The Sirijonga side is still blocked by 1 Gorkhapatra page, machine-converted
unreviewed GT, non-frozen/non-claim-eligible rows, claim_safe=false, and 79
unresolved review packet rows. The top-level two-script Limbu gate now has 8 gaps:
for both scripts, book-candidate evidence exists but is not claim-safe, and the
normalization, recognizer, and page recipes are not claim-ready. This is an honest
registration upgrade, not a quality claim.
The recipe validator now independently audits the page-marker-v2 evidence chain:
score_limbu_book_ocr_candidate wrapper report, admission report, metrics, champion
metrics, review-coverage report, and unresolved packet hashes. It blocks missing
champion evidence, missing review-coverage counts, champion regressions, uncovered
missed/extra review regions, mismatched evidence paths, and scorer commands that omit
or redirect the recorded --book-output, --book-audit, --champion-metrics, or
--review-coverage-report inputs. The current registry still validates, but future
"current best" registrations cannot rely on command-string convention alone or pair old
evidence with a weaker replay command.
The Sirijonga page recipe is now executable as a replay rather than a placeholder.
tools/copy_page_ocr_document.py copies a pre-materialized page OCR document bundle
into a recipe-run documents directory, verifies the manifest input path against the
copied document's source_path, and writes a hash report. The
limbu-sirijonga-gorkhapatra-page-current-router recipe now uses it to replay
operational-image-line-width480-minwidth005-conf070-documents, has
requires_operator_resolution=false, and no longer carries unresolved-command or
missing-model-artifact blockers. Smoke run:
outputs/eval-recipe-smoke-limbu-sirijonga-gorkhapatra-page-current-router/.
It audits cleanly but remains non-claim-safe: 1 sidecar-fixture page, codepoint CER
0.739361, grapheme CER 0.768262, WER 0.918969, line F1 0.872458,
matched-line codepoint CER 0.046585, reading-order accuracy 0.775478, and
0 claim-eligible rows. This is harness reproducibility progress only; the quality
blocker remains reviewed/frozen Sirijonga page data plus page assembly/layout.
The current selected Sirijonga diagnostics now have textless unresolved-region review
packets in outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/. Gorkhapatra
operational conf070 has 69 review rows/crops: 67 missed references and 2 extra
predictions. Its packet audit passes with 0 hidden-reference leaks; the missed text is
center-column dominated, with missed reference text contributing 83.8643% of
attributed codepoint error mass. Sikkim same-row merge has 22 review rows/crops: 9
missed references and 13 extra predictions. Its packet audit also passes with 0
hidden-reference leaks; Sikkim remains mixed, with missed reference text at 31.6140%
and extra prediction text at 28.1198% of attributed codepoint error mass. These
packets are review/recovery inputs, not claim evidence.
The page-level review loop is now mechanically closed with
tools/apply_unresolved_page_review_packet.py. Refreshed packets include blank
corrected_text cells; pending rows remain no-ops. Accepted/corrected missed-reference
rows require non-empty corrected_text before insertion by geometry, accepted/corrected
unmatched predictions update the existing OCR line only when corrected_text is
supplied, and false-positive predictions are deleted only with explicit
drop/delete statuses. Pending application smokes for Gorkhapatra and Sikkim score
identically to their selected source outputs (0.739361 and 0.418057 codepoint CER,
respectively), proving the path is scoreable without changing OCR text before review.
tools/audit_unresolved_page_review_coverage.py now independently checks exact
sample_id + line-id coverage against line-alignment.csv without writing hidden
reference text. Current Sirijonga packets are coverage-complete: Gorkhapatra covers
67/67 missed references and 2/2 extra predictions, while Sikkim covers 9/9 missed
references and 13/13 extra predictions, with zero duplicate or extra packet keys.
The Gorkhapatra Sirijonga pruning check rejects threshold-chasing as the next local
move. The unpruned width-480 run recovers more references than operational conf070
(261/303 matches, 42 misses) but adds 62 extra predictions and worsens page
codepoint CER to 0.761144; the operational conf070 route keeps only 2 extras and
scores 0.739361 CER but misses 67 references. Aggressive conf085 pruning lowers
page CER to 0.730446 by deleting difficult lines, but recall falls to 199/303
matches with 104 missed references, which is not acceptable for book digitization.
The tuned conf070/conf080 routes sit in the same tradeoff band, and auto split
tall recovers 285 references but adds 54 extras and worsens CER to 0.778638.
Conclusion: keep the current operating region for text output and attack
spatial/structural missed-line recovery plus review application, especially the
center-column Gorkhapatra misses. Do not spend the next cycle on global pruning or
plain tall-split threshold sweeps.
The Gorkhapatra Sirijonga candidate-line recovery diagnostic now makes that boundary
measurable. Comparing selected conf070 page documents against the rejected
auto-split-tall candidate set produced 100 candidate-only rows; the packet audit passes
with 0 issues (candidate-line-review-packet.json SHA-256
68f678a77e0b2056c25ecbc1595cf62e63c4ef72aa923d79355a4a9886998ff4, audit SHA-256
ac645bb4755bab878c4e65749735d86856c0dcb9e473399bf064db869e8df56c). The
diagnostic-only oracle overlap selector picked 49 rows against the 67 currently missed
references (selection report SHA-256
45fa5a781d62736a2aa3ef5c97762755d0b65ad99f1c7984856e585ab30e5128, selected JSONL
SHA-256 06f6439e09ce6a1b8c292dcaf57e5672bdd674ddca58bb03fad5d7f5d1fc9a57).
tools/apply_candidate_page_line_reviews.py applies such accepted candidate-only page
rows into a separate scoreable page-document projection; in the oracle-overlap
diagnostic it inserted 48 candidate-text lines and skipped 1 empty accepted row
(candidate-page-line-review-application.json SHA-256
b0c8d076df3bc6e19b85c3819ff1611abde1d691b71f59607a53506fef00ad7e). Geometry improves
strongly: line F1 0.964346, precision 0.993007, recall 0.937294, and missed
references drop to 19. Text quality rejects automatic promotion: page codepoint CER
worsens to 0.766779, grapheme CER to 0.798907, and matched-line reading-order
accuracy is only 0.594983 (metrics SHA-256
0467192299d469d02e3ad1b14c9be669a6561e08263098da858d337ea3267da8). The safe
non-oracle artifact is the prioritized review queue at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-prioritized-candidate-line-review/:
100 body-line candidates, 99 high priority, 1 medium, audit passed with
operational_review_queue_safe=true and SHA-256
0c9cde56dbec68f4a146cf417cee033271814ce7c5887058841028798f2a6985. Treat this as a
review/transcription queue or crop-recognition target, not an OCR output candidate.
tools/prepare_candidate_line_text_crop_probe.py has now prepared a non-oracle crop
probe for that queue at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-text-crop-probe/:
100 body-line candidates processed, 44 refined by trimming top bleed from multi-row
crops, and 56 copied as full crops. Report / JSONL / label file / HTML SHA-256:
dc5de12cda741533cc31e343147b4e12488b4ee1611e1e8cdb81a038206725c3 /
9ad687ca53e229565132dc579a79efd6adb6572aadad54fc5c61294fceca3fc5 /
2c5ae66ea3468f97f456ebcfa4f979703e65ded921f3850054b357cbeb015f2b /
46ae06c2412961b02fb9f9f8db7b30fd81afcc702cb8fb79b672fe7fe41094c8. A sampled
refinement isolates the candidate bottom line from a multi-row crop, so this is a
reasonable next recognizer/reviewer probe input; it is still not automatic OCR text.
tools/predict_recognizer_eval.py now flattens embedded newlines and tabs in
recognizer output before writing TSV, after the Sirijonga crop probe exposed malformed
multi-line prediction rows. Re-running the packaged v4 Sirijonga recognizer on the 100
original candidate crops and 100 refined crop-probe images with /tmp/ocrtech-paddle311
produced 0 failures and valid 3-column TSVs. Original/refined prediction TSV SHA-256:
8f32f8895ded0a4a1c405333206ebfc55df37606ff1b288ef4319c7067060fc0 /
66c4fa2d25b46bf24c99ad9936000ec0b73a932478cf1e74e613cabe09f79fd3. The comparison at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-crop-recognizer-probe-comparison/
has 100 rows, 44 refined rows, 40 changed predictions, max probe confidence
0.992109, and automatic_application_safe=false (report / JSONL SHA-256
8b2133b39c3d139b8a83d10858301bcad0e92cd821a4042bbf4aeb0bdf27064d /
f45d8d9866d05681202a221741612fb263312b6725aeac01d1fe6dcc12029464).
A small non-oracle recognizer-probe threshold sweep found no clean operational
promotion. min_refined_confidence=0.45, min_confidence_delta=0.05 accepts 21 rows
but worsens page codepoint CER to 0.747351. The strict 0.90 / 0.05 setting
accepts 6 rows and improves page codepoint CER slightly to 0.737090, but line F1
drops to 0.866545 from selected conf070's 0.872458, missed references are still
66, and extra predictions rise to 7 from 2 (selection / selected JSONL / application /
metrics SHA-256:
31ee7a4dc399cab13ded5665cdf0998e132b136ad60202b57a3636b8cfd0f042 /
072af68c1e59f31406f1e8df350b45e5c1040136f4b966f5adda9bb14b90b603 /
99ace4d56307e488e71250247e456aad651f99b87ff621ccda3ec1ecd60535a8 /
ee62806b3a3b3198544aeea273e9164d99bdf8059024af07775e53f0c36b3326). This is useful
evidence for reviewer-assisted recovery and crop-recognition work, not a default
book-OCR output policy.
tools/audit_candidate_page_line_application_effect.py now turns that policy boundary
into an auditable gate. It compares a candidate page-line application to the selected
baseline on page CER, line F1, missed/extra/blank rates, reading order, and inserted
line alignment, and writes an inserted-line TSV without reference text. The strict
0.90 / 0.05 recognizer-probe application is rejected with
book_ocr_objective_safe=false: 6 inserted rows, 1 matched row, 5 unmatched extras;
the matched insert has line codepoint CER 0.888889, grapheme CER regresses by
0.002485, line F1 regresses by 0.005914, extras increase by 5, and reading-order
accuracy regresses by 0.150648 (effect audit / TSV SHA-256
08eff93810712cafe46009ac8aa5a1d6dcd00c3206f2cd5b88aff527c7e0cd82 /
581c9d96f99ffee1be638a90ec662aefc6b6837baf04a1af75a50ba06a7a9181). The loose
0.45 / 0.05 application is also rejected: 21 inserted rows, 9 matched, 12 extras,
and worse codepoint CER, grapheme CER, line F1, extras, and reading order (audit / TSV
SHA-256 9c8865eb655fe74593295e241863f746ca565a50a80948412f0c67e5473f77dd /
5e1dc16eeb64f22db879e2f8a631fa5a7b693dce8a674a0956eebb9447a7aa49). This audit is
now the guardrail for page-candidate insertion diagnostics: a CER-only improvement is
not sufficient for the Limbu book OCR goal.
The follow-up oracle-feature analysis shows why the recognizer-probe selector failed
and what the next non-oracle selector should test.
tools/analyze_candidate_line_review_oracle_features.py compares the same 100
candidate-only auto-split-tall rows against the oracle selected/rejected labels, while
emitting only safe features at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-oracle-feature-analysis/
(report / TSV SHA-256
4c45a640338b6510b63bced5b903b847e4725ab87f544fb36ba7def7ac2e2bb0 /
21e31ff240b2a40220b70b9f029237064b83c951509b6bea62ec9fc4855098bf). The 49
oracle-selected rows have much longer candidate text (mean/median 29.306/37) and
wider boxes (mean/median 295.531/307) than the 51 rejected rows (text
mean/median 4.471/1, width mean/median 127.137/74). They also skew toward
middle x-band and middle/bottom y-bands. Refined recognizer confidence points the
wrong way: selected rows have mean/median refined confidence 0.382649/0.146180,
while rejected rows have 0.601181/0.751105. The tested refined-confidence/delta
thresholds have only 0.142857-0.500000 oracle precision and at most 0.183673
recall. So the next serious automatic recovery experiment is not "raise confidence";
it is a geometry/text-feature selector, probably starting with candidate text length,
box width, and body-region bands, scored through the effect audit before promotion.
That selector is now implemented as
tools/select_candidate_line_reviews_by_geometry_text_features.py, and the result is
exactly the split the product goal cares about: it is a strong missing-line finder but
still bad automatic OCR text. The precision-first setting
(min_candidate_text_length=30, min_bbox_width=80, y-band middle,bottom_or_right)
selected 31 rows; the oracle-overlap diagnostic labels all 31 as useful missing-line
candidates. Applying their machine candidate text gives line F1 0.933566, precision
0.992565, recall 0.881188, 36 missed references, 2 extra predictions, and 31/31
inserted rows matched to references. It is rejected for OCR output because page
codepoint CER worsens to 0.761733, grapheme CER to 0.792281, matched-line CER to
0.091202, and reading-order accuracy to 0.604292 (selection / metrics /
effect-audit SHA-256
9469f67f8eb1afb300e25bc332e9566fd92f583b77fbc054953027b9883dd444 /
22689ccec2f4cdf7b0f9f0e4679ef13756b3423d81b5c45d65b9b9779de60e6d /
0ba763fb14cfd1e63ff6eb49313ea668ed908986c2533319e18aa4d148364988). The
recall-heavier min_candidate_text_length=10 version selects 38 rows with oracle
diagnostic precision 35/38, raises line F1 to 0.936097, but worsens page CER to
0.764760, adds 3 extra inserted predictions, and fails the effect audit (selection /
metrics / effect-audit SHA-256
d3cf35e7049c113fc9793e1da85d98ac8104ac403064fcf5c5765ee9adfe06bd /
605dee2ba5cd5cb06578ea786feeb225efceb7a72b3825049b81d031f2464356 /
5f4aaef0a472f95757342a3da00ca932eab3453b286d9afb40ba61940cb4d6f2). The right next
move is not another selector threshold. It is to use the 31 precision-first rows as a
review/crop-recognition worklist, improve their text, and fix insertion reading order.
Candidate insertion ordering is now explicit in
tools/apply_candidate_page_line_reviews.py. The default geometry strategy preserves
old diagnostics but sorts all page lines by raw page/y/x geometry, which caused the
large reading-order drop on Gorkhapatra. preserve_source_append keeps the selected
source block order and appends candidate insertions. On the 31-row precision selection,
it keeps line F1 0.933566, recall 0.881188, extras 2, and 31/31 inserted rows
matched, while improving reading-order accuracy from 0.604292 to 0.750472; it is
still unsafe because page codepoint CER is 0.762153, grapheme CER 0.798575, and
reading order remains below the conf070 baseline (application / metrics /
effect-audit SHA-256
66c98b29c9b919333add26d72802e237715317fc331e844d119958d91167b85c /
5f11ba613d1ea7e10ecec40cde78424994edb4258069023541384e193b267ed3 /
56068accf83105e4bd0f526f2ddac0d68fb9a234d52899e35b3ec4bae5ce2743). The
source_reading_order mode re-sorts with the source metadata's
wide_column_right_to_left order but is worse here: codepoint CER 0.773003 and
reading order 0.722227 (effect-audit SHA-256
b62ae4e0be13247a0a8c6af34656058c9412c3bb4fce1b2afd922189511c658e). This narrows the
remaining problem: use append/preserve semantics for reviewed insertions unless a
better slotting policy is proven, and fix the inserted text.
tools/prepare_candidate_line_transcription_worklist.py now turns the 31
precision-first rows into a pending reviewer/crop-recognizer worklist at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist/.
It emits JSONL, TSV, HTML, and recognizer labels; all rows are pending with blank
corrected_text, no oracle/reference fields are emitted, and all 31 HTML crop links
resolve. tools/audit_candidate_line_transcription_worklist.py now independently
audits the packet; the real audit passes with 31 JSONL rows, 31 TSV rows, 31 crop links,
31 label rows, 0 missing crops, 0 crop hash mismatches, 0 forbidden fields, 0 nonblank
review fields, and 0 issues. Report / JSONL / TSV / HTML / labels / audit SHA-256:
7a92ed3d85faee782f22353a778fa36434c6456ca29182a462349ffcc630b180 /
a98488a08ed6d28944b171fe50577a9958960bd31cb6cc7180a58b3e5b270ea1 /
f405d1ede74174f0def525bfa3c5bb9819a3c929671cc74d366a5769e48a31b6 /
3971d08d5bca15286fcceae6326014ad5edbab89a8be1f2656d5e69703cbee72 /
1bcf9418af1ee056ccf32bfe56fccc7d51d26ffdef52f51c1f24397eadced5c0 /
86678953c170569196585aaaf25d6d44fb8d031a7c35f705ad791797d1a0d541.
The reviewed-worklist bridge now exists too. tools/apply_candidate_line_transcription_worklist.py
compiles a filled candidate-line transcription worklist into accepted candidate-page
rows, while enforcing non-empty corrected_text, non-machine reviewer identity, crop
hash validation, and pending-row no-op semantics. The current real 31-row worklist is
still pending-only, so the pending compile at
outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist-application-pending/
has accepted_count=0, skipped_count=31, issues=[], and
application_ready=false (report / accepted JSONL / skipped JSONL SHA-256
a5116e0509ab3612fb83f21298397d57f10a92f32a4f2936f6a2284050577fce /
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 /
2f2aaf09512a659629a6cd54928261a3971a4fce07ec0ecc51009331ce18c39e). Applying that
empty accepted JSONL through tools/apply_candidate_page_line_reviews.py with
preserve_source_append inserts 0 rows and scores exactly as selected conf070:
codepoint CER 0.739361, grapheme CER 0.768262, line F1 0.872458, 67 missed
references, 2 extras, and reading-order accuracy 0.775478 (application / metrics
SHA-256 b2c69ab5c835d83fa9814df1c1e0a0f8797f29bf1f28c53230a74b3027324533 /
5e75442ea41ea8600ef6dd06fc2b246ad61615f5cea6a3107b7e0e3cb090e32b). This closes the
mechanical loop: once reviewed text is added to the 31 rows, it can be compiled,
inserted, scored, and effect-audited without admitting pending machine text.
The immediate local product target is now a Limbu-first OCR pipeline that handles both
Devanagari-written Limbu and Sirijonga-script Limbu from imperfect page captures. This
is not a video-to-page task; video frames are only one possible capture source. The
router at outputs/limbu-first-router-v1/model-card.json is now a guarded
script_select_composite: Tesseract nep+eng is the Devanagari primary, exported v4
PaddleOCR Sirijonga is the secondary, routing_signal_engine=secondary lets Sirijonga
Unicode trigger secondary routing, and primary_guard_script=devanagari prevents clear
Devanagari primary lines from being overwritten by Sirijonga hallucinations. The v4
component was exported from outputs/limbu-recognizer-v4-2026-06-18/limbu-v4.tgz and
packaged under outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/.
Runtime preflight passes in /tmp/ocrtech-paddle311 Python 3.11.
Current evidence is operational, not claim-grade: the Sikkim Herald Sirijonga line crop improved to diagnostic codepoint CER 0.095 against the demo GT but remains in review; the full Sikkim Herald Sirijonga page still fails quality with 20/22 lines in review; the Gorkhapatra candidate page runs with 98 lines, 22 review lines, and 7 guarded Devanagari primary lines, but its references are still pending manual labels. The next real work is full-page Sirijonga improvement and frozen page-level eval, not another headline recognizer number.
Post-correction is now admission-gated: deterministic Limbu correction profiles can be
audited with ocrtech audit-limbu-post-correction-profile, profile IDs/rules are shape
checked, missing admission evidence marks the profile experimental, and profiles that
claim validated status must carry hashed held-out eval evidence with non-regressing
before/after metrics or parse-limbu-doc refuses to apply them. No validated Limbu
profile exists yet; the review queue remains the correction gate.
Output manifests now bind critical inputs by hash: model cards, fallback model cards,
capture-prep metadata, post-correction profiles, source documents, and review TSVs are
recorded in limbu-output-manifest.json when supplied, and audit-limbu-output
recomputes those hashes. This moves the Limbu output bundle closer to claim-grade
provenance, but page-level scoring against frozen references is still missing.
PER-LANGUAGE CER — goal scorecard (RE-GROUNDED 2026-06-18 with verified micro CER)
Goal state: a per-script PP-OCR recognizer system supporting Nepal/diaspora languages, each with a measurable per-language CER.
METRIC CORRECTION (eval-rigor pass). Every "CER" in earlier versions of this table was
1 − PaddleOCR norm_edit_dis. That is NOT CER.norm_edit_disis a per-sample-normalized macro average; on datasets with many short header/caption lines a single error on a 3-char line costs0.33, which drags the macro mean far above the true (micro) error rate. Re-scoring every model through4–8× better than reported**; Newa/Tirhuta (long, uniform synthetic lines, where macro ≈ micro) are roughly as reported and genuinely weak. Grapheme-cluster CER (UAX#29) is the honest headline.infer_rec.py+tools/eval_metrics.py(micro CER = total edit distance ÷ total reference length, NFC-normalized) gives the numbers below. Predictions were eyeball-verified against GT (e.g. magar GTसमावेशीकरणको दिशामा एउटा फड्को :→ PRED identical). Result: the Devanagari/Limbu recognizers are **
| # | Language | Script | graph CER | cp CER | exact-line | Eval | prev (mislabeled) |
|---|---|---|---|---|---|---|---|
| 1 | Limbu/Sirijonga | Limbu (native) | 0.041 | 0.021 | 0.521 | real Gorkhapatra (303 crops) | 0.176 |
| 2 | Majhi | Devanagari | 0.075 | 0.047 | 0.173 | real Gorkhapatra | 0.26 |
| 3 | Magar | Devanagari | 0.078 | 0.047 | 0.287 | real (⚠ selection-leaked) | 0.22 |
| 4 | Sunuwar (Deva) | Devanagari | 0.080 | 0.054 | 0.178 | real Gorkhapatra | 0.24 |
| 5 | Magahi | Devanagari | 0.083 | 0.054 | 0.158 | real Gorkhapatra | 0.21 |
| 6 | Bhojpuri | Devanagari | 0.101 | 0.061 | 0.157 | real Gorkhapatra | 0.19 |
| 7 | Thami | Devanagari | 0.120 | 0.068 | 0.120 | real Gorkhapatra | 0.33 |
| 8 | Wambule | Devanagari | 0.121 | 0.064 | 0.125 | real Gorkhapatra | 0.30 |
| 9 | Darai | Devanagari | 0.129 | 0.083 | 0.073 | real Gorkhapatra | 0.24 |
| 10 | Tirhuta | Tirhuta (native) | 0.343 | 0.250 | 0.084 | held-out synthetic | ~0.35 |
| 11 | Newa/Prachalit | Newa (native) | 0.613 | 0.418 | 0.071 | held-out synthetic | 0.40 |
Headline (honest): on real Gorkhapatra print, Limbu grapheme CER 0.041 (52% of lines exact) and 8 Devanagari languages grapheme CER 0.075–0.129 (codepoint 0.047– 0.083). These are strong, real-print results. Newa (0.613) and Tirhuta (0.343) are weak even on their own synthetic eval — they fail to fit the synthesis distribution they trained on (transliterated text + complex native conjuncts + long lines). Newa is the clear priority to improve.
The lever was DATA, not the model — real data must be same-font/same-domain (the
Sikkim Herald different-font Limbu fine-tune hurt the Gorkhapatra eval; same-font
Gorkhapatra crops helped). Artifacts: outputs/limbu-recognizer-v4-2026-06-18/,
outputs/devanagari-recognizer-v2-2026-06-18/, outputs/newa-recognizer-v3-2026-06-18/,
outputs/tirhuta-recognizer-v1-2026-06-18/.
Residual caveats on the strong numbers: (a) labels are machine-converted legacy→
Unicode — a low CER partly reflects model+GT sharing the converter's systematic biases,
so true CER vs human-correct text needs native-reader verification; (b) magar was the
deva-v2 selection set (mild leak) but sits mid-range, so the leak isn't inflating the
headline; (c) Devanagari/Limbu are real-print, Newa/Tirhuta are synthetic — not directly
comparable.
Data floor raised (the enabler, 2026-06-18)
Real/large training data harvested this session: Limbu 303→6,867 (real, same-font); Newa 3.7k→468k (Devanagari→Prachalit transliteration); Tirhuta 53→111k (Maithili→ Tirhuta translit); Devanagari synth→79k real; Sunuwar 692→4,674; Kirat Rai 887→4,343 (+1,011 eval); Sunuwar/Kirat Rai/Gurung Khema +54k synthetic each (varied conditions). Transliteration via an Aksharamukha py3.10 sidecar venv is the scalable lever for any script with a Devanagari corpus. Training pipeline (push N): Tirhuta training now; Newa-retrain (468k), Sunuwar, Kirat Rai, Gurung Khema queued.
Honest caveats: Limbu + 8 Devanagari are real-page CER (machine-converted labels,
human-verification pending; Devanagari was epoch-9 budget-capped + has a train/eval font
gap, so conservative). Newa's number is held-out synthetic printed Nepal Lipi — a real
manuscript image set exists (Zenodo, 3,180 crops) but is handwritten/cross-domain, so it
needs a manuscript-trained model for a fair real number. The Newa long-line squish (480px)
caps quality; wider rec_image_shape + more printed-register data are the known levers.
Staged next experts — real-eval-crop status (2026-06-18, all from Sikkim Herald legacy→Unicode, machine GT, human-verification-pending):
- Sunuwar (Jenticha) — ✅ 429 real crops, 368 unique, 5,132 letters (NEW,
data/sunuwar-real-eval-pack/,tools/build_sunuwar_real_eval_pack.py; 3 mukhia editions; fully_confirmed bytes only, 88.6% byte coverage;|→U+11BC5 left unapplied). Best of the staged set; crops visually verified as clean single-line newspaper text. - Lepcha — ✅ 132 real crops (
data/lepcha-real-eval-pack/, 4 editions). - Kirat Rai — ✅ 70 real crops (capped: independent typeface, 778 unmapped bytes — cannot easily expand from the 4,343 raw lines).
- Gurung Khema — ◐ real TEXT exists, no real IMAGES. The
gurung-language-toolkitholds ~13k real monolingual Gurung lines (Tamu Times news, CDC textbooks, bible) — but all in modified Devanagari (gvr orthography, ZWJ clusters), not Khema script. The old synth corpus was syllable-salad (text_source: generatedfrom a 58-char dict). Built a Devanagari→Gurung Khema transliterator (tools/devanagari_to_gurungkhema.py) — Khema is a Brahmi abugida (inherent /a/, U+1612F THOLHOMA = virama; Aksharamukha has NO Khema target so the map is hand-derived from Unicode 16 names + Pandey L2/22-157, lossy: sh/ss/s→SA, retroflex/dental/nya nasals→NA, no visarga). Validated: 8 real lines → 0 dotted-circles, renders cleanly. Transliterated the full corpus → 8,291 unique Khema lines, 397,799 chars (data/real-text/gurungkhema-real.txt; p50=44/p90=87 chars). Caveat: eval is synthetic-only (no real Khema print exists anywhere — revival script); single font (NotoSansGurungKhema); GT is machine-transliterated via a self-built, non-native-validated map → CER measures fit-to-our-transliteration, not true Khema reading.
Each of the three with crops needs its expert trained (synth corpora ready in
data/synth-big/) then scored on its real pack → first-ever real-crop CER. GPU-gated
behind Newa v4-wide.
Lessons baked in for the next training pass: warm-start refines need LOW LR (~2e-5,
not 1e-4); augmentation must be MODERATE (heavy aug degrades clean-eval accuracy and slows
from-scratch convergence); filter scriptio-continua lines to ≤ max_text_length; always
verify the GPU is clear of stale processes before launch (a lingering process caused an
OOM). GPU instance i-0fa19dff7b233a353 stopped via OS shutdown (SSO expired at teardown —
user should confirm stopped on next aws sso login --profile playaround).
EVAL-RIGOR PASS (2026-06-18, after a critical Codex review) — RE-SCORE DONE
RESULT OF THE RE-SCORE (the headline finding): the metric correction went the
opposite way from what was feared. Codex was right that the numbers were not what
they seemed — but the error was that I was over-stating CER, not under-stating it.
1 − norm_edit_dis is not CER; the true micro CER is much lower on the short-line
Devanagari/Limbu evals (Limbu 0.176→0.041 grapheme; Devanagari ~0.25→0.075–0.129
grapheme). Newa/Tirhuta (long uniform lines, macro≈micro) stayed weak. All four
models re-scored via infer_rec.py + tools/eval_metrics.py; predictions
eyeball-verified against GT. The scorecard above is now the honest, defensible table.
Still-open gaps below.
Codex review flagged that our headline CER numbers are softer than presented. Honest gaps to fix BEFORE pushing N further (ranked impact/effort):
- Model-selection-on-test leakage (Very High / Low). Best checkpoints were picked
on the eval page itself → optimistic. Fix: per-source train/dev/frozen-test
splits; select on dev, report on a frozen test never used for selection.
- Devanagari: 7 of 8 packs are already clean (only
magarwas the selection set during deva-v2 training). Re-score the 7 as clean test; flag magar as leaked or re-select on a held-out dev. - Limbu: 0.176 WAS selection-leaked (picked on the one eval page). Needs a 3-way
by-edition split (24 editions) + retrain. NOTE:
data/limbu-gorkhapatra-extra/labels lack per-edition tags → re-extract with edition provenance first.
- Devanagari: 7 of 8 packs are already clean (only
- Codepoint vs grapheme-cluster CER (Very High / Low). Built
tools/eval_metrics.py— reports BOTH codepoint CER and grapheme-cluster CER (UAX#29 viaregex \X). For combining-mark scripts a dropped vowel-sign is 1 codepoint but corrupts a whole akshara, so codepoint CER under-counts: e.g. 2 dropped vowel signs in a 7-cluster / 15-codepoint line = codepoint CER 0.13 but grapheme CER 0.29. Report grapheme CER as the honest headline; our numbers will tick UP. - Separate synthetic-held-out from real-page CER in every claim (Very High / Low). Limbu 0.176 (real page) vs Newa 0.40 / Tirhuta 0.35 (held-out synthetic) are NOT comparable. Newa real test = the Zenodo manuscript (clean, never trained/selected) — eval v3 on it. Tirhuta = synthetic-only, label as such.
- Label verification via active learning (Very High / Med). Labels are machine- converted (legacy→Unicode); a model can learn the converter's systematic errors and report fake progress. Verify high-uncertainty/checkpoint-disagreement lines first.
- Build a text detector (High / Med). Recognition-only CER won't predict page OCR; detector misses are the largest absent error source.
- Transliteration = pretraining, not GT (High / Low). Fine-tune/eval Newa+Tirhuta on REAL native text so they're document recognizers, not transliteration-font recognizers.
- Blind spots: consistent NFC + grapheme-cluster eval; native-reader error review (errors on meaningful marks/names matter more than CER); licensing/provenance for any public dataset/model release. "First-ever OCR" claim should narrow to "recognition model under a defined eval" until detector + verified labels + frozen test land.
Status:
- ✅ (2) Codepoint + grapheme-cluster CER —
tools/eval_metrics.pybuilt; all 4 models re-scored on instance via/home/ubuntu/re_score.py. Scorecard re-grounded. - ✅ (3) Synthetic vs real separated — table now labels each eval; Newa/Tirhuta flagged synthetic. (Newa-on-Zenodo real test still pending — manuscript domain.)
- ◐ (1) Selection leakage — Devanagari: 7/8 packs confirmed clean test,
magarflagged leaked (mid-range, not inflating). Limbu still selection-leaked → by-edition 3-way split + retrain still TODO (labels need per-edition provenance re-extract). - ☐ (4) Label verification (native-reader / active-learning), (5) detector, (6) real-text fine-tune for Newa/Tirhuta, (7) NFC + provenance discipline — open.
Next concrete moves: (a) Newa is the weakest real lever — retrain with wider
rec_image_shape + real printed-register Nepal Lipi (not just transliteration); (b) Limbu
by-edition frozen-test split to kill the last selection leak; (c) native-reader spot-check
of ~100 Devanagari lines to bound the converter-bias question.
Newa v4-wide (RUNNING 2026-06-18) — Codex-prescribed width fix
Codex review + a no-GPU render audit pinned the Newa failure (0.613 graph CER on its OWN
single-font synthetic eval) to a mechanical squish, not fonts: v3 rendered at native
widths p50=454 / p90=1051 / p99=1292 px but the recipe resized to 48×480, crushing
45% of training images (p90 = 2.2× horizontal compression) so the dense Prachalit
conjunct stacks smear illegibly. Render fidelity itself is fine (clean glyphs, no tofu);
the source images are already full-res, so no re-synthesis was needed. The "lots of fonts"
premise is false — only 2 Unicode Newa fonts exist (NotoSansNewa, NepalLipiUnicode) and v3
used one; fonts are not the lever until the model first fits its own single-font set.
v4 = controlled width experiment: image_shape 480→640 (800 OOM'd at batch 64),
RecConAug disabled, batch 48, 40 epochs, same data/font/warm-start as v3.
runs/newa-v4-wide-2026-06-18/.
VERDICT (epoch 8, stopped early): width alone did NOT rescue Newa. Re-scored micro CER
cp=0.628 / graph=0.794 — worse than v3 (cp 0.418/graph 0.613), despite an easier
640px eval; norm_edit_dis plateauing ~0.55. v4 is undertrained (8 vs v3's 20 epochs) so
this is not a clean negative, but it is clearly not on track for Codex's hoped 0.25–0.40 —
confirming the real bottleneck is transliteration/data fidelity, not pixels. Stopped to
free the GPU for the staged-expert queue; best_accuracy checkpoint preserved for a
full-length resume if a clean width verdict is wanted later. The real Newa v5 fix =
better/native-validated text + real printed-register data, not just shape.
Staged-expert queue (2026-06-18→19, autonomous; stopped after Sunuwar per user)
run_queue.sh warm-started from the Devanagari pretrain, 640px, batch 32, 6h-cap, re-scored
on the REAL eval pack, then shutdown -h now. User redirected mid-run to stop after Sunuwar
(Kirat Rai/Lepcha skipped, corpora staged on EBS for later).
Sunuwar — FIRST-EVER real-crop CER: cp_CER = graph_CER = 0.342, exact-line 0.007 (n=429 real Sikkim Herald crops; ~epoch 16, 6h cap). cp==graph because Jenticha is an alphabet (no combining-mark clusters). 34% CER = not usable yet, and the cause is the predicted synth→real FONT GAP: trained on synthetic NotoSansSunuwar but the real crops are the Sikkim Herald newspaper typeface.
SAME-FONT RETRAIN (2026-06-19) — data-floor thesis CONFIRMED for Sunuwar
Harvested 3,010 real same-font Sunuwar crops from 22 Herald editions
(tools/harvest_sunuwar_train_crops.py; converter verified uniform 87–90% across all 25
editions; 3 mukhia editions held out as the 429-crop eval — clean by-edition split). Fine-tuned
the synth model on them (warm-start, LR 2e-5, 60 epochs).
| Sunuwar | synth-only | same-font |
|---|---|---|
| codepoint CER | 0.342 | 0.121 |
| grapheme CER | 0.342 | 0.121 |
| exact-line | 0.007 | 0.184 |
0.342 → 0.121 (−65% relative), exact 0.7%→18.4% — purely from real same-font data, the same lever that took Limbu to 0.041. Not as deep (3,010 crops vs Limbu's 6,867; converter ~89% byte-confirmed → some label noise). More editions / higher byte-coverage would push lower.
Lepcha + Kirat Rai stay blocked for same-font: only their eval-set editions decode (Lepcha's converter returns 0 on all 26 non-eval editions; Kirat Rai's typeface uncracked) — deferred pending font-decoding work. Newa got a separate fix: trained on 4,456 real native Prachalit lines (not transliteration) — result pending. Devanagari + Limbu untouched.
Verified state at start (read from code + disk, not assumed)
Sources: Agent sweeps over training.py, models.py, source_audit.py,
limbu_legacy.py, language_registry.py, languages/nepal-v1.json,
docs/REMOTE_TRAINING.md, and disk inventory of data/, runs/, outputs/.
Training
ocrtech train-recognizeris a recipe/config generator (writes PaddleOCR label files, dictionary,paddle_recognizer.yml, summary) with an optional--runthat shells out topython PaddleOCR/tools/train.py. Infra is complete: remote host audit/bootstrap/sync + AWS G6 GPU training have all been exercised.- All recorded fine-tunes are the OLD mixed Nepali/English PP-OCRv3 effort, and
all failed admission.
outputs/recognizer-failure-review-paddle-ppocrv3recommendsstop_family(paired hard-50:0wins /545losses /7ties). HF TrOCRrecognizer-next-v1trained (eval CER 0.188) but still lost to Tesseract on benchmark. - No script-specific (Limbu/Sirijonga or any Nepal-language) recognizer has ever
been trained.
runs/sirijonga-limbu-paddle-{ablation-clean,ablation-lighting,ablation-camera,recipe-smoke}*are recipes only (labels + yml + summary); there are zero.pdparamscheckpoints anywhere underruns/. - Lesson (do not repeat): the dead family is mixed-Devanagari PP-OCRv3 mobile fine-tuning. The new approach must be script-specific.
Real validation (the claim's test set)
- Zero verified real references exist for any language. The one identified
Limbu page (
gorkhapatra-language-page-0001-01, date 2083-02-12, PDF page 6, fontNamdhinggoSILL, 304 target spans) is stuck atpending=298manual line reviews. - The verification pipeline (
prepare-…-verification-bundle→apply-…-verification-bundle) only setsclaim_evidence_eligible=trueafter per-line humanreview_status. It does not ingest PDF-native text. - But
extract-gorkhapatra-language-page-pdf-textalready extracts legacy Namdhinggo spans (bbox/font/codepoints) andlimbu_legacy.pyalready converts them to Unicode Limbu (~9,957 chars). This converted text is not wired into the verified-reference path — so the pipeline currently demands transcription it does not need. (Matches the handoff pitfall: don't ask for manual transcription when PDF-native extraction already yields usable Unicode.)
Per-language evidence (triage so far)
- Limbu/Sirijonga — only confirmed native-script evidence; text layer is legacy-font (Namdhinggo), deterministically convertible to Unicode.
- Live-pack (untriaged,
script=unknown, PDFs on disk): संस्कृत (Sanskrit, pages 6–7), मगर (Magar), वाम्बुले (Wambule), राना थारु (Rana Tharu) — 8 page candidates across epapers2083-02-31and2083-03-01. - Most other registry languages are Devanagari-transcribed synthesis text with no Gorkhapatra page identified yet.
Critical path to the goal
- Triage matrix — classify each Gorkhapatra page's text-layer form per
language:
unicode_native/legacy_font/devanagari_transcription/latin_notation/no_text_layer. Sets N and per-language script form. - First real eval (Limbu) — wire PDF-native legacy→Unicode into the verified reference path so the Limbu reference becomes claim-eligible without 298 from-scratch transcriptions (pre-fill + human sign-off, not manual typing).
- Baselines on the real Limbu page — stock PP-OCRv5/v6, Tesseract, Surya, PaddleOCR-VL. Records the bar the fine-tune must beat.
- First script-specific fine-tune (Limbu/Sirijonga) — train the existing
Sirijonga recipe on a GPU host (AWS G6 path exists); eval on the real Limbu page
- held-out synthetic with no text-hash leakage.
- Generalize per qualifying language — repeat triage → real eval → baseline → fine-tune. Decide single-multilingual vs per-script model from the matrix.
Checklist
Phase 1 — Triage matrix (evidence decides N)
- Inventory existing Gorkhapatra packs + confirm PDFs on disk.
- Run
extract-gorkhapatra-language-page-pdf-texton the live-pack (all 8 pages, 4 candidate languages). Result:passed=True samples=8/8 spans=3486. - Classify each page's text-layer form. All pages are
legacy_font— no Unicode text layer exists anywhere in the Gorkhapatra epapers. - Record the matrix (below) + key findings.
- Disambiguate live-pack language→page alignment (2 languages currently map to the same physical page — see findings).
Phase 2 — First real eval (Limbu, no manual transcription)
Design (locked, grounded in code): Add a PDF-native legacy-conversion source
for the draft reference's reading_order. New command
prepare-gorkhapatra-language-page-pdf-native-references emits a draft
.ref.json per accepted page whose reading_order items are
{id, text=<legacy→Unicode>, bbox, confidence} from the converted target-font
spans (limbu_legacy.convert_gorkhapatra_limbu_pdf_text rows), with
reference_status=draft, claim_evidence_eligible=false, machine_generated=true,
requires_human_review=true, assisted_source=pdf_native_legacy_conversion.
Then the existing chain runs unchanged, preserving every claim gate:
prepare-…-verification-bundle (pre-fills ocr_text=converted Unicode + crops +
dashboard) → human accept/correct in dashboard → audit-…-verification-csv →
apply-…-verification-bundle --require-codepoint-range U+1900..U+194F --tables-reviewed --figures-reviewed --captions-reviewed →
claim_evidence_eligible=true. Reviewer effort drops from "transcribe 298 lines"
to "accept pre-filled correct Unicode".
Why this shape: apply builds verified text from the verification CSV's ocr_text
(on accept) keyed by line_id, and pulls tables/figures from the draft ref
(source_audit.py:3679-3745). Sourcing the draft reading_order from the
conversion is the only change; gates, dashboard, and audits are reused as-is.
Implementation gotchas to handle:
bbox format: pdf-text spans look like
[x0,y0,x1,y1]; the bundle's_clean_bbox_xywh/_write_xywh_cropexpect[x,y,w,h]. Convert.conversion completeness: 36/304 Limbu spans have
unmapped_codepoints. Flag those rows innotes(and/or lower confidence) so the reviewer scrutinizes them; do not present them as clean. Clean rows (268) are confident accepts.bboxes are already in rendered-image pixel space (1985×3062 ≈ span extent), so no DPI rescale is required.
Decide the integration shape — new
prepare-…-pdf-native-referencescommand feeding the existing verification→apply→audit chain (above).Implement the command + reading_order-from-conversion builder (bbox xywh conversion via per-sample image/PDF-point scale, unmapped-codepoint flagging, generic
review_notesurfaced in the verification CSV). Code:source_audit.py(prepare_gorkhapatra_language_page_pdf_native_references,_pdf_native_language_page_reference,_bbox_points_to_image_xywh,_image_pdf_point_scale); CLIprepare-gorkhapatra-language-page-pdf-native-references.Generate the Limbu draft + verification bundle on real data: draft
converted=1(303 spans, 11,890 Unicode chars, scale≈2.0008, 36 spans flagged — almost all benign smart-quote punctuation U+2018/U+2019, not failures); bundlelines=303 crops=303 missing_bboxes=0, all 303ocr_textpre-filled. Artifacts:data/limbu-gorkhapatra-language-page-pdf-native-references-2026-06-16/,data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/.Add fixture tests (
tests/test_source_audit.py): bbox scaling; draft is Unicode + non-claim-ready; an accepted draft passes the apply claim gate with theU+1900..U+194Frange check. Focused suites green;git diff --checkclean.Human sign-off (now a light accept pass, not transcription) — open the confirmed PDF text dashboard, spot-check converted Unicode against the clickable bounded spans/crops, scrutinize the 36 flagged spans, set
review_statusper line in the downstream verification CSV, then runapply(commands below). Agents must NOT fabricate this review; the verified reference asserts a human checked it.After sign-off: pass
audit-gorkhapatra-language-page-review --require-verified-references.
Commands to finish the Limbu real eval (human sign-off)
# 1. Review in the confirmed browser dashboard: full PDF page + clickable boxes
open /Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html
# use that dashboard to verify/correct the downstream verification-lines.csv;
# do not open the deleted/broken pdf-native verification-review.html
# 2. Dry-run the status counts
uv run ocrtech audit-gorkhapatra-language-page-verification-csv \
data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-lines.csv \
--out outputs/limbu-pdf-native-verification-csv-audit
# 3. Apply -> claim-eligible verified reference (note: --require-codepoint-range U+1900..U+194F)
uv run ocrtech apply-gorkhapatra-language-page-verification-bundle \
data/limbu-gorkhapatra-language-page-pack-2026-06-16/manifests/gorkhapatra-language-page-pack.jsonl \
data/limbu-gorkhapatra-language-page-review-2026-06-16/gorkhapatra-language-page-review.csv \
data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-lines.csv \
data/limbu-gorkhapatra-language-page-pdf-native-references-2026-06-16 \
--candidate-language लिम्बू \
--require-codepoint-range U+1900..U+194F \
--tables-reviewed --figures-reviewed --captions-reviewed \
--out data/limbu-gorkhapatra-language-page-verified-2026-06-16
Phase 3 — Baselines on real Limbu page
- Real-eval → recognizer bridge built + validated. New command
prepare-gorkhapatra-language-page-recognizer-evalturns a verified reference'sreading_orderinto tight line crops + a recognizer eval manifest (ManifestEntry, split=eval, slices incl.real/hard_eval/recognizer_eval/language:<X>), mirroringcrops.pyconventions so it plugs intoaudit-recognizer-corpus/benchmark. Smoke on the Limbu draft (--allow-draft): 303 tight line crops (e.g. 177×29 px), emitted manifest passesaudit-manifest(303 samples). Honest gating: draft rows areclaim_evidence_eligible=false; only verified references yield claim-grade real-eval rows. Tests added; suites green. - After Limbu sign-off: re-run the bridge without
--allow-drafton the verified reference → claim-eligible real recognizer eval manifest. - Run stock PP-OCRv5/v6, Tesseract, Surya, PaddleOCR-VL on the real Limbu line crops; record CER/line-accuracy (expected: stock engines largely fail on Sirijonga — that failure is the baseline the fine-tune must beat).
Phase 4 — First Limbu/Sirijonga fine-tune
GPU host (found): ocrtech-g6-training-20260615-v4 = i-0fa19dff7b233a353,
g6.xlarge, stopped, us-east-1a, AWS account 798006441552. Auth: aws CLI works
via --profile playaround (the admin/default profile is for a different account).
Starting the instance costs money (~$0.8/hr g6.xlarge) → confirm before launch.
Base-model correction (important): the existing recipes under
runs/sirijonga-limbu-paddle-* use base
devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams — which is both the dead
family (recognizer-failure-review → stop_family) and script-mismatched
(Devanagari weights for a U+1900–194F script). Per
MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md §3, Limbu needs a separate
recognizer + dictionary, not the Devanagari base. So regenerate the recipe with
the Limbu dict and no Devanagari pretrained head (from-scratch / backbone-init).
Inputs ready (free prep done):
- Limbu dict:
data/dictionaries/limbu-sirijonga-v1.txt(76 chars; covers all 68 required Limbu-range chars; 0 missing vs labels). - Augmented rendered split:
outputs/sirijonga-limbu-rendered-split-namdhinggo-augmented-2026-06-16/— 900 train / 162 eval rows (six degradation profiles, text-hash isolated).
Validated training approach (research fan-out, 2026-06-16, cited). A 4-agent research pass against current PaddleOCR sources overturned the initial from-scratch PP-OCRv3 assumption:
- Harness is stale.
training.pyhardcodes PP-OCRv3 mobile (MobileNetV1Enhance+ SAR head) +devanagari_PP-OCRv3_mobile_rec_pretrained. Current shipping gen is PP-OCRv6 (paddleocr3.7.0, 2026-06-11); PP-OCRv5 is the documented Brahmic-validated baseline. All current rec is SVTR-based; CRNN/MobileNetV1+SAR are legacy.tools/train.py -c config.ymlstays (current). - Fine-tune, don't train from scratch. For a disjoint new script, warm-start from a pretrained rec, reuse the SVTR backbone, reset only the head for the Limbu dict (PaddleOCR skip-loads the mismatched head). Backbone transfers across disjoint scripts; from-scratch overfits our small set.
- Recipe: base
configs/rec/PP-OCRv5/multi_language/devanagari_PP-OCRv5_mobile_rec.yaml(SVTR_LCNet / PPLCNetV3 + CTC+NRTR MultiHead), pretrainedPP-OCRv5_mobile_rec_pretrained.pdparams(ordevanagari_PP-OCRv5...for a Brahmic warm-start). Mobile tier (server/SVTRv2 overfit on <9k lines). LR 1e-4,max_text_length50 (verify vs labels),rec_image_shape [3,48,320], MultiScaleSampler, RecConAug+RecAug, bs 64, ~100 epochs @9k / 200–300 @900. - Data caveats: ~900 augmented lines is below PaddleOCR's ~5,000 finetune
floor → smoke/pipeline check only, not a deliverable. Scale toward the ~8,800
unique Limbu lines (render the fuller limdic corpus, not the 200-hash subset).
Enforce one canonical Limbu codepoint ordering (NFC + subjoiner/vowel reorder —
reuse
limbu_legacy.py) across all labels AND CER. Dedup train/eval by image hash AND (text, font); freeze the real newspaper eval. Heavy degradation aug (Augraphy/ocrodeg-style) + a small real-data fine-tune close the synthetic→real gap. - Greenfield: no published Limbu OCR / Tesseract model exists — first of its kind. Closest proven recipe: IIIT Indic CRNN+CTC, synthetic pretrain → small real fine-tune.
DATA-PLAN CORRECTION (methodology review, 2026-06-16). Sampling exposed a
killer hole: the limdic synthesis corpus is dictionary headwords (69% single
words, median 7 chars/1 word) while the real eval is newspaper prose (median 39
chars/5 words). "More data" of the same kind deepens the mismatch. Root cause: the
synthesis used the worst available source. Investigation found native-Sirijonga
prose corpora already migrated under
…/limbu-speech-toolkit/data/external/ that were never used:
pustakalaya/limbu_unicode/limbu-bhasha-kaksha-7.txtetc. (school textbooks): 5,127 lines, median 40 chars / 7 words ≈ exact eval match; thousands more across grades/subjects +cdc_moecdc,curriculum_*,grammars,theses_tu.web_limbu/limbu_sentences.txt(535, 25/4) andweb_limbu/raw/sikkim_herald_decoded.tsv(~514) — Sikkim Herald newspaper = best register match, different paper → no eval leakage.ebible_lif/biblenlp_lif-lifNT_sirijanga.txt(7,873 verses, long).- (IGT corpus is romanized Latin; LIFWBT Bible is Devanagari-script — NOT Sirijonga.
A
DevUni→LimUniSIL map exists to transliterate the 9,283 Devanagari verses if more is needed.)
Corrected data plan: build the Limbu training corpus from textbook + newspaper + web Sirijonga prose (eval-matched line length), keep limdic only for lexical coverage, track per-source license_status (textbooks/Bible = pending_review).
HARVEST DONE (4 sonnet agents, 2026-06-16). Extracted + consolidated native
Sirijonga prose → outputs/limbu-prose-harvest/limbu-prose-corpus.jsonl:
10,285 unique lines, median 47 chars / 8 words (≈ eval's 39/5), 0 eval
leakage (NFC-normalized, deduped, excluded the 303 Gorkhapatra eval lines).
Sources: 9,710 textbooks (E-Pustakalaya/CDC) + 514 Sikkim Herald newspaper + 27
Wikipedia incubator + 25 pangloss + 8 UDHR. Archive.org yielded 0 (all English/
Devanagari about Limbu); academic mostly romanized/IPA. License tiers (research
use only; clear before any redistribution): textbooks CC-BY-NC-ND 3.0 (derivative
text — internal only); Sikkim Herald = govt bulletin, silver (legacy-decoded,
not gold); Wikipedia/Wiktionary CC-BY-SA; pangloss CC-BY-NC-SA; UDHR free.
More available later: offline Crúbadán (~340k words), empty textbook grades 2&5.
Note for recipe: eval lines run to 114 chars / 16 words → set max_text_length ~120
and consider a wider rec_image_shape than 320px for long newspaper lines.
SHAPING HOLE (Hole 8, critical, must fix before GPU). The line renderer
(synthesis_resources.py:2354) uses plain Pillow draw.text with no
layout_engine=LAYOUT_RAQM, and PIL.features.check('raqm') is False in this
env → no HarfBuzz complex shaping. Limbu stacks subjoined consonants (U+1929–192B) +
reorders vowel signs; basic layout has no GPOS/GSUB guarantee, so synthetic Sirijonga
crops may be mis-shaped vs the correctly-typeset real page — and prior "passed" render
audits never checked visual shaping (only font SHA / label coverage / degradation).
A basic-render vs real-crop visual diff looked broadly similar (Namdhinggo may
position marks acceptably without GPOS), but this is unverified-safe. Fix: render with
LAYOUT_RAQM (libraqm trivial on the Ubuntu G6) and visually verify a sample against
real crops before the full render+train. Add a shaping check to the render audit.
SHAPING RESOLVED (2026-06-16) — locally, no GPU needed. brew install libraqm
(+ fribidi) then rebuilt Pillow from source (uv pip install --reinstall --no-binary pillow pillow with PKG_CONFIG_PATH to brew) → PIL.features.check('raqm') is now
True. Renderer (synthesis_resources.py) switched to ImageFont.Layout.RAQM when
available (falls back to BASIC). Finding: for Namdhinggo, basic≈raqm (a visual diff
of identical text matched) — so prior Limbu renders were NOT broken; but raqm is
essential for Devanagari (the other languages) and is now the default. Validated:
the 10,285-line prose corpus → synthesis-text manifest
(data/synthesis-text/cdjk-limbu-prose-2026-06-16/limbu-prose-text.jsonl) → rendered
with raqm produces correct Sirijonga (e.g. 'ᤁᤡᤖᤠᤋ᤻ ᤕᤠᤰᤌᤢᤱ ᤐᤠᤏ᤻'). Corrected
training-data pipeline (right corpus + correct shaping + PaddleOCR labels) is proven.
Dict fix (Hole 3, verified): data/dictionaries/limbu-sirijonga-v1.txt is
missing 6 punctuation chars the real page uses — U+0027 ', U+0028 (, U+002A *,
U+002F /, U+2018/U+2019 ' '. The recognizer can't emit them until added.
(The dict was audited against synthetic dictionary labels, not real prose.)
v1 scope decisions (autonomy): single Gorkhapatra page as honestly-scoped v1 proof-of-concept (expand editions before any paper claim); clean-digital recognizer first (matches the clean-PDF eval; defer scan/photo-degradation + multi-font robustness to v2). Eval is recognizer-only with oracle (PDF) line boxes and machine-converted+human-verified labels — label the claim accordingly.
Code change required: write_paddle_recognizer_recipe (training.py) must emit
the PP-OCRv5 mobile skeleton (PPLCNetV3 + CTC/NRTR MultiHead, MultiScaleSampler,
LR 1e-4, pretrained_model → PP-OCRv5 mobile rec) instead of the hardcoded PP-OCRv3
config. The from-scratch PP-OCRv3 recipe generated earlier
(runs/sirijonga-limbu-paddle-rec-fromscratch-2026-06-16) is superseded — discard.
Sequence (corrected): get the recipe + data right, then one GPU run.
- Update
training.pyto emit the PP-OCRv5 mobile recipe (PPLCNetV3 + CTC/NRTR, NRTRLoss,gtc_encode: NRTRLabelEncode, anchoredmax_text_length) + addedPADDLE_PPOCRV5_MOBILE_REC_PRETRAINEDand switched the function + CLI--base-modeldefaults off the dead PP-OCRv3 base. Guard tests updated to pass the stopped base explicitly;test_training_benchmark.pygreen (49 tests). - Built the right corpus + correct shaping: 10,285 prose lines → leakage-safe split (train 9,209 / eval 1,076) → raqm render (18,418 train + 1,076 eval images).
- Assembled the full PP-OCRv5 training package:
runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/(recipe yml w/ PP-OCRv5 pretrained- v2 dict; 18,418 train + 1,076 eval labels). Ready to ship to GPU.
- [
] G6 run IN PROGRESS (2026-06-16): instance i-0fa19dff7b233a353 running at 3.235.87.54 (NVIDIA L4 23GB, CUDA driver 13.0, Python 3.10); SSH key `/.ssh/ocrtech-g6-training-20260615.pem, userubuntu. Fresh instance — env setup running (/setup.sh/setup.log→ venv + paddlepaddle-gpu cu126 + PaddleOCR main + requirements, logging). Bundleoutputs/limbu-ppocrv5-bundle-2026-06-16/limbu-ppocrv5-training.tar.gz(230MB, 19,501 files) transferring via scp. Next: extract → download PP-OCRv5 pretrained →tools/train.py -c paddle_recognizer.yml` (nohup, ~1-2h) → export → eval. - [
] TRAINING LIVE (2026-06-16 ~18:12): PP-OCRv5 mobile fine-tune running on the L4, no errors. ~0.94 s/iter, ~68 samples/s; by epoch 2/limbu-train`.norm_edit_dis≈ 0.13 and rising, loss falling (CTC ~43 + NRTR ~2.9), GPU mem 13/23GB. PaddleOCR ETA ~7h24m for full 100 epochs (≈$6). Decision: leave at 100 epochs, stop early on eval plateau (per-epoch checkpoints + best-metric tracking → no loss from early stop). Background watcher polls eval metric → pulls me back on plateau / ~28-min check-in / done / error to evaluate the best checkpoint against the real Limbu recognizer-eval manifest. Launch cmd:python ~/PaddleOCR/tools/train.py -c <recipe> -o Global.pretrained_model=pretrain/PP-OCRv5_mobile_rec_pretrainedfrom ` - Eval on real Limbu recognizer-eval manifest (post sign-off) + held-out synthetic; record CER/norm_edit_dis. (Stock baseline deferred — known to fail on Sirijonga.)
- Pass
audit-augmentation-ablationwith real verified evidence present.
Phase 5 — Second language (Devanagari): converter SOLVED + real-eval STAGED
Update (2026-06-17): the verified-converter blocker is resolved.
- Built
nepal-ttf2utf(public: github.com/Ampixa/nepal-ttf2utf, MIT) — legacy ASCII-font → Unicode for Nepal scripts: Devanagari (preeti/kantipur/sagarmatha/pcs/ fontasy via testednpttf2utf) + nayanepal/Gorkhapatra extensions (ƒ→र, †→्, validated 97–99% clean Devanagari on real pages) + Limbu/Sirijonga (SIL Namdhinggo map). 9 tests green. Blog draft submitted to Ampixa (passed style check, queued). - Magar Devanagari real-eval STAGED:
outputs/magar-real-eval-pack/— 383 line crops- Unicode ground truth (via the converter), 13,561 chars. This is the first real Gorkhapatra Devanagari-language eval set (couldn't exist before the converter).
- Remaining for N≥2: train a Devanagari PP-OCRv5 recognizer (harvest Devanagari prose like Limbu → render w/ raqm → train on GPU) → eval on the Magar real-eval pack → per-language CER. The converter/eval-data blocker is now gone; this is one GPU campaign.
Phase 5 — Second language (Devanagari): (prior) blocked on a verified Preeti converter
Investigated (2026-06-16) to reach N≥2. The real Gorkhapatra Devanagari language pages (Sanskrit/Magar/Wambule/Rana Tharu) have no available Unicode ground truth:
nayanepal/Gorkhapatra-Boldfonts have generic glyph names → no derivable map.- The web article body is published as an image, not text (only the Nepali title is Unicode) — so no web cross-reference for the body.
- No verified Preeti/Devanagari-legacy converter in the migration or pip-installable here.
The spans are Preeti (
g]kfn→नेपाल,du/→मगर,dxfjL/ k\"g du/→महावीर पून मगर), but a hand-built Preeti map is only ~80% accurate (garbles e.g.गूमँ७ाअाे— digit/sign errors). Decision: do NOT build a Devanagari real-eval on noisy labels (would conflate model error with ~20% label noise — violates the real-validation discipline). N≥2 is a scoped follow-up needing ONE verified component: a tested Preeti converter (or an expert/anchor- validated table). Then all 4 Devanagari pages convert and the ready ~200k-row Devanagari corpus trains a recognizer → languages #2–5 in one pass. Status: N=1 (Limbu) done; N≥2 pending verified converter.
Phase 5 — Generalize (hold until the Limbu loop closes)
Recon (2026-06-16): no nayanepal/Preeti/Devanagari legacy map exists in the repo
(only data/mappings/limbu-legacy/Limbu.map). Two paths to Devanagari-page references:
- (A) nayanepal→Unicode map — deterministic, bbox-aligned like Limbu, but requires
sourcing/deriving the real font encoding (embedded
nayanepal.ttffiles are saved under the pdf-textfonts/dirs). Do NOT assume Preeti-compatibility — verify. - (B) article web-text cross-reference — each page has an
article_url(gorkhapatra.../news/<id>) and the website serves Unicode; align web text to legacy PDF spans for bbox-level line text. Likely cleaner than reverse-engineering the font. - Resolve live-pack language→page ambiguity (2 languages per page) first.
- Decide path A vs B for Devanagari references; build the converter/aligner.
- Repeat triage → real eval → baseline → fine-tune per qualifying language.
- Decide single-multilingual vs per-script model shape.
- Per-language support table (native vs fallback, CER) — the deliverable.
OVERNIGHT — multi-language push (2026-06-17, autonomous)
Mission: deliver multiple languages with real-page CER by morning. Strategy: one
Devanagari PP-OCRv5 recognizer evaluated on each Devanagari language's real
Gorkhapatra page (ground truth via the new nepal-ttf2utf converter) → Magar + Wambule;
plus the existing Limbu recognizer → 3 languages.
PRINT CORPUS FOUND (Naya Nepal archive). Gorkhapatra's Naya Nepal publishes a
daily page in a different language — a scan of ~100 recent issues found 50 distinct
languages (वाम्बुले, मगही, सुनुवार, थामी, राजवंशी, कुलुङ, बान्तावा, मैथिली, मगर, भोजपुरी,
माझी, शेर्पा, सन्थाली, याक्खा, धिमाल, गुरुङ, ल्होमी, लिम्बू, तामाङ, चाम्लिङ राई, उर्दू…).
Each is a real printed page (epaper PDF). Almost all are Devanagari → our ONE Devanagari
recognizer + nepal-ttf2utf scores ~47 of them. Genuinely non-Devanagari: Limbu
(Sirijonga, done), Urdu (Perso-Arabic — separate recognizer needed), and a few
Tibetan-sphere langs (likely Devanagari in print).
9 languages staged for real-eval (real Gorkhapatra pages → Unicode ground truth via the
converter): Limbu (Sirijonga) + 8 Devanagari — Magar, Wambule, Darai, Magahi,
Sunuwar, Majhi, Bhojpuri, Thami (362–444 line crops each). 8 Devanagari packs shipped to the
G6 (~/deva-train/real_eval/); scored by the one recognizer once it plateaus.
State:
- Converter enabler shipped:
nepal-ttf2utf(public, blogged). Unlocks Devanagari real ground truth from legacynayanepalpages (97–99% clean). - Real-eval packs staged (3 languages):
outputs/{limbu,magar,wambule}-real-eval-pack/(Limbu 303 / Magar 383 / Wambule 383 line crops + Unicode labels). Magar & Wambule are distinct Devanagari language pages (मगर भाषा / वाम्बुले राई). - Devanagari corpus: 411,996 eval-matched prose lines (median 40 chars / 7 words ≈ eval
39/6; segmented from migration Devanagari sources, 0 eval leakage). Sampled 40k →
split train 35,937 / eval 4,063 → rendered 71,874 + 4,063 images (raqm, Noto Serif
Devanagari). Dict
devanagari-v1.txt(141 chars, 75 Devanagari). - Recipe: PP-OCRv5 mobile, same-script
devanagari_PP-OCRv5warm start (verified HTTP 200), LR 1e-4. - GPU: ~4h budget on playaround (user-authorized start/stop). Capacity-retry poller restarting the G6; bundle building. Plan: sync → train (stop-early on plateau, ~1–2h like Limbu) → eval on Magar + Wambule packs → per-language CER → stop instance.
- Honest caveat: Devanagari renders use Noto Serif, eval is the
nayanepalnewspaper face → a train/eval font gap (unlike font-matched Limbu), so Devanagari real CER will be conservative. Labels are machine-converted (human-verification-pending), single page each.
NON-DEVANAGARI NOVEL SCRIPTS — the real headline (2026-06-17)
User direction: "those which aren't Devanagari will be our proper headline" and "don't care we can synthesize. we will find other sources for validation" — so corpus/validation scarcity is not a blocker. With synthesis-from-font, the only hard gates are (1) the script is Unicode-encoded (a target to OCR into) and (2) a usable font exists (to render training data). Validation pages are sourced separately (Sikkim Herald prints Kirat Rai / Sunuwar / Lepcha editions, etc.).
Research: 7 sonnet agents (Magar Akkha, Kirat Rai, Newa/Tirhuta, then Gurung Khema+Sunuwar, Tibetan, Lepcha), primary-source-verified. Feasibility matrix:
| Script | Languages | Unicode | Font (for synthesis) | Existing OCR? | Verdict |
|---|---|---|---|---|---|
| Limbu/Sirijonga | Limbu | 4.0 | Namdhinggo (SIL) | none | ✅ DONE — real CER 0.21 |
| Kirat Rai | Bantawa/Kiranti | 16.0 (U+16D40) | Kanchenjunga OFL + AKRS TECkit map | none | ✅ novel |
| Sunuwar/Jenticha | Sunuwar | 16.0 (U+11BC0) | Noto Sans Sunuwar (Google Fonts) | none | ✅ novel — simplest |
| Lepcha (Róng) | Lepcha (Ilam) | 5.1 (U+1C00) | Noto Sans Lepcha + Mingzat (SIL) | none | ✅ novel |
| Tirhuta | Maithili | 7.0 (U+11480) | Noto Sans Tirhuta | none | ✅ novel |
| Gurung Khema | Gurung | 16.0 (U+16100) | Noto (built from SFD source via fontforge — 58/58 covered) | none | ✅ novel |
| Newa/Prachalit | Newar | 9.0 (U+11400) | Noto Sans Newa | ⚠️ exists (3 papers, CER 2.6%) | ✅ feasible, not first |
| Tibetan | Sherpa/Tamang(Tamyig)… | 2.0 (U+0F00) | Noto/Jomolhari/BabelStone | ⚠️ mature (BDRC/Namsel/Monlam) | ❌ not novel; Nepal langs mostly print Devanagari |
| Magar Akkha | Magar | ❌ not encoded (2011 proposal shelved) | none | n/a | ❌ blocked — no Unicode target |
Headline reframe: not 1–2 non-Devanagari languages but six first-ever non-Devanagari OCR recognizers are feasible — Limbu (done) + Kirat Rai + Sunuwar + Lepcha + Tirhuta + Gurung Khema — all confirmed zero prior OCR, all Unicode-encoded, all with a usable font. A suite of first-ever OCR for endangered Himalayan scripts. Carve-outs are honest: Tibetan/Urdu are mature elsewhere (out); Magar Akkha is not in Unicode (blocked until encoded); Newa already has OCR.
Synthesis pipeline built (CPU, no GPU):
- Fonts downloaded + verified (
data/fonts/himalayan-scripts/): Noto Sans Sunuwar/ Lepcha/Tirhuta/Newa (from google/fonts), Kanchenjunga 2.001 + Mingzat 1.200 (SIL). cmap coverage == assigned-character counts exactly (Sunuwar 44, Lepcha 74, KiratRai 58, Tirhuta 82, Newa 97); render proofs inoutputs/script-proofs/show clean glyphs, no dotted-circle tofu, via raqm shaping. - Dicts generated from cmap by Unicode category (Python 3.14.2 ships Unicode 16.0.0,
so even the 2024 scripts classify):
data/dictionaries/{kiratrai,sunuwar,lepcha, tirhuta,newa}-v1.txt. - New generic synthesizer
tools/synthesize_script_corpus.py: classifies dict by category (base L* / combining Mn,Mc / digits Nd / punct P*), generates structurally valid syllables (combining mark always follows a base; at most one mark per base), renders newspaper-length lines (~22–46 chars) with raqm + degradation profiles (clean/ scan/low_light/bright_uneven/rotate), writes PaddleOCR labels (img\tlabel). Linear alphabets (Sunuwar, Kirat Rai) use mark-prob 0; abugidas (Lepcha/Tirhuta/Newa/Gurung Khema) attach vowel signs. Train/eval de-duplicated by line text → no leakage. - Shaping validation (bug found + fixed): an early 2-marks-per-base path made raqm
insert U+25CC dotted-circle placeholders for invalid mark stacks — glyphs absent from
the label, corrupting (image, label) pairs. Fix: cap at one mark and a
uharfbuzzShapeCheckershapes every candidate line and rejects any producing a dotted-circle/ .notdef glyph in any render font (shaping_validated=true). All 6 final corpora are provably dotted-circle-free. Per-script lines filtered: Kirat Rai/Sunuwar/Lepcha/ Gurung Khema 0, Tirhuta 426, Newa 6,802 — Newa's combining marks have the strictest valid-cluster rules, so the validator kept only valid clusters (linear scripts Kirat Rai & Sunuwar have no marks, never affected). - Corpora generated:
data/synth-corpora/{kiratrai,sunuwar,lepcha,tirhuta,newa, gurungkhema}/— 6 scripts, 7,000 train + 900 eval images each. Training-ready.
Honest scope: synthetic text is structurally valid but not natural-language (no real corpus exists) — it teaches the glyph→Unicode mapping, which is the core OCR task; the Limbu precedent (synthetic-only train → real CER 0.21) shows this transfers. Real-page validation is the user-sourced next step (Sikkim Herald editions, etc.). GPU training of these is sequential and gated by budget — staged launch-ready, trained as GPU frees after the Devanagari run.
REAL-TEXT REALITY + corrected strategy (2026-06-17, after user pushback)
User correctly flagged: the staged novel-script corpora use generated structurally- valid syllables, not real language — weaker than real prose rendered in the same font (the same lesson Limbu taught: dictionary-words vs newspaper-prose mismatch hurt). Also: these scripts aren't Gorkhapatra-gated — real text/images exist elsewhere. A 5-agent harvest (codepoint-verified, non-Gorkhapatra) found:
| Script | Real Unicode text | Best real validation images |
|---|---|---|
| Newa | 668 lines / 35k cp (nepal-lipi.com) | ✅ Zenodo 6967421 — 683 manuscript pages + Unicode PAGE-XML GT, CC-BY 4.0, 503MB |
| Lepcha | 498 lines / 65k cp (SIL Mingzat folk tale + combos + wiki) | 2 Wikimedia manuscript imgs (no GT); EAP281 (browser-gated) |
| Tirhuta | 53 lines (4 real wiki + 49 Aksharamukha Deva→Tirhuta) | LDCIL 21,412 Tirhuta sentences (paid); Varna Ratnakara MS |
| Sunuwar | 114 lines / 1.8k cp (r12a + GFonts specimen) | Sikkim Herald PDFs — legacy font, no public converter |
| Kirat Rai | 30 lines / 651 cp (SIL sample + r12a) | Sikkim Herald — legacy AKRS; TECkit maps exist (silnrsi/wsresources: Deva2Krai.map, kiratraifontnew.map) |
| Gurung Khema | 0 lines — genuinely none (2024 script) | offline textbooks only |
Harvested text saved under data/real-text/<script>.txt. |
Two paths that mirror the Limbu success (real text → real CER), beating random synth:
- Sikkim Herald legacy conversion = the eastern "Gorkhapatra." Sikkim's govt weekly prints Lepcha/Limbu/Sunuwar/Kirat Rai editions, all legacy-font (AKRS/PUA), convertible to Unicode via TECkit maps (Kirat Rai's exist). Yields real training text AND real validation crops+GT — the proper Limbu playbook. Caveat: Herald PDFs use CID/PUA fonts → extraction needs CIDToGIDMap handling (non-trivial, like the nayanepal work).
- Aksharamukha transliteration of abundant Devanagari corpora → native script at
scale (e.g. Leipzig Newari 36k sentences = Devanagari → transliterate to Prachalit;
Maithili Devanagari → Tirhuta). Gives real-language content at volume. Plumbing: pip
pkg breaks on py3.14 (
ast.Strremoved); use the appspot API or a py3.10 sidecar venv.
Per-script call:
- Newa — best near-term real-validated non-Devanagari target (real text + the Zenodo GT image set). Caveat: Newa OCR already exists (less "first-ever").
- Lepcha / Kirat Rai / Sunuwar — first-ever; pursue via Sikkim Herald conversion for real text + real eval. Scale training text with transliteration where possible.
- Tirhuta — scale via Aksharamukha (Maithili Devanagari is abundant).
- Gurung Khema — genuinely zero real text; synthetic-syllable corpus is the only option (the random-synth approach is correct here, uniquely).
Corrected plan: replace random-synth as the primary with real/transliterated text (keep synth for glyph coverage + augmentation); acquire the Newa Zenodo real-eval set; build the Sikkim-Herald legacy→Unicode converter for the eastern scripts. Training awaits GPU re-auth (SSO expired at last teardown).
SIKKIM HERALD CONVERTERS BUILT (2026-06-17) — real text + real eval, the Limbu playbook
The Sikkim Herald (Sikkim govt weekly) prints Limbu/Kirat Rai/Sunuwar/Lepcha editions in
legacy fonts. Built legacy→Unicode converters + real-eval packs per script (PDFs under
data/sikkim-herald/<script>/, converted text under data/real-text/sikkim-herald/):
| Script | Converter | Real text | Real eval | Open gap |
|---|---|---|---|---|
| Kirat Rai | src/ocrtech/kiratrai_legacy.py (SIL TECkit) |
887 lines | data/kiratrai-real-eval-pack/ (70 clean) + train pkg runs/kiratrai-ppocrv5-mobile/ |
6 bytes unmapped (f…) → 778 lines pending |
| Limbu (2nd src) | reused limbu_legacy.py + Srijonga override map (byte 0x51→U+1922) |
2,269 lines / 46k cp | 2,175 clean | ~solved (0.018% unmapped) |
| Sunuwar | src/ocrtech/sunuwar_legacy.py — font decoded from scratch by glyph-shape identity (no public map existed) |
692 clean lines (88.6%) | buildable | 1 byte | (shape=shyele vs dist=u) → native reader |
| Lepcha | src/ocrtech/lepcha_herald_legacy.py — Herald font decoded from scratch (JG map was wrong font) |
132 clean lines | data/lepcha-real-eval-pack/ (132 crops) |
2 bytes ],% → ~349 lines pending |
| Newa | n/a (Unicode native) | 3,779 lines / 308k cp | data/newa-real-eval-pack/ (3,180 manuscript crops, Zenodo) |
manuscript HTR; OCR exists |
Method note: Sunuwar + Lepcha Herald fonts had no public byte→Unicode map; both were
reverse-engineered by rendering each legacy byte's glyph and matching it to the Unicode
font's glyph (Noto), round-trip-validated against the printed ink. Genuinely-ambiguous
bytes are flagged (CONFIRMED vs UNCERTAIN tiers), never guessed — closing them needs a
native reader or the source keyboard table. New tests: tests/test_{kiratrai,sunuwar, lepcha_herald}_legacy.py (all green).
nepal-ttf2utf extended (public lib): ported Kirat Rai + Sunuwar + Lepcha converters
into /Users/cdjk/github/llm/nepal-ttf2utf/ (2→5 scripts), strict-mode surfaces the
pending bytes, 35 tests pass, honest coverage matrix. Working-tree only — push pending
user sign-off. (Also fixed a pre-existing pyproject.toml force-include build bug.)
Blocked on user: (1) GPU re-auth (aws sso login --profile playaround) to train; (2)
native-reader disambiguation of a few bytes (Sunuwar |, Lepcha ]/%, Kirat Rai 6) to
reach ~100%; (3) sign-off to push nepal-ttf2utf.
RESULT — Limbu/Sirijonga recognizer v1 (2026-06-16)
First-ever Limbu/Sirijonga OCR recognizer trained and evaluated on a real page.
- Model: PP-OCRv5 mobile fine-tune, 16 epochs (~1h, early-stopped on synthetic plateau), trained 100% on synthetic rendered prose (9,209 unique lines).
- Real Gorkhapatra page (303 line crops):
norm_edit_dis= 0.790 (≈ char accuracy 79% / CER ≈ 0.21), exact-line acc = 6.6%. - Held-out synthetic eval:
norm_edit_dis= 0.644 — lower than real because the synthetic eval has longer lines (median 47 vs 39 chars) crushed by the 320px width resize. So the synthetic "plateau" under-reported the model; the real page is the truer measure. - Baseline context: stock PaddleOCR / Tesseract have no Limbu → ~0 on Sirijonga. This is from-zero on a brand-new script.
- Preserved:
outputs/limbu-sirijonga-recognizer-v1-2026-06-16/limbu-v1-model.tar.gz(best checkpoint + exported inference model + recipe). GPU instance stopped.
Honesty scope (v1): preliminary — real-eval labels are machine-converted + human-verification-pending; single page (N=1); recognizer-only with oracle (PDF) line boxes; single font; clean-digital. Proves the script is learnable and the pipeline works end-to-end; not yet a paper-grade claim.
v2 levers (clear next gains): widen rec_image_shape beyond 320px (fixes long-line
squish — biggest lever per the synthetic-vs-real gap); human sign-off → claim-grade
labels; more Gorkhapatra editions (beat N=1); add degradation/multi-font for robustness;
run stock + PaddleOCR-VL baselines to quantify the "stock fails" bar.
RESULT — Devanagari recognizer v1, 8 languages (2026-06-17)
One PP-OCRv5 mobile Devanagari recognizer evaluated on 8 real Gorkhapatra
language pages (Unicode ground truth via nepal-ttf2utf). Trained 100% on
synthetic (Noto Serif Devanagari renders of 35,937 eval-matched prose lines),
warm-started from devanagari_PP-OCRv5. Early-stopped at epoch 9 by the
wall-clock budget cap (still improving — synthetic norm_edit_dis 0.737 and rising).
| Language (Devanagari page) | lines | norm_edit_dis | CER ≈ |
|---|---|---|---|
| Magahi | 425 | 0.655 | 0.35 |
| Bhojpuri | 362 | 0.626 | 0.37 |
| Magar | 383 | 0.622 | 0.38 |
| Sunuwar (Devanagari) | 444 | 0.607 | 0.39 |
| Darai | 427 | 0.606 | 0.39 |
| Majhi | 393 | 0.575 | 0.43 |
| Thami | 396 | 0.534 | 0.47 |
| Wambule | 383 | 0.522 | 0.48 |
| mean | 3,213 | ≈0.593 | ≈0.41 |
Artifact: outputs/devanagari-recognizer-v1-2026-06-17/deva-model.tgz (best
checkpoint + recipe + dict + eval results).
Honest caveats: epoch 9 only (budget-capped mid-improvement — more epochs would
lift this materially); train/eval font gap (Noto Serif Devanagari vs the
nayanepal newspaper face) — unlike font-matched Limbu (0.21), so these are
conservative; machine-converted labels (human-verification-pending); single page per
language. These are preliminary real-page numbers, not paper-grade.
Budget/ops note (lesson): the cap-watcher stopped training cleanly at 00:07 UTC
(DEADLINE-STOP), but defunct train.py processes lingered in pgrep, so the local
poller never detected the stop and the GPU sat idle ~73 min before I returned —
budget waste. Fix next time: poll GPU utilization or the deva_train_stopped.sentinel,
not just pgrep. Also: the staged run_real_eval.sh first failed silently (python
not on the non-interactive PATH; the venv is /home/ubuntu/ocr-venv/bin/python) — the
in-script grep swallowed the error. Fixed to use the venv interpreter + capture logs.
AWS SSO expired at teardown so the instance was stopped via OS shutdown -h now
(default behavior = stop, EBS preserved); user should confirm instance state =
stopped (not running/terminated) on next aws sso login --profile playaround.
REAL same-font Devanagari training corpus (2026-06-18) — closes the font gap
The font gap above is the single biggest lever on the 0.35–0.48 CER, and we now
have real same-font training data to close it. tools/build_gorkhapatra_devanagari_real.py
harvests the nayanepal newspaper face (the exact eval face) straight from the
locally-downloaded Gorkhapatra epaper PDFs and converts it to Unicode via
nepal_ttf2utf.convert_devanagari(font="nayanepal") (Preeti base map + ƒ/† ext +
NFC), the same ~97–99% converter behind the eval GT.
- Output:
data/gorkhapatra-devanagari-real/—images/+labels.txt(PaddleOCR recognition format),summary.json. - Scope reality: the task brief said "636 editions"; only 24 epaper PDFs are
physically on disk (
_listings/probe_results.jsoncatalogues ~432 remote, but only 24 were downloaded — 492 MB matches the 24, not 636). Harvested those 24. - Design call:
nayanepalis the body face of the whole paper, not just the rotating "Naya Nepal" language page (verified per-page; the eval pages use it too). So we harvest everynayanepalspan on every page and only SKIP each edition's nativeNamdhinggoSILLLimbu page (already harvested, not Devanagari). One span = one printed line; cropped tight at 288 dpi from the span bbox. - Result: 79,364 unique real same-font Devanagari line crops, 2,813,427 Devanagari
chars (median 42 chars/line), from 114,434 nayanepal spans seen. De-duplicated by
cosmetic-normalized text key across all editions (21,731 dups dropped). Eval
leakage excluded against ALL 8 Devanagari eval packs — 6 in
outputs/multilang-eval/*/manifests/(Thami, Bhojpuri, Majhi, Sunuwar, Magahi, Darai) plus Magar + Wambule inoutputs/{magar,wambule}-real-eval-pack/manifests/(these two are NOT under multilang-eval — an easy-to-miss split; the loader globs*-real-eval-packroots too, also harmlessly catching Lepcha/Urdu non-Deva packs). 3,707 leakage keys; 155 lines dropped; final corpus has 0 eval overlap (re-verified against the full 3,707-key set). Leakage matching uses a normalized key (NFC + whitespace-collapse + smart-punct fold) because the eval manifests keep U+202F etc. that the converter folds — raw exact-match would miss most overlaps. - Conversion quality (honest): 5.9% span conversion-gap rate — spans whose
converted text still carries a residual non-Devanagari glyph (top offenders
‚U+201A,m,ì,Þ,Œ— known nayanepal extension bytes the Preeti base map doesn't cover) are DROPPED, not kept dirty. Kept lines are 100% Devanagari + allow-set (digits/danda/punct), enforced on the final Unicode like the Limbu pack. - Known GT artifact (preserved on purpose): ~18% of lines carry an intra-word kerning
space (e.g.
नार ासाथforनारासाथ). These spaces are in the source PDF text layer (the Gorkhapatra typesetter inserts them) and are also present in the eval-pack GT — so keeping them keeps train/eval GT conventions identical. Residual NBSP/narrow-NBSP are folded to ASCII space; true mid-word spaces are left as-is. - Per-language attribution: NOT recoverable per line — this is a pooled real Devanagari corpus (Nepali general news + rotating minority-language pages), which is exactly what the shared Devanagari recognizer wants. Fine for retraining.
- Next: warm-start the Devanagari recognizer on this real corpus (LOW LR ~2e-5, moderate aug per the baked-in lessons), eval on the unchanged 8 held-out packs. Expectation: meaningful CER drop from closing the Noto-Serif→nayanepal font gap.
Tally — languages with real-page recognition (2026-06-17)
9 languages with real Gorkhapatra-page CER: Limbu/Sirijonga (own recognizer, native Sirijonga script, CER 0.21) + 8 Devanagari (one shared recognizer, CER 0.35–0.48). Plus 6 first-ever non-Devanagari native-script recognizers staged training-ready (Limbu done; Kirat Rai + Sunuwar launch-ready on the now-stopped GPU; Lepcha + Tirhuta + Gurung Khema corpora local). Next GPU session: train Kirat Rai / Sunuwar (recipes + corpora already staged) → first-ever own-script CER; resume Devanagari past epoch 9 to lift the 8-language numbers.
Triage matrix (Phase 1 result)
| Language | Gorkhapatra page | Text-layer form | Font | Convertible→Unicode | Native-script? | Status |
|---|---|---|---|---|---|---|
| Limbu/Sirijonga | 2083-02-12 p6 | legacy_font | NamdhinggoSILL | yes (limbu_legacy) |
yes (Sirijonga) | converter exists, page disambiguated → first target |
| Sanskrit | 2083-03-01 p6/p7 | legacy_font | nayanepal (~85–89% ASCII) | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Magar) |
| Magar | 2083-03-01 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Sanskrit) |
| Wambule | 2083-02-31 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Rana Tharu) |
| Rana Tharu | 2083-02-31 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Wambule) |
Strategic implications from triage
- The whole Gorkhapatra epaper text layer is legacy-font, not Unicode. Real
references for every language must go through legacy→Unicode conversion. The
Limbu/Namdhinggo path is the template; Devanagari pages need a
nayanepal→Unicode converter (a Preeti-family legacy map — do not assume one exists; verify the actual glyph map, or cross-reference the article web text which is served in Unicode atgorkhapatraonline.com/news/<id>). - Live-pack language alignment is unreliable — 2 candidate languages resolve to the same physical page. Disambiguation (which language is actually on the page) is required before any Devanagari-language real eval. The Naya Nepal card → epaper edition mapping picked overlapping editions.
- Limbu is the only clean first eval: native script, existing converter,
already-disambiguated page (p6 Limbu vs p7 non-Limbu). Phase 2 builds on it and
the capability generalizes to
nayanepalafterward.
Log
- 2026-06-16: Goal set. Mapped real state (training infra complete but only the dead mixed-PP-OCRv3 family trained; zero verified real references; only Limbu has native-script evidence). Wrote this checklist. Starting Phase 1 triage.
- 2026-06-16: Phase 1 triage done. Ran PDF-native extractor on the live-pack
(8 pages). Finding: all Gorkhapatra epaper text is legacy-font (Namdhinggo for
Limbu,
nayanepal~85–89% ASCII for Devanagari pages) — no Unicode layer anywhere. Live-pack alignment is ambiguous (Sanskrit≡Magar page, Wambule≡Rana Tharu page). Confirmed no trained recognizer checkpoints exist. Limbu confirmed as first target. Next: Phase 2 — wire Limbu PDF-native legacy→Unicode into the verified-reference path (pre-fill + human sign-off), avoiding 298 from-scratch transcriptions. - 2026-06-16: Phase 2 tooling shipped. Added
prepare-gorkhapatra-language-page-pdf-native-references(legacy→Unicode draft with image-pixel bboxes + unmapped-codepoint flags) and a generic per-linereview_notepass-through in the verification bundle. Ran end-to-end on real Limbu data: 303 pre-filled spans, bundle built (303 crops). Tests added and green; cli + source_audit + limbu_legacy suites pass;git diff --checkclean. Remaining for the first real eval: human sign-off pass (light accept, not transcription) →apply→ claim-eligible verified Limbu reference. Then Phase 3 baselines + Phase 4 fine-tune. - 2026-06-16: Phase 3 backbone shipped. Added
prepare-gorkhapatra-language-page-recognizer-eval(verified reference → tight line crops + recognizer eval manifest). Smoke on Limbu draft: 303 crops, passesaudit-manifest. Tests added;source_audit/limbu_legacy/clisuites green;git diff --checkclean. User confirms an AWS G6 host is available for Phase 4, but the localawsSSO token is expired —aws sso loginneeded before I can query or launch it. Next unblocked work: Phase 4 launch-readiness (rendered split + recipe + documented GPU launch command), pending AWS re-auth and the Limbu sign-off for the real eval. - 2026-06-16: AWS re-authed (playaround profile). Found host
ocrtech-g6-training-20260615-v4(i-0fa19dff7b233a353, g6.xlarge, us-east-1a); start hit transientInsufficientInstanceCapacity, a background poller acquired it, then I stopped it (no idle cost) per "research first, don't train on assumption". - 2026-06-16: Ran a 4-agent research fan-out on the correct PaddleOCR recognizer
approach. Overturned the from-scratch PP-OCRv3 assumption → validated approach is
fine-tune PP-OCRv5 mobile (devanagari multilingual config) from pretrained, reuse
backbone, reset head for the Limbu dict (see "Validated training approach" above).
Harness is 2–3 gens stale; needs a
training.pyPP-OCRv5 recipe update + a fuller ~8,800-line rendered corpus before the GPU run. Instance stays stopped until ready. - 2026-06-22: For the current Sirijanga Gorkhapatra page, the 31-row geometry/text
missed-line worklist has an oracle-reference ceiling. It improves line coverage
when inserted with perfect text (line F1
0.933566, misses 36), but append/geometry ordering damages page OCR (0.771993codepoint CER with preserve/append,0.771825with geometry, versus selectedconf070at0.739361).candidate_document_orderis now the best diagnostic slotting policy: codepoint CER0.739024, line F10.933566, misses 36, extras 2, reading-order accuracy0.775506. It still fails admission because grapheme CER worsens to0.778367. The next blocker is grapheme/text normalization on top of candidate-document-order slotting, not just transcription. The non-oracle machine-text version under the same order mode still fails at0.763078codepoint CER and0.797416grapheme CER, so the machine candidate text remains rejected. - 2026-06-22: The 31-row Sirijonga oracle ceiling was ablated by span group and single
row; no subset improves both codepoint and grapheme page CER under
candidate-document order. A diagnostic oracle reference-order projection over the
same rows does pass strongly (
0.102355codepoint CER,0.136823grapheme CER, line F10.933566, 0 effect-audit issues). This proves the remaining Gorkhapatra Sirijonga blocker is deployable page reading order/layout assembly, not detector recall or line recognizer text in isolation. - 2026-06-22: Added
tools/analyze_page_reading_order_geometry.pyand profiled the oracle reference-order ceiling. The page decomposes into 21 reference-order geometry chunks and 20 large jumps, so the next deployable reading-order policy must reason about article/block regions, not only columns. - 2026-06-22: Added
tools/apply_page_reading_order_geometry_profile.pyand tested the 21-chunk oracle-derived profile as a page-ordering mechanism. On the 31-row oracle-text projection it reaches0.095711codepoint CER and0.129700grapheme CER with line F10.933566; on the non-oracle machine-text projection it reaches0.143146codepoint CER,0.198112grapheme CER, line F10.933566, recall0.881188, misses 36, extras 2, and reading-order accuracy1.0, with the effect audit safe. This is not deployable evidence because the profile is derived from hidden eval alignment; it proves the next two-day goal should be non-oracle layout profile calibration, not more detector or recognizer churn. - 2026-06-22: Added
tools/build_page_reading_order_geometry_profile.py, the non-oracle bridge from geometry-only line-box review to the profile applier. It rejects text-bearing geometry references, blocks pending packets unless explicitly allowed, and blocks mixed multi-source profiles unless explicitly allowed. Applying the current Sirijonga line-box packet as pending seed geometry gives 2 samples and 389 line boxes; the separate Gorkhapatra pending seed profile has 303 lines, 26 chunks, andprofile_claim_safe=false. Applied to the 31-row machine-text candidate projection, that Gorkhapatra seed profile reproduces the strong profile-order score exactly: codepoint CER0.143146, grapheme CER0.198112, line F10.933566, recall0.881188, misses 36, extras 2, reading-order accuracy1.0, metrics SHA-2568181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189. This changes the next task from "invent a layout profile path" to "complete/review the geometry order packet and rebuild this profile without--allow-pending." - 2026-06-22: Added
tools/audit_page_reading_order_geometry_profile.pyand the--require-profile-claim-safegate ontools/apply_page_reading_order_geometry_profile.py. The current Gorkhapatra pending seed profile passes structural audit but hasoperational_profile_safe=false; the required-safe audit fails, and the applier refuses it under the new flag. Structural audit / required-safe audit SHA-256:9aecaa40b5ee509ad12e495777a857a76fea7d4ad68793e9055e721a163b8877/694486b5a3f28bcdb3093c085814fdb76daf835be5599f26319daca80b70b3ce. This keeps the strong seed-profile diagnostic from being accidentally promoted before geometry review is complete. - 2026-06-22: Added
tools/admit_page_reading_order_profile_application.py, the promotion gate that binds profile audit, profile application, and page-effect audit. On the real pending Gorkhapatra seed-profile projection it rejects admission withadmitted_operational=falseeven thoughbook_ocr_objective_safe=true, because the profile provenance is pending/non-claim-safe. Admission SHA-256:02bcd20dc1a5598f92b9a251dd5ae4669171ba8d9429df4eb9c06cf33bc354b6. The operational promotion sequence is now explicit: reviewed geometry profile -> required-safe profile audit -> required-safe profile application -> page score/effect audit -> profile-application admission. - 2026-06-22: Extended
tools/apply_page_reading_order_geometry_profile.pywith--profile-mapfor multi-page/profile-per-template routing, and extendedtools/admit_page_reading_order_profile_application.pyto accept multiple--profile-auditinputs. The current pending seed map (7e94909dacbe9be20d18a70ddbac5fcc0421b0c2a35c2055482b1f2859895397) maps Gorkhapatra and Sikkim to separate profiles; applying it to the Gorkhapatra 31-row machine-text projection reproduces metrics SHA-2568181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189. - 2026-06-22: Added
tools/audit_page_reading_order_profile_map.pyand bound it into profile-application admission. The current pending seed map passes structural audit (1090ec5254bad3e5d0ec492739488de5a65d684673d33cb36d50f1dde8c7d7d7) but fails required-safe audit (bb4c07c4fcb7f2031ee085313d3a537eda01650b77f3af31ce23d605af136d06) because both mapped profiles are pending. Map-driven admission now rejects the Gorkhapatra projection with SHA-2568764a46251c9e0bc1b2415de9089f5eb67032e9bf9eccf11167f54da8c18aaf6. This closes the obvious multi-page footgun: do not use one layout profile across a book unless a profile map or template selector chooses it per page, and do not promote a map-driven run without a safe profile-map audit. - 2026-06-22: Integrated reading-order profile maps into
tools/parse_limbu_book_pages.py. The book wrapper now applies--reading-order-profile-mapafter optional line assembly and before aggregate Markdown/review queues/score-documents/are written. A sidecar test proves the profile map changes bothbook.mdand the score projection consistently. The wrapper also hash-binds supplied profile-map audits to the exact map being applied and refuses--require-reading-order-profile-map-safewithout an audit, so stale map-audit artifacts cannot silently authorize a book run.tools/audit_limbu_book_output.pynow verifies the profile-map/application report, assignments JSONL, optional map audit, and per-page profile source-document hashes. This moves the layout-profile work from standalone diagnostics into the actual multi-page book OCR wrapper and its normal bundle audit. - 2026-06-23: Re-scoped the work after user pushback: stop treating eval harness work
as the main deliverable. The next target is real Limbu book OCR throughput and review,
with only thin guardrail audits. Re-ran the local source inventory at
outputs/limbu-page-source-inventory-2026-06-23-realbook-pivot/: 36 records, 3 real Devanagari-written Limbu CDC page candidates, 3 Devanagari text-only sources, 1 synthetic smoke pack, andready=True. The 4TB host atcdjk@100.110.148.43rejected SSH auth in this session, but the mirrored CDC PDFs are present locally. Existing CDC page draft output has 3 real rendered CDC pages with unreviewed curriculum references; the current multi-page CDC book audit is structurally clean (page_count=3,line_count=35,review_line_count=27) but remains a book OCR artifact audit, not an OCR-quality claim. Next work should be a real-page OCR/review loop and reviewed-only training export, not more generic scoring infrastructure. - 2026-06-23: Added operational PDF-book intake to
tools/build_limbu_book_page_manifest.py. The tool now accepts--pdfas an alternative to--image-dir, renders selected PDF pages to PNG, and writes the same page manifest shape consumed by capture prep and book OCR, with source PDF path/hash, page number/index, rendered image hash, and render DPI recorded per row. Real smoke ondata/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/mero_ganit_grade1_limbu.pdfrendered pages 8..10 at 220 DPI tooutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-pages.jsonl(pages=3, no source-attention flags, 1870x2420 PNGs). That manifest then passed throughtools/prepare_limbu_book_captures.pywith 3 pages and 0 failed capture audits, producingoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/capture-prep/prepared-page-manifest.jsonl. This is the next real-book front door for CDC PDF pages: feed the prepared manifest intotools/parse_limbu_book_pages.pyand inspect layout/review packets. - 2026-06-23: Ran that CDC PDF front door through the full operational book wrapper
using the current v4 router, with recognizers held fixed:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-square-nontext-guidance/. It emits editable book Markdown, page layout overlays, page triage, layout-gap review, and a passing book audit over 3 real rendered CDC pages. Counts: 93 preserved lines, 21 book-line review rows, 3 page-layout review rows, 3 page-triage rows, 9 suspicious layout-gap rows, andbook_audit_passed=True. Visual overlay inspection showed most high-priority empty detections were square page furniture, especially 60x61 registration boxes, not missed body text. Updatedtools/parse_limbu_book_pages.pyso square empty detections are guided asinspect_non_text_or_drop_empty_detectioninstead of generic transcription work. On the real rerun, action counts became 9 non-text inspect/drop, 1 transcribe/drop empty detection, and 11 low-confidence spotchecks, with audit still passing. Remaining practical blocker on this artifact is the 9 layout-gap rows (7possible-text ink,2sparse ink), which need review as missed text vs page graphics before applying queues or exporting reviewed training rows. - 2026-06-23: Inspected those CDC layout-gap rows against real crop/context evidence
instead of adding more eval machinery. The only remaining high-priority gap,
gap-0001-cdc-maths-p0008-003, was a wide dense box-border/illustration fragment, not missed Limbu text. Updatedtools/run_limbu_book_ocr.pyso wide low-density border/graphic bands are classified aspage_graphic_or_illustrationwhile the existing compact missed-line proxy still stayscandidate_text_band. Real rerun:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-gap-graphic-border-guidance/. Counts stayed stable at 3 pages and 93 extracted lines; wrapper book-line review rows stayed 21 (review_line_count=22in the parse summary), layout-gap rows stayed 9, andbook_audit_passed=True. The gap queue is now 6 page-graphic/illustration rows and 3 rule/spacing rows, with 0 candidate-text gap rows; page triage accepts all 3 pages andoperator_attention_pages=0. This is a review-noise reduction on real CDC pages, not an OCR-quality claim. Next useful work is to apply/resolve the remaining line-review queue and export reviewed-only training rows, not to expand the eval harness. - 2026-06-23: Inspected the 21-row CDC book-line review queue via a crop contact sheet.
Three rows that were previously low-confidence text spotchecks were actually 60x60
page-corner registration marks with OCR hallucination text (
े?). Updatedtools/parse_limbu_book_pages.pyso low-confidence rows with the same square non-text geometry are guided asinspect_non_text_or_drop_low_confidence_detection, while text-shaped low-confidence rows remainspotcheck_low_confidence_text. Real rerun:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/. Counts stayed stable at 3 pages, 93 extracted lines, 21 book-line review rows, 9 layout-gap rows,operator_attention_pages=0, andbook_audit_passed=True; standalone audit also passed atoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-output-audit-lowconf-nontext-guidance/. The current line-review action counts are 9inspect_non_text_or_drop_empty_detection, 3inspect_non_text_or_drop_low_confidence_detection, 1transcribe_or_drop_empty_detection, and 8spotcheck_low_confidence_text. The contact sheet lives atoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png. This is the current CDC review packet; next work is human review/application/export, not more harness expansion. - 2026-06-23: Turned the obvious non-text rows from that packet into an auditable
cleaned projection, without pretending they are training labels. The proposal queue
at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops.tsvmarks 12 rows asdrop: 9 empty non-text detections and 3 low-confidence non-text registration-corner hallucinations. It leaves 9 rows pending: the one transcribe/drop row and 8 low-confidence text/graphic spotchecks. Applying it withtools/apply_limbu_book_line_reviews.pyproducedoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/withapplied_count=12,dropped_count=12,skipped_count=9,line_count=69, and a passing standalone book-output audit. The apply tool also had a real chainable output bug fixed: reviewed projections now write their ownoutput_dirinbook-run.json, sotools/audit_limbu_book_output.pyno longer fails on stale source-output provenance. Running the reviewed-only training exporter on the cleaned projection still correctly fails withno training rows were exported, because no human accepted/corrected text labels exist yet. - 2026-06-23: Prepared the human review station from the cleaned CDC
projection. Command:
tools/prepare_limbu_book_unresolved_review_packet.py --book-output-dir .../review-applied-obvious-nontext-drops --book-line-review-queue .../manual-review-proposals/book-line-review-obvious-nontext-drops.tsv. Packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/pending-text-review-packet/. HTML review station:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/pending-text-review-packet/book-unresolved-review-packet.html. The packet audit passes with 0 issues. Counts: 9 existing-line rows, 12 non-pending source rows skipped, 9 crops, 9 page contexts, and 8 prefilled text rows. Review guidance is now preserved in the packet as 8spotcheck_low_confidence_textrows and 1transcribe_or_drop_empty_detectionrow;tools/prepare_limbu_book_unresolved_review_packet.pynow maps book-linereview_priority/review_recommended_actioninto the packet'sreview_category/suggested_review_actionfields. The packet TSV/export now also preservespage_context_path,page_context_sha256, andneighbor_context, instead of keeping that evidence only inside the HTML. After the HTML is reviewed and exported asbook-unresolved-review-packet.reviewed.tsv, run the existingtools/apply_limbu_book_reviewed_packet_to_training_pack.pywrapper against the cleaned projection. - 2026-06-23: Inspected the 9-row packet visually with crop plus page context. One
remaining blank
transcribe_or_drop_empty_detectionrow (img-line-0013) was objectively another wavy illustration/rope segment, not visible text. Created v2 proposal queueoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsvand applied it tooutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/. Counts:applied_count=13,dropped_count=13,skipped_count=8,line_count=68, standalone book-output audit passed, and reviewed-only export still correctly refuses withno training rows were exported. The current review packet is nowoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/; it audits cleanly with 8 pending rows, 8 crops, 8 page contexts, allspotcheck_low_confidence_text. - 2026-06-23: Removed another operational bottleneck instead of expanding eval
machinery:
tools/run_limbu_book_ocr.pynow accepts--pdfdirectly. The underlying manifest builder still prefers PyMuPDF, but now falls back to Poppler (pdfinfo+pdftoppm) when running inside the Paddle Python 3.11 environment wherefitzis unavailable. The run summary records the source kind, source PDF hash, PDF page window, DPI, andpdf_renderer. Focused tests cover image-dir behavior, PyMuPDF rendering, and the Poppler fallback. - 2026-06-23: Used that PDF front door to create the next larger real CDC review
target:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/. Command ran the Maths Limbu PDF pages 11..15 at 220 DPI through the current v4 router with recognizers held fixed. It rendered 5 pages with Poppler, preserved 189 OCR lines, wrote 58 pending book-line review rows, wrote 15 layout-gap rows, accepted all 5 pages in page triage, and passed the book-output audit. Review action counts are 22inspect_non_text_or_drop_empty_detection, 17inspect_non_text_or_drop_low_confidence_detection, 3transcribe_or_drop_empty_detection, 2verify_non_limbu_or_drop, and 14spotcheck_low_confidence_text; gap actions are 6 page graphic/illustration and 9 rule/spacing rows. Prepared unresolved packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pending-text-review-packet/. Packet audit passes with 0 issues. It has 58 pending rows, 58 crops, 58 page contexts, 58 neighbor contexts, and 33 prefilled OCR text rows. This is a better label production target than the earlier 8-row smoke, but still no training rows exist until a non-machine reviewer accepts/corrects text or drops non-text rows. - 2026-06-23: Reduced that larger packet's review burden with one conservative visual
drop pass, still without minting labels. Built a crop+page-context contact sheet at
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pending-text-review-packet/book-unresolved-review-contact-sheet.jpg. The only automatic proposal marks repeated 60px page-corner registration marks/page furniture asdrop, including the top-rightे?hallucinations; text-shaped labels and ambiguous exercise graphics remain pending. Proposal queue:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-corner-registration-drops-v1.tsv. Applying it producedoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/with 20 dropped rows and 38 skipped/pending rows. The cleaned book-output audit passes with 0 issues, and reviewed-only training export still correctly refuses withno training rows were exported. Current cleaned packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/pending-text-review-packet/. Packet audit passes with 0 issues. It has 38 pending rows, all with crop, page-context path/hash, and neighbor context; 28 rows have prefilled OCR text. Remaining actions: 7inspect_non_text_or_drop_empty_detection, 12inspect_non_text_or_drop_low_confidence_detection, 3transcribe_or_drop_empty_detection, 2verify_non_limbu_or_drop, and 14spotcheck_low_confidence_text. - 2026-06-23: Made a second conservative visual drop pass on the 38-row packet after
generating a larger remaining-only sheet:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/pending-text-review-packet/book-unresolved-review-remaining-contact-sheet.jpg. The v2 proposal keeps all text-shaped/glyph rows pending and drops only 13 additional obvious non-text objects: arrows, blank answer rectangles, and standalone illustrations. Proposal queue:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv. Applying v2 from the original 58-row book output producedoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/with 33 dropped rows and 25 skipped/pending rows. The cleaned book-output audit passes with 0 issues, and reviewed-only training export still correctly refuses withno training rows were exported. Current cleaned packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/. Packet audit passes with 0 issues. It has 25 pending rows, all with crop, page-context path/hash, neighbor context, and prefilled OCR text. Remaining actions: 13spotcheck_low_confidence_text, 11inspect_non_text_or_drop_low_confidence_detection, and 1verify_non_limbu_or_drop. - 2026-06-23: Added a small reviewer-context bridge instead of expanding the eval
harness.
tools/build_limbu_pdf_text_context.pyextractspdftotext -bbox-layoutlines from legacy-font Limbu PDFs, uses word x-gaps to rejoin tiny legacy glyph fragments before conversion, converts throughsrc/ocrtech/limbu_legacy.py, and writes JSONL/TSV/Markdown plus per-page raw/unicode text sidecars. It marks every row aspdf_text_layer_legacy_converted_review_context_not_gold,is_gold_text=false, andtraining_use=forbidden_without_human_alignment_review. Running it on CDC Maths PDF pages 11..15 with the rendered page manifest producedoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/: 82 text-layer lines, 81 with converted Limbu codepoints, 0 unmapped-codepoint lines, and rendered bboxes mapped to the samecdc-maths-p0011..cdc-maths-p0015page IDs. This is useful for human review of the current 25-row unresolved packet, but it is not an auto-label source. - 2026-06-23: Connected that context directly to the current review station.
tools/prepare_limbu_book_unresolved_review_packet.pynow accepts--pdf-text-context-jsonland attaches up to 3 nearest PDF text-layer rows to each packet row using page ID plus rendered bbox y/overlap geometry. The packet TSV keeps advisory columns (pdf_text_context_*) and the HTML shows a "PDF text-layer context" panel with a warning that it is not gold and cannot be used as a label without human alignment. Regenerated assisted packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-review-packet-with-pdf-context/. Counts: 25 packet rows, 25 matched PDF-context rows, 0 unmatched rows, 82 source context rows, 3 context rows max per packet row. Packet audit passes with 0 issues. - 2026-06-23: Added a source-text coverage check for missed-line recovery.
tools/build_limbu_pdf_text_line_coverage.pycompares converted PDF text-layer line boxes with the cleaned OCR document line boxes and emits review-only candidate rows when a source text line has weak y coverage or weak x coverage from current OCR boxes. On the same five CDC Maths pages, conservative thresholds (min_y_overlap_ratio=0.40,min_x_coverage_ratio=0.20) producedoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pdf-text-line-coverage/: 82 source PDF text lines, 68 covered, 1 no-Limbu/no-text skipped, and 13 candidate rows. Candidate categories: 8pdf_text_line_missing_y_coverageand 5pdf_text_line_partial_x_coverage, mostly isolated Limbu numerals/page-structure fragments. The derived missing-line review queue at.../pdf-text-missing-line-review/has 13 rows, 13 page contexts, and audits cleanly with 0 issues. - 2026-06-23: Unified the existing 25 unresolved text rows and the 13 source-text
coverage candidates into the first combined review packet:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context/. It has 38 pending rows: 25existing_book_line, 13candidate_missing_line, 38 PDF-context matches, 0 unmatched rows, and a passing packet audit with 0 issues. This packet is still not training data until a human reviewer accepts/corrects rows; it is superseded by the strict no-prefill packet below as the current review target. - 2026-06-23: Hardened the current review station against accidental label promotion.
tools/prepare_limbu_book_unresolved_review_packet.pyno longer pre-fills pendingcorrected_textfrom OCR/candidate text by default;--prefill-corrected-textis now an explicit opt-in. The HTML still has a deliberateAccept textaction that copies OCR text intocorrected_textbefore marking a row accepted, but a reviewer must take that action or type a correction. Regenerated strict current packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context-strict-v2/. It has 38 pending rows, 25existing_book_line, 13candidate_missing_line, 38 PDF-context matches, 0 unmatched rows, 0 pending prefilled corrected-text rows, and a passing packet audit with 0 issues. This strict packet supersedes the earlier prefilled combined packet as the current review target. - 2026-06-23: Made the no-prefill policy an audit invariant.
tools/audit_limbu_book_unresolved_review_packet.pynow counts pending rows with nonblankcorrected_text, compares that count against the packet report, and fails strict original-packet audits whenpending_corrected_text_prefill_enabled=falsebut pending corrected text is present. The explicit--prefill-corrected-textpacket path still audits cleanly when its report declares the prefill. Re-auditing the real strict packet above still passes: 38 rows, 38 pending, 0 accepted/dropped, andpending_prefilled_corrected_text_count=0. - 2026-06-23: Hardened the PDF text-layer context boundary in the same packet audit.
tools/audit_limbu_book_unresolved_review_packet.pynow validates that packet-levelpdf_text_context_training_usestaysforbidden_without_human_alignment_review, each embedded PDF context row keepscontext_status=pdf_text_layer_legacy_converted_review_context_not_gold, each embedded context row keeps training use forbidden, and no embedded context row is marked as gold. It also compares attached/unmatched PDF-context counts against the packet report. Re-auditing the real strict packet still passes with 38 PDF-context matches and 0 unmatched rows. This keeps PDF text useful for review while preventing it from silently becoming training truth. - 2026-06-23: Removed the remaining candidate-text fallback from missing-line
application.
tools/apply_candidate_line_reviews.pynow requires every accepted candidate insertion to carry explicit nonblankcorrected_text; it no longer insertscandidate_textwhen reviewed text is blank.tools/split_limbu_book_unresolved_review_packet.pynow rejects acceptedcandidate_missing_linepacket rows without explicitcorrected_textinstead of warning that the applicator will use candidate text. Focused tests cover direct candidate JSONL/TSV application, unified-packet splitting, reviewed-packet application, and reviewed training handoff. The real strict packet remains audit-clean because all rows are still pending and have blankcorrected_text. - 2026-06-23: Closed the analogous existing-line split fallback. The unified packet
splitter no longer fills accepted existing-line
corrected_textfrom OCR text; an accepted existing line now needs explicitcorrected_textor explicitsplit_corrected_textsbefore it can be passed totools/apply_limbu_book_line_reviews.py. This keeps "accept current OCR" as an explicit reviewer action in the HTML export rather than a status-only TSV edit. Focused review-chain tests pass with 42 tests, and the real strict CDC packet still audits with 0 issues. - 2026-06-23: Hardened the downstream candidate-application audit for old or tampered
projections.
tools/audit_candidate_line_review_application.pynow inspects insertedcandidate_line_review_insertionmetadata for the current review file and fails when reviewedcorrected_textis missing,inserted_textis missing, or the stored line text no longer matches the reviewed/inserted text after normalization. Focused review-chain tests now pass with 43 tests, and the real strict CDC packet still audits with 0 issues. - 2026-06-23: Pivoted from eval-harness hardening to actual recognizer training
input. Added
tools/export_limbu_pdf_text_training_pack.py, which takes converted PDF text-layer context rows and crops the corresponding full rendered PDF line from the page image. This avoids training on current OCR guesses and avoids assigning a full PDF line label to a partial OCR word crop. The export is explicitly silver, not gold: rows carrygt_status=pdf_text_layer_silver,claim_eligible=false, and the summary says it is not human-reviewed ground truth or CER/WER claim evidence. Focused tests cover crop geometry, Paddle label emission, non-Limbu skipping, and silver provenance. - 2026-06-23: Materialized the first real Limbu PDF-text training pack from CDC Maths
pages 11..15:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/. It exports 81 labeled rendered-line crops from 82 PDF text-layer rows, skipping 1 no-Limbu row. Counts: 64sirijanga_limbu, 17mixed_sirijanga_devanagari, 1509 Sirijonga codepoints, 18 Devanagari codepoints, and 55 dictionary entries. The pack manifest and label file both have 81 rows. Becauseocrtech train-recognizerneeds image paths resolvable from repo root, a root-path manifest was written at.../manifests/train-root-paths.jsonl. - 2026-06-23: Split the silver PDF-text pack for training. The first attempt grouped
only by
sample_idand failed leakage because duplicate crop/text signatures crossed train/dev; that was the correct failure. Re-splitting with--group-by sample_signature --stratify-by slices --eval-ratio 0.15produced a clean split under.../splits/: 69 train rows, 12 dev rows,leakage_passed=True. Generated a PP-OCRv5 mobile recognizer recipe atruns/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/paddle_recognizer.ymlusing the pack dictionary, 20 epochs, batch size 8, and learning rate0.00005. - 2026-06-23: Started training in the only defensible way available now: a one-epoch
local CPU smoke on the silver PDF-text pack before any longer run. Command used
/tmp/ocrtech-paddle311/bin/pythonand/tmp/PaddleOCR-v3.7.0/tools/train.pywithGlobal.use_gpu=False,Global.epoch_num=1,Train.loader.drop_last=False, andnum_workers=0. It completed, loaded PP-OCRv5 pretrained weights, reported 9 train iters and 2 dev iters, and savedbest_accuracy,latest, anditer_epoch_1checkpoints underruns/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/smoke-checkpoints/. The final log line records devnorm_edit_dis=0.06734576621095845andacc=0.0. Treat this as a training-path proof, not a quality claim. Next work is a full longer/GPU run, export/package, router integration, and frozen-page review-burden comparison. - 2026-06-23: Completed the generic PP-OCRv5 silver run and rejected it as the useful
candidate. The run lives at
runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/, trained for 20 local CPU epochs, exported and packaged cleanly, and passed model audit. The final dev metric was weak (norm_edit_dis=0.0903799184964279,acc=0.0), and a runtime smoke prediction was visibly bad. The likely mistake is architectural, not just duration: the 55-character pack dictionary changed the recognition head, so the run threw away the useful existing Sirijonga head. Keep it as negative evidence; do not promote it to the book router. - 2026-06-23: Trained the better candidate by warm-starting from the existing Sirijonga
v4 checkpoint and dictionary instead of the generic pack dictionary. Recipe:
runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/paddle_recognizer.yml; base checkpoint:outputs/limbu-first-router-v1/source-archives/limbu/checkpoints/best_accuracy.pdparams; dictionary:data/dictionaries/limbu-sirijonga-v2.txt; 20 local CPU epochs, batch size 8, LR0.00002. Paddle loaded the base checkpoint without head-shape mismatch warnings. Best dev checkpoint was epoch 14 withacc=0.49999958333368055andnorm_edit_dis=0.7922192746296381; final epoch drifted toacc=0.4166663194447338,norm_edit_dis=0.7741729695385723, so the admitted checkpoint isfull-checkpoints/best_accuracy, notlatest. - 2026-06-23: Exported, packaged, audited, and smoke-tested that v4 warm-start
candidate. Inference export is under
runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/export-best/; package/model card isruns/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/package-best/model-card.json; model audit passed atpackage-best/model-audit/. Runtime smoke on held-out cropcdc-maths-p0011-pdf-line-0001.pngreturned exactly the silver label with average confidence0.9353305101394653. A narrow package-level score over the 12-row silver eval split is atruns/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/eval-best/score/: codepoint CER0.012618296529968454, grapheme CER0.023529411764705882, exact-line accuracy0.75, 12/12 matched predictions, and no GT codepoint gaps in the 91-character dictionary. This is admission evidence only, not a benchmark claim: the pack is PDF-text silver, only 12 scoring rows, not human verified, not unseen for checkpoint selection, and has acknowledged train/dev text overlap. The immediate next task is to run a frozen-page router comparison against current v4 and judge review burden/regressions on real pages. Stop expanding eval-harness surface before that. - 2026-06-23: Ran that frozen-page router comparison and rejected global promotion of
the warm-start recognizer. Created router variant
outputs/limbu-router-v5-ppocr-devanagari-v4warm-sirijonga-silver-2026-06-23/model-card.jsonby keeping the current Devanagari primary and routing policy fixed while swapping only the Sirijonga secondary to the warm-start package;ocrtech audit-modelpassed. On CDC Maths pages 11..15, the v5 comparison run atoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v5-v4warm-silver-cdc-compare/passed the book audit and preserved the same 189 lines as router v4, but review rows rose from 58 to 68. PDF-text line coverage did not improve: both routes cover 68 PDF-text lines, leave 13 missing/partial candidates, and skip 1 no-Limbu row. A quick geometry-aligned PDF-text check was only marginally better in aggregate for v5 (1730 -> 1708 codepoint edits over 1841 reference codepoints) and exposed several long-line regressions. - 2026-06-23: Also checked the same v5 router on the real-print Gorkhapatra two-page
packet. The run at
outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-router-v5-v4warm-silver-compare/passed the book audit and preserved the same 692 lines, but review rows rose from 599 to 665. The usable Limbu page worsened from 230 to 296 review rows; the rejected non-Limbu page stayed at 369 review rows. The comparison summary isoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/router-v5-v4warm-admission-comparison.json. Decision: keep router v4 as operational default. The warm-start recognizer is useful evidence that PDF-text silver training can move crop recognition, but it should only be used as a selective probe or further training input until it improves real-page review burden. - 2026-06-23: Moved back to the actual bottleneck: line preservation/layout with the
current v4 recognizer held fixed. Added
tools/sweep_limbu_pdf_text_line_detector_params.py, covered bytests/test_sweep_limbu_pdf_text_line_detector_params.py, as a narrow detector-only diagnostic against rendered PDF text-layer geometry. It is not an OCR-quality harness; it only asks whether detected line boxes cover source text-layer lines. On CDC Maths pages 11..15 the sweep atoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/evaluated 216 simple morphology settings. Current defaults rank 154th: 68/81 Limbu source lines covered, 13 missing/partial candidates, 189 detector boxes. The top recall setting covers 76/81 with only 5 candidates but emits 335 boxes, so it is not acceptable as a default without review-burden evidence. - 2026-06-23: Ran the balanced detector candidate through the real book OCR path, not
just the geometry sweep. The derived manifest is
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-balanced-candidate-v1/book-pages-balanced-detector.jsonland changes only detector parameters (horizontal_kernel=23,vertical_kernel=3,min_width=15,min_height=10,min_area=50,max_height=100,detector_padding=2). The OCR run atoutputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/passes the book audit with 214 lines and 60 review rows. PDF-text coverage improves from 68 covered / 13 candidates under the current default to 73 covered / 8 candidates, and all missing-y rows become partial-x rows. The cost is real: +25 OCR lines, +2 review rows, and 2 operator-attention pages. Decision: keep the current default for now, but use this balanced config as the next real-page probe for pages where missed lines matter more than slight review-row growth. - 2026-06-23: Made that probe usable without hand-editing page manifests. The
one-command runner now accepts
--detector-profile current|balanced-pdf-text.currentis the default and writes no detector overrides.balanced-pdf-textwrites a hash-bound profiled manifest under<run>/profiled-page-manifest/book-pages.jsonlbefore capture prep, merges the balanced detector settings into per-pageparse_kwargs, records the profile and profiled manifest hashes inlimbu-book-ocr-run.json, and refuses to silently override conflicting row-level detector settings. Focused runner tests now cover profile materialization and conflict rejection. This is an operational OCR knob for real photographed/PDF pages, not another eval harness. - 2026-06-23: Ran the new
--detector-profile balanced-pdf-textswitch on the real Gorkhapatra two-unique-page smoke with stable page IDs by reusing the current baseline raw manifest. The run isoutputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-balanced-detector-profile-probe-stable-pageids/; it passes the book audit, records source manifest SHAb8bb7138a12c42c1a919339bbfad5165f5162ab768f4c8c00874f777b364384c, and writes the profiled manifest SHAb8f1be0a77ba7e86bf34cd15c3bd4592cbdbca252ca4c00ae20c30bf49e42010. Compared withgorkhapatra-2unique-runner-v4-current-py311-priority-dashboard, totals move from 692 lines / 599 review rows / 1 layout-gap row to 690 lines / 596 review rows / 5 layout-gap rows. On the usable Limbu page, lines move 323 -> 325, review rows 230 -> 231, Sirijonga-classified rows 286 -> 290, and other rows 6 -> 5. On the rejected non-Limbu page, review rows fall 369 -> 365 but layout-gap warnings rise 0 -> 3. Decision stays conservative:balanced-pdf-textis safe as an explicit line-preservation probe, but it is not proven better as the default. - 2026-06-23: Added a review surface for detector-profile geometry deltas:
tools/compare_limbu_book_line_geometry.py, covered bytests/test_compare_limbu_book_line_geometry.py. It compares two completed book OCR runs by page/line bbox IoU, writes matched-line TSV plus added/removed-line TSV, and copies crop plus full-page context images for unmatched lines. It also writesline-geometry-delta-contact-sheet.pngfor fast visual triage. It is not a text scorer or admission gate; it exists so layout-profile changes are visually reviewed before promotion. Running it on current Gorkhapatra default vs the stablebalanced-pdf-textprobe producedoutputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-balanced-detector-line-geometry-delta-review/: 679 matched lines, 11 candidate-only lines, 13 baseline-only lines. On the usable Limbu page specifically: 317 matched, 8 candidate-only, 6 baseline-only. A spot check of candidate-only cropcrops/added_in_candidate-gorkhapatra-unique-current-p0001-0094-img-line-0094.pngconfirms it is real visible page text, but still a fragment-level geometry change. The regenerated report now records contact sheet SHAb164f9d3e01a64c74081272a6923921442b4a4023fa3142360fa2268e51ffeb6. Visual pass: the profile adds small real Sirijonga fragments on the usable page (ᤛᤢᤱ,ᤁᤡᤖᤠᤋ᤻,ᤂᤢᤏᤧ, etc.), but it also removes larger baseline text/header detections. Decision remains: usebalanced-pdf-textonly as an opt-in line-preservation probe; do not make it the default from aggregate counts. - 2026-06-23: Added a safer additive detector profile:
--detector-profile current-plus-balanced-rescue. Unlikebalanced-pdf-text, it keeps the current detector defaults as the primary pass and adds the balanced PDF-text settings only asimage_line_rescue_detector_passes, so baseline boxes are not replaced. Focused runner tests verify this profile does not set replacementimage_line_min_width/image_line_max_heightoverrides and instead writes a rescue pass. Real Gorkhapatra stable-page-ID run:outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-current-plus-balanced-rescue-profile-probe-stable-pageids/. It passes the book audit with 703 lines / 606 review rows / 1 layout-gap row. Source pass counts: usable pageprimary=323,rescue-1=8; rejected pageprimary=369,rescue-1=3. Compared to current default, usable page lines move 323 -> 331, review rows 230 -> 234, Sirijonga rows 286 -> 292, and the earlierpossible_missed_line_gapflag disappears while layout-gap count stays 1. - 2026-06-23: Geometry-delta packet for current default vs
current-plus-balanced-rescue:outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-current-plus-balanced-rescue-line-geometry-delta-review/. It reports 692 matched default lines, 11 candidate-only rescue lines, and 0 removed lines. Usable page: 323 matched, 8 added, 0 removed. Contact sheet SHA isbe12597dc1f32ce446b0f122af530cd04b512dd9861db1b33ab9643f6d381544. Visual read: additions are mixed, but mostly small real Sirijonga fragments on the usable page; since no baseline lines are lost and layout-gap warnings do not increase, this is the best opt-in line-preservation profile so far. Still do not make it default until it is checked on the actual next book/photo batch. - 2026-06-23: Replayed
current-plus-balanced-rescueon the CDC Maths pages 11..15 PDF-derived book run:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-current-plus-balanced-rescue-cdc-compare/. It passes audit with 219 lines / 65 review rows / 14 layout-gap rows. Against the current default it preserves all 189 baseline boxes and adds 30 rescue boxes; PDF-text coverage improves from 68/81 to 73/81 covered source lines, matching the replacementbalanced-pdf-textrun while avoiding its baseline-line deletion risk. Geometry delta packet:outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/current-plus-balanced-rescue-line-geometry-delta-review/reports 189 matched, 30 added, and 0 removed lines, with contact sheet SHA838d1868b445f9ea8a11cf6e34db95fadcb9cb092098ff2321f2bec5680af22b. This is enough operational evidence to use additive rescue as the next opt-in probe, not enough to change defaults or spend more time expanding generic eval harness. - 2026-06-23: The one-command runner now writes detector-profile workflow guidance
directly into
limbu-book-ocr-run.jsonandREADME.md. Each run records the profile usage note and whether the profile is default/additive/replacement. Defaultcurrentruns also includedetector_profile_probe_command, a ready-to-run command that reruns the exact source page manifest with--detector-profile current-plus-balanced-rescueinto a sibling output directory, plusdetector_profile_probe_compare_commandto generate the line-geometry delta packet against that sibling rescue run. This is deliberately operator glue, not a default-profile promotion. - 2026-06-23: The generated OCR run README now links its "After review" section to
docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md, the current reviewed TSV -> audited final book -> reviewed-only training pack handoff. This keeps new OCR runs connected to the accepted/corrected-review blocker instead of leaving the operator at raw queue files. - 2026-06-23: Fresh OCR runs with nonzero review rows and no page-triage
source-reject/reshoot blockers now also emit
browser_review_station_commandinlimbu-book-ocr-run.jsonand aBrowser review station:README section. The command creates a pending unresolved-review HTML/TSV packet from the run's focused review queue viatools/prepare_limbu_book_unresolved_review_packet.py. The run now also emitsbrowser_review_station_apply_command, which expects the exportedbook-unresolved-review-packet.reviewed.tsvin the browser review station directory and applies it throughtools/apply_limbu_book_unresolved_review_packet.pywith--packet-dirprovenance. It also emitsbrowser_review_station_training_pack_command, which exportsbrowser-review-station-applied/final-bookthroughtools/export_limbu_book_training_pack.pyafter the reviewed packet has been applied. Clean runs with 0 review rows do not emit empty browser-station commands, and pages that need source rejection or reshoot must be handled in page triage before line review. This closes the setup gap from completed OCR run to browser review/application/training-pack handoff, while preserving the rule that no labels exist until an exported reviewed TSV is audited and applied. The focused runner tests now cover both sides: source-reject pages do not emit browser review commands, while a reviewable low-confidence Limbu run executes the generated packet-builder target and checks the generated apply/training-pack command paths. - 2026-06-23: The browser review station now includes suspected missed-line layout
gaps instead of leaving them in a separate queue. When
layout_gap_review_row_countis nonzero, the generated command passes the layout-gap queue as--missing-line-review-queue;tools/prepare_limbu_book_unresolved_review_packet.pymaps pendinggap_row_idrows tocandidate_missing_line, derives a page-wide bbox fromfrom_y/to_y, copies the gap crop, renders highlighted page context, and attaches neighboring OCR-line context. The new tests prove both the packet-builder conversion and the runner-generated command on a synthetic suspicious-gap page. This is intentionally book-OCR operator glue, not another generic eval-harness layer. - 2026-06-23: Proved the layout-gap browser station is not a dead-end artifact. The
suspicious-gap runner test now continues through reviewer TSV export/application
using the generated apply-command paths: it marks the generated
candidate_missing_lineas accepted with explicitcorrected_text, saves the TSV atbrowser_review_station_apply_command --review-packet, applies it viatools/apply_limbu_book_unresolved_review_packet.pyusing the original packet directory as provenance, and verifies the final book projection contains the inserted Unicode line with insertion metadata tied back to the originalgap_row_id. This closes the practical loop for missed-line recovery: detect suspicious gap -> review/transcribe in browser -> audited final-book insertion. - 2026-06-23: The Limbu recognizer now has a measured real-heavy fine-tune champion on
the frozen 255-line Gorkhapatra PDF-backed silver eval. Baseline combined 48,507-row
checkpoint:
runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/scored codepoint CER0.016498, grapheme CER0.038266, exact-line accuracy0.662745. Real-heavy low-LR fine-tune:runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/scored codepoint CER0.011336, grapheme CER0.024736, exact-line accuracy0.729412. That is51fewer codepoint edits and64fewer grapheme edits on the same image paths and identical GT text, with a35.36%relative grapheme error reduction. Key artifacts: training auditoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-training-audit-2026-06-23/, model packageruns/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json, package auditoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-package-audit-2026-06-23/, scoreoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-eval-2026-06-23/score/, failure analysisoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-eval-2026-06-23/score/failure-analysis.md, official paired comparisonoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-vs-combined-best-comparison-2026-06-23/comparison.md, and path-keyed comparisonoutputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-vs-combined-best-path-keyed-comparison-2026-06-23/comparison.md. Caveat remains strict: this is not human-ground-truth claim-safe because labels are PDF text-layer silver and the checkpoint was selected on this eval surface. The top remaining recognizer failures are omitted Limbu vowel/mark codepoints, quote or ASCII punctuation mismatches, omitted spaces, and omitted sentence punctuation.tools/compare_recognizer_eval_runs.pynow falls back to uniqueimage_pathpairing when sample IDs are rewritten; the real-heavy comparison pairs all 255 rows, audits cleanly, and keepsclaim_safe=falsebecause of silver labels, selected-on-eval status, differing manifest files, and the older baseline's stale row-levelsplit=trainmetadata.