|
Download ocr-tech/docs/RECOGNIZER_GOAL_PROGRESS.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 334 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/RECOGNIZER_GOAL_PROGRESS.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/RECOGNIZER_GOAL_PROGRESS.md
-
curl -L -o RECOGNIZER_GOAL_PROGRESS.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/RECOGNIZER_GOAL_PROGRESS.md
334 kB
# Limbu Book OCR Goal Progress
Living checklist for the active goal. Update often as work lands.
**Active goal:** build a usable Limbu book OCR system that accepts scanned or
photographed real Limbu book pages and outputs editable Unicode text with page and line
structure preserved. It must support both Sirijonga Limbu and Devanagari-written Limbu.
The current bottleneck is page layout, reading order, missed-line recovery, and review
burden with the current recognizers held fixed. Evaluation is a guardrail, not the
product: keep it to frozen real-book acceptance checks such as page CER/WER, line
detection precision/recall/F1, reading-order accuracy, blank/extra/missed rates, and
review-packet coverage. Do not expand a general eval harness unless it directly proves
one of those book OCR acceptance checks.
**Historical context:** the earlier recognizer-language program remains useful evidence,
but it is not the active thread goal. Any next work should make the book OCR loop more
true: cleaner source selection, stronger page preparation, better line preservation,
lower manual review burden, safer correction application, or measured improvement over
the current diagnostic baselines.
**2026-06-24 controlled recognizer ablation verdict:** after the targeted+Unicode bulk
run regressed, a five-cell ablation grid was trained on the AWS G6 L4 host and scored
against the current champion's exact frozen Gorkhapatra eval identity. The artifact
root is `outputs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`; the launcher
and scoring script are in `runs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`.
Results on `255` matched lines: champion reference remains codepoint CER
`0.011336032388663968`, grapheme CER `0.024735729386892176`, exact `0.7294117647058823`.
Control v2 replay at LR `1e-6` scored `0.011234817813765181` / `0.02452431289640592`
/ `0.7333333333333333`, a one-edit non-significant improvement and therefore only a
sanity check. v2 plus `256` capped targeted rows scored worse:
`0.01194331983805668` / `0.026004228329809725` / `0.7294117647058823`. v2 replay at
LR `1e-5` tied codepoint CER but worsened grapheme CER:
`0.011336032388663968` / `0.026004228329809725` / `0.7372549019607844`. Both v3
dictionary cells failed catastrophically, around `0.98` codepoint CER and `0.0`
exact-line accuracy, because expanding the dictionary from the v2 checkpoint resizes
recognition-head parameters. Do not promote any ablation checkpoint. Keep the
real-heavy v2 dictionary champion as the current recognizer. Next recognizer work
should not add dictionary characters by warm-starting directly from the v2 head, should
not chase Paddle `norm_edit_dis` alone, and should focus on real/book-page line
harvesting or a separate from-scratch/rehead training plan if full Unicode coverage is
actually required.
Started: 2026-06-16. Owner: Claude session.
---
## LIMBU OPERATIONAL OCR PIPELINE PIVOT (2026-06-21)
**2026-06-23 recognizer corpus correction:** do not describe the current Limbu
training inventory as only the `1,866` Gorkhapatra train rows. That count is only
the train split of
`outputs/limbu-gorkhapatra-pdf-backed-silver-2026-06-23-v3/`, a PDF-backed
silver real-domain pack with `255` held-out eval rows. The larger local Sirijonga
training inventory already includes `18,418` rendered Namdhinggo Limbu prose train
rows in `outputs/limbu-prose-rendered-train-2026-06-16/`, `28,155` augmented
rendered train rows in `data/limbu-aug-train/train.txt`, and the prior
`runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/` checkpoint trained from the
rendered prose pack. The `1,866`-row Gorkhapatra G6 run was therefore a narrow
real-domain fine-tune, not a full-corpus Limbu recognizer run. The corrected
training policy is: use synthetic/rendered rows for glyph, word, augmentation,
and font coverage; use Gorkhapatra PDF-backed rows for real-domain adaptation and
checkpoint selection; keep synthetic rows out of headline CER claims. The
dedicated builder `tools/build_limbu_combined_recognizer_pack.py` materializes
this as an auditable combined pack, rewrites legacy label paths to repo-root
paths, tags source components, and drops exact normalized train-text overlaps
against the held-out Gorkhapatra eval text before training.
**2026-06-23 combined corpus G6 launch:** the corrected combined training pack is
`outputs/limbu-combined-synth-real-gorkhapatra-v1-2026-06-23/`. It has `48,507`
train rows and `255` eval rows after dropping `13` exact normalized train-text
overlaps with the Gorkhapatra eval text. Train composition is `28,155`
augmented rendered rows, `18,418` rendered Namdhinggo prose rows, `1,853`
Gorkhapatra PDF-backed silver rows, and `81` CDC Sirijonga PDF-backed silver
rows. Eval is only the held-out Gorkhapatra PDF-backed silver real-print split.
Both manifests passed `ocrtech audit-manifest`. The generated recipe is
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-g6-l4-ppocrv5-2026-06-23/`
with pack hash
`1f77f7a0f947fc475afcef5568f08a1f0649a92dc8ee1a7eb0620956b734085b`, base
checkpoint
`runs/limbu-gorkhapatra-pdf-backed-silver-v4warm-20ep-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams`,
batch size `64`, learning rate `0.00001`, `15` epochs, warmup `1`, GPU enabled,
and `train_drop_last=false`. The first generated run used Paddle's default
`main_indicator=acc`; that was wrong for the current CER-oriented goal because
it can overwrite a lower-edit-distance checkpoint with a higher exact-line-acc
checkpoint. `src/ocrtech/training.py` and the CLI now expose
`--main-indicator`, and the active corrected recipe is
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/`
with `main_indicator=norm_edit_dis`. The AWS G6 instance
`i-0fa19dff7b233a353` was restarted on 2026-06-23, current public IP
`35.175.116.142`, remote workspace `/opt/ocrtech`. An initial exact-acc run was
stopped after proving the selection-rule problem, then a norm-edit run initially
hit OOM because the old parent process was still holding GPU memory. After
clearing the GPU, the active corrected run was launched as remote PID `2404` via
`/opt/ocrtech/runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/train-g6.sh`.
Startup sanity passed: zero missing label images, Paddle `3.3.1` sees one NVIDIA
L4, training dataloader has `758` iters, eval dataloader has `4` iters, and the
checkpoint loaded successfully from the previous Gorkhapatra best model. First
eval at global step `200` saved by norm-edit with `norm_edit_dis=0.8731627801677526`
and `acc=0.1686274443675512`.
**2026-06-24 CER improvement path: targeted fabrication, not generic synthetic
volume.** The combined recognizer pack already has heavy synthetic coverage:
`46,573/48,507` train rows are synthetic/rendered and only `1,934` are
real-or-silver line crops. The norm-edit run selected epoch `1` as best
(`norm_edit_dis=0.8731627801677526`), while later evals hovered lower, so dumping
more random synthetic rows into the same mixture is the wrong next move. The next
CER work should build a targeted fabrication factory:
- render from real Limbu/Sirijonga text, not random Unicode strings, using verified
renderable upstream Namdhinggo fonts and any CDC/book fonts we can verify. Do not use
extracted PDF subset fonts unless they pass visible-ink renderability checks;
- match the real line-width distribution, especially long newspaper/book lines
that stress the recognizer input width and CTC/attention decoding;
- apply phone/page degradations learned from the June 23 PXL photos and prepared
images: slight perspective, blur, JPEG, uneven illumination, stroke thickening/
thinning, paper background, local crop padding variation, and downsampled
low-resolution edges;
- oversample hard graphemes and substitutions from `confusions.json`,
`worst-lines.csv`, and current phone/cdc/gorkhapatra OCR outputs instead of
uniformly sampling the dictionary;
- keep source tags and sampling weights explicit so synthetic rows can be downweighted
or capped relative to real/silver rows;
- never score claims on synthetic rows. Evaluate every ablation on held-out
Gorkhapatra PDF-backed silver and separate CDC/phone-page review packs.
Practical acceptance gate for the next fabrication run: create a new
`limbu-targeted-fabrication-v1` component, train as a short warm-start ablation,
and keep it only if it improves held-out real/silver micro CER without degrading
CDC and phone-page matched-line CER. If it only improves synthetic validation or
exact-line `acc` while `norm_edit_dis`/CER stagnates, reject it.
**2026-06-24 targeted fabrication v1 materialized and corrected:** the new generator
`tools/build_limbu_targeted_fabrication_pack.py` now builds the targeted synthetic
component described above. It reads verified synthesis-text JSONL/plain text, optional
`worst-lines.csv`, optional `confusions.json`, verified upstream Namdhinggo fonts, and
an optional real line-crop width profile. It writes Paddle labels, rendered images,
`manifests/train.jsonl`, `dictionary.txt`, `rejects.jsonl`, and
`targeted-fabrication-summary.{json,md}`. Every row is explicitly tagged
`claim_evidence_eligible=false`, `gt_status=synthetic_text`, and
`source_training_component=limbu-targeted-fabrication-v1`; synthetic rows are
training-only and must not be used for headline CER claims. The first usable pack is
`outputs/limbu-targeted-fabrication-v1-2026-06-24/`: `9,996` phone-photo degraded
line images from `5,000` selected base text lines, `4` render rejects caught by the ink
guard, `1,866` Gorkhapatra train line widths used as the width profile, four verified
upstream Namdhinggo fonts, current combined-run `worst-lines.csv`/`confusions.json`,
and RAQM shaping enabled. Crop dimensions are width min/p50/max `89/608/2460`, height
min/p50/max `32/61/147`, dark-pixel min/p50/max `28/2082/113592`, and min-gray
min/p50/max `0/16/129`. Correction: the earlier generated pack under the same name
used extracted PDF subset fonts and was effectively blank; those fonts do not render
visible Unicode Limbu through Pillow/RAQM and are now rejected by an explicit font
renderability test plus a per-crop ink guard. The generator has focused tests in
`tests/test_build_limbu_targeted_fabrication_pack.py`, including this blank-font
regression.
The combined recognizer builder now accepts this component through
`--targeted-fabrication-train-manifest`. The first combined pack is
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-v1-2026-06-24/`: `58,467`
train rows and `255` held-out Gorkhapatra eval rows after dropping `49` exact eval-text
overlaps. Train composition is `28,155` augmented rendered rows, `18,418` rendered
Namdhinggo prose rows, `9,960` targeted-fabrication rows, `1,853` Gorkhapatra
PDF-backed silver rows, and `81` CDC Sirijonga PDF-backed silver rows. Both
`train.jsonl` and `eval.jsonl` passed `ocrtech audit-manifest` with hash verification.
Next action is not to trust the new synthetic component; train a short warm-start
ablation from the current best checkpoint and keep it only if held-out real/silver CER
improves without phone/CDC regression.
**2026-06-24 Unicode/diacritic coverage hardening:** the statistical pass is
documented in `docs/LIMBU_UNICODE_DIACRITIC_STATS.md`. Unicode 17.0.0 UCD was checked
against the local Python `unicodedata` view; the Limbu block remains `68` assigned
codepoints in `U+1900..U+194F`. A HarfBuzz/Pillow Namdhinggo shaping probe rendered
`21,891/21,891` assigned/component-order cases cleanly, which proves renderability but
also proves the font is too permissive to be a grammar oracle. The corpus-stat pass on
the targeted combined pack found `61/68` codepoint coverage, with seven missing
codepoints and low counts for `U+1919`, `U+1924`, `U+1926`, `U+1928`, `U+192B`,
`U+1944`, `U+194E`, and `U+194F`. It also showed that `NFC`/`NFD` do not repair these
labels because most Limbu marks have combining class `0`; only `U+193B`, `U+1939`, and
`U+193A` have non-zero combining classes.
The coverage generator `tools/build_limbu_unicode_coverage_text.py` now uses the
measured corpus order instead of random permutations: `base+subjoined+vowel`,
`base+vowel+KEMPHRENG+final`, and no generator-created `final+sign` context. It also
fixes the earlier omission of `U+1932` from finals. The generated source
`data/synthesis-text/limbu-sirijonga-unicode-coverage-v1-2026-06-24/coverage-text.jsonl`
has `1,936` rows and passes the new corpus-stat audit with `warning_counts={}`. The
rendered component `outputs/limbu-unicode-coverage-fabrication-v1-2026-06-24/` has
`3,872` mixed-degradation line crops, `0` render rejects, and dark-pixel min/p50/max
`33/1880/49435`. The combined pack
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-2026-06-24/`
contains `62,339` train rows and `255` held-out eval rows, passed train/eval manifest
audits, covers `68/68` assigned Limbu codepoints, and has no assigned Limbu codepoint
below `1,000` occurrences after dedupe. Existing inherited corpus warnings remain in
older rows; the new Unicode coverage supplement itself added none. Next training should
treat `unicode-coverage-fabrication-v1` as an explicit ablation component.
**2026-06-24 targeted+Unicode G6 training launch:** training is now running on the
combined existing pack plus targeted fabrication plus Unicode coverage supplement, not
on the coverage supplement alone. The run directory is
`runs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-24/`.
It warm-starts from
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams`,
uses append-only dictionary `data/dictionaries/limbu-sirijonga-v3.txt` (`98` chars;
v2 prefix plus `U+1909`, `U+190A`, `U+191A`, `U+191D`, `U+191E`, `U+1932`,
`U+1940`), batch size `64`, LR `0.00001`, `15` epochs, warmup `1`,
`main_indicator=norm_edit_dis`, and eval every `200` steps. The staged G6 workspace is
`/opt/ocrtech` on AWS instance `i-0fa19dff7b233a353`, public IP `3.219.242.144`,
training PID `11133`. Remote preflight resolved `62,339` train rows and `255` eval
rows with zero missing images and zero malformed rows; Paddle `3.3.1` sees one NVIDIA
L4, with `975` train iters and `4` eval iters. Startup head-shape warnings are
expected because the dictionary grew by seven chars. First eval at global step `200`
saved the initial best with `norm_edit_dis=0.04608837027816626`, `acc=0.0`, and
`fps=243.75362537495823`; this is a warmup checkpoint, not the final CER answer. After
completion, pull back the run, audit training completion, export/package
`best_accuracy`, run frozen Gorkhapatra recognizer eval, and compare real CER against
the real-heavy champion before deciding whether the synthetic additions helped.
**2026-06-24 targeted+Unicode G6 training result: reject.** The 15-epoch G6 run
completed without crash or OOM and the local completion audit passed with `issue_count=0`,
`warning_count=1` at
`outputs/limbu-combined-targeted-unicode-v1-15ep-training-audit-2026-06-24/`; the warning
is only that the audit regex did not extract the best validation metric even though the
training log records it. Paddle selected epoch `14` with
`norm_edit_dis=0.8278816454641351`, `acc=0.1450980335255673`, and
`fps=245.24747110530726`. The exported/package-best model scored worse on the frozen
Gorkhapatra recognizer eval:
`outputs/limbu-combined-synth-real-gorkhapatra-targeted-unicode-v1-15ep-normedit-best-eval-2026-06-24/score/`
has `255` matched lines, codepoint CER `0.024696356275303644` (`244/9880`), grapheme CER
`0.053276955602537` (`252/4730`), and exact-line accuracy `0.4666666666666667`. Against
the real-heavy champion at
`outputs/limbu-targeted-unicode-v1-best-vs-real-heavy-champion-comparison-2026-06-24/comparison/`,
it is worse by `0.013360323886639675` codepoint CER and `0.028541226215644824`
grapheme CER, with paired wins/losses/ties `21/102/132` and `22/98/135`. Against the
older combined baseline at
`outputs/limbu-targeted-unicode-v1-best-vs-combined-baseline-comparison-2026-06-24/comparison/`,
it is worse by `0.008198380566801618` codepoint CER and `0.015010570824524318`
grapheme CER. Do not promote this package into book OCR. Treat it as a negative
ablation showing that the current targeted synthetic/Unicode supplement diluted the
real-heavy signal instead of improving real print CER. The eval remains non-claim-safe
silver PDF-text ground truth, and the stale row-level `split=train` metadata warning in
the candidate eval manifest should be fixed before the next serious comparison.
**2026-06-22 measured-page correction:** the active goal is no longer pipeline
plumbing. The Devanagari-written Limbu PP-OCRv5 run was stopped at `best_accuracy`
after dev regression, completion-audited, exported, packaged, and scored. The sealed
31-line recognizer eval is non-claim-safe but useful: codepoint CER `0.118219`,
grapheme CER `0.256822`, exact-line accuracy `0.290323`. On the three real CDC
Devanagari pages, the trained recognizer cut image-line Tesseract page codepoint CER
from `0.865400` to `0.480602`; a directed detector cleanup
(`horizontal_kernel=31`, `vertical_kernel=5`, `min_width=80`, `max_height=80`) moved it
to `0.458432` with grapheme CER `0.645161`, WER `0.824242`, line F1 `0.593750`, and
matched-line CER `0.177895`. The paired detector comparison is positive but
non-claim-safe (`3` pages; codepoint CER improvement `0.022169`). The selected failure
analysis still shows `12/31` reference lines missed and dominant error mass from
missed reference text (`47.58%`). The next Devanagari work is targeted missed-line
recovery/layout modeling with the PP-OCRv5 recognizer held fixed, not another broad
pipeline pass or blind recognizer run. Current evidence lives under
`outputs/limbu-measured-page-baseline-2026-06-22/` and
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/`.
**2026-06-22 router full-page smoke:** the Devanagari PP-OCRv5 package is now wired
into an operational Limbu router instead of the failed Tesseract primary. The first
artifact was `outputs/limbu-router-v2-ppocr-devanagari-sirijonga-2026-06-22/`: primary
`paddleocr` Devanagari CDC PP-OCRv5, secondary Sirijonga v4 PP-OCRv5, line-level
script selection, Devanagari primary guard, and hashed component model cards. It passed
`ocrtech audit-model` and `preflight-limbu-pipeline` under the Paddle Python 3.11
runtime, but the guard was too blunt for Sirijonga pages. A fresh two-page v2
Gorkhapatra book smoke at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2p-router-v2-ppocr/` completed
but is not usable OCR: `book-run.json` reports `692` detected lines and `671` review
rows across 2 pages. Page 1 alone has `323` image-line detections with unfiltered
morphology parameters (`horizontal_kernel=23`, `min_width=35`, `filter_enabled=false`);
page 2 has `369`. Page 1 is visually Sirijonga, but v2 classified `317/323` lines as
Devanagari because the Devanagari primary guard blocked the Sirijonga secondary after
low-confidence Devanagari hallucinations. `ScriptSelectCompositeEngine` now supports
`primary_guard_min_confidence`, preserving old behavior unless configured. The current
working router is
`outputs/limbu-router-v4-ppocr-devanagari-sirijonga-confidence-guard-2026-06-22/`,
with `primary_guard_min_confidence=0.8`; it passes model audit, runtime preflight, and
focused `script_select_composite` tests. On the first Sirijonga page, the runnable
filtered smoke
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-filter-conf060/`
keeps `152` lines and `80` review rows after `drop_empty`, `require_script=sirijonga`,
and `min_confidence=0.6`; kept script counts are `151` Sirijonga, `1` mixed, `0`
Devanagari, `0` other. Treat that filtered output as diagnostic only, not selected book
OCR: inspection showed many removed rows are real Sirijonga text with low recognition
confidence, so confidence filtering silently drops page content. The selected operational
baseline for the same page is now the v4 preserve-mode run at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-preserve/`.
It keeps all `323` detected lines, writes `230` review rows, reports script counts
`286` Sirijonga, `27` Devanagari, `4` mixed, and `6` other, and passes the strict
`--require-no-dropped-image-lines --min-line-count 25` audit at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-confidence-guard-preserve/no-dropped-audit/`.
The filtered run fails that same audit with `removed_line_count=171`. This proves the
trained recognizers are packaged and callable in the book pipeline and that the prior
router was suppressing Sirijonga. The remaining failure is page quality and review
burden: preserve mode keeps the text but many lines are low-confidence or noisy on this
complex newspaper sheet. For tomorrow's book pages, default to preserve mode plus review
queue; use script/confidence filtering only for diagnostics or optional cleanup views,
not as the digitization source of truth. Do not start another broad recognizer training
pass from this evidence.
**2026-06-22 photographed-page prep path:** raw multi-page capture preparation is now
wired into the book OCR path via `tools/prepare_limbu_book_captures.py`. The tool reads
a raw page JSONL manifest, prepares each page with the existing capture pipeline
(`auto_deskew`, crop, perspective, grayscale/autocontrast options), audits every capture
bundle, and emits `prepared-page-manifest.jsonl` rows whose `image_path` points at the
prepared image and whose `parse_kwargs.capture_prep_metadata` points at the matching
metadata file. This closes the operational gap between photographed/slanted pages and
`tools/parse_limbu_book_pages.py`; it is not another eval harness. A real one-page
Sirijonga smoke produced
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-capture-prep-preserve-manifest/prepared-page-manifest.jsonl`
with capture audit pass and prepared-image quality
`1985x3062`, entropy `2.894443`, luminance stddev `76.480214`, edge stddev `74.158261`.
Auto-deskew skipped on that already-straight page because there was insufficient
near-horizontal line evidence. Running v4 preserve OCR from that prepared manifest at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-capture-prep-preserve/`
kept `323` lines, wrote `230` review rows, dropped `0` image-line rows, and passed the
strict capture-plus-preserve audit at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-router-v4-capture-prep-preserve/strict-capture-no-dropped-audit/`.
Use this capture-prep -> preserve-OCR -> strict audit chain as tomorrow's default for
book page photos. The book capture-prep wrapper now also writes
`capture-prep-dashboard.html`, a raw/prepared contact sheet with operation summaries,
prepared-image quality, and capture-audit status per page; the dashboard path, hash,
and card count are recorded in `book-capture-prep-run.json` and surfaced by the
one-command runner. This gives the operator a fast visual gate for crop, deskew, and
perspective mistakes before OCR review.
**2026-06-23 fragmented-text deskew fallback:** `src/ocrtech/capture.py` now keeps the
existing Hough near-horizontal line deskew path, but if it cannot find enough long line
segments it falls back to row-projection scoring over the thresholded text ink. This
targets photographed pages where Limbu glyph rows are fragmented into short components
rather than long strokes. Blank/no-evidence pages still skip auto-deskew. The focused
capture test now covers a slanted page made of separated short text fragments and
passes through `prepare_limbu_capture(..., auto_deskew=True)` with an applied angle in
the expected range.
**2026-06-22 current one-page runner verification:** the current one-command runner was
rerun on the known one-page Sirijonga folder at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir-current/`.
It completed under `/tmp/ocrtech-paddle311`, generated one raw-page manifest row,
prepared one page, and passed the book artifact audit with zero issues. Top-level
counts: `1` page, `323` preserved lines, `230` book-line review rows, script counts
`286` Sirijonga / `27` Devanagari / `4` mixed / `6` other, and
`operator_attention_pages=1` with `high_review_burden`. The capture prep dashboard is
hash-bound in `limbu-book-ocr-run.json` with one page card and two images (raw and
prepared); the book review dashboard has `230` cards and `230` crop images. The
book-line review queue has `230` rows and includes `source_crop_paths` and
`source_crop_sha256s`. This is still not an OCR quality claim, but it proves the current
operator path emits the review/capture surfaces needed for real page triage.
**2026-06-22 current multi-page runner verification:** the current one-command runner
was also rerun on the full four-image Gorkhapatra source folder at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-4p-runner-v4-preserve-image-dir-current/`.
It completed under `/tmp/ocrtech-paddle311` and passed the book artifact audit with zero
issues. Counts: `4` pages, `1384` preserved lines, `1198` book-line review rows, script
counts `682` Sirijonga / `64` Devanagari / `30` mixed / `608` other. The top-level
source summary reports `4` discovered images, `4` kept images, `0` skipped images, and
`2` duplicate SHA-256 groups; the runner now also records this as the source attention
flag `duplicate_source_images_present`, so the known source contamination is visible at
the run front door. The capture prep dashboard has `4` page cards and `8` raw/prepared
images; the book review dashboard has `1198` cards and `1198` crop images; the
book-line review queue has `1198` rows with crop paths and crop hashes. Operator
attention flags expose the page pattern: pages 1 and 3 are `high_review_burden`, while
pages 2 and 4 are `all_lines_need_review` plus `high_other_script_share`. This is valid
multi-page plumbing and triage evidence for the current code, not a clean Limbu quality
target.
**2026-06-22 refreshed operational run after scope correction:** because the default
repo `uv run` environment is Python `3.14` and cannot import the Paddle runtime, the
current book runner was executed with the documented Paddle runtime
`/tmp/ocrtech-paddle311/bin/python`. The one-page folder run at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-current-py311/`
completed with `1` page, `323` preserved lines, `230` book-line review rows,
`1` layout-gap review row, and a passing book artifact audit. The cleaner duplicate
skipping multi-page run at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311/`
completed with `2` unique pages, `692` preserved lines, `599` book-line review rows,
`1` layout-gap review row, and `passed=true` in
`book-output-audit/limbu-book-output-audit.json`. Source selection discovered `4`
images, kept `2`, skipped `2` duplicate images, and recorded
`source_images_skipped,duplicate_source_images_removed`. Page 1 is the usable
Sirijonga operational target (`323` lines, `230` review rows, one possible missed-line
gap); visual inspection of that gap crop suggests a section/spacing break rather than
an obvious missing line, so it remains review triage, not automatic insertion. Page 2 is
a Nepali newspaper page, not a Limbu OCR target: it has `369/369` review rows,
`298` other-script lines, and the operator flags `all_lines_need_review` plus
`high_other_script_share`. The immediate product bottleneck is therefore source
selection plus review-burden reduction on real Limbu pages, not another eval harness.
**2026-06-22 page-level triage packet:** `tools/run_limbu_book_ocr.py` now writes a
page-level operator packet before line-level review under `page-triage-review/`:
`page-triage-review-queue.tsv` and `page-triage-review-dashboard.html`. The TSV keeps
page image path/hash, operator flags, per-page script counts, review ratio, layout-gap
count, and an advisory `recommended_action` such as
`source_reject_candidate`, `inspect_layout_gap_then_line_review`,
`review_low_confidence_lines`, or `accept_page`. It does not delete lines or make a
quality claim; it is meant to stop a bad source page from turning into hundreds of
line-review tasks. A refreshed duplicate-skipping real run at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-page-triage-links/`
emits `2` page-triage rows: page 1 is `inspect_layout_gap_then_line_review` (`323`
lines, `230` review rows, one layout-gap warning), while page 2 is
`source_reject_candidate` (`369/369` review rows, `298` other-script lines,
other ratio `0.8076`). The page-triage dashboard image links were verified to resolve
relative to the dashboard directory (`../capture-prep/...`).
**2026-06-22 page-triage application path:** `tools/apply_limbu_book_page_triage.py`
now turns explicit page-level reject/reshoot/drop decisions into a chainable reviewed
book projection. It copies only kept page documents, rewrites aggregate `book.md` /
`book.body.md`, writes new `score-documents/`, filters `book-line-review-queue.tsv`
and `layout-gap-review-queue.tsv` to kept pages, and records rejected pages plus source
hashes in `page-triage-application-audit.json`. Advisory recommendations alone do not
remove pages; an operator must explicitly set `operator_decision` or `review_status` to
a reject-style value. `tools/apply_limbu_book_review_queues.py` now applies this
page-triage projection first when a page-triage queue is present, then applies book-line
corrections and layout-gap insertions to the filtered projection. A real smoke copied
the two-page page-triage queue, explicitly rejected
`gorkhapatra-unique-current-p0002`, and ran the combined review command to
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-page-triage-links-reviewed-p1-only/`.
The final book contains only `gorkhapatra-unique-current-p0001` (`323` lines), the page
2 Nepali text is absent from `final-book/book.md`, the page-triage audit filtered the
line-review queue from `599` rows to `230` rows by removing `369` rejected-page rows,
and the final layout-gap application consistency audit passed with zero issues.
**2026-06-22 line-review priority guidance:** `tools/parse_limbu_book_pages.py` now
adds `review_priority_rank`, `review_priority`, and `review_recommended_action` to
`book-line-review-queue.tsv`, and the book review dashboard displays those values as
operator-facing pills. The goal is not to remove review rows or call them correct; it is
to make the review packet usable by separating empty/no-script detections, mixed-script
risks, and ordinary low-confidence spot checks. A refreshed two-page run at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-2unique-runner-v4-current-py311-review-priority/`
still has `599` review rows, but they are now bucketed as `304` high-priority
`transcribe_or_drop_empty_detection`, `15` medium-priority
`verify_mixed_script_text`, and `280` low-priority
`spotcheck_low_confidence_text`. After page-triage rejection of the known non-Limbu page,
the usable Sirijonga page has the review work shape we actually care about: `6`
high-priority empty detections, `4` medium mixed-script rows, and `220` low-priority
low-confidence spot checks. This is a practical review-burden reduction mechanism
without silently dropping OCR text.
**2026-06-23 scope correction and current priority packet:** the user correctly pushed
back that we were overdoing the eval-harness direction. Keep evaluation as acceptance
evidence for the book OCR loop, not as the main product. The current freshest real-book
run is
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/`,
executed under `/tmp/ocrtech-paddle311/bin/python` because the repo `uv run` Python
3.14 environment cannot import Paddle. It processes `2` unique source pages after
duplicate skipping, preserves `692` OCR lines, writes `599` review rows, and passes the
book artifact audit. The priority queue/dashboard are operator views only:
`book-line-review-queue.tsv` remains in page/reading order for application, while
`book-line-priority-review-queue.tsv` is a sorted working copy. Both have `599` rows;
the priority copy starts with rank `020` high-priority
`transcribe_or_drop_empty_detection` rows. Counts are `304` high
`transcribe_or_drop_empty_detection`, `15` medium `verify_mixed_script_text`, and
`280` low `spotcheck_low_confidence_text`. This confirms the next practical work is
reducing review burden and recovering/preserving real page lines, not expanding generic
metrics.
**2026-06-23 triaged projection cleanup:** page-level triage application now filters
all page-scoped review queues, not only the canonical book-line queue. Explicitly
rejected pages are removed from `book-review-queue.tsv`,
`book-line-review-queue.tsv`, `book-line-priority-review-queue.tsv`, and
`layout-gap-review-queue.tsv`; aggregate review counts, priority/action counts, script
counts, and source-document provenance hashes are recomputed for the kept pages.
Inherited review-dashboard paths are cleared from the triaged `book-run.json` so a
filtered projection cannot accidentally point an operator back to rejected-page cards.
The runner README now tells the operator to inspect page triage and set
`operator_decision=reject`, `reshoot`, or `drop` before applying line/gap reviews. A
real check on the 2026-06-23 two-page run with page 2 explicitly rejected writes
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard/manual-triage-check/triaged-p1-only/`:
`1` kept page, `323` lines, `230` review rows, priority counts `6` high / `4` medium /
`220` low, all filtered queues contain only `gorkhapatra-unique-current-p0001`, and
`tools/audit_limbu_book_output.py` passes with zero issues. This is an operational
source-selection and review-burden improvement, not an OCR quality claim.
**2026-06-23 non-text empty-detection guidance:** the book-line review queue now records
`book_line_bbox` and `book_line_aspect_ratio`, and empty OCR rows whose geometry is a
large square/near-square detection are separated from text transcription as
`inspect_non_text_or_drop_empty_detection`. This is intentionally conservative: it does
not drop the row or change OCR text; it prevents obvious portraits/page furniture from
being mixed into the "transcribe visible text" bucket. A real rerun at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/`
passes the book audit with `2` pages, `692` lines, and `599` review rows. Across both
pages it separates `6` non-text empty detections, leaving `298`
`transcribe_or_drop_empty_detection` rows. After explicitly rejecting the non-Limbu
second page, the p1-only projection at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/`
passes `tools/audit_limbu_book_output.py` with zero issues and has the actionable
review shape: `5` empty visible-text transcription/drop rows, `1` non-text inspect/drop
row, `4` mixed-script verification rows, and `220` low-confidence spot checks. This is
review-packet quality work, not a CER/WER improvement.
**2026-06-23 full-page layout overlay review:** `tools/run_limbu_book_ocr.py` now emits
a page-layout review packet under `page-layout-review/` before page triage and
line-level transcription. It writes `page-layout-review-manifest.tsv`, a
`page-layout-review-dashboard.html`, and one full-page overlay PNG per OCR page with
line boxes and order indices drawn over the prepared page image. Overlay colors are
green for accepted text, amber for review text, red for empty OCR, and purple for
structural text. This directly targets the active layout/missed-line bottleneck: an
operator can now inspect whole-page line coverage and reading order before spending time
on individual crop transcription. A real two-page Paddle run at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-layout-overlay/`
passes the book audit, writes `2` layout rows and `2` overlay images, and records page 1
as `323` lines / `6` empty lines / `230` review lines with reading order
`wide_column_right_to_left`. Visual inspection of
`page-layout-review/overlays/layout-0001-gorkhapatra-unique-current-p0001.png` confirms
the overlay is nonblank and shows the current failure surface: many low-confidence body
rows plus a small number of empty OCR/page-furniture detections. This is review and
layout-debug evidence, not a text-quality claim.
**2026-06-23 reviewed-line training-pack bridge:** `tools/export_limbu_book_training_pack.py`
now turns a reviewed Limbu book projection into line crops, PaddleOCR-style labels,
a JSONL recognizer manifest, dictionary, and script inventory. The default is
deliberately strict: only lines with explicit book-line review correction, split-review,
or layout-gap insertion metadata are exported. Pending/unreviewed OCR rows are skipped
so the training set does not silently become a copy of current recognizer output. The
exporter now revalidates that metadata before writing labels: status must be
accepted/corrected, reviewer must be present and non-machine, and the evidence-specific
reviewed text must be present and match the exported line. Candidate insertions require
explicit `corrected_text`, so replayed machine candidate text cannot leak into
recognizer training. The OCR runner README now continues the operator workflow after review application:
`uv run python tools/export_limbu_book_training_pack.py <run>-reviewed/final-book --out
<run>-training-pack-reviewed`. A real default export against the older p1-only
final-book correctly refused to write a pack because there were no accepted line/gap
review rows. A bootstrap diagnostic export from the latest p1-only triaged projection
used `--include-unreviewed` and wrote
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/training-pack-bootstrap-p1-unreviewed/`:
`317` crops/labels, `6` empty rows skipped, dictionary with `97` characters, and script
sample counts `221` Sirijonga / `27` Devanagari / `69` mixed. That pack is explicitly
marked `gt_status_counts={"unreviewed_ocr_projection": 317}` and is not training truth.
The practical next blocker is accepted reviewer corrections/insertions, not more eval
harness.
**2026-06-23 layout-gap ink guidance:** layout-gap rows now include image evidence, not
only geometry. `tools/run_limbu_book_ocr.py` writes `gap_ink_dark_pixel_count`,
`gap_ink_density`, `gap_ink_bbox`, `gap_ink_row_bands`,
`gap_candidate_text_band_crop_paths`, `gap_candidate_text_band_crop_sha256s`,
`gap_candidate_text_band_count`, `gap_horizontal_rule_count`,
`page_context_path`, `page_context_sha256`, `gap_review_priority`, and
`gap_recommended_action` into
`layout-gap-review-queue.tsv`, and the dashboard/README surface priority/action counts.
The page context image highlights the suspicious y-band on the full prepared page, so a
reviewer can reject rules, headers, and spacing before treating a crop as a missed text
line. When a gap has a candidate text-like band, the dashboard shows focused
candidate-band crops in addition to the full gap crop so transcription can target the
missed line instead of a wide spacing band.
The applicator remains compatible because these are additional review columns; accepted
insertions still require explicit `review_status` and `corrected_text`. Re-running the
packet writer against the latest p1-only triaged projection at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/layout-gap-review-ink-guidance/`
kept the one suspicious gap but downgraded it to low-priority
`likely_rule_or_spacing`: `5737` dark pixels, density `0.038029`, `0` candidate text
bands, `3` horizontal-rule bands, and no candidate-band crops. This corrects the
earlier geometry-only warning: the p1 gap is likely section rule/spacing, not an actual
missed body line. Page triage now consumes the gap-review summary before assigning
operator flags. The refreshed
real p1 triage check at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-nontext-review-guidance/manual-triage-check/triaged-p1-only/ink-guided-page-triage-check/`
keeps `suspicious_gap_count=1` for auditability, but writes
`candidate_text_gap_count=0`, `horizontal_rule_gap_count=1`, removes
`possible_missed_line_gap`, and recommends `review_low_confidence_lines` instead of
layout-gap inspection.
**2026-06-23 layout-gap review evidence hardening:** accepted layout-gap insertions now
preserve focused candidate text-band crop paths/hashes in both inserted-line metadata
and `layout-gap-insertion-pairs.jsonl` when those crops exist. The applicator validates
the full gap crop plus every candidate-band crop before accepting a row; the independent
application audit now revalidates the same candidate-band crop hashes after the fact.
It also checks every insertion-pair row against the actual inserted document line and
insertion metadata: page id, gap row id, inserted text, bbox, full crop path/hash, and
candidate-band crop lists must agree.
This keeps reviewed missing-line insertions tied to the actual image evidence that
prompted them, and prevents stale/tampered review TSV rows from feeding the later
training-pack exporter. It is deliberately scoped to OCR review integrity, not a new
general eval harness.
**2026-06-23 focused low-confidence review view:** the book parser now writes
`book-line-focused-review-queue.tsv` and `book-focused-review-dashboard.html` in
addition to the exhaustive canonical line-review queue and full priority queue. The
focused queue includes every critical/high/medium row and only a deterministic,
evenly spaced sample of `spotcheck_low_confidence_text` rows per page (default `25`,
configurable with `--focused-low-confidence-limit-per-page`). This is review-burden
control, not line dropping: the canonical queue still preserves every low-confidence
flag and remains available for exhaustive review/application. Page triage filters the
focused queue along with the canonical and priority queues after explicit page
rejection, and the book-output audit now verifies the focused queue hash, focused
summary counts, and focused-dashboard card count. The combined review-application
wrapper records the focused queue and summary for provenance, but it still applies the
canonical review queue; using the sampled focused queue for application would silently
skip rows. The book-output audit now also proves every focused row is present in the
canonical queue, so a detached or manually edited focused TSV cannot pass just because
its hash and card count match. The combined review wrapper refuses an explicit focused
queue as `--book-line-review-queue`; the canonical queue is the only application
source. To make the focused view usable without hand-copying edits,
`tools/merge_limbu_book_focused_review_queue.py` overlays reviewed focused rows back
onto the full canonical queue while requiring identical headers and identical
non-review/provenance fields. Only review fields are merged, and detached/tampered
focused rows are refused.
Refreshed real-book evidence:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-runner-v4-current-py311-focused-review-view-stable-pageids/`
was run under `/tmp/ocrtech-paddle311/bin/python` with the v4 router held fixed,
duplicate-source skipping, and stable `gorkhapatra-unique-current-p000*` page IDs. It
passes the book-output audit with `2` pages, `692` preserved lines, and `599` canonical
book-line review rows. The new focused queue contains `369` rows: all `319`
must-review rows plus `50` sampled low-confidence spot checks, omitting `230`
low-confidence rows from the bounded working view while keeping them in the canonical
queue. After explicit source rejection of the known non-Limbu page
`gorkhapatra-unique-current-p0002`, the p1-only projection at
`.../manual-triage-check/triaged-p1-only/` passes `audit_limbu_book_output` with zero
issues. The actual Limbu page now has `230` canonical review rows and `35` focused
operator rows (`10` must-review rows plus `25` low-confidence spot checks). The
page-triage projection now regenerates filtered review dashboards instead of leaving
the operator without a visual surface after stale dashboards are cleared; the p1-only
artifact has canonical and priority dashboards with `230` cards each and a focused
dashboard with `35` cards, with no rejected-page card. This is the packet to review
next; it does not claim recognition improved. A no-op merge of the current pending
focused queue into the full canonical p1-only queue writes
`.../manual-triage-check/triaged-p1-only/book-line-review-queue.focused-merged.tsv`
with `230` output rows, `35` focused rows consumed, and `0` changed rows; once focused
review is filled, that merged full queue is the one to pass to review application.
Codex made a conservative first reviewed projection from the focused contact sheets by
marking only obvious non-body/page-furniture rows as drops: `img-line-0179`
footer/contact metadata, `img-line-0180` date/header metadata, `img-line-0193`
portrait detection, `img-line-0001` masthead, `img-line-0002` website, and
`img-line-0039` decorative page ornament/icon. The edited focused queue is
`.../manual-triage-check/triaged-p1-only/book-line-focused-review-queue.codex-obvious-drops-v2.tsv`;
the merged full canonical queue is
`.../manual-triage-check/triaged-p1-only/book-line-review-queue.codex-obvious-drops-v2-merged.tsv`
with `230` output rows and `6` changed rows. Applying that full queue produces
`.../manual-triage-check/triaged-p1-only/reviewed-obvious-drops-v2/final-book/`: `6`
book-line drops, `317` remaining lines, `0` layout-gap insertions, and a passing
`audit_limbu_book_output` with zero issues. Exporting a reviewed training pack from
this drop-only projection correctly refuses with "no training rows were exported";
that is desired because no accepted text correction exists yet.
After the scope correction, the useful next packet is not another eval harness but a
small pending operator packet. Reusing the existing unresolved-review packet writer on
the edited focused queue writes
`.../manual-triage-check/triaged-p1-only/pending-focused-review-packet-v2/`. It has
`29` pending rows, skips the `6` already-dropped focused rows, contains only existing
book-line checks, and audits cleanly with
`tools/audit_limbu_book_unresolved_review_packet.py` (`audit_passed=true`,
`issues=0`). Every row is currently `existing_low_confidence_line` with suggested
action `correct_existing_line_or_accept`. The packet HTML is now a real review station,
not just a static contact sheet:
`.../pending-focused-review-packet-v2/book-unresolved-review-packet.html` shows each
crop with editable `review_status`, prefilled `corrected_text`, optional split fields,
reviewer, notes, local browser autosave, per-row quick actions, and an "Export reviewed
TSV" button that preserves the packet TSV schema for the existing applicator. The quick
actions accept current text, mark corrected text, drop existing lines, keep pending, and
jump to the next pending row; accept/correct refuses blank text in the browser UI.
Candidate missing-line rows now also show neighboring OCR lines around the proposed
geometry insertion point, which gives reviewers local reading-order context before
accepting or correcting a missed-line crop.
Browser autosave is only refresh protection; the exported TSV remains the review
handoff artifact. Export is browser-preflighted for missing reviewer names, blank
accepted text, machine-looking reviewer names, invalid statuses, and invalid drop/reject
rows; the command-line packet audit remains authoritative.
`tools/audit_limbu_book_unresolved_review_packet.py --packet-tsv` and
`tools/apply_limbu_book_unresolved_review_packet.py --packet-dir` now support applying
an exported `book-unresolved-review-packet.reviewed.tsv` against the original packet
provenance without overwriting the source TSV. The exact commands live in
`docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md`.
`tools/apply_limbu_book_reviewed_packet_to_training_pack.py` is now the preferred
post-review command: it applies the reviewed TSV to the cleaned book, audits the final
book, exports the reviewed-only training pack, and fails if zero reviewed training rows
are produced. This is the immediate blocker before training: a reviewer must accept or
correct those line texts in the packet HTML/TSV, then run the handoff wrapper against
the already-cleaned `reviewed-obvious-drops-v2/final-book/` base. A non-claim
mechanical smoke with one copied OCR row marked accepted exposed stale chainable
document-hash recording in the book-line/candidate applicators; after fixing those
hashes, the smoke passed reviewed packet audit, application audit, final book-output
audit, and exported `training_rows=1`. The smoke artifact was deleted and is not
training data; the same contract is now covered by a strict synthetic wrapper
integration test. The handoff wrapper now writes a `README.md` beside the report with
artifact paths and the next `split-manifest` / `train-recognizer` commands, while
stating that the handoff is not a CER/WER quality claim. A pending-only
mechanical proof now exists at
`.../pending-focused-review-packet-v2/application-pending-on-reviewed-drops-base/`:
`29` packet rows, status counts `{"pending": 29}`, `0` book-line rows applied, `0`
candidate rows applied, final line count `317`, packet-application audit passed, and
book-output audit passed. The packet splitter now also passes the reviewed packet crop
path/hash into the generated book-line review TSV as `source_crop_paths` and
`source_crop_sha256s`, so accepted packet corrections keep crop evidence in correction
metadata and later training provenance. The unified packet audit now has a reviewed
mode (`--allow-reviewed`) and the unified applicator runs that audit before splitting
or applying the TSV. Pending-only packets still pass as no-ops, but accepted/corrected
rows require explicit reviewer text that does not look machine-generated and explicit
`corrected_text` or `split_corrected_texts`; drop/reject rows require reviewer identity
and must target existing book lines. The current pending proof records
`reviewed_application_ready=false`, which is correct because no human text labels exist
yet. The unified application audit now also verifies the packet-audit file record,
`review_packet_audit_passed`, `reviewed_application_ready`, accepted/drop status
counts, and requires `reviewed_application_ready=true` whenever the application changes
book text. The unified applicator now runs that application audit itself after writing
the final report, writes `book-unresolved-review-packet-application-audit.json`, and
fails the command if the audit does not pass. Do not detour into broader scoring until
those accepted corrections exist.
The reviewed training-pack exporter now runs `tools/audit_limbu_book_output.py` on the
source book output before it writes labels and records that audit in
`source-book-output-audit/`. A real retry against
`reviewed-obvious-drops-v2/final-book/` wrote a passing source-book audit at
`.../reviewed-obvious-drops-v2/training-pack-reviewed-after-source-audit/source-book-output-audit/`
(`passed=true`, `issues=[]`) and then correctly refused the export with "no training
rows were exported" because the projection still has only drops, not accepted text
corrections. This is the desired state before human review: source consistency is
proven, but no labels are minted.
The same exporter now recognizes `candidate_line_review_insertion` metadata as
explicit review evidence, in addition to book-line corrections/splits and layout-gap
insertions. This matters for missed-line recovery: accepted candidate/missing lines
inserted through the unified review packet will now become reviewed training rows
instead of being silently skipped after application.
Direct candidate-line review application now also validates crop evidence whenever a
review row supplies `crop_path` or `crop_sha256`: both fields must be present, the crop
must exist, and the SHA-256 must match before insertion. The inserted-line metadata and
`candidate-line-insertion-pairs.jsonl` now preserve `crop_path` and `crop_sha256`, so
reviewed missed-line training rows stay tied to the crop evidence that justified them.
The reviewed-application path was hardened after this real run exposed stale metadata.
Book-line and layout-gap chainable outputs now clear inherited focused queues and
review dashboards, because old focused rows cease to be a true subset once canonical
review statuses change. They also recompute `source_book_document_json_sha256` when
rewriting the source document pointer. The book-output audit now checks markdown beside
the recorded `document_json`, while keeping `output_dir` tied to the original OCR page
bundle. This makes corrected/drop projections auditable without pretending old review
dashboards still describe them.
**2026-06-22 crop-bound correction application:** `tools/apply_limbu_book_line_reviews.py`
now validates `source_crop_paths` and `source_crop_sha256s` on every actionable
book-line review row before applying accepted/drop corrections. A stale or tampered crop
hash fails the apply step, and successful corrections carry the crop paths/hashes into
line metadata and correction pairs. This keeps the unresolved-line review loop tied to
the actual reviewed crop images, instead of accepting detached TSV edits.
**2026-06-22 operational missed-line triage:** `tools/run_limbu_book_ocr.py` now reads
the final per-page `document.json` geometry and records a `layout_gap_summary` for each
book page. Pages with unusually large vertical row gaps get the operator flag
`possible_missed_line_gap`, and pages whose geometry cannot be read get
`line_geometry_unavailable`. This is intentionally not another eval harness layer; it
is a front-door run-summary warning that a page may have preserved confident text while
still missing a visible line region. The runner now also writes
`layout-gap-review/layout-gap-review-queue.tsv` and
`layout-gap-review/layout-gap-review-dashboard.html`, with hash-bound crop bands for
each suspicious vertical gap, so the operator can inspect likely missed-line regions
directly from the run output. `tools/apply_limbu_book_layout_gap_reviews.py` now applies
accepted rows from that queue as inserted reviewed lines in a new chainable book
projection, requiring non-empty `corrected_text` and validating the gap crop SHA-256
before it writes `book.md`, `book.body.md`, `book-documents/`, `score-documents/`,
`book-run.json`, and `layout-gap-review-application-audit.json`.
`tools/audit_limbu_book_layout_gap_review_application.py` verifies that application
bundle: source run hash, review queue hash, accepted crop hashes, corrected document
hashes, score-document projections, insertion-pair count, and current-versus-inherited
inserted-line counts.
`tools/apply_limbu_book_review_queues.py` now wraps the reviewed-book operator flow:
given a top-level runner output, it applies book-line corrections first, then
layout-gap insertions, audits the final gap application, and writes the final reviewed
book under `final-book/` with source/intermediate/final hashes in
`limbu-book-review-queues-application.json`. The initial OCR runner README now prints
the exact `apply_limbu_book_review_queues.py` command for the run, and the wrapper
writes its own top-level README pointing at the final editable text, final `book-run`,
and consistency audit.
**2026-06-22 operator runner correction:** after the eval-harness work started to
outgrow the immediate product need, the default path was collapsed into one operational
command instead of another scoring layer. `tools/run_limbu_book_ocr.py` now takes a raw
book-page manifest, prepares captures, runs the selected v4 preserve-mode router over
the prepared manifest, writes book-level editable text and review queues, runs the
existing book artifact audit, and emits `limbu-book-ocr-run.json`. It does not change
detector or recognizer behavior and it is not an OCR-quality claim. The live Sirijonga
one-page smoke at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve/` produced
the same operational counts as the manual flow: `1` page, `323` preserved lines, `230`
book-line review rows, script counts `286` Sirijonga / `27` Devanagari / `4` mixed /
`6` other, and a passing book artifact audit with zero issues. Use this runner for
tomorrow's raw page batches unless a page needs manual crop/perspective values in the
manifest. The runner now also emits operator attention flags in the top-level JSON,
README, and CLI output for pages with no detected lines, all/high review burden, high
`other` script share, or failed page artifact audit. These flags are triage cues for
inspection/reshoot/review, not eval metrics.
**2026-06-22 folder ingestion path:** `tools/build_limbu_book_page_manifest.py` now
turns a directory of page images into the raw JSONL page manifest with deterministic
natural filename ordering, generated page IDs, and source image hashes. The book runner
also accepts `--image-dir` directly, generates `raw-page-manifest/book-pages.jsonl`
inside the run directory, and then follows the same capture-prep -> preserve-OCR ->
book-audit path. A real one-page folder-input smoke at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir/`
matches the manifest-based runner: `1` page, `323` preserved lines, `230` book-line
review rows, script counts `286` Sirijonga / `27` Devanagari / `4` mixed / `6` other,
and a passing book artifact audit with zero issues. This is the practical entry point
for tomorrow's captured page folders; hand-edit the generated manifest only when page
order, crop, or perspective needs per-page correction.
**2026-06-22 book review dashboard:** multi-page book runs now emit
`book-ocr/book-review-dashboard.html`, a book-level contact sheet built from
`book-line-review-queue.tsv` and the existing per-page line-crop manifests. The
dashboard is hash-bound in `book-run.json`, and `tools/audit_limbu_book_output.py`
checks both the dashboard hash and card count against the book-line review queue. The
book-line TSV now also carries source crop paths, crop SHA-256s, and crop warnings for
each review row, and the book audit verifies those crop hashes; for line-assembled or
reading-order-profiled documents, these crop references resolve back to the original
per-page OCR bundle instead of the derived document directory. The
real one-page folder-input smoke at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-1p-runner-v4-preserve-image-dir-dashboard/`
has `230` dashboard cards and `230` crop images, matching the `230` book-line review
rows. This closes the immediate review-surface gap for tomorrow's page folders: the
operator no longer has to open one per-page dashboard per page just to review weak
book OCR lines.
**2026-06-22 real 4-page folder run:** the current folder-based runner has now processed
the full 4-image Gorkhapatra input directory end-to-end at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-4p-runner-v4-preserve-image-dir/`.
It generated the raw page manifest, prepared all captures, ran v4 preserve-mode OCR,
wrote book Markdown/body Markdown, score documents, `book-line-review-queue.tsv`, and
the book dashboard, and passed the book artifact audit with zero issues. Aggregate
counts: `4` pages, `1384` preserved lines, `1198` book-line review rows, script counts
`682` Sirijonga / `64` Devanagari / `30` mixed / `608` other. Per-page counts expose
the next real failure pattern: pages 1 and 3 each have `323` lines and `230` review rows
with mostly Sirijonga output, while pages 2 and 4 each have `369` lines and all `369`
lines in review, including `298` `other` rows. Visual inspection and the source-pack
metadata showed this is not a clean four-page Limbu book target: page 2 is a Nepali /
Devanagari `नयाँ नेपाल` page, the pack records both `लिम्बू` and `माझी` candidates, and
the `-01`/`-02` image pairs contain duplicate SHA-256 content across candidates. Treat
this output as a useful end-to-end book plumbing smoke, not an OCR-quality result or a
detector failure diagnosis. `tools/run_limbu_book_ocr.py` now records per-page line,
review, script, and audit counts in `limbu-book-ocr-run.json` and the run README so this
kind of bad-page pattern is visible without opening `book-run.json`.
**2026-06-22 source-folder correction:** the operational fix is source selection, not a
larger eval harness. `tools/build_limbu_book_page_manifest.py` and
`tools/run_limbu_book_ocr.py` now support `--include-filename-regex`,
`--exclude-filename-regex`, and `--skip-duplicate-images`. The manifest summary records
discovered, skipped, duplicate, and kept images, including duplicate SHA-256 groups, so
mixed export folders cannot silently masquerade as clean book batches. The runner also
copies those source-selection counts into `limbu-book-ocr-run.json`, the run `README.md`,
and CLI output, so an operator does not have to open the lower-level manifest summary to
notice skipped pages or duplicate captures. On the current Gorkhapatra pack, filtering
to `.*-01\.png$` and enabling duplicate skipping produced
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-filtered-01-dedup-manifest/book-pages.jsonl`
with exactly one unique kept target image; the later `0002-01` file was skipped as a
duplicate of `0001-01`, and the `-02` files were skipped by the filename filter. The
practical rule for tomorrow is: first build or let the runner build a hash-bound page
manifest, inspect the summary for skipped/duplicate/mixed-language evidence, then OCR
only the selected page set. Do not spend more time expanding metrics until the input
batch is actually the book pages we intend to digitize.
The manifest builder itself now emits the same source attention flags before OCR:
`source_images_skipped`, `duplicate_source_images_removed`, and
`duplicate_source_images_present`. Use `tools/build_limbu_book_page_manifest.py` as the
cheap preflight when a folder may be an export pack rather than a clean book sequence.
The current Gorkhapatra source preflight at
`outputs/limbu-live-book-smoke-2026-06-22/gorkhapatra-current-source-preflight/`
returns `pages=4` and `source_attention_flags=duplicate_source_images_present` without
running OCR.
**Missed-line rescue follow-up:** generic secondary morphology rescue was tested and
rejected. The compact long-line rescue raised mean line F1 from `0.605257` to
`0.626903` but worsened page codepoint CER from `0.458432` to `0.464766`; broader
normal/small-fragment rescues regressed to `0.537609` and `0.832146` codepoint CER by
reintroducing noisy extra predictions. Evidence and the geometry review packet are in
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/`. The next move should split
oversized low-IoU detections and recover missed bottom/right and top/center lines, not
add another undirected rescue pass. A narrower split-wide component option was then
implemented in both the materializer and operational `parse-limbu-doc` image-line
path, but CDC scoring showed it was a no-op: every tested split-wide variant kept
`{"primary": 33}` detector source counts and the same page codepoint CER `0.458432`
(metric SHA-256 `b1d993e9fcf09a7cff18d31c3c7e43ee2ff34cd8a57eaae96526e9f977d4c5cb`).
Do not spend the next pass on simple wide-component whitespace splitting; use the
geometry review packet and failure analysis to drive page-region/row recovery. The
concrete CDC target list is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/devanagari-cdc-page-region-recovery-targets.md`:
12 misses split into 3 no-overlap rows, 4 overmerged rows, 2 partial detections, 1
low-IoU refinement, and 2 page markers. Follow-up existing-row repair tests were also
negative: same-row merge gap80 regressed page codepoint CER to `0.463975`, detected-row
split gap24/min700 regressed to `0.467933`, and detected-row split min900 was a no-op.
The detected-row split path is now wired into both the standalone materializer and
operational `parse-limbu-doc` / book-manifest image-line runs, but it is rejected as a
default by that CDC evidence; next work should target absent-row and short page-marker
recovery from page regions. Region-filtered rescue is now the first
positive result in that direction: the materializer and runtime image-line rescue pass
parsers support `region_min_x_ratio`, `region_max_x_ratio`, `region_min_y_ratio`, and
`region_max_y_ratio`, and the tight bottom-long + top-title CDC diagnostic improves
page codepoint CER from `0.458432` to `0.452890`, grapheme CER from `0.645161` to
`0.637481`, mean line F1 from `0.605257` to `0.635724`, and matched references from
`19` to `21` with 36 detected lines. Score:
`outputs/limbu-page-eval-drafts/limbu-devanagari-cdc-curriculum-linebox-2026-06-21/score-image-line-ppocrv5-detector-h31-v5-minw80-maxh80-region-rescue-bottom-long-top-title-tight/metrics.json`
(SHA-256 `1c787c10392edf4f8a27dac018b5406b3a960b963a419aa4a64e9ec1c0acc0af`). The
conservative page-marker extension is a page-CER tie and the current line-structure
best: it keeps codepoint CER `0.452890`, raises mean line F1 to `0.659689`, and
matches `22` references with 37 detections by recovering the social-page `२` marker
(score SHA-256 `19846ebae905411387ccbb88ee9520021387aac9f3debcc4ce81e88a87e1c836`).
The runtime image-line path and standalone materializer now tag conservative
bottom-margin digit-only small-box detections as `structural_role=page_marker`; a fresh
h20 structural rerun marks exactly the recovered social-page `२` marker and keeps the
same metric SHA-256, so the change is metadata-only for current scoring. It is not yet
script-count exclusion or final page-order handling. `export_document` now also writes
`document.body.md`, which omits structural blocks while preserving the full `document.md`
and `document.json`; the one-page operational social run at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/operational-parse-social-page-marker-structural`
keeps the marker in full markdown, excludes it from body markdown, binds the body file
in `limbu-output-manifest.json`, and passes `audit_limbu_output`.
The remaining h20 unresolved regions now have a textless review packet at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/`:
24 crop rows, split into 9 missed-reference regions and 15 extra-prediction regions.
Its JSONL/HTML hide reference text, suppress 2 exact machine-text echoes of hidden
reference strings, and currently have zero hidden-reference text leaks against the
failure analysis. The packet now has an independent audit tool,
`tools/audit_unresolved_page_review_packet.py`; the current audit artifact
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/unresolved-region-review-h20-page-markers/unresolved-region-review-audit.json`
passes with 24 rows, 0 hidden-reference leaks, and 0 issues (SHA-256
`33105c0144c5cfa56ee20554a6aee82434f9c4cb6c19a60cc9533882a918db5b`). This
satisfies the review-surface requirement for the current CDC diagnostic, not the
underlying missed-line recovery requirement.
A targeted h20 + same-row merge gap120 follow-up was tested for the remaining
top-title/fragment near misses and rejected. It raises mean line F1 to `0.694815`, but
page codepoint CER regresses to `0.465558`, grapheme CER to `0.654378`, matched-line
CER to `0.190031`, and reading-order pair accuracy to `0.571429`; the refreshed
failure analysis still has 9 unmatched references and dominant missed-reference error
mass. Do not chase broader same-row merging as the next default path.
A narrower capped same-row merge (`gap160`, `width500`, `y_tolerance=25`) was also
tested to merge only short top fragments. It produced 34 lines with 2 row merges,
improved micro line F1 to `0.707692`, and reduced missed references to `8`, but page
codepoint CER still regressed to `0.460808`, grapheme CER to `0.643625`, and
matched-line CER to `0.185259` (score SHA-256
`61129eaa91445c0d8a86f66c5a5827ffa342b331bffe032d12f60d9476ad0b32`). Reject it as a
default path too; merged crops are damaging recognition even when the geometry looks
better.
The lower-height h10 marker extension recovers more single-digit markers and matches
`23` references, but it regresses page codepoint CER to `0.454473`, grapheme CER to
`0.639017`, WER to `0.844707`, and reading-order pair accuracy to `0.538462`; keep it
diagnostic-only, not the default path. The refreshed failure analysis is in
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-failure-analysis-devanagari-cdc-region-rescue-bottom-long-top-title-tight/`;
missed reference text is still dominant (`45.78%` of codepoint error mass), so this is
progress, not finished book OCR.
Combining h10 markers with source-break post-recognition line assembly was also tested
and rejected as the default: page codepoint CER is `0.452890`, line recall rises to
`0.806452`, but precision falls to `0.675676`, extra predictions rise to `12`, two
blank predictions appear, and reading-order pair accuracy is only `0.553191`. Score:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-cdc-h10-page-markers-line-assembly-gap160-width500/metrics.json`
(SHA-256 `6ccf1bce5263f0eed1fc966a6d2f9c3c6567534fdd750e7a05549462a12b6c57`).
The detected-tall geometry candidate has now been scored and rejected as the default.
Operational `parse-limbu-doc` supports primary/rescue `min_aspect_ratio` and
`max_aspect_ratio`, plus opt-in `image_line_split_detected_tall_*` settings. The
geometry probe on the science page split the known bad `rescue-1` box
`[319, 1856, 1075, 115]` into `[319, 1856, 1052, 37]` and `[317, 1944, 1051, 29]`.
Probe:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-split-geometry-probe/detected-tall-split-geometry-probe.json`
(SHA-256 `51b364b4211f40d54714c903b9038fab284fc83e7853efa85ccc3ebb32225c64`).
The Paddle follow-up was run through the book wrapper in a dedicated local Python 3.13
Paddle environment. It produces 3 pages, 38 source lines, 28 review rows, and a passing
book artifact audit (`limbu-book-output-audit.json` SHA-256
`ceb9d681e4ee50ac48880f41b5c0dc25e084cd4cfcc43b0774d573366634210e`), but the score
regresses page codepoint CER to `0.478226`, grapheme CER to `0.674347`, and WER to
`0.882958`. It improves line recall/F1 to `0.838710` / `0.776119` and cuts missed
references to `5`, but matched-line CER worsens to `0.191242` and reading-order pair
accuracy is only `0.578431`. The candidate admission gate rejects it
(`admitted_operational=false`, admission SHA-256
`2d56eaaf0071fa17341d9905aaa7bc46d5bb89fa40c1428e936645e7727665dc`) because it
regresses against the `0.458432` codepoint-CER baseline and misses the `0.60`
reading-order threshold. Keep detected-tall split as recall/review diagnostic evidence,
not default book OCR text.
That reuse path now exists: `tools/prepare_candidate_line_review_packet.py` compares a
selected document set against a secondary candidate document set and writes
candidate-only line crops whose max selected-line IoU is below a configured threshold;
`tools/audit_candidate_line_review_packet.py` checks hashes, crops, and row shape. The
detected-tall-vs-current-best packet at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-only-review-packet/`
has 4 rows (maths marker `१`, two science bottom-row split candidates, and science
marker `२`) and passes audit with 0 issues (packet SHA-256
`f20438d05d158377387a05df9197cc3a6e7e9afec573a60668c1d429d1b83128`, audit SHA-256
`919615e48fe66c24810e6137b5bb65dd64b348ebfee0b5817af38b47f38c2f40`). This is now the
preferred way to exploit high-recall rejected geometry: review it as missing-line
evidence without contaminating selected book text.
Accepted candidate-only rows now have an application lane:
`tools/apply_candidate_line_reviews.py`. It inserts reviewed secondary-candidate lines
into a new book projection with provenance metadata, writes full/body markdown,
score-documents, and insertion pairs, and leaves the measured selected run untouched.
The real detected-tall packet is still all pending; the pending smoke at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-candidate-line-review-application-pending/`
has `review_rows=4`, `applied_count=0`, `skipped_count=4`, empty insertion pairs
(SHA-256 `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`), and
preserves selected `book.md` / `book.body.md` byte-for-byte. Audit SHA-256:
`95521bc3c5ad86bd6cdf19350cfa1323a972ce817ee363451ff79349ccc4f69f`. The application
output now also has an independent artifact audit,
`tools/audit_candidate_line_review_application.py`; the real pending application audit
passes with 0 issues and SHA-256
`0e6addc547c1ef8df3ae9fe54d6599a16341fb2d57168d3b785da4cbe38cf452`.
The pending application projection has also been scored and admitted through the same
book candidate gate, using that independent application audit instead of a normal book
audit. The score is byte-identical to the current selected h20 source-break
line-assembly metrics (SHA-256
`184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244`): page codepoint
CER `0.446556`, line F1 `0.750000`, missed/extra/blank counts `7` / `9` / `0`, and
reading-order pair accuracy `0.611765`. Admission passes operationally with
`admitted_operational=true`, `claim_safe=false`, and SHA-256
`eb0e559908046322343a7507a0143ae2d724876c069eb2528f0d1950085fd5b5`. This proves
reviewed candidate-line application outputs can use the same audit/score/admission
lane; it does not add missing lines until rows are actually accepted.
An automatic candidate-line selector now exists, but it is not a default OCR policy.
`tools/select_candidate_line_reviews.py` implements `page_marker_v2`, limited to
bottom-margin digit-only structural page markers with confidence, selected-line IoU,
size, and duplicate-text guards. On the real detected-tall packet, v2 selected only
the maths marker `१` and rejected the other 3 rows (selection report SHA-256
`928aaecd9a102c5615e1085b3d29a80188d419c35e17bbfc9c743876f7e20872`; selected JSONL
SHA-256 `8152ba724a67a4ef25296c6dcd47a02c7050262356065d6c0c0979fa4f366867`; rejected
JSONL SHA-256 `a3a571ad12c8efade971d0b89baa0ef9b1345cb15d97beba3ec284f5089b3e85`).
The applied v2 projection passes application audit (SHA-256
`b0e377bec4f0fca9f4506fa9126a65c57014adf7d419a04e08f98d50bf3b8743`) and improves line
recall/F1 to `0.806452` / `0.769231`, but page codepoint CER regresses to `0.448139`
and reading-order pair accuracy drops to `0.553191`. Admission rejects it
(`admitted_operational=false`, admission SHA-256
`9cd826a51b38aa29facab4ec332a4084421ac25bf26365dc04f954e6f1b5a76f`). Treat this as proof that page
markers need separate structural export/scoring semantics; do not insert them blindly
into the selected editable text stream.
`tools/run_page_ocr_eval.py` now exposes that separation as diagnostics, without
changing the all-line metrics used by admission. It counts non-text
`metadata.structural_role` predictions, matches, unmatched predictions, exact text
matches, and per-role totals. Structural-metric replays show current h20 assembly has
one exact matched structural page marker (metrics SHA-256
`f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`), while the v2
auto-marker application has two exact matched structural page markers (metrics SHA-256
`2634d0f70f6dac3cac08844ea1530aff08c26271a92a7eb2c725a879415ce5b9`). That added marker
explains the improved line recall/F1 (`0.806452` / `0.769231`), but it does not justify
promotion because page CER and reading order still regress. Use these structural
metrics as evidence, not as a substitute admission gate.
The candidate-only review lane now has an explicit oracle/reference-overlap diagnostic
for body-line recovery: `tools/select_candidate_line_reviews_by_reference_overlap.py`
selects rows only when they overlap currently unmatched reference lines, and labels the
output `oracle_reference_overlap_v1`. On the real detected-tall packet it selected one
text row, the science bottom line, with IoU `0.860889` to
`limbu-devanagari-cdc-science-p0009-l0009` (selection SHA-256
`d11dcdc1cc0b07626e6d85b97356c86cadd670796b9e926708e2be7585344d05`). Applying that row
with reference text as corrected text gives the review-ceiling score: page codepoint CER
`0.420428`, line recall/F1 `0.806452` / `0.769231`, missed/extra counts `6` / `9`, and
reading order `0.634409` (metrics SHA-256
`247c4186059047fc9de16019f358172bc46261de497127581f63335f986cb521`). Applying the
same row with candidate OCR text gives page codepoint CER `0.445764` but matched-line
codepoint CER worsens to `0.191413` (metrics SHA-256
`21fd79e6a0645c0084cac959743354f52c224b28b15e4e60be7f56e3ac664d75`). The audit and
admission path now blocks these oracle/reference-assisted applications from operational
admission: application audits record `operational_review_safe=false` and admissions
reject them (`c8f69e7d011c217e74218ffe280e901420ab99ebefa7f329344af776a6e779c7` for
reference text, `2a97b7f01425efdd514887324c8c23c6b4dd46c48376b99c2299fad736d88b45`
for candidate text). This proves the geometry surface contains real missed-line
evidence, but direct candidate text is not enough; the next useful path is reviewer
correction or better recognition on detected-tall body candidates.
A non-oracle reviewer priority queue now exists for that next path:
`tools/prioritize_candidate_line_reviews.py`. It consumes the candidate-only review
JSONL and ranks rows using only packet-visible evidence: structural role, candidate
text, confidence, selected-line IoU, bbox size/position, and crop links. The real
prioritized queue at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/`
has 4 pending rows: 2 high-priority body-line candidates and 2 structural page-marker
checks. Artifact SHA-256 values: prioritization JSON
`ef3fc75efc52597c97ea2e28196164b31ef3ceb18414ef5b47e559cc4343e51a`, JSONL
`9a53ad3d3ada7e61211d5406ce35c9ed9a10e86a41ea09ea0f4edcb3766ad49e`, TSV
`753adcd61f70f490b0429d6ff9d25ff88b0bc1ac8ec20a2f9ae21ec1f514fa19`, HTML
`5fbd7f39801ac7e29b0503314475ef26e6e61345b26330720bf840ad1e2dcdef`. The independent
queue audit at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review/prioritized-candidate-line-review-audit.json`
passed with 0 issues (SHA-256
`d2fe0e33e3bd5f78698632892581651a3e95332936b7e9ddfa926c1ecc3fcd22`), confirming 4
source rows, 4 JSONL rows, 4 TSV rows, no oracle/reference-assisted review rows, and
`operational_review_queue_safe=true`. Ranks 1-2 are the two science bottom-row text
candidates; both remain `pending` with suggested action
`transcribe_visible_line_or_reject`. This is the safe reviewer-facing artifact to use
before applying candidate lines; it is not an automatic selector and does not change
the current selected OCR candidate.
A text-focused crop probe now exists for those body-line rows:
`tools/prepare_candidate_line_text_crop_probe.py`. It is non-oracle and only prepares
recognizer/reviewer triage crops; it does not alter selected OCR output. On the real
prioritized queue it processed 2 body-line candidates and refined both: rank 1 was
top-bleed-trimmed from `1100x85` to `1100x63`, and rank 2 was trimmed left of a
detected horizontal rule from `1099x77` to `466x77`, removing the rule and right-side
illustration (probe SHA-256
`7f1dfc09d4cf612e7516dbdea07eb6bc19608e637ad11efe41e8a8112393c79a`, JSONL SHA-256
`a7d7cced3cdca1a223aa6c6d00ccc70acabee90183107eac6b70ff61b630f011`, label-file
SHA-256 `47b3594dac7a97f516d21f271c1fe1e794f6d621bfc17539d1cf6b8e540ef4e0`, HTML SHA-256
`10db1161085a49e722ad373b88ff9656aaf87b5423dad2ec88e8f1c48f1f18b6`). The Devanagari
PP-OCRv5 line recognizer was then run locally over the refined crops in both
`.venv-paddle313` and `/tmp/ocrtech-paddle311`; both produced identical predictions
(TSV SHA-256 `734b208860106d84a7dbd7473e18927293a29b51e8b67298e173c5b2c626bbee`, 2 rows,
0 failures). The original-vs-refined comparison at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-crop-recognizer-probe-comparison/`
shows this is not promotable: max probe confidence is only `0.469282`, rank 1 improves
from `0.251062` to `0.469282`, rank 2 improves from `0.363574` to `0.390961`, prediction
text changes twice, and
`automatic_application_safe=false` (comparison SHA-256
`18cdfb099ae6d93ee96ee350faec0305837349d65142ebaaad05db1553c04eb7`, JSONL SHA-256
`f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d`). This confirms
the refined crop is useful reviewer/recognizer evidence, not an automatic OCR-quality
improvement.
The application path now supports that TSV workflow directly:
`tools/apply_candidate_line_reviews.py --candidate-review-file` accepts JSONL or TSV
review files, and `tools/audit_candidate_line_review_application.py` audits both. A
real pending TSV smoke at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/detected-tall-prioritized-candidate-line-review-application-pending/`
left all 4 rows pending (`applied_count=0`), preserved selected `book.md` and
`book.body.md` byte-for-byte, passed application audit with
`operational_review_safe=true` (audit SHA-256
`e36036c0e3a9177e7c526465cf16c4a066f54ae37fe920bfb97c1a3bb9278c5c`), scored with
current-best metrics (score SHA-256
`f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`), and passed
operational admission (SHA-256
`453226159011a944a606e0c951c5b7dceda3bee73c76f44d195c44e2d61672c1`,
`claim_safe=false`). The next non-oracle improvement requires actual accepted reviewer
rows in this TSV, not another automatic promotion.
The first multi-page book wrapper now exists as `tools/parse_limbu_book_pages.py`. It
reads an ordered JSONL page manifest, runs the existing `parse_limbu_document` path for
each page, and aggregates per-page bundles into `book.md`, `book.body.md`,
`book-review-queue.tsv`, `book-run.json`, and scorer-compatible `score-documents/`
with page hashes, audit status, script counts, total line counts, and review-row
counts. A persistent sidecar smoke run at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-book-sidecar-smoke`
has 2 pages, 4 lines, 1 aggregate review row, 0 page audit failures, and script counts
`devanagari_limbu=1`, `limbu_sirijonga=2`, `mixed_limbu_devanagari=1`, `other=0`
(`book-run.json` SHA-256
`7f9da4cec883e0fc9e72e16f62c1ac633046daea4c86091d25129643376f8819`). The selected
h20 structural CDC diagnostic has now been replayed through that book wrapper using
real CDC page images and sidecars derived from the materialized selected page
documents. The prep artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-replay-inputs`
(summary SHA-256 `c28d0511bc42b855125c8e6a2063715ff15a0d085869a2107e32e7dfb4f411cf`);
the book output is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay`
(`book-run.json` SHA-256 `9654cebe89ab68618cf5bf36419e5bd3a9513195faf9d4f9eb4ce5481adb3e8d`).
It has 3 pages, 35 lines, 27 aggregate review rows, 0 page audit failures, and script
counts `devanagari_limbu=35`, `limbu_sirijonga=0`, `mixed_limbu_devanagari=0`,
`other=0`. The score projection evaluates cleanly against the replay manifest with
3/3 scored pages, 0 missing documents/references, 0 source-path mismatches, page
codepoint CER `0.446556`, page grapheme CER `0.626728`, matched-line codepoint CER
`0.175130`, and matched-line reading-order pair accuracy `0.600000`
(`score-multi-page-cdc-h20-structural-book-replay/metrics.json` SHA-256
`1a353b2d06b89caf081d190ad79add2eee57d1d51b70cb6144be40e53d082d75`). The replay prep
now drops blank selected-output lines by default and records them; in this CDC replay
it dropped 2 science-page blank lines (`img-line-0006`, `img-line-0007`). The scorer
now also exposes the objective's line-count rates directly: 31 references, 35
predictions, 22 matches, 9 missed references, 13 extra predictions, 0 blank
predictions, micro line precision/recall/F1 `0.628571`/`0.709677`/`0.666667`, and
missed/extra/blank rates `0.290323`/`0.371429`/`0.000000`. This is replay evidence
from existing selected outputs, not fresh OCR inference, but it closes the book-level
packaging/scoring path on real pages.
The replay also exposed a body-export bug: sidecar-replayed structural page markers
could be grouped into body text blocks. `src/ocrtech/structure.py` now emits
structural lines as their own blocks before normal paragraph grouping, so full
`book.md` keeps the social-page `२` marker and `book.body.md` omits it. The underlying
OCR quality is still not solved: 27 review rows remain and matched-line CER is
unchanged from the h20 diagnostic.
A confidence-threshold replay sweep confirms that pruning is not the next main lever.
Using `tools/filter_page_ocr_documents.py` on the CDC h20 structural book replay,
`confidence >= 0.20` removes one science-page false positive and improves micro line
F1 to `0.676923` while tying page CER at `0.446556`; it is only a possible
review-triage cleanup. `confidence >= 0.30` and `0.40` improve precision and F1 but
worsen page CER to `0.447348` and `0.448139`, and `confidence >= 0.50` drops matched
lines, reducing recall to `0.645161`. The sweep is documented in
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md`. Do not promote a
confidence threshold as default book OCR output; recover missed/split page lines
instead.
The first post-recognition line assembly diagnostic now exists as
`tools/assemble_page_ocr_lines.py`. It merges adjacent same-row OCR line records after
recognition, preserving source line ids/texts/bboxes/confidences in metadata, and uses
no reference text. On the CDC h20 structural book replay, `gap160 width500` merged only
two top-fragment pairs, reducing 35 OCR lines to 33. It improves micro line
precision/recall/F1 to `0.727273` / `0.774194` / `0.750000`, drops missed references
from `9` to `7`, and drops extra predictions from `13` to `9`. After preserving
original block order, the source-break variant keeps page codepoint CER tied at
`0.446556` and page grapheme CER tied at `0.626728`, while matched-line CER becomes
`0.177052`. The space-join variant is better as editable one-line text but nudges page
CER to `0.447348`; combining space-join assembly with `confidence >= 0.20` worsens page
CER to `0.451306`. Treat source-break assembly as the current best score projection
for line structure, and treat space-join assembly as an export/review question. The
source-break artifact has a replay audit from `tools/audit_page_ocr_line_assembly.py`;
it passes with 0 issues and SHA-256
`69eb0db004f0bbabdead620bf63dfc138fb02deb8e3003fb9ec668a88d26afb1`. Artifacts are
documented in `outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/README.md`.
`tools/parse_limbu_book_pages.py` now exposes this as an opt-in operational book
projection with `--line-assembly`. The implementation deliberately does not mutate the
audited per-page OCR bundles; it writes `line-assembly-source-documents/`,
`line-assembled-documents/`, `line-assembly-manifest.jsonl`, an assembly replay audit,
and points `book.md`, `book.body.md`, and `score-documents/` at the assembled
documents. It also writes `book-line-review-queue.tsv`, an assembly-aware review
surface that groups source review rows by editable book line while preserving source
line ids. The default book-wrapper assembly settings are the measured source-break
projection (`y_tolerance=25`, `max_gap=160`, `max_merged_width=500`,
`max_component_width=250`, `min_vertical_overlap_ratio=0.30`, newline joiner).
The integrated CDC replay artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/multi-page-cdc-h20-structural-book-replay-line-assembly-integrated`
(`book-run.json` SHA-256
`f2591d47ace1ceb5bc676d3d8c645f6905ab250e1e04fcfbe083ac4cac9f1e25`). It has 27
source review rows and 26 book-line review rows because the two reviewed maths top
fragments collapse into one assembled book line. Its score has the same metric SHA-256
as the standalone source-break diagnostic,
`184783816a6f45058d134ddb9ad50818001427fd5f2bf305d153ad38333e9244`.
The aggregate book bundle now has an independent artifact audit,
`tools/audit_limbu_book_output.py`; the integrated CDC replay audit passes with 0
issues (`limbu-book-output-audit.json` SHA-256
`f5470c9ca04d7a3004f8ed1e3ea3fa280da768f29b65e1fd60fa7f8d42f2f6f5`). This proves
book-run hashes, aggregate markdown, review queues, score projection, page output
audits, and line-assembly audit consistency. It does not certify OCR quality.
Accepted `book-line-review-queue.tsv` edits can now be applied with
`tools/apply_limbu_book_line_reviews.py`, which writes a corrected editable book-line
projection plus `book-line-correction-audit.json` and
`book-line-correction-pairs.jsonl`. The correction is intentionally applied to the
assembled book line and records source line ids; it does not pretend that a reviewed
assembled-line edit can always be split back into original OCR fragments.
`tools/admit_limbu_book_ocr_candidate.py` now gates book OCR candidates against the
objective metric surface: book artifact audit, page CER/WER, line precision/recall/F1,
reading-order accuracy, blank/extra/missed-line rates, and improvement over the
`0.458432` CDC baseline. The integrated CDC line-assembly run passes this operational
gate (`limbu-book-ocr-candidate-admission.json` SHA-256
`8e0110ad320cbe09f37893a30725c25db0cd3d40a2b1c77cc8f0b0ab982c6b9d`) with page
codepoint CER `0.446556`, line F1 `0.750000`, and absolute codepoint-CER improvement
`0.011876`. It remains `claim_safe=false` because the eval has `claim_eligible_rows=0`
and only 3 scored pages; this is the current operational candidate, not a final claim.
The candidate gate now also supports explicit grapheme-CER and WER non-regression
baselines (`--baseline-grapheme-cer`, `--baseline-wer`; zero regression allowed when
supplied). This was added after the recognizer-probe application exposed a real gate
hole. Applying the rank-1 refined-crop machine prediction improves page codepoint CER
to `0.442597`, line recall/F1 to `0.806452` / `0.769231`, missed references to `6`,
and reading order to `0.634409`, but it regresses grapheme CER to `0.639017`, WER to
`0.866565`, and matched-line CER to `0.186047`. The paired comparison against the
current best is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-detected-tall-recognizer-probe-vs-current-best/comparison.json`
(SHA-256 `5dff9bbce7158c56fd0489452f60ddd30f8b3cd04ddf31301852665ef4e5fbc8`) and is
not claim-safe: codepoint improvement is only `0.003959` over 3 pages with CI low
`0.000000`, while grapheme CER regresses by `0.012289`. The strict multi-metric
admission at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/admission-detected-tall-recognizer-probe-application-conf045-delta005-strict-multi-metric/limbu-book-ocr-candidate-admission.json`
(SHA-256 `812fb6e5d19553441e97eada4154879c369e4fcd3924d5717e537f77ef066cdb`) correctly
rejects it for grapheme and WER regressions. Keep the h20 source-break line-assembly
integrated projection as current best; the recognizer-probe row is review evidence,
not a promotable OCR policy.
The missing-line review evidence is now visible at book level instead of living only
beside the selected OCR artifact. New tool:
`tools/prepare_limbu_book_missing_line_review_queue.py`. It binds candidate-only rows
from the detected-tall prioritized queue to the current best book pages, validates crop
paths, hashes each crop, and writes a separate pending review queue without altering
`book.md` or `score-documents/`. Current code also writes `page_context_path` and
`page_context_sha256` for each candidate when the source page image is available, with
the candidate bbox highlighted on the full page so the reviewer can reject marginalia,
rules, or spacing before accepting an insertion. Newly generated queues also include
`book-missing-line-review-dashboard.html`, a static reviewer dashboard that shows each
candidate crop next to its highlighted page context and is hash/card-count bound in the
queue report. This is reviewer workflow for book OCR cleanup, not a new eval-harness
track. The real historical artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-detected-tall-prioritized/`
with 4 rows: 2 high-priority body-line candidates and 2 structural page-marker
candidates. Report SHA-256:
`081cabed352868c26ad5d35fbc77bafc6d95b8ff27281de4d39484e15eadfc20`; TSV SHA-256:
`4ee99c552a00282e3f1bee0b4511d23b90b072d51adac15e829f9054d7237bf5`; Markdown
SHA-256: `fd318583ec70ba178fce876cb6f622ca9e7e35ac2aee77a4cf4af754ad66d01a`. It now has
an independent audit via `tools/audit_limbu_book_missing_line_review_queue.py`; the
real audit passes with 4 pending rows, 0 accepted rows, 4 verified crop hashes, no
issues, and SHA-256 `13c9ba67c47ac3bc9159606c72d3e77ce3d4fac1657ea563102d6b7078b49a58`.
Newly generated queues also audit any recorded page-context image hashes.
This is movement toward the objective's review-packet requirement, not an OCR-quality
promotion. The next real quality step is still reviewer acceptance/correction or a
recognizer/layout path that converts these rows into non-regressing page text.
That reviewer-to-application lane now accepts the book-level missing-line TSV directly:
`tools/apply_candidate_line_reviews.py` understands the queue aliases
`candidate_row_id`/`candidate_sample_id` as `row_id`/`sample_id`. A pending smoke from
the audited queue produced
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-missing-line-review-queue-application-pending/`
with `applied_count=0`, `skipped_count=4`, empty insertion pairs
(`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`), and a passing
application audit (SHA-256 `cd824e7d7b1b7f7a4a439fe398716c9849acc2bcd3694a3f73124b6466268488`).
`book.md` and `book.body.md` are byte-identical to the current best h20 assembly
(`6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3` and
`7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff`). This proves
accepted/corrected missing-line reviews can flow into the existing insertion projection
without changing pending OCR output.
There is now one unified unresolved review packet instead of separate queues:
`tools/prepare_limbu_book_unresolved_review_packet.py` combines the current best
book-line review queue with the detected-tall missing-line queue, renders crops for
existing low-confidence book lines from source page images, copies candidate-only
missing-line crops, and records crop hashes in one TSV/HTML packet. The real packet at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/`
has 30 rows and 30 verified crop files: 26 existing low-confidence book lines plus 4
candidate-only missing-line candidates. Report SHA-256
`c49c5d69105b24d36556b414f187626516353d2fc15de53fd5f098e7ad0d3adb`; TSV SHA-256
`d03c811db6fbac072491dd63bd48b36329eb30f873fac0f31495a6f907486a58`; HTML SHA-256
`f7f1f514fe77c92d763d8a59d19ce14c39019123b14e0aeecf6794caa077b45f`. A direct
integrity check over the TSV found 0 missing crops and 0 crop-hash mismatches. This is
now independently audited by `tools/audit_limbu_book_unresolved_review_packet.py`; the
real audit passes with 30 pending rows, 30 verified crops, no issues, no warnings, and
SHA-256 `3254beec46d99f9c5b6af9c72b39feefca8d97948e7e60f126088ee949886289`. This is
the current reviewer-facing packet for unresolved lines; it does not use reference text
or change OCR metrics.
The packet-to-application bridge is now wired. `tools/prepare_limbu_book_unresolved_review_packet.py`
preserves `candidate_structural_role`, so candidate page markers remain distinguishable
from body text after review. It also carries explicit `split_corrected_texts` and
`split_bboxes` columns for reviewer-approved split-line repairs. The regenerated
packet has report SHA-256
`3af6d2b767aca7d566529a01f909e850231fca33cba5636c9b042008b07ed5c4`, TSV SHA-256
`9ecd72b9f804124357cef566e58cc655cac676d0e4c81421093aa6a98d416fed`, and audit
SHA-256 `396dc11cdb95ed3d632698fa9bf777e65f0ba260abbcd3debe1be2fec390e91c`
(`audit_passed=true`, 30 pending rows, 30 verified crops, no issues/warnings). New
tool `tools/split_limbu_book_unresolved_review_packet.py` splits the unified packet
into the two applicator-specific TSVs under
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-current-best-plus-detected-tall/split-for-application/`:
book-line TSV SHA-256 `83a40bdd0863d2cb38ed347c100493e24dcbf71b48e01b2d2fb373f6d2fbd890`,
candidate-line TSV SHA-256 `d6b5426520cccb95be66b9f0ef8c7daf220aa5fd0d9f4896b3682678fa6bca39`,
split report SHA-256 `586c688f704b1cc09bd93c096d51dc8ab1c7b252cc1cf0b6ae0eb436cf323050`.
`tools/apply_limbu_book_line_reviews.py` now writes a chainable corrected
`book-run.json`, so the candidate-line inserter can run after book-line corrections
instead of requiring the original source run. The pending chain source run SHA-256 is
`738e38a500450f7bd2fb1a516e1a42d08b0e688af9b6bb7b89ab87dfe71e07ef`; pending
application audit SHA-256 values are
`d06a569829174db6aa680e4a9716b9587485fcc6487860cd147fc5897b3e9411` for book-line
correction and `09dfec36a41bff5ee6c1b0287099734da7de264379f5d544da3cce16169c3eaf`
for the chained candidate-line application. Pending-row smoke applications with
`tools/apply_limbu_book_line_reviews.py` and `tools/apply_candidate_line_reviews.py`
skipped all rows and kept `book.md` / `book.body.md` byte-identical to current best
(`6727bcb7a25b799afaefe1c1fbf3db81541915f65731dda39b7fe3c5995debc3` /
`7b517c1e19bc79783f59dfffbd795d5c728355b9b2f4895179876b8c12dc18ff`). This closes
the mechanical gap from one reviewer-facing packet to a corrected-and-inserted book
projection, but it still does not improve OCR until rows are actually reviewed.
The review workflow is now one command via
`tools/apply_limbu_book_unresolved_review_packet.py`: split unified packet, apply
existing book-line corrections, then apply candidate missing-line insertions against
the corrected projection. `tools/apply_candidate_line_reviews.py` now writes a final
chainable `book-run.json` with updated page/document paths and line counts. Real
pending application output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-application-pending-unified/`.
`tools/apply_limbu_book_line_reviews.py` can now deliberately split one accepted
existing book-line row into multiple `TextLine` records via `split_corrected_texts`,
using reviewer-supplied `split_bboxes` when available. Wrapper report SHA-256
`d557307628c2b2c39a9df05d1dba056608003f10999021990457be163c70fbb6`;
final `book-run.json` SHA-256 `b9b7696471e28ca702385987a00ae05fa8a9df3b6f2018d894d75c1d83f421aa`;
final independent candidate-application audit SHA-256
`245b56e4be9e9f1c89f777b45d1f7583f113c9e13d113dbaf412fb4ac2cd0f34`. The wrapper
itself is now audited by `tools/audit_limbu_book_unresolved_review_packet_application.py`;
real audit SHA-256 `cb6e3b9748c20282ca045df766c8220293ef274bbf7cc3f665717be6a1fc2913`
with `audit_passed=true`, `operational_application_safe=true`, 30 packet rows, 0
accepted rows, 0 book-line applications, 0 candidate-line applications, 0 issues, and
0 warnings. The real run split 30 rows cleanly (26 existing book-line, 4
candidate-missing-line), applied 0 because every row is still pending, and kept final
`book.md` / `book.body.md` byte-identical to current best. This removes operator glue
from the review loop; it does not promote text until rows are reviewed.
The missed-reference review coverage audit now exists:
`tools/audit_limbu_book_missed_line_review_coverage.py`. It compares frozen page-eval
`reference_unmatched` rows from `line-alignment.csv` against the unresolved review
packet by bbox overlap, so it is diagnostic-only and must not leak reference text into
OCR output. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-current-best-plus-unresolved-packet/`;
report SHA-256 `743d5c446797e0df0c360a78cea516bbcbb42cfb8de70843504ee0ac784ded07`, TSV
SHA-256 `ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37`.
Result: all 7 frozen unmatched reference rows have review-packet coverage at the
region level: 2 `covered_by_candidate_missing_line`, 2
`covered_by_existing_review_region`, and 3 `swallowed_by_existing_review_region`.
This changes the next engineering move: stop chasing broad detector recall for this
candidate. The remaining missed-line failures are mostly geometry/segmentation inside
already surfaced regions, especially oversized maths rows swallowing short right-side
fragments. Next work should target bbox tightening / row-split handling for existing
book lines, then rescore on the frozen pages.
`tools/prepare_limbu_book_line_split_suggestions.py` now generates a bounded review
aid for that split work. It reads the unified unresolved packet, verifies crop hashes,
and emits visual column-projection subspans for oversized pending `existing_book_line`
rows so a reviewer can populate `split_corrected_texts` / `split_bboxes` without
starting from a blank crop. It also writes an HTML review page with original crops
beside overlay PNGs. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-suggestions-current-unresolved-packet/`;
report SHA-256 `428eb35644115487aa48530db7b27d3854fcdb3c2603c49f1b3376c5fdba2f6d`,
TSV SHA-256 `b5890b32a355718666530c23085197e7a2404524bdbb78f04bc8da6921681c43`,
HTML SHA-256 `a9b416efb8ea5045e6bdd8af9723916274914b95f30621b05c616b84e0c129d1`.
It now has an independent audit via `tools/audit_limbu_book_line_split_suggestions.py`;
audit SHA-256 `9b7bdf8ae83ed79b2ee36b3d8031ebe26ce930a6321f996761bbd8dab9f81671`,
`audit_passed=true`, `operational_review_aid_safe=true`, 19 suggestion rows, 109 total
visual subspans, 19 overlay PNGs, 0 issues, and 0 warnings. It produced 19 suggestion rows, including the swallowed-row packet IDs
`existing-0005-limbu-devanagari-cdc-maths-p0008-img-line-0007` and
`existing-0011-limbu-devanagari-cdc-maths-p0008-img-line-0013`; the narrow
right-fragment case `existing-0007-limbu-devanagari-cdc-maths-p0008-img-line-0008`
did not produce a multi-segment suggestion. Boundary: this is not automatic OCR
recovery. Some suggestions are word/span splits or non-text visual spans, not logical
line splits, so reviewers must verify/edit them before applying.
`tools/prefill_limbu_book_split_suggestions.py` now closes the next operator-glue gap:
it copies those audited visual suggestions into a new unified unresolved packet by
pre-filling `split_bboxes` for matching pending `existing_book_line` rows, while
leaving `review_status`, `corrected_text`, and `split_corrected_texts` unchanged. Real
output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled/`;
report SHA-256 `9bc4d3f245133bc572c218536425ce3935b0839a5f0983176dce9853865437e7`,
TSV SHA-256 `b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4`.
It has 30 packet rows, all still pending, with 19 prefilled `split_bboxes` rows. The
prefilled packet was run through the existing one-command wrapper at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-split-prefilled-application-pending/`;
wrapper report SHA-256 `507bc6d51b6d1a1c299ec62e9f3ddaeb12773ecdbcda827caaaa7168f5a26598`,
wrapper audit SHA-256 `13763cd2fdb437e7addb6a830a808a957a756b1ed4eb88aa3fafb7e6891d1be1`,
final `book-run.json` SHA-256 `abb937646cfa736da056dc46d5edfba57392435b6be642388a16f6cd783a88ca`.
The wrapper audit passes with 30 packet rows, `accepted_counts={}`, 0 book-line
applications, 0 candidate-line applications, 0 issues, and 0 warnings; final `book.md`
and `book.body.md` remain byte-identical to current best. The next improvement now
requires actual reviewed split text, not another bridge script.
`tools/prepare_limbu_book_split_transcription_worksheet.py` now turns that prefilled
packet into a segment-level worksheet so reviewers do not have to hand-build JSON
arrays. It crops each suggested segment from the source page image, records crop
hashes, and writes TSV/HTML grouped by source packet row. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-current/`;
report SHA-256 `5eaa08ad0b2535d21989f2ece15dc65d8d36a46695670b5c326776b72f8cb01b`,
worksheet TSV SHA-256 `0b6e75a24515d404a7d8e33a2097b882f99af22732aca6eee1fdf3f1d84f2c0e`,
worksheet HTML SHA-256 `8da8216f287924df90cd206153e64cb9f75bdf35fe8f5cd6ccafc5c274732ebd`.
It contains 19 source packet rows, 109 segment worksheet rows, and 109 segment crops.
Some segment crops are non-text visual spans, so reviewers must use `segment_status`
to reject those instead of turning every visual segment into OCR text.
`tools/prefill_limbu_book_split_transcription_worksheet.py` now adds a recognizer
candidate-text layer to that worksheet without touching review fields. It verifies the
worksheet report and all segment crop hashes, writes a recognizer label file, and
merges predictions into `candidate_segment_text` / confidence / status columns while
leaving `segment_text` empty and every `segment_status` pending. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-recognizer-prefill-current/`;
prefill report SHA-256 `60796a0265abff4117701d489a3982dfdfeaf66f9050e2efff4302c8caf608f9`,
label file SHA-256 `1076d642bce34ddb2b4ff9059c42ced0728ba359bd19e1f2625b33ece7390b84`,
prediction TSV SHA-256 `ecf15827b3f8ce58338984421f93889f7150baf3cd2ac82c1824b209278b4ce7`,
prediction summary SHA-256 `c03e881417fbf26e9862b1d9316065cdaa904d18ab111c17f79e03546700b4af`,
augmented TSV SHA-256 `8a7cfe05fa8434178e6ee2464326e25e7ced465ee5da7594bf954d336c6c75e0`,
and HTML SHA-256 `fe8012dce16587551fc548ef6a870c58cf401692a66909f1127e754546355948`.
It is independently audited by
`tools/audit_limbu_book_split_transcription_worksheet_prefill.py`; audit SHA-256
`2fa090dbdad4d6a36078c254828eec38799dfef3d50687a67af2b53315b3f32c`,
`audit_passed=true`, `operational_review_aid_safe=true`, 109 label rows, 109
prediction rows, 0 changed review fields, 0 issues, and 0 warnings.
The Devanagari CDC PP-OCRv5 recognizer ran over 109 segment crops with 0 failures:
100 predictions are above `min_confidence=0.65`, 8 are low-confidence, and 1 is empty
(non-empty confidence min/median/max `0.174888` / `0.8584135` / `0.990676`). This is
review acceleration, not auto-application. The compiler now accepts this prefill
report as an alternate worksheet source after review edits, and the unchanged real
prefill report compiles to 0 rows with output TSV byte-identical to the split-prefilled
packet (pending compile report SHA-256
`f8cd51be1baaa288182a8ad30cf5966e2a1639fb6b0f9dbaa0e2b631e7119b4b`).
`tools/apply_limbu_book_split_transcription_worksheet.py` now compiles completed
worksheet groups back into review-packet rows. It writes `split_corrected_texts` and
matching pruned `split_bboxes`, excluding segment rows marked non-text/drop/rejected,
and can mark completed packet rows corrected via `--mark-status corrected`. The current
real worksheet is still unreviewed, so the pending compile at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-transcription-worksheet-application-pending/`
has report SHA-256 `2f148e4b870be32648d5ee545c71b79626464f4457f3772f30ad100c7e168ec1`,
output TSV SHA-256 `b998d4fa0e744f730e5f6ca54f38c12daf392f12d14da3aba32ca4b6c94dfcf4`,
`compiled_count=0`, 19 worksheet packet groups skipped as `pending_segments`, and a
byte-identical output TSV to the split-prefilled packet. Tests cover the completed
path: a reviewed group with one non-text segment compiles to two text lines/two bboxes
and the existing packet splitter preserves it as an accepted existing-book-line row.
`tools/apply_limbu_book_split_worksheet_review.py` now provides the one-command path
from reviewed worksheet to corrected book projection: compile worksheet -> apply the
unified review packet -> audit the application. Real pending run:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-worksheet-review-application-pending/`;
wrapper report SHA-256 `96a5974c02d72c59d200895027a89a6069b701e9ee4d291ed5732c2bf8eadec3`,
compiled-packet report SHA-256 `137b53deaad0a0716fecc3d043382a88a0d9de0a716f47acf008111de50cf0ad`,
application report SHA-256 `fc3fe47f0badda0651689e92c2c45560e288840921ada0d8738bc4e14c4f75c3`,
application audit SHA-256 `d82737206bf8238dbb949470f58df7d55006f4a6ce532ae459360c89db8fa7a7`.
Because the real worksheet remains unreviewed, this run has `compiled_count=0`,
`accepted_counts={}`, 0 book-line applications, 0 candidate-line applications, and
final markdown byte-identical to current best. Fixture tests prove the reviewed path
does apply a split and the pending path stays no-op. After real review, the remaining
step is now one command: `tools/score_limbu_book_ocr_candidate.py` scores the
resulting `application/final-book/` against the frozen page set and runs the book
candidate admission gate. Admission was tightened to accept unified unresolved-review
application audits only by matching the audit's final-book identity to the candidate
`book-run.json` SHA-256; the application audit now also records
`final_book_output_dir` / `output_dir` for `application/final-book/`. The real pending
worksheet application was scored/admitted at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-worksheet-review-application-pending/`;
wrapper report SHA-256 `f546cb6e4cf007c1a32b8e6799ce3203d1060eafd327cc86a72921217a59a832`,
metrics SHA-256 `f96d421ebcac7d262a25d3f5e22ff313e57db4d5593a3b42e56ed0407e52609f`,
page-eval-run SHA-256 `3356fb45524953c8837b6ab237c701866285adf11661101de9e634de8b4c153b`,
and admission SHA-256 `7fadfc25c0191fa71abb9d93e6266c0e0078fea015df7a02bd067d02556db966`.
It passes operational admission with `claim_safe=false` and the same metric values as
current best: codepoint CER `0.446556`, grapheme CER `0.626728`, WER `0.839243`,
micro line F1 `0.750000`, missed/extra/blank rates `0.225806` / `0.272727` /
`0.000000`, and reading-order accuracy `0.611765`. This closes the post-review
scoring gate; it is not quality movement until segment rows are actually reviewed.
The recognizer-prefilled segment worksheet was then tested in two automatic
diagnostic modes. The direct split promotion path
(`tools/promote_limbu_book_split_transcription_prefill.py`) is rejected: it promoted
13 complete groups into 89 text segment rows, improved WER to `0.711653`, but collapsed
line F1 to `0.171429`, reading order to `0.421053`, missed-line rate to `0.612903`,
and extra-line rate to `0.889908`; strict admission failed. The useful path is the
new line-text diagnostic, `tools/promote_limbu_book_split_prefill_as_line_text.py`,
which joins complete high-confidence segment suggestions into the original book-line
text and clears split fields so layout is preserved. Real output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups/`;
promotion report SHA-256 `abebf54f05c80c78d9c9d60459a132d4274e0dfb0d454a38c40bdaad782091bf`,
packet TSV SHA-256 `72dee88f38b8d3f217b0cc706f3b8d3b5d2bdcb376c8331cf29a6c1ceddea8dc`.
Applied output:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-application/`;
application audit SHA-256 `c482d60733a43e0981b38c8a84d8a8b4ed69167494d1117f064edd291f67c968`,
final `book-run.json` SHA-256 `930f0b0475a8b9459e104a145ca5f7e9634e68cb739d9df60b0e4829bc8509a0`.
Strict score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups/`;
metrics SHA-256 `d277042a488bcc2956d2492150504614a88bde077e30f34a0a417afb7fc620c1`,
admission SHA-256 `a0279029f9c346bf51cbbdb5a3c89edfc9db68f0d37dfed49dd3d73e4c21a598`.
It passes with `admitted_operational=true`, `claim_safe=false`, no admission issues,
codepoint CER `0.399842`, grapheme CER `0.511521`, WER `0.711653`, matched-line
codepoint CER `0.123640`, line precision/recall/F1 `0.727273` / `0.774194` /
`0.750000`, missed/extra/blank rates `0.225806` / `0.272727` / `0.000000`, and
reading-order accuracy `0.611765`. This is now the best operational diagnostic
candidate, but the next serious step is not to trust automatic text blindly; it is to
turn these admitted suggestions into review UX and calibrate/verify the six skipped
low-confidence groups plus any pages outside this frozen set.
That review bridge now exists. `tools/promote_limbu_book_split_prefill_as_line_text.py`
now writes an HTML evidence packet and supports `--review-status pending`, which keeps
all rows unapplied while placing the 13 joined recognizer suggestions in
`corrected_text` for reviewer inspection. A new audit tool,
`tools/audit_limbu_book_line_text_review_packet.py`, verifies the pending packet hashes,
status safety, crop/document references, and that every TSV packet row appears in the
HTML. Real pending review packet:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending/`;
promotion report SHA-256 `ce389aa01e11f95e1815cef01f2acbe6ff63debb7f74f58045c234c12ee4fada`,
TSV SHA-256 `ea2f56874371b0b146217d7fa83e883f5a2793ce5672b0038066a5837314d13e`,
HTML SHA-256 `08679a25155a34ac6578a8f0e55d0e153d770f8e22d6930e22282ea755d7f215`,
audit SHA-256 `acac7ff3b890c9cf888a1ca24ed1e3142d3ae55d097651c5ec65c3fcb2d4c8f3`.
The audit passes with `operational_review_packet_safe=true`, 13 pending suggestions,
30 pending rows, 0 accepted statuses, 0 issues, and 0 warnings; the four
`candidate_missing_line` rows are visible in the HTML under "Other Pending Packet
Rows". The audit now has a reviewed mode for the post-review step:
`--allow-reviewed` permits accepted rows only when they carry explicit `corrected_text`
and a non-machine reviewer identity; edited TSV hash drift is recorded as a warning
rather than suppressing row/crop/document validation. Applying this pending packet
through the normal unresolved-packet applicator is a verified no-op:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-review-pending-application-pending/`;
application audit SHA-256 `187d1e7653d97e869ba72128879f7dfaf5017dcd5539edfbddc3eb075901d4b2`,
0 book-line applications, 0 candidate-line applications, and final markdown hashes
byte-identical to the current best. This is the artifact to hand to a reviewer before
trying to turn the diagnostic CER gain into claimable OCR text.
There is now a guarded post-review application wrapper:
`tools/apply_limbu_book_line_text_review_packet.py`. It runs the line-text packet audit
with `--allow-reviewed`, refuses pending-only packets, refuses accepted rows with
missing or machine reviewer identity, refuses accepted rows with blank
`corrected_text`, refuses dropped rows with missing or machine reviewer identity,
applies the unified unresolved packet, audits the application, and can optionally run
frozen-page score/admission with `--score-out`. Real smoke:
invoking it on both current pending line-text packets correctly fails with
`line-text reviewed packet has no audited accepted/corrected/drop rows ready for
application`, because no human-reviewed rows exist yet. This closes the mechanical gap
between review and scored application without making pending machine suggestions
look reviewed.
The same tool now has an explicit low-confidence calibration switch,
`--allow-low-confidence-above`, without changing default behavior. At threshold `0.60`
it admits three borderline segments (`0.637542`, `0.609537`, `0.600665`) and promotes
16 rows instead of 13. The auto-applied threshold-0.60 diagnostic passes operational
admission but is mixed: codepoint CER regresses to `0.406176` versus the strict
`0.399842`, while grapheme CER improves to `0.508449`, WER to `0.706189`, and
matched-line codepoint CER to `0.114738`; line metrics stay unchanged. Evidence:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-threshold060-complete-groups/`;
metrics SHA-256 `f4e1aa3c32e93ec94c845aab7e4f335a9876c22e69cdd6f41ee57797abdd0f81`,
admission SHA-256 `f6c095b40839f3657aa507af2eecf1fc849b1eaa351f39c4bf4f88787afb5731`.
Because of that metric tradeoff, threshold `0.60` should not automatically replace the
stricter 13-row diagnostic, but it should be reviewer-visible. The richer pending
review packet is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-review-pending/`;
report / TSV / HTML / audit SHA-256
`26cdf80ca5280fdc07440ab7bbaa73ab82d6b6aa633801e26de54312e24cc64c` /
`6f86cd9966d219e98f8182ef8a090adff41715a8492a573eb1316092142bb3b3` /
`4c907d94c090cc79d83af760c8876d8584968a8ff25b5f4678a9b1fd24a59893` /
`7ea916b354cc3e74761261f5adb7e6c8ceabc992956bf66463c2af189eff8405`.
It audits cleanly with 16 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. Pending application is a verified no-op with application audit
SHA-256 `688820c7d9ce6b552b9dc53db68eeb65d809ef9206f11a76c4cfdec4055f85f5` and
byte-identical final markdown. Its HTML also exposes all four missing-line candidate
crops. Use this richer packet when review time allows borderline segment inspection;
otherwise use the stricter 13-suggestion packet.
A still better reviewer packet now exists for the actual missed-line workflow:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending/`.
It is generated with `--allow-low-confidence-above 0.60 --suggest-candidate-missing-text`,
so it keeps the 16 existing-line suggestions and also copies the four pending
`candidate_missing_line` machine texts into `corrected_text` as editable suggestions.
All rows remain pending. Report / TSV / HTML / audit SHA-256:
`4ae24b061e5d2c36bc467bafe31134c1c1d1e51de405ab21ebf25c8690f1f07c` /
`d4f4438160031a799cde0eecfb7cea4f8e85b662b3c4b356363131880379e364` /
`3dd21d3dd677b68cc415d7df6da8892c1527bd626600e17f9282666c120b722b` /
`42fb1372dc06c7db3d6484044dfc2fc752a50573c9363880257b1206c43ba76e`.
The audit passes with 20 pending suggestions, 30 pending rows, 0 accepted statuses, 0
issues, and 0 warnings. Pending application is a verified no-op:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-suggestions-review-pending-application-pending/`;
application report / audit SHA-256
`aa2a106ef8719c5e8f3094dced4ced1883b987a9db13fe6c45e0416acfaa49cc` /
`3ec74b70dec0233d8569b3dcc784aa4e1ca859673288fd76b666ca8cd1f7d51d`, 0 applied rows,
and final markdown byte-identical to current best. This is the preferred packet when a
reviewer will inspect both existing-line corrections and missing-line insertions.
The preferred packet also now has a missed-reference coverage audit:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-threshold060-line-text-suggestions/`.
Report / TSV SHA-256:
`68565c47c276d53a00af08002b0f734d1cc4094da2ebb82ec7b88a4d2ee401c4` /
`ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37`.
Against the threshold-0.60 line-text score, all 7 frozen missed-reference rows are
review-visible: 2 covered by `candidate_missing_line`, 2 covered by existing review
regions, 3 swallowed by larger existing review regions, and `uncovered_or_weak_count=0`.
This is review-coverage evidence only; the rows remain pending and are not claim-safe
OCR until a human accepts/corrects them and the final book is rescored.
That audit exposed one remaining packet gap: the previous preferred packet covered
all missed references but left one high-confidence extra prediction uncovered,
`limbu-devanagari-cdc-social-p0009/img-line-0001` (`थिक्सिङ् - थिक्`). The raw
unresolved packet generator now supports `--line-alignment-extra-predictions`, which
adds `prediction_unmatched` book lines as `extra_prediction_line` review rows unless
they are already present in the confidence-driven book-line review queue. Focused
tests cover this path. The superseding reviewer packet is:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-pending/`.
It has 31 pending rows, 20 pending suggestions, 0 accepted statuses, 0 audit issues,
and 0 warnings. Promotion report / TSV / HTML / audit SHA-256:
`d5650dc1b67ae9881e6b4ccad5a5c1cacf8a744f02fc6af11bce3ce587bcda44` /
`5e0e618adc8229fea734b8356be7aa3fe6f2c4f09737f9a5685ed15c284ac468` /
`978fb40593d192b23c78f4f240e75c754cf29f404957ad104e6c4683d885ea50` /
`e1053e353a4408c33008926a4923d3d25c93decbd943441d6e8fcc892f398b4d`.
The raw unresolved packet now audits cleanly too: report / TSV / audit SHA-256
`55ed8e0c6cfceeb609aeea3cf85862819488c6a59df422b935e817d36350d155` /
`866e40ae4c5d037a347a3740c57610601f54071aa3deeefeed994f1a48d3a601` /
`92d66a8333c387478b4d5bcb5d0fdbf51b231671aa5e1ce60c1ac8b0e26db5a2`, with 31
pending rows, 4 candidate-missing rows, 27 existing-book-line rows, exactly 1
`extra_prediction_line`, 0 issues, and 0 warnings. The raw-packet audit now accounts
for `extra_prediction_line_row_count` separately and verifies the frozen
line-alignment CSV hash used to append extra predictions.
Its structural review coverage artifact is
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/line-review-coverage-threshold060-line-text-missing-extra-suggestions/`;
report / missed TSV / extra TSV SHA-256:
`ae9b7903a719110e77cfbe8089fc0cc0f025bd6f93804cdec4ac1d5a7b24b775` /
`ce7eefcb4fb43b0e1ec74661adbad00c00b2f11b2b08e60dfc063735592dad37` /
`ad610c8019489fc03c041dfa1f0eb35e31bf76a304f8cf5731b3469752a21d51`.
Coverage is now complete for the frozen line-structure errors: 7/7 missed references
review-visible and 9/9 extra predictions review-visible. The guarded application
wrapper correctly rejects the packet while all rows are pending, so this is a human
review packet only.
The application side now supports actual extra-line rejection instead of merely
showing the row. `tools/apply_limbu_book_line_reviews.py` accepts explicit drop/reject
statuses for existing book lines, removes the line from the document, removes its
line id from blocks, drops emptied text blocks, and records `dropped_counts` /
`book_line_dropped_count` through the split, unified application, line-text wrapper,
and independent application audit. A real smoke copied the superseding packet to
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra/`,
marked only `limbu-devanagari-cdc-social-p0009/img-line-0001` rejected by
`cdjk-review-smoke`, then applied and scored it. Smoke packet report / TSV SHA-256:
`1a8fc27bfc3fd9a3a3c47abc766df7abfb97301855c7f4b3725e2ef2722cb1b8` /
`91287f19aaecf89c7735fad7221b71789c605d59bfc71296d7002ac959167120`.
Application:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-missing-extra-suggestions-review-smoke-reject-extra-application/`;
application report / audit SHA-256:
`1806662aa1318a340df70067990e345a0e79e363a04ea8891c1858da837a5a01` /
`bcfd4ef72fc93ed7c0a08d26a67df0f2953e44450249747f5a0c4f06f1326f9b`.
Result: line-text packet audit passed, application audit passed,
`accepted_status_count=0`, `dropped_status_count=1`,
`dropped_counts={"existing_book_line": 1}`, and `book_line_dropped_count=1`. Smoke
score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-missing-extra-smoke-reject-extra/`;
score/admission report / metrics SHA-256:
`7501fe80695cd1adb87428a94ef3f9d7c6b416200279f3a6d0b16d57e37af08b` /
`7bccf3236644d2d73d2ddc7864f85de64dbe7681b91d41ad3fab7d48d8a45e89`.
Operational admission passes but `claim_safe=false`; codepoint CER `0.436263`,
grapheme CER `0.615975`, WER `0.823739`, line F1 `0.761905`, extra-line rate
`0.250000`, missed-line rate `0.225806`, and reading-order accuracy `0.611765`.
This is synthetic review-smoke evidence, not human-review claim evidence.
A second mixed smoke copied the superseding pending packet to
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke/`,
then applied one existing-line correction, one accepted `candidate_missing_line`
insertion with text `१`, and one rejected extra prediction. Mixed packet report / TSV
SHA-256: `14ca2c76ea261c79730d96480d6e7a4bb901a807d24a306cc4aabb490d50983d` /
`7641025877f89c519d371bc3df613b4a98fa9a4874bf06c9b0ff2993a8602fff`.
Application:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-application/`;
application report / audit SHA-256:
`8b1b983b34c65dba327788959a3200e176dc86639bb50a15f22b792cb5637c31` /
`58095326d648202df10aecdda244b470fba2601c9bf5d82e2c9edeb563dd9617`.
The mixed application audits cleanly with `accepted_status_count=2`,
`dropped_status_count=1`, `accepted_counts={"candidate_missing_line": 1,
"existing_book_line": 1}`, `dropped_counts={"existing_book_line": 1}`,
`book_line_applied_count=2`, `candidate_line_applied_count=1`, and
`book_line_dropped_count=1`. It then scores via
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke/`;
score/admission report / metrics SHA-256:
`03adcbacac5f4f1fac68d9f73bf9c85c33f024931c6cff33ab3106f5878bfff1` /
`e3ddf0d31b7bed313f9e22a05be0583e3afa7187cc8a29785b406f6b84b2cc5c`.
Operational admission fails and `claim_safe=false`: codepoint CER `0.438638`,
grapheme CER `0.614439`, line F1 `0.781250`, extra-line rate `0.242424`,
missed-line rate `0.193548`, matched-line CER `0.174901`, and reading-order accuracy
`0.553191`. Treat this as pipeline-plumbing evidence only. The next concrete gap is
accepted missing-line insertion placement/reading-order calibration; otherwise a
seemingly correct insertion can improve recall while failing operational admission.
That gap is now patched for structural page markers. `tools/apply_candidate_line_reviews.py`
places reviewed candidate insertions with structural role `page_marker` before body
text, leaves ordinary text insertions geometry-placed, and records
`placement_policy` in insertion metadata. A rerun of the same mixed packet with this
front-matter policy is:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-application/`;
application report / audit / final book-run SHA-256:
`bac62fc812eaed939ceecf7e36fd7ccb0a2329ae7644d2755df0077487c1b179` /
`fc264875a4876dceadccbbdf221b3771726000a4e50518fba9df60d92b63ba40` /
`879547c4827fee26900f5f7d49a7462df0793fd6d7a689bfecf90618f507b56a`.
It audits cleanly with the same action counts as the failing mixed smoke:
`accepted_status_count=2`, `dropped_status_count=1`,
`accepted_counts={"candidate_missing_line": 1, "existing_book_line": 1}`,
`dropped_counts={"existing_book_line": 1}`, `book_line_applied_count=2`,
`candidate_line_applied_count=1`, and `book_line_dropped_count=1`. Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter/`;
admission report / metrics SHA-256:
`43705c1d33ce7c5f23d04452a364cf1316efa084b90ea6b67b35f458c754a60a` /
`a3718bbab380b2c44d406d938db00bf3fdaa2021d24d1f683eb944419a76d65d`.
Operational admission now passes, `claim_safe=false`: codepoint CER `0.435471`,
grapheme CER `0.608295`, WER `0.812810`, line F1 `0.781250`, extra-line rate
`0.242424`, missed-line rate `0.193548`, matched-line CER `0.174901`, and
reading-order accuracy `0.648936` (`61/94` matched-line pairs). This is still
synthetic review evidence; it is useful because it validates the mixed review
application path without weakening the admission gate.
The structural ordering policy was then generalized in the actual projection paths:
`src/ocrtech/structure.py`, `src/ocrtech/image_lines.py`, and
`tools/apply_limbu_book_line_reviews.py` now sort explicit `page_marker` blocks before
body text and also treat bottom-margin Devanagari digit-prefixed rows like `२ ...` as
front-matter rows. The latter handles page numbers fused into neighboring title/header
text, while avoiding a broad "bottom text first" rule. Rerun artifact:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-threshold060-mixed-review-smoke-frontmatter-row-application/`;
application report / audit / final book-run SHA-256:
`c7ea956cdec75c2aefc9260e0e30c30710f77f5e84bc19b6be53948b3e103408` /
`8efc363d1895ed955e8fc692fb10ae1a6ff312f9f146464065e8eb6696a9a657` /
`d012af48bb7c64d65572cfa70ff135a6a8321aa1658c29ae1c26467fb82fedba`.
Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-text-threshold060-mixed-review-smoke-frontmatter-row/`;
admission report / metrics SHA-256:
`47ee4098aaa2fc92c9d053c1fc8cfeb3093528a5556dcf450656af8c6b356837` /
`7201ed8390b0bce13ecc5f7aca6643b1b6d77f7be99c1b7b12548460f3150182`.
Operational admission passes, `claim_safe=false`: codepoint CER `0.455265`,
grapheme CER `0.632873`, WER `0.821451`, line F1 `0.781250`, extra-line rate
`0.242424`, missed-line rate `0.193548`, matched-line CER `0.174901`, and
reading-order accuracy `0.787234` (`74/94` matched-line pairs). The tradeoff is real:
reading order improves sharply, but raw page CER regresses versus page-marker-only
ordering because low-confidence unreviewed text is now placed in the reference order.
Do not promote this as claim evidence; use it as layout-order evidence and keep the
human review boundary intact.
The text-first candidate was also rerun through the structural ordering code:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-application/`.
It applies the same 13 existing-book-line corrections as the previous best-text
candidate, leaves 17 rows pending, and applies no candidate-missing insertions.
Application report / audit / final book-run SHA-256:
`d3821317c37f4c914febdfce29e9e113f383d074d2132bd85b619fa68add7ef3` /
`5ebad12995fdb520dcd98daf02c614f0e3d8d6fbff23db441bead8804d4ce05b` /
`902dce10d52e2b54ee9bf3e6ed2e2e0b5d15ef746ad921a28df5c20610fc06bb`.
Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order/`;
score/admission report / metrics SHA-256:
`f24b4f8a3abd66ac89c475ed9cc98e54a402e2369b587a0780dc19cbf85473a7` /
`979827dc2ccdbc4f6c3ea45ee97c1bcc0151200f8c5eaeafb31a7928c72524d4`.
Operational admission passes, `claim_safe=false`: codepoint CER `0.404592`,
grapheme CER `0.520737`, WER `0.722582`, line F1 `0.750000`, extra-line rate
`0.272727`, missed-line rate `0.225806`, matched-line CER `0.123640`, and
reading-order accuracy `0.682353` (`58/85` matched-line pairs). This is the better
text-first operational candidate after structural ordering: it gives up a small amount
of raw CER versus the old text-first run (`0.399842` -> `0.404592`) but improves
reading order (`0.611765` -> `0.682353`). It still does not solve the missing-line
problem because no candidate-missing rows are accepted.
The conservative page-marker selector has now been wired to the unified TSV review
packet without weakening its guards. `tools/select_candidate_line_reviews.py` now reads
candidate-missing TSV rows, derives page size from the row's `book_document`, parses
`max_selected_iou` from `review_reasons`, and rejects a page marker when the same digit
would still be embedded at the start of an overlapping selected line after accepted
existing-line corrections from the same TSV are applied. On the complete-groups packet
this selects the maths `१` marker and the science `२` marker; the science marker is
allowed because accepted review row `existing-0019` removes the leading digit from
`img-line-0014`. The two body candidates remain rejected. Selection report / selected
JSONL / rejected JSONL SHA-256:
`f52f61ab09ba6f2101ae8049815a01257dd5b57e93c9aab492f2125fefbf2058` /
`7e7b37359d5b4dc8972220054b33f2ffeb388ff7d02a621972035a19a573baa9` /
`40a9ae5a0fb174694cdc1e04d108adb52900b2864bce43aabf4557878f21feaf`.
Applied on top of the structural-order best-text book, the new projection lives at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-application/`.
Applicator audit / independent application audit / book-run SHA-256:
`26dc4049979618330eb8016fad699af82465719d7e88749540309adcc40261cc` /
`e135fcb2faa7b1a125c2f706fae41e1fb448dfe918390f434f5075895fc540d4` /
`61d6dfc09e1b7e09f0fd1cf57b5aac988b6c8e65f567f2eda6c19e33928ff775`.
Score/admission report / metrics / admission SHA-256:
`9a4f3bca208d515b1e7538536a1abb54000749d4694e2317a981ca3588064ba3` /
`fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4` /
`1bbd1d216d3c3b23def126332aecf1052abf254a0f15891dd58048c103cc2205`.
Operational admission passes, `claim_safe=false`: codepoint CER `0.402217`, grapheme
CER `0.516129`, WER `0.711653`, line precision/recall/F1 `0.714286` / `0.806452` /
`0.757576`, extra-line rate `0.285714`, missed-line rate `0.193548`, matched-line CER
`0.123518`, and reading-order accuracy `0.712766` (`67/94` matched-line pairs). It has
3 structural page-marker predictions, 2 matched structural lines, and 2 exact
structural-text matches. This is the current best operational layout candidate, but it
is not a full OCR-quality solution: body-line candidate text is still too noisy, and
the next gain has to come from reviewed body-line recovery or a real line split/remove
path, not more blind marker insertion.
The body-line recognizer-probe lane was refreshed against that current best and exposed
one more audit bug. The refreshed crop probe
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-text-crop-probe-from-complete-groups-page-marker-v2/`
uses the two body-line rejections from the TSV-backed page-marker selector. It again
refines both body rows (`trim_top_bleed` and `left_of_horizontal_rule`), with probe /
JSONL SHA-256 `9e64044df9ec4070dcc28b78a2e0f48c2fd6cb8009fdab1de41a36eb2591a61c` /
`31e8ecbf17ab207c67c3f46510f66771e4f2802315cf052ee52aaaa858ae68d4`. Existing
recognizer predictions are valid because the refined crop hashes are byte-identical to
the older probe. The refreshed comparison
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-candidate-crop-recognizer-probe-comparison-from-complete-groups-page-marker-v2/`
has comparison / JSONL SHA-256
`12fd030c336f3d1cdef882de8512f56a5f58e7bd513d301b4dab0fd95418b329` /
`f1d184db8267c93270b5451cce692bf0695f9af7548470f0b8af771236a6c38d`. The non-oracle
machine selector
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/body-line-recognizer-probe-selection-conf045-delta005-from-complete-groups-page-marker-v2/`
selects only the first body row (`confidence=0.469282`, delta `0.218220`) and rejects
the second for low confidence/delta; selection / selected JSONL SHA-256:
`41655846f1845d1a966a72e7def29b38e58bee53bf5472493a6c3bda536e1dc1` /
`205415b49c984145cbf573aa1e7a65547a931a202bc8789daa58ebe10d64c2b4`.
`tools/audit_candidate_line_review_application.py` now distinguishes current-review
insertions from inherited candidate insertions in chained applications; the real body
probe application has `applied_count=1`, `inserted_line_count=1`,
`total_inserted_line_count=3`, and `inherited_inserted_line_count=2`.
Application audit / book-run SHA-256:
`d89b9c8966c5c1a01e74bb5edbe3ab19c08054b9f915d9aa5fad1863199f6d99` /
`abe8f59007aa77fb13bf690070230346aa14dc88040fa2b18b8d80ef2fd332de`.
Score/admission:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe/`;
score report / metrics / admission SHA-256:
`b52a853cf05196c618a416fc981cf747b05cc178a6435fc35a0d497380611992` /
`e2307c02624fd601f2ad5674b3405e62620c2565993d3dcbf2425e5b8fd74f79` /
`2b55cb5073c523c2aa69c69f52251452a80ab89e762554074b29d21361db8b6c`.
Operational admission passes against the old CDC baseline, `claim_safe=false`, with
codepoint CER `0.441013`, grapheme CER `0.562212`, WER `0.760834`, line F1
`0.776119`, missed-line rate `0.161290`, and reading-order accuracy `0.656863`.
Do **not** promote it over page-marker-v2: paired comparison against the current best
at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/compare-body-recognizer-probe-vs-page-marker-v2-current-best/`
shows codepoint CER regresses by `0.038797` and grapheme CER regresses by `0.046083`
(comparison / paired TSV / markdown SHA-256:
`1bafa22b69296c98f9186ade50cce68d509491e0b77de8be80c45bb9a4d3a209` /
`cd89984c2dfc9a8e6792b15b1b193348716d9a5c0c140d048278843fbead732f` /
`38104641a4c8744233fa90669395f994047121e21aba25c7af7602741dccf3a8`). Treat this as
review evidence and an audit-chainability fix, not the selected OCR policy.
That promotion mistake is now guarded in code. `tools/admit_limbu_book_ocr_candidate.py`
accepts `--champion-metrics`, and `tools/score_limbu_book_ocr_candidate.py` passes it
through. When supplied, admission rejects candidates that regress codepoint CER,
grapheme CER, or WER against the champion beyond the configured `--max-champion-*`
tolerances. Real verification against the page-marker-v2 champion:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-complete-groups-structural-order-page-marker-v2-body-recognizer-probe-champion-gated/`
fails admission with champion deltas `-0.038797` codepoint CER, `-0.046083` grapheme
CER, and `-0.049180` WER (wrapper/admission SHA-256
`4126d1ca9c4e644190d862301f199f3f46335fb1c7b48191209c6e70e3f46214` /
`ff1425910714bacd54f96640d8f2281c08d78c93f07b0d43c7be3117d9a1c4e1`). The
page-marker-v2 self-check at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-gated-self/`
passes with zero champion regression (wrapper/admission SHA-256
`6103f40ec19bdde710fd1438ff8547382118fbbdadc9f96ce65425767a5ce75d` /
`3b22736a064654783b9a8cee5a455936d7e86cc1d759d1e7536bb5ea69eb727c`). From now on,
do not call a candidate "current best" unless it passes this champion-gated admission,
not merely the older `0.458432` CDC baseline gate.
The reviewer-facing unresolved packet has been refreshed against the actual
page-marker-v2 current best. `tools/prepare_limbu_book_unresolved_review_packet.py`
now accepts selector-style JSONL for `--missing-line-review-queue` and filters
non-pending rows out of unresolved packets, reporting skipped counts; this matters
because the page-marker-v2 book run points at a queue with 13 already-corrected rows.
Real packet:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-extra-predictions/`.
It uses the page-marker-v2 book output, its own `line-alignment.csv`, and the
page-marker-v2 selector's rejected body-candidate JSONL. The packet has 18 pending
rows and 18 verified crops: 13 existing low-confidence book lines, 3 extra prediction
lines from current-best alignment, and 2 candidate missing body lines. It skipped 13
non-pending existing rows. Report / TSV / HTML / audit SHA-256:
`1e004bb5c6b206a2ad3faf147e1f54cea3c7d442300b71be295a3c2d9743c5ab` /
`fe8908da4050e90f17d4a6ccbf111ba04b5752491bccb19a01d11b5853bdf84a` /
`17694e6aa1224dec5e3edc12cb742e07d0be9989aa66f9718399d48e3fef23c7` /
`10e95db8204c47bb4fec39264e2914510fed82544221f541f486bebf3f3d26be`. Audit passes
with 0 issues and `operational_review_packet_safe=true`. The pending application at
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/book-unresolved-review-packet-page-marker-v2-current-best-application-pending/`
applies 0 book lines and 0 candidate lines, audits cleanly (application/audit/final
book-run SHA-256
`4fd8812729cbd73e5332d0dbf21bcb6959221caa7c515d0e818cbcdf3f2bcb19` /
`239610a43b40f765543d168561b5c3e7b378c89e08e129b0e012b67f8d83764a` /
`cf0b2970cbc3aa56dd72ffe7fc6dc68b43b5ffa2ca4e99e31d1f7f8c52a632bc`), and final
`book.md` / `book.body.md` are byte-identical to page-marker-v2 current best. This is
now the current packet to hand to a reviewer before further body-line recovery.
Coverage has been refreshed for that packet too:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/missed-line-review-coverage-page-marker-v2-current-best-packet/`.
Against page-marker-v2 `line-alignment.csv`, there are 6 unmatched reference lines:
1 is covered by a candidate missing-line packet row, 2 are covered by existing review
regions, and 3 are swallowed by larger existing review regions. `uncovered_or_weak_count`
is `0`. All 10 unmatched predictions are covered by existing review regions, with
`extra_uncovered_or_weak_count=0`. Report / missed coverage TSV / extra coverage TSV
SHA-256:
`32026349ec549e142036042cf3b6bc2713a41b0438f4c5220b3afb082d7e7abc` /
`596c77aa6770a28467dafa89a8f7122bd080bb274ef98bacdfba7521566d8ad8` /
`95f41b789b6ae9aadf93b4273d1c96b7a39f2e119f1500e7e8187ec9ee350b91`. This proves
the current packet covers the current missed/extra line surfaces for review. It does
not prove OCR completion; the remaining work is correction or safe automatic recovery
of those reviewed regions, followed by champion-gated scoring/admission.
Admission now knows about that requirement directly. `tools/admit_limbu_book_ocr_candidate.py`
accepts `--review-coverage-report` plus strict default maxima for uncovered missed and
extra lines, and `tools/score_limbu_book_ocr_candidate.py` forwards those inputs. The
current page-marker-v2 candidate was rerun with both champion metrics and review
coverage:
`outputs/limbu-book-ocr-missed-line-recovery-2026-06-22/score-admit-book-line-split-recognizer-prefill-line-text-complete-groups-structural-order-page-marker-v2-champion-coverage-gated/`.
It passes operational admission with `claim_safe=false`, zero champion regression, and
review coverage values `{missed_reference_count: 6, uncovered_or_weak_count: 0,
extra_prediction_count: 10, extra_uncovered_or_weak_count: 0}`. Wrapper / metrics /
admission SHA-256:
`0018bfb5ee2f48821c3c9b1101d0921bbad54c41cd63be120dc72437fe9f1344` /
`fad040788e620c7beaaeb923e7b870a6c1123982c777f2335102b816edcd5eb4` /
`b9ece419592b3bfc3dbc892880c206a2cbcb24a8777e0c245d1b7614afa30f43`. Future
promotion attempts should supply both `--champion-metrics` and
`--review-coverage-report`; passing the old baseline-only gate is no longer enough
evidence for book OCR.
The eval harness registry now reflects current book-candidate evidence for both
Limbu scripts. Page-level recipes
`limbu-devanagari-cdc-page-marker-v2-champion-coverage-gated` and
`limbu-sirijonga-gorkhapatra-book-conf070-coverage-gated` are registered in
`configs/eval-harness-recipes-v1.json`, and the `limbu-first-two-script-ocr`
targets in `configs/eval-target-coverage-v1.json` point at them with
`eval_status=registered_real_pack`. Validator status:
`outputs/eval-harness-recipes-current-validation.json` reports
`recipes=13 valid=True claim_ready=0`; `outputs/eval-target-coverage-current.json`
reports `valid=True coverage_ready=False target_sets=3 gaps=52`. Both book
candidates are structurally valid and operationally admitted, but neither is
claim-ready. The Devanagari side is still blocked by only 3 CDC pages, 0 claim-ready
references, `curriculum_text_unreviewed` GT, non-frozen/non-claim-eligible rows,
missing hash-verifiable model artifacts, `claim_safe=false`, and 18 unresolved review
rows. The Sirijonga side is still blocked by 1 Gorkhapatra page, machine-converted
unreviewed GT, non-frozen/non-claim-eligible rows, `claim_safe=false`, and 79
unresolved review packet rows. The top-level two-script Limbu gate now has 8 gaps:
for both scripts, book-candidate evidence exists but is not claim-safe, and the
normalization, recognizer, and page recipes are not claim-ready. This is an honest
registration upgrade, not a quality claim.
The recipe validator now independently audits the page-marker-v2 evidence chain:
`score_limbu_book_ocr_candidate` wrapper report, admission report, metrics, champion
metrics, review-coverage report, and unresolved packet hashes. It blocks missing
champion evidence, missing review-coverage counts, champion regressions, uncovered
missed/extra review regions, mismatched evidence paths, and scorer commands that omit
or redirect the recorded `--book-output`, `--book-audit`, `--champion-metrics`, or
`--review-coverage-report` inputs. The current registry still validates, but future
"current best" registrations cannot rely on command-string convention alone or pair old
evidence with a weaker replay command.
The Sirijonga page recipe is now executable as a replay rather than a placeholder.
`tools/copy_page_ocr_document.py` copies a pre-materialized page OCR document bundle
into a recipe-run `documents` directory, verifies the manifest input path against the
copied document's `source_path`, and writes a hash report. The
`limbu-sirijonga-gorkhapatra-page-current-router` recipe now uses it to replay
`operational-image-line-width480-minwidth005-conf070-documents`, has
`requires_operator_resolution=false`, and no longer carries unresolved-command or
missing-model-artifact blockers. Smoke run:
`outputs/eval-recipe-smoke-limbu-sirijonga-gorkhapatra-page-current-router/`.
It audits cleanly but remains non-claim-safe: 1 sidecar-fixture page, codepoint CER
`0.739361`, grapheme CER `0.768262`, WER `0.918969`, line F1 `0.872458`,
matched-line codepoint CER `0.046585`, reading-order accuracy `0.775478`, and
0 claim-eligible rows. This is harness reproducibility progress only; the quality
blocker remains reviewed/frozen Sirijonga page data plus page assembly/layout.
The current selected Sirijonga diagnostics now have textless unresolved-region review
packets in `outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/`. Gorkhapatra
operational `conf070` has 69 review rows/crops: 67 missed references and 2 extra
predictions. Its packet audit passes with 0 hidden-reference leaks; the missed text is
center-column dominated, with missed reference text contributing `83.8643%` of
attributed codepoint error mass. Sikkim same-row merge has 22 review rows/crops: 9
missed references and 13 extra predictions. Its packet audit also passes with 0
hidden-reference leaks; Sikkim remains mixed, with missed reference text at `31.6140%`
and extra prediction text at `28.1198%` of attributed codepoint error mass. These
packets are review/recovery inputs, not claim evidence.
The page-level review loop is now mechanically closed with
`tools/apply_unresolved_page_review_packet.py`. Refreshed packets include blank
`corrected_text` cells; pending rows remain no-ops. Accepted/corrected missed-reference
rows require non-empty `corrected_text` before insertion by geometry, accepted/corrected
unmatched predictions update the existing OCR line only when `corrected_text` is
supplied, and false-positive predictions are deleted only with explicit
`drop`/`delete` statuses. Pending application smokes for Gorkhapatra and Sikkim score
identically to their selected source outputs (`0.739361` and `0.418057` codepoint CER,
respectively), proving the path is scoreable without changing OCR text before review.
`tools/audit_unresolved_page_review_coverage.py` now independently checks exact
`sample_id` + line-id coverage against `line-alignment.csv` without writing hidden
reference text. Current Sirijonga packets are coverage-complete: Gorkhapatra covers
67/67 missed references and 2/2 extra predictions, while Sikkim covers 9/9 missed
references and 13/13 extra predictions, with zero duplicate or extra packet keys.
The Gorkhapatra Sirijonga pruning check rejects threshold-chasing as the next local
move. The unpruned width-480 run recovers more references than operational `conf070`
(`261/303` matches, 42 misses) but adds 62 extra predictions and worsens page
codepoint CER to `0.761144`; the operational `conf070` route keeps only 2 extras and
scores `0.739361` CER but misses 67 references. Aggressive `conf085` pruning lowers
page CER to `0.730446` by deleting difficult lines, but recall falls to `199/303`
matches with 104 missed references, which is not acceptable for book digitization.
The tuned `conf070`/`conf080` routes sit in the same tradeoff band, and auto split
tall recovers 285 references but adds 54 extras and worsens CER to `0.778638`.
Conclusion: keep the current operating region for text output and attack
spatial/structural missed-line recovery plus review application, especially the
center-column Gorkhapatra misses. Do not spend the next cycle on global pruning or
plain tall-split threshold sweeps.
The Gorkhapatra Sirijonga candidate-line recovery diagnostic now makes that boundary
measurable. Comparing selected `conf070` page documents against the rejected
auto-split-tall candidate set produced 100 candidate-only rows; the packet audit passes
with 0 issues (`candidate-line-review-packet.json` SHA-256
`68f678a77e0b2056c25ecbc1595cf62e63c4ef72aa923d79355a4a9886998ff4`, audit SHA-256
`ac645bb4755bab878c4e65749735d86856c0dcb9e473399bf064db869e8df56c`). The
diagnostic-only oracle overlap selector picked 49 rows against the 67 currently missed
references (selection report SHA-256
`45fa5a781d62736a2aa3ef5c97762755d0b65ad99f1c7984856e585ab30e5128`, selected JSONL
SHA-256 `06f6439e09ce6a1b8c292dcaf57e5672bdd674ddca58bb03fad5d7f5d1fc9a57`).
`tools/apply_candidate_page_line_reviews.py` applies such accepted candidate-only page
rows into a separate scoreable page-document projection; in the oracle-overlap
diagnostic it inserted 48 candidate-text lines and skipped 1 empty accepted row
(`candidate-page-line-review-application.json` SHA-256
`b0c8d076df3bc6e19b85c3819ff1611abde1d691b71f59607a53506fef00ad7e`). Geometry improves
strongly: line F1 `0.964346`, precision `0.993007`, recall `0.937294`, and missed
references drop to 19. Text quality rejects automatic promotion: page codepoint CER
worsens to `0.766779`, grapheme CER to `0.798907`, and matched-line reading-order
accuracy is only `0.594983` (metrics SHA-256
`0467192299d469d02e3ad1b14c9be669a6561e08263098da858d337ea3267da8`). The safe
non-oracle artifact is the prioritized review queue at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-prioritized-candidate-line-review/`:
100 body-line candidates, 99 high priority, 1 medium, audit passed with
`operational_review_queue_safe=true` and SHA-256
`0c9cde56dbec68f4a146cf417cee033271814ce7c5887058841028798f2a6985`. Treat this as a
review/transcription queue or crop-recognition target, not an OCR output candidate.
`tools/prepare_candidate_line_text_crop_probe.py` has now prepared a non-oracle crop
probe for that queue at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-text-crop-probe/`:
100 body-line candidates processed, 44 refined by trimming top bleed from multi-row
crops, and 56 copied as full crops. Report / JSONL / label file / HTML SHA-256:
`dc5de12cda741533cc31e343147b4e12488b4ee1611e1e8cdb81a038206725c3` /
`9ad687ca53e229565132dc579a79efd6adb6572aadad54fc5c61294fceca3fc5` /
`2c5ae66ea3468f97f456ebcfa4f979703e65ded921f3850054b357cbeb015f2b` /
`46ae06c2412961b02fb9f9f8db7b30fd81afcc702cb8fb79b672fe7fe41094c8`. A sampled
refinement isolates the candidate bottom line from a multi-row crop, so this is a
reasonable next recognizer/reviewer probe input; it is still not automatic OCR text.
`tools/predict_recognizer_eval.py` now flattens embedded newlines and tabs in
recognizer output before writing TSV, after the Sirijonga crop probe exposed malformed
multi-line prediction rows. Re-running the packaged v4 Sirijonga recognizer on the 100
original candidate crops and 100 refined crop-probe images with `/tmp/ocrtech-paddle311`
produced 0 failures and valid 3-column TSVs. Original/refined prediction TSV SHA-256:
`8f32f8895ded0a4a1c405333206ebfc55df37606ff1b288ef4319c7067060fc0` /
`66c4fa2d25b46bf24c99ad9936000ec0b73a932478cf1e74e613cabe09f79fd3`. The comparison at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-candidate-line-crop-recognizer-probe-comparison/`
has 100 rows, 44 refined rows, 40 changed predictions, max probe confidence
`0.992109`, and `automatic_application_safe=false` (report / JSONL SHA-256
`8b2133b39c3d139b8a83d10858301bcad0e92cd821a4042bbf4aeb0bdf27064d` /
`f45d8d9866d05681202a221741612fb263312b6725aeac01d1fe6dcc12029464`).
A small non-oracle recognizer-probe threshold sweep found no clean operational
promotion. `min_refined_confidence=0.45`, `min_confidence_delta=0.05` accepts 21 rows
but worsens page codepoint CER to `0.747351`. The strict `0.90` / `0.05` setting
accepts 6 rows and improves page codepoint CER slightly to `0.737090`, but line F1
drops to `0.866545` from selected `conf070`'s `0.872458`, missed references are still
66, and extra predictions rise to 7 from 2 (selection / selected JSONL / application /
metrics SHA-256:
`31ee7a4dc399cab13ded5665cdf0998e132b136ad60202b57a3636b8cfd0f042` /
`072af68c1e59f31406f1e8df350b45e5c1040136f4b966f5adda9bb14b90b603` /
`99ace4d56307e488e71250247e456aad651f99b87ff621ccda3ec1ecd60535a8` /
`ee62806b3a3b3198544aeea273e9164d99bdf8059024af07775e53f0c36b3326`). This is useful
evidence for reviewer-assisted recovery and crop-recognition work, not a default
book-OCR output policy.
`tools/audit_candidate_page_line_application_effect.py` now turns that policy boundary
into an auditable gate. It compares a candidate page-line application to the selected
baseline on page CER, line F1, missed/extra/blank rates, reading order, and inserted
line alignment, and writes an inserted-line TSV without reference text. The strict
`0.90` / `0.05` recognizer-probe application is rejected with
`book_ocr_objective_safe=false`: 6 inserted rows, 1 matched row, 5 unmatched extras;
the matched insert has line codepoint CER `0.888889`, grapheme CER regresses by
`0.002485`, line F1 regresses by `0.005914`, extras increase by 5, and reading-order
accuracy regresses by `0.150648` (effect audit / TSV SHA-256
`08eff93810712cafe46009ac8aa5a1d6dcd00c3206f2cd5b88aff527c7e0cd82` /
`581c9d96f99ffee1be638a90ec662aefc6b6837baf04a1af75a50ba06a7a9181`). The loose
`0.45` / `0.05` application is also rejected: 21 inserted rows, 9 matched, 12 extras,
and worse codepoint CER, grapheme CER, line F1, extras, and reading order (audit / TSV
SHA-256 `9c8865eb655fe74593295e241863f746ca565a50a80948412f0c67e5473f77dd` /
`5e1dc16eeb64f22db879e2f8a631fa5a7b693dce8a674a0956eebb9447a7aa49`). This audit is
now the guardrail for page-candidate insertion diagnostics: a CER-only improvement is
not sufficient for the Limbu book OCR goal.
The follow-up oracle-feature analysis shows why the recognizer-probe selector failed
and what the next non-oracle selector should test.
`tools/analyze_candidate_line_review_oracle_features.py` compares the same 100
candidate-only auto-split-tall rows against the oracle selected/rejected labels, while
emitting only safe features at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-oracle-feature-analysis/`
(report / TSV SHA-256
`4c45a640338b6510b63bced5b903b847e4725ab87f544fb36ba7def7ac2e2bb0` /
`21e31ff240b2a40220b70b9f029237064b83c951509b6bea62ec9fc4855098bf`). The 49
oracle-selected rows have much longer candidate text (mean/median `29.306`/`37`) and
wider boxes (mean/median `295.531`/`307`) than the 51 rejected rows (text
mean/median `4.471`/`1`, width mean/median `127.137`/`74`). They also skew toward
middle x-band and middle/bottom y-bands. Refined recognizer confidence points the
wrong way: selected rows have mean/median refined confidence `0.382649`/`0.146180`,
while rejected rows have `0.601181`/`0.751105`. The tested refined-confidence/delta
thresholds have only `0.142857`-`0.500000` oracle precision and at most `0.183673`
recall. So the next serious automatic recovery experiment is not "raise confidence";
it is a geometry/text-feature selector, probably starting with candidate text length,
box width, and body-region bands, scored through the effect audit before promotion.
That selector is now implemented as
`tools/select_candidate_line_reviews_by_geometry_text_features.py`, and the result is
exactly the split the product goal cares about: it is a strong missing-line finder but
still bad automatic OCR text. The precision-first setting
(`min_candidate_text_length=30`, `min_bbox_width=80`, y-band `middle,bottom_or_right`)
selected 31 rows; the oracle-overlap diagnostic labels all 31 as useful missing-line
candidates. Applying their machine candidate text gives line F1 `0.933566`, precision
`0.992565`, recall `0.881188`, 36 missed references, 2 extra predictions, and 31/31
inserted rows matched to references. It is rejected for OCR output because page
codepoint CER worsens to `0.761733`, grapheme CER to `0.792281`, matched-line CER to
`0.091202`, and reading-order accuracy to `0.604292` (selection / metrics /
effect-audit SHA-256
`9469f67f8eb1afb300e25bc332e9566fd92f583b77fbc054953027b9883dd444` /
`22689ccec2f4cdf7b0f9f0e4679ef13756b3423d81b5c45d65b9b9779de60e6d` /
`0ba763fb14cfd1e63ff6eb49313ea668ed908986c2533319e18aa4d148364988`). The
recall-heavier `min_candidate_text_length=10` version selects 38 rows with oracle
diagnostic precision `35/38`, raises line F1 to `0.936097`, but worsens page CER to
`0.764760`, adds 3 extra inserted predictions, and fails the effect audit (selection /
metrics / effect-audit SHA-256
`d3cf35e7049c113fc9793e1da85d98ac8104ac403064fcf5c5765ee9adfe06bd` /
`605dee2ba5cd5cb06578ea786feeb225efceb7a72b3825049b81d031f2464356` /
`5f4aaef0a472f95757342a3da00ca932eab3453b286d9afb40ba61940cb4d6f2`). The right next
move is not another selector threshold. It is to use the 31 precision-first rows as a
review/crop-recognition worklist, improve their text, and fix insertion reading order.
Candidate insertion ordering is now explicit in
`tools/apply_candidate_page_line_reviews.py`. The default `geometry` strategy preserves
old diagnostics but sorts all page lines by raw page/y/x geometry, which caused the
large reading-order drop on Gorkhapatra. `preserve_source_append` keeps the selected
source block order and appends candidate insertions. On the 31-row precision selection,
it keeps line F1 `0.933566`, recall `0.881188`, extras `2`, and 31/31 inserted rows
matched, while improving reading-order accuracy from `0.604292` to `0.750472`; it is
still unsafe because page codepoint CER is `0.762153`, grapheme CER `0.798575`, and
reading order remains below the `conf070` baseline (application / metrics /
effect-audit SHA-256
`66c98b29c9b919333add26d72802e237715317fc331e844d119958d91167b85c` /
`5f11ba613d1ea7e10ecec40cde78424994edb4258069023541384e193b267ed3` /
`56068accf83105e4bd0f526f2ddac0d68fb9a234d52899e35b3ec4bae5ce2743`). The
`source_reading_order` mode re-sorts with the source metadata's
`wide_column_right_to_left` order but is worse here: codepoint CER `0.773003` and
reading order `0.722227` (effect-audit SHA-256
`b62ae4e0be13247a0a8c6af34656058c9412c3bb4fce1b2afd922189511c658e`). This narrows the
remaining problem: use append/preserve semantics for reviewed insertions unless a
better slotting policy is proven, and fix the inserted text.
`tools/prepare_candidate_line_transcription_worklist.py` now turns the 31
precision-first rows into a pending reviewer/crop-recognizer worklist at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist/`.
It emits JSONL, TSV, HTML, and recognizer labels; all rows are pending with blank
`corrected_text`, no oracle/reference fields are emitted, and all 31 HTML crop links
resolve. `tools/audit_candidate_line_transcription_worklist.py` now independently
audits the packet; the real audit passes with 31 JSONL rows, 31 TSV rows, 31 crop links,
31 label rows, 0 missing crops, 0 crop hash mismatches, 0 forbidden fields, 0 nonblank
review fields, and 0 issues. Report / JSONL / TSV / HTML / labels / audit SHA-256:
`7a92ed3d85faee782f22353a778fa36434c6456ca29182a462349ffcc630b180` /
`a98488a08ed6d28944b171fe50577a9958960bd31cb6cc7180a58b3e5b270ea1` /
`f405d1ede74174f0def525bfa3c5bb9819a3c929671cc74d366a5769e48a31b6` /
`3971d08d5bca15286fcceae6326014ad5edbab89a8be1f2656d5e69703cbee72` /
`1bcf9418af1ee056ccf32bfe56fccc7d51d26ffdef52f51c1f24397eadced5c0` /
`86678953c170569196585aaaf25d6d44fb8d031a7c35f705ad791797d1a0d541`.
The reviewed-worklist bridge now exists too. `tools/apply_candidate_line_transcription_worklist.py`
compiles a filled candidate-line transcription worklist into accepted candidate-page
rows, while enforcing non-empty `corrected_text`, non-machine reviewer identity, crop
hash validation, and pending-row no-op semantics. The current real 31-row worklist is
still pending-only, so the pending compile at
`outputs/limbu-sirijonga-page-unresolved-review-2026-06-22/gorkhapatra-auto-split-tall-geometry-text-selection-len30-w80-y-midbot-transcription-worklist-application-pending/`
has `accepted_count=0`, `skipped_count=31`, `issues=[]`, and
`application_ready=false` (report / accepted JSONL / skipped JSONL SHA-256
`a5116e0509ab3612fb83f21298397d57f10a92f32a4f2936f6a2284050577fce` /
`e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855` /
`2f2aaf09512a659629a6cd54928261a3971a4fce07ec0ecc51009331ce18c39e`). Applying that
empty accepted JSONL through `tools/apply_candidate_page_line_reviews.py` with
`preserve_source_append` inserts 0 rows and scores exactly as selected `conf070`:
codepoint CER `0.739361`, grapheme CER `0.768262`, line F1 `0.872458`, 67 missed
references, 2 extras, and reading-order accuracy `0.775478` (application / metrics
SHA-256 `b2c69ab5c835d83fa9814df1c1e0a0f8797f29bf1f28c53230a74b3027324533` /
`5e75442ea41ea8600ef6dd06fc2b246ad61615f5cea6a3107b7e0e3cb090e32b`). This closes the
mechanical loop: once reviewed text is added to the 31 rows, it can be compiled,
inserted, scored, and effect-audited without admitting pending machine text.
The immediate local product target is now a Limbu-first OCR pipeline that handles both
Devanagari-written Limbu and Sirijonga-script Limbu from imperfect page captures. This
is not a video-to-page task; video frames are only one possible capture source. The
router at `outputs/limbu-first-router-v1/model-card.json` is now a guarded
`script_select_composite`: Tesseract `nep+eng` is the Devanagari primary, exported v4
PaddleOCR Sirijonga is the secondary, `routing_signal_engine=secondary` lets Sirijonga
Unicode trigger secondary routing, and `primary_guard_script=devanagari` prevents clear
Devanagari primary lines from being overwritten by Sirijonga hallucinations. The v4
component was exported from `outputs/limbu-recognizer-v4-2026-06-18/limbu-v4.tgz` and
packaged under `outputs/limbu-first-router-v1/components/limbu-sirijonga-paddle-v4/`.
Runtime preflight passes in `/tmp/ocrtech-paddle311` Python 3.11.
Current evidence is operational, not claim-grade: the Sikkim Herald Sirijonga line crop
improved to diagnostic codepoint CER 0.095 against the demo GT but remains in review;
the full Sikkim Herald Sirijonga page still fails quality with 20/22 lines in review;
the Gorkhapatra candidate page runs with 98 lines, 22 review lines, and 7 guarded
Devanagari primary lines, but its references are still pending manual labels. The next
real work is full-page Sirijonga improvement and frozen page-level eval, not another
headline recognizer number.
Post-correction is now admission-gated: deterministic Limbu correction profiles can be
audited with `ocrtech audit-limbu-post-correction-profile`, profile IDs/rules are shape
checked, missing admission evidence marks the profile experimental, and profiles that
claim validated status must carry hashed held-out eval evidence with non-regressing
before/after metrics or `parse-limbu-doc` refuses to apply them. No validated Limbu
profile exists yet; the review queue remains the correction gate.
Output manifests now bind critical inputs by hash: model cards, fallback model cards,
capture-prep metadata, post-correction profiles, source documents, and review TSVs are
recorded in `limbu-output-manifest.json` when supplied, and `audit-limbu-output`
recomputes those hashes. This moves the Limbu output bundle closer to claim-grade
provenance, but page-level scoring against frozen references is still missing.
---
## PER-LANGUAGE CER — goal scorecard (RE-GROUNDED 2026-06-18 with verified micro CER)
**Goal state: a per-script PP-OCR recognizer system supporting Nepal/diaspora
languages, each with a measurable per-language CER.**
> **METRIC CORRECTION (eval-rigor pass).** Every "CER" in earlier versions of this
> table was `1 − PaddleOCR norm_edit_dis`. **That is NOT CER.** `norm_edit_dis` is a
> per-sample-normalized *macro* average; on datasets with many short header/caption
> lines a single error on a 3-char line costs ~0.33, which drags the macro mean far
> above the true (micro) error rate. Re-scoring every model through `infer_rec.py` +
> `tools/eval_metrics.py` (micro CER = total edit distance ÷ total reference length,
> NFC-normalized) gives the numbers below. Predictions were eyeball-verified against
> GT (e.g. magar GT `समावेशीकरणको दिशामा एउटा फड्को :` → PRED identical). Result: the
> Devanagari/Limbu recognizers are **~4–8× better than reported**; Newa/Tirhuta
> (long, uniform synthetic lines, where macro ≈ micro) are roughly as reported and
> genuinely weak. Grapheme-cluster CER (UAX#29) is the honest headline.
| # | Language | Script | **graph CER** | cp CER | exact-line | Eval | prev (mislabeled) |
|---|---|---|---|---|---|---|---|
| 1 | **Limbu/Sirijonga** | Limbu (native) | **0.041** | 0.021 | 0.521 | real Gorkhapatra (303 crops) | 0.176 |
| 2 | Majhi | Devanagari | **0.075** | 0.047 | 0.173 | real Gorkhapatra | 0.26 |
| 3 | Magar | Devanagari | **0.078** | 0.047 | 0.287 | real (⚠ selection-leaked) | 0.22 |
| 4 | Sunuwar (Deva) | Devanagari | **0.080** | 0.054 | 0.178 | real Gorkhapatra | 0.24 |
| 5 | Magahi | Devanagari | **0.083** | 0.054 | 0.158 | real Gorkhapatra | 0.21 |
| 6 | Bhojpuri | Devanagari | **0.101** | 0.061 | 0.157 | real Gorkhapatra | 0.19 |
| 7 | Thami | Devanagari | **0.120** | 0.068 | 0.120 | real Gorkhapatra | 0.33 |
| 8 | Wambule | Devanagari | **0.121** | 0.064 | 0.125 | real Gorkhapatra | 0.30 |
| 9 | Darai | Devanagari | **0.129** | 0.083 | 0.073 | real Gorkhapatra | 0.24 |
| 10 | **Tirhuta** | Tirhuta (native) | **0.343** | 0.250 | 0.084 | held-out **synthetic** | ~0.35 |
| 11 | **Newa/Prachalit** | Newa (native) | **0.613** | 0.418 | 0.071 | held-out **synthetic** | 0.40 |
**Headline (honest):** on real Gorkhapatra print, **Limbu grapheme CER 0.041** (52% of
lines exact) and **8 Devanagari languages grapheme CER 0.075–0.129** (codepoint 0.047–
0.083). These are strong, real-print results. **Newa (0.613) and Tirhuta (0.343) are
weak even on their own synthetic eval** — they fail to fit the synthesis distribution
they trained on (transliterated text + complex native conjuncts + long lines). Newa is
the clear priority to improve.
The lever was DATA, not the model — real data must be **same-font/same-domain** (the
Sikkim Herald *different*-font Limbu fine-tune *hurt* the Gorkhapatra eval; same-font
Gorkhapatra crops helped). Artifacts: `outputs/limbu-recognizer-v4-2026-06-18/`,
`outputs/devanagari-recognizer-v2-2026-06-18/`, `outputs/newa-recognizer-v3-2026-06-18/`,
`outputs/tirhuta-recognizer-v1-2026-06-18/`.
**Residual caveats on the strong numbers:** (a) labels are machine-converted legacy→
Unicode — a low CER partly reflects model+GT sharing the converter's systematic biases,
so true CER vs human-correct text needs native-reader verification; (b) `magar` was the
deva-v2 selection set (mild leak) but sits mid-range, so the leak isn't inflating the
headline; (c) Devanagari/Limbu are real-print, Newa/Tirhuta are synthetic — not directly
comparable.
### Data floor raised (the enabler, 2026-06-18)
Real/large training data harvested this session: Limbu 303→6,867 (real, same-font);
Newa 3.7k→**468k** (Devanagari→Prachalit transliteration); Tirhuta 53→**111k** (Maithili→
Tirhuta translit); Devanagari synth→**79k real**; Sunuwar 692→4,674; Kirat Rai 887→4,343
(+1,011 eval); Sunuwar/Kirat Rai/Gurung Khema +**54k synthetic each** (varied conditions).
Transliteration via an Aksharamukha py3.10 sidecar venv is the scalable lever for any
script with a Devanagari corpus. **Training pipeline (push N):** Tirhuta training now;
Newa-retrain (468k), Sunuwar, Kirat Rai, Gurung Khema queued.
**Honest caveats:** Limbu + 8 Devanagari are real-page CER (machine-converted labels,
human-verification pending; Devanagari was epoch-9 budget-capped + has a train/eval font
gap, so conservative). Newa's number is held-out *synthetic* printed Nepal Lipi — a real
manuscript image set exists (Zenodo, 3,180 crops) but is handwritten/cross-domain, so it
needs a manuscript-trained model for a fair real number. The Newa long-line squish (480px)
caps quality; wider `rec_image_shape` + more printed-register data are the known levers.
**Staged next experts — real-eval-crop status (2026-06-18, all from Sikkim Herald
legacy→Unicode, machine GT, human-verification-pending):**
- **Sunuwar (Jenticha)** — ✅ **429 real crops, 368 unique, 5,132 letters** (NEW,
`data/sunuwar-real-eval-pack/`, `tools/build_sunuwar_real_eval_pack.py`; 3 mukhia
editions; fully_confirmed bytes only, 88.6% byte coverage; `|`→U+11BC5 left unapplied).
Best of the staged set; crops visually verified as clean single-line newspaper text.
- **Lepcha** — ✅ 132 real crops (`data/lepcha-real-eval-pack/`, 4 editions).
- **Kirat Rai** — ✅ 70 real crops (capped: independent typeface, 778 unmapped bytes —
cannot easily expand from the 4,343 raw lines).
- **Gurung Khema — source assessment superseded 2026-07-15.** The historical
`gurung-language-toolkit` and Devanagari→Khema transliterator remain synthetic
training inputs: the map is lossy and non-native-validated, and any CER on those
renders measures fit to the project's transliteration. That route produced 8,291
unique Khema lines / 397,799 characters with
`tools/devanagari_to_gurungkhema.py`; those counts remain synthetic render inputs.
However, the earlier claim that no real Khema print exists is false. Two official
2022 State Portal Sikkim Herald Gurung Editions provide eight real printed pages
and 749 mechanically convertible line candidates. They are not yet an eval pack:
the independent provenance reviewer, adjudication, native pixel-label, crop, split,
and leakage gates remain open. See `RESEARCH.md` r13 and the frozen
`outputs/sikkim-herald-gurung-khema-*-2026-07-15-r13/` artifacts.
Each of the three with crops needs its expert trained (synth corpora ready in
`data/synth-big/`) then scored on its real pack → first-ever real-crop CER. GPU-gated
behind Newa v4-wide.
**Lessons baked in for the next training pass:** warm-start refines need LOW LR (~2e-5,
not 1e-4); augmentation must be MODERATE (heavy aug degrades clean-eval accuracy and slows
from-scratch convergence); filter scriptio-continua lines to ≤ `max_text_length`; always
verify the GPU is clear of stale processes before launch (a lingering process caused an
OOM). GPU instance i-0fa19dff7b233a353 stopped via OS shutdown (SSO expired at teardown —
user should confirm `stopped` on next `aws sso login --profile playaround`).
---
## EVAL-RIGOR PASS (2026-06-18, after a critical Codex review) — RE-SCORE DONE
**RESULT OF THE RE-SCORE (the headline finding):** the metric correction went the
*opposite* way from what was feared. Codex was right that the numbers were not what
they seemed — but the error was that I was *over*-stating CER, not under-stating it.
`1 − norm_edit_dis` is not CER; the true micro CER is much lower on the short-line
Devanagari/Limbu evals (Limbu 0.176→**0.041 grapheme**; Devanagari ~0.25→**0.075–0.129
grapheme**). Newa/Tirhuta (long uniform lines, macro≈micro) stayed weak. All four
models re-scored via `infer_rec.py` + `tools/eval_metrics.py`; predictions
eyeball-verified against GT. The scorecard above is now the honest, defensible table.
Still-open gaps below.
Codex review flagged that our headline CER numbers are softer than presented. Honest
gaps to fix BEFORE pushing N further (ranked impact/effort):
1. **Model-selection-on-test leakage (Very High / Low).** Best checkpoints were picked
on the eval page itself → optimistic. Fix: per-source train/**dev**/**frozen-test**
splits; select on dev, report on a frozen test never used for selection.
- Devanagari: **7 of 8 packs are already clean** (only `magar` was the selection set
during deva-v2 training). Re-score the 7 as clean test; flag magar as leaked or
re-select on a held-out dev.
- Limbu: 0.176 WAS selection-leaked (picked on the one eval page). Needs a 3-way
by-edition split (24 editions) + retrain. NOTE: `data/limbu-gorkhapatra-extra/`
labels lack per-edition tags → re-extract with edition provenance first.
2. **Codepoint vs grapheme-cluster CER (Very High / Low).** Built `tools/eval_metrics.py`
— reports BOTH codepoint CER and **grapheme-cluster CER** (UAX#29 via `regex \X`).
For combining-mark scripts a dropped vowel-sign is 1 codepoint but corrupts a whole
akshara, so codepoint CER *under*-counts: e.g. 2 dropped vowel signs in a 7-cluster /
15-codepoint line = codepoint CER 0.13 but grapheme CER 0.29. Report grapheme CER as
the honest headline; our numbers will tick UP.
3. **Separate synthetic-held-out from real-page CER in every claim (Very High / Low).**
Limbu 0.176 (real page) vs Newa 0.40 / Tirhuta 0.35 (held-out *synthetic*) are NOT
comparable. Newa real test = the Zenodo manuscript (clean, never trained/selected) —
eval v3 on it. Tirhuta = synthetic-only, label as such.
4. **Label verification via active learning (Very High / Med).** Labels are machine-
converted (legacy→Unicode); a model can learn the converter's systematic errors and
report fake progress. Verify high-uncertainty/checkpoint-disagreement lines first.
5. **Build a text detector (High / Med).** Recognition-only CER won't predict page OCR;
detector misses are the largest absent error source.
6. **Transliteration = pretraining, not GT (High / Low).** Fine-tune/eval Newa+Tirhuta on
REAL native text so they're document recognizers, not transliteration-font recognizers.
7. **Blind spots:** consistent NFC + grapheme-cluster eval; native-reader error review
(errors on meaningful marks/names matter more than CER); licensing/provenance for any
public dataset/model release. "First-ever OCR" claim should narrow to "recognition
model under a defined eval" until detector + verified labels + frozen test land.
**Status:**
- ✅ **(2) Codepoint + grapheme-cluster CER** — `tools/eval_metrics.py` built; all 4
models re-scored on instance via `/home/ubuntu/re_score.py`. Scorecard re-grounded.
- ✅ **(3) Synthetic vs real separated** — table now labels each eval; Newa/Tirhuta
flagged synthetic. (Newa-on-Zenodo real test still pending — manuscript domain.)
- ◐ **(1) Selection leakage** — Devanagari: 7/8 packs confirmed clean test, `magar`
flagged leaked (mid-range, not inflating). Limbu still selection-leaked → by-edition
3-way split + retrain still TODO (labels need per-edition provenance re-extract).
- ☐ **(4) Label verification** (native-reader / active-learning), **(5) detector**,
**(6) real-text fine-tune for Newa/Tirhuta**, **(7) NFC + provenance discipline** — open.
Next concrete moves: (a) Newa is the weakest real lever — retrain with wider
`rec_image_shape` + real printed-register Nepal Lipi (not just transliteration); (b) Limbu
by-edition frozen-test split to kill the last selection leak; (c) native-reader spot-check
of ~100 Devanagari lines to bound the converter-bias question.
### Newa v4-wide (RUNNING 2026-06-18) — Codex-prescribed width fix
Codex review + a no-GPU render audit pinned the Newa failure (0.613 graph CER on its OWN
single-font synthetic eval) to a **mechanical squish**, not fonts: v3 rendered at native
widths p50=454 / p90=1051 / p99=1292 px but the recipe resized to **48×480**, crushing
**45% of training images** (p90 = 2.2× horizontal compression) so the dense Prachalit
conjunct stacks smear illegibly. Render fidelity itself is fine (clean glyphs, no tofu);
the source images are already full-res, so no re-synthesis was needed. The "lots of fonts"
premise is false — only 2 Unicode Newa fonts exist (NotoSansNewa, NepalLipiUnicode) and v3
used one; fonts are not the lever until the model first fits its own single-font set.
**v4 = controlled width experiment:** image_shape 480→**640** (800 OOM'd at batch 64),
RecConAug **disabled**, batch 48, 40 epochs, same data/font/warm-start as v3.
`runs/newa-v4-wide-2026-06-18/`.
**VERDICT (epoch 8, stopped early): width alone did NOT rescue Newa.** Re-scored micro CER
cp=**0.628** / graph=**0.794** — *worse* than v3 (cp 0.418/graph 0.613), despite an easier
640px eval; `norm_edit_dis` plateauing ~0.55. v4 is undertrained (8 vs v3's 20 epochs) so
this is not a clean negative, but it is clearly not on track for Codex's hoped 0.25–0.40 —
confirming the real bottleneck is **transliteration/data fidelity, not pixels**. Stopped to
free the GPU for the staged-expert queue; `best_accuracy` checkpoint preserved for a
full-length resume if a clean width verdict is wanted later. The real Newa v5 fix =
better/native-validated text + real printed-register data, not just shape.
### Staged-expert queue (2026-06-18→19, autonomous; stopped after Sunuwar per user)
`run_queue.sh` warm-started from the Devanagari pretrain, 640px, batch 32, 6h-cap, re-scored
on the REAL eval pack, then `shutdown -h now`. User redirected mid-run to stop after Sunuwar
(Kirat Rai/Lepcha skipped, corpora staged on EBS for later).
**Sunuwar — FIRST-EVER real-crop CER: cp_CER = graph_CER = 0.342, exact-line 0.007**
(n=429 real Sikkim Herald crops; ~epoch 16, 6h cap). cp==graph because Jenticha is an
alphabet (no combining-mark clusters). **34% CER = not usable yet, and the cause is the
predicted synth→real FONT GAP:** trained on synthetic NotoSansSunuwar but the real crops are
the Sikkim Herald newspaper typeface.
### SAME-FONT RETRAIN (2026-06-19) — data-floor thesis CONFIRMED for Sunuwar
Harvested **3,010 real same-font Sunuwar crops** from 22 Herald editions
(`tools/harvest_sunuwar_train_crops.py`; converter verified uniform 87–90% across all 25
editions; 3 mukhia editions held out as the 429-crop eval — clean by-edition split). Fine-tuned
the synth model on them (warm-start, LR 2e-5, 60 epochs).
| Sunuwar | synth-only | **same-font** |
|---|---|---|
| codepoint CER | 0.342 | **0.121** |
| grapheme CER | 0.342 | **0.121** |
| exact-line | 0.007 | **0.184** |
**0.342 → 0.121 (−65% relative), exact 0.7%→18.4%** — purely from real same-font data, the
same lever that took Limbu to 0.041. Not as deep (3,010 crops vs Limbu's 6,867; converter
~89% byte-confirmed → some label noise). More editions / higher byte-coverage would push lower.
**Lepcha + Kirat Rai stay blocked** for same-font: only their eval-set editions decode
(Lepcha's converter returns 0 on all 26 non-eval editions; Kirat Rai's typeface uncracked) —
deferred pending font-decoding work. **Newa** got a separate fix: trained on 4,456 real *native*
Prachalit lines (not transliteration) — result pending. Devanagari + Limbu untouched.
## Verified state at start (read from code + disk, not assumed)
Sources: `Agent` sweeps over `training.py`, `models.py`, `source_audit.py`,
`limbu_legacy.py`, `language_registry.py`, `languages/nepal-v1.json`,
`docs/REMOTE_TRAINING.md`, and disk inventory of `data/`, `runs/`, `outputs/`.
### Training
- `ocrtech train-recognizer` is a **recipe/config generator** (writes PaddleOCR
label files, dictionary, `paddle_recognizer.yml`, summary) with an optional
`--run` that shells out to `python PaddleOCR/tools/train.py`. Infra is complete:
remote host audit/bootstrap/sync + AWS G6 GPU training have all been exercised.
- **All recorded fine-tunes are the OLD mixed Nepali/English PP-OCRv3 effort, and
all failed admission.** `outputs/recognizer-failure-review-paddle-ppocrv3`
recommends `stop_family` (paired hard-50: `0` wins / `545` losses / `7` ties).
HF TrOCR `recognizer-next-v1` trained (eval CER 0.188) but still lost to
Tesseract on benchmark.
- **No script-specific (Limbu/Sirijonga or any Nepal-language) recognizer has ever
been trained.** `runs/sirijonga-limbu-paddle-{ablation-clean,ablation-lighting,ablation-camera,recipe-smoke}*`
are **recipes only** (labels + yml + summary); there are **zero `.pdparams`
checkpoints** anywhere under `runs/`.
- **Lesson (do not repeat):** the dead family is mixed-Devanagari PP-OCRv3 mobile
fine-tuning. The new approach must be script-specific.
### Real validation (the claim's test set)
- **Zero verified real references exist for any language.** The one identified
Limbu page (`gorkhapatra-language-page-0001-01`, date 2083-02-12, PDF page 6,
font `NamdhinggoSILL`, 304 target spans) is stuck at `pending=298` manual line
reviews.
- The verification pipeline (`prepare-…-verification-bundle` →
`apply-…-verification-bundle`) only sets `claim_evidence_eligible=true` after
per-line human `review_status`. It does **not** ingest PDF-native text.
- But `extract-gorkhapatra-language-page-pdf-text` already extracts legacy
Namdhinggo spans (bbox/font/codepoints) and `limbu_legacy.py` already converts
them to Unicode Limbu (~9,957 chars). **This converted text is not wired into the
verified-reference path** — so the pipeline currently demands transcription it
does not need. (Matches the handoff pitfall: don't ask for manual transcription
when PDF-native extraction already yields usable Unicode.)
### Per-language evidence (triage so far)
- **Limbu/Sirijonga** — only confirmed **native-script** evidence; text layer is
**legacy-font** (Namdhinggo), deterministically convertible to Unicode.
- **Live-pack (untriaged, `script=unknown`, PDFs on disk):** संस्कृत (Sanskrit,
pages 6–7), मगर (Magar), वाम्बुले (Wambule), राना थारु (Rana Tharu) — 8 page
candidates across epapers `2083-02-31` and `2083-03-01`.
- Most other registry languages are Devanagari-transcribed synthesis text with no
Gorkhapatra page identified yet.
---
## Critical path to the goal
1. **Triage matrix** — classify each Gorkhapatra page's text-layer form per
language: `unicode_native` / `legacy_font` / `devanagari_transcription` /
`latin_notation` / `no_text_layer`. Sets N and per-language script form.
2. **First real eval (Limbu)** — wire PDF-native legacy→Unicode into the verified
reference path so the Limbu reference becomes claim-eligible **without** 298
from-scratch transcriptions (pre-fill + human sign-off, not manual typing).
3. **Baselines on the real Limbu page** — stock PP-OCRv5/v6, Tesseract, Surya,
PaddleOCR-VL. Records the bar the fine-tune must beat.
4. **First script-specific fine-tune (Limbu/Sirijonga)** — train the existing
Sirijonga recipe on a GPU host (AWS G6 path exists); eval on the real Limbu page
+ held-out synthetic with no text-hash leakage.
5. **Generalize per qualifying language** — repeat triage → real eval → baseline →
fine-tune. Decide single-multilingual vs per-script model from the matrix.
---
## Checklist
### Phase 1 — Triage matrix (evidence decides N)
- [x] Inventory existing Gorkhapatra packs + confirm PDFs on disk.
- [x] Run `extract-gorkhapatra-language-page-pdf-text` on the live-pack (all 8
pages, 4 candidate languages). Result: `passed=True samples=8/8 spans=3486`.
- [x] Classify each page's text-layer form. **All pages are `legacy_font`** — no
Unicode text layer exists anywhere in the Gorkhapatra epapers.
- [x] Record the matrix (below) + key findings.
- [ ] Disambiguate live-pack language→page alignment (2 languages currently map to
the same physical page — see findings).
### Phase 2 — First real eval (Limbu, no manual transcription)
**Design (locked, grounded in code):** Add a PDF-native legacy-conversion *source*
for the draft reference's `reading_order`. New command
`prepare-gorkhapatra-language-page-pdf-native-references` emits a draft
`.ref.json` per accepted page whose `reading_order` items are
`{id, text=<legacy→Unicode>, bbox, confidence}` from the converted target-font
spans (`limbu_legacy.convert_gorkhapatra_limbu_pdf_text` rows), with
`reference_status=draft`, `claim_evidence_eligible=false`, `machine_generated=true`,
`requires_human_review=true`, `assisted_source=pdf_native_legacy_conversion`.
Then the **existing** chain runs unchanged, preserving every claim gate:
`prepare-…-verification-bundle` (pre-fills `ocr_text`=converted Unicode + crops +
dashboard) → human accept/correct in dashboard → `audit-…-verification-csv` →
`apply-…-verification-bundle --require-codepoint-range U+1900..U+194F
--tables-reviewed --figures-reviewed --captions-reviewed` →
`claim_evidence_eligible=true`. Reviewer effort drops from "transcribe 298 lines"
to "accept pre-filled correct Unicode".
Why this shape: `apply` builds verified text from the verification CSV's `ocr_text`
(on `accept`) keyed by `line_id`, and pulls tables/figures from the draft ref
(`source_audit.py:3679-3745`). Sourcing the draft `reading_order` from the
conversion is the *only* change; gates, dashboard, and audits are reused as-is.
Implementation gotchas to handle:
- bbox format: pdf-text spans look like `[x0,y0,x1,y1]`; the bundle's
`_clean_bbox_xywh`/`_write_xywh_crop` expect `[x,y,w,h]`. Convert.
- conversion completeness: 36/304 Limbu spans have `unmapped_codepoints`. Flag
those rows in `notes` (and/or lower confidence) so the reviewer scrutinizes them;
do not present them as clean. Clean rows (268) are confident accepts.
- bboxes are already in rendered-image pixel space (1985×3062 ≈ span extent), so no
DPI rescale is required.
- [x] Decide the integration shape — new `prepare-…-pdf-native-references` command
feeding the existing verification→apply→audit chain (above).
- [x] Implement the command + reading_order-from-conversion builder (bbox xywh
conversion via per-sample image/PDF-point scale, unmapped-codepoint flagging,
generic `review_note` surfaced in the verification CSV). Code:
`source_audit.py` (`prepare_gorkhapatra_language_page_pdf_native_references`,
`_pdf_native_language_page_reference`, `_bbox_points_to_image_xywh`,
`_image_pdf_point_scale`); CLI `prepare-gorkhapatra-language-page-pdf-native-references`.
- [x] Generate the Limbu draft + verification bundle on real data:
draft `converted=1` (303 spans, 11,890 Unicode chars, scale≈2.0008, 36 spans
flagged — almost all benign smart-quote punctuation U+2018/U+2019, not failures);
bundle `lines=303 crops=303 missing_bboxes=0`, all 303 `ocr_text` pre-filled.
Artifacts: `data/limbu-gorkhapatra-language-page-pdf-native-references-2026-06-16/`,
`data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/`.
- [x] Add fixture tests (`tests/test_source_audit.py`): bbox scaling; draft is
Unicode + non-claim-ready; an accepted draft passes the apply claim gate with the
`U+1900..U+194F` range check. Focused suites green; `git diff --check` clean.
- [ ] **Human sign-off (now a light accept pass, not transcription)** — open the
confirmed PDF text dashboard, spot-check converted Unicode against the clickable
bounded spans/crops, scrutinize the 36 flagged spans, set `review_status` per
line in the downstream verification CSV, then run `apply` (commands below).
Agents must NOT fabricate this review; the verified reference asserts a human
checked it.
- [ ] After sign-off: pass
`audit-gorkhapatra-language-page-review --require-verified-references`.
#### Commands to finish the Limbu real eval (human sign-off)
```bash
# 1. Review in the confirmed browser dashboard: full PDF page + clickable boxes
open /Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html
# use that dashboard to verify/correct the downstream verification-lines.csv;
# do not open the deleted/broken pdf-native verification-review.html
# 2. Dry-run the status counts
uv run ocrtech audit-gorkhapatra-language-page-verification-csv \
data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-lines.csv \
--out outputs/limbu-pdf-native-verification-csv-audit
# 3. Apply -> claim-eligible verified reference (note: --require-codepoint-range U+1900..U+194F)
uv run ocrtech apply-gorkhapatra-language-page-verification-bundle \
data/limbu-gorkhapatra-language-page-pack-2026-06-16/manifests/gorkhapatra-language-page-pack.jsonl \
data/limbu-gorkhapatra-language-page-review-2026-06-16/gorkhapatra-language-page-review.csv \
data/limbu-gorkhapatra-language-page-pdf-native-verification-bundle-2026-06-16/gorkhapatra-language-page-verification-lines.csv \
data/limbu-gorkhapatra-language-page-pdf-native-references-2026-06-16 \
--candidate-language लिम्बू \
--require-codepoint-range U+1900..U+194F \
--tables-reviewed --figures-reviewed --captions-reviewed \
--out data/limbu-gorkhapatra-language-page-verified-2026-06-16
```
### Phase 3 — Baselines on real Limbu page
- [x] **Real-eval → recognizer bridge built + validated.** New command
`prepare-gorkhapatra-language-page-recognizer-eval` turns a verified reference's
`reading_order` into tight line crops + a recognizer eval manifest
(`ManifestEntry`, split=eval, slices incl. `real`/`hard_eval`/`recognizer_eval`/
`language:<X>`), mirroring `crops.py` conventions so it plugs into
`audit-recognizer-corpus`/benchmark. Smoke on the Limbu draft (`--allow-draft`):
303 tight line crops (e.g. 177×29 px), emitted manifest passes `audit-manifest`
(303 samples). Honest gating: draft rows are `claim_evidence_eligible=false`; only
verified references yield claim-grade real-eval rows. Tests added; suites green.
- [ ] After Limbu sign-off: re-run the bridge **without** `--allow-draft` on the
verified reference → claim-eligible real recognizer eval manifest.
- [ ] Run stock PP-OCRv5/v6, Tesseract, Surya, PaddleOCR-VL on the real Limbu line
crops; record CER/line-accuracy (expected: stock engines largely fail on Sirijonga
— that failure is the baseline the fine-tune must beat).
### Phase 4 — First Limbu/Sirijonga fine-tune
**GPU host (found):** `ocrtech-g6-training-20260615-v4` = `i-0fa19dff7b233a353`,
g6.xlarge, **stopped**, us-east-1a, AWS account 798006441552. Auth: `aws` CLI works
via `--profile playaround` (the `admin`/default profile is for a different account).
Starting the instance costs money (~$0.8/hr g6.xlarge) → confirm before launch.
**Base-model correction (important):** the existing recipes under
`runs/sirijonga-limbu-paddle-*` use base
`devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams` — which is **both the dead
family** (`recognizer-failure-review` → `stop_family`) **and script-mismatched**
(Devanagari weights for a U+1900–194F script). Per
`MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md` §3, Limbu needs a **separate
recognizer + dictionary**, not the Devanagari base. So regenerate the recipe with
the Limbu dict and no Devanagari pretrained head (from-scratch / backbone-init).
**Inputs ready (free prep done):**
- Limbu dict: `data/dictionaries/limbu-sirijonga-v1.txt` (76 chars; covers all 68
required Limbu-range chars; 0 missing vs labels).
- Augmented rendered split: `outputs/sirijonga-limbu-rendered-split-namdhinggo-augmented-2026-06-16/`
— 900 train / 162 eval rows (six degradation profiles, text-hash isolated).
**Validated training approach (research fan-out, 2026-06-16, cited).** A 4-agent
research pass against current PaddleOCR sources overturned the initial from-scratch
PP-OCRv3 assumption:
- **Harness is stale.** `training.py` hardcodes PP-OCRv3 mobile
(`MobileNetV1Enhance` + SAR head) + `devanagari_PP-OCRv3_mobile_rec_pretrained`.
Current shipping gen is **PP-OCRv6** (`paddleocr` 3.7.0, 2026-06-11); PP-OCRv5 is
the documented Brahmic-validated baseline. All current rec is **SVTR-based**;
CRNN/MobileNetV1+SAR are legacy. `tools/train.py -c config.yml` stays (current).
- **Fine-tune, don't train from scratch.** For a disjoint new script, warm-start
from a pretrained rec, **reuse the SVTR backbone, reset only the head** for the
Limbu dict (PaddleOCR skip-loads the mismatched head). Backbone transfers across
disjoint scripts; from-scratch overfits our small set.
- **Recipe:** base `configs/rec/PP-OCRv5/multi_language/devanagari_PP-OCRv5_mobile_rec.yaml`
(SVTR_LCNet / PPLCNetV3 + CTC+NRTR MultiHead), pretrained
`PP-OCRv5_mobile_rec_pretrained.pdparams` (or `devanagari_PP-OCRv5...` for a
Brahmic warm-start). **Mobile tier** (server/SVTRv2 overfit on <9k lines). LR
**1e-4**, `max_text_length` **50** (verify vs labels), `rec_image_shape [3,48,320]`,
MultiScaleSampler, RecConAug+RecAug, bs 64, ~100 epochs @9k / 200–300 @900.
- **Data caveats:** ~900 augmented lines is **below PaddleOCR's ~5,000 finetune
floor** → smoke/pipeline check only, not a deliverable. Scale toward the ~8,800
unique Limbu lines (render the fuller limdic corpus, not the 200-hash subset).
Enforce **one canonical Limbu codepoint ordering** (NFC + subjoiner/vowel reorder —
reuse `limbu_legacy.py`) across all labels AND CER. Dedup train/eval by image hash
AND (text, font); freeze the real newspaper eval. Heavy degradation aug
(Augraphy/ocrodeg-style) + a small real-data fine-tune close the synthetic→real gap.
- **Greenfield:** no published Limbu OCR / Tesseract model exists — first of its
kind. Closest proven recipe: IIIT Indic CRNN+CTC, synthetic pretrain → small real
fine-tune.
**DATA-PLAN CORRECTION (methodology review, 2026-06-16).** Sampling exposed a
killer hole: the limdic synthesis corpus is **dictionary headwords** (69% single
words, median 7 chars/1 word) while the real eval is **newspaper prose** (median 39
chars/5 words). "More data" of the same kind deepens the mismatch. Root cause: the
synthesis used the *worst* available source. Investigation found native-Sirijonga
**prose** corpora already migrated under
`…/limbu-speech-toolkit/data/external/` that were never used:
- `pustakalaya/limbu_unicode/limbu-bhasha-kaksha-7.txt` etc. (school textbooks):
5,127 lines, **median 40 chars / 7 words ≈ exact eval match**; thousands more
across grades/subjects + `cdc_moecdc`, `curriculum_*`, `grammars`, `theses_tu`.
- `web_limbu/limbu_sentences.txt` (535, 25/4) and `web_limbu/raw/sikkim_herald_decoded.tsv`
(~514) — **Sikkim Herald newspaper** = best register match, different paper → no
eval leakage.
- `ebible_lif/biblenlp_lif-lifNT_sirijanga.txt` (7,873 verses, long).
- (IGT corpus is romanized Latin; LIFWBT Bible is Devanagari-script — NOT Sirijonga.
A `DevUni→LimUni` SIL map exists to transliterate the 9,283 Devanagari verses if
more is needed.)
Corrected data plan: build the Limbu training corpus from **textbook + newspaper +
web Sirijonga prose** (eval-matched line length), keep limdic only for lexical
coverage, track per-source license_status (textbooks/Bible = pending_review).
**HARVEST DONE (4 sonnet agents, 2026-06-16).** Extracted + consolidated native
Sirijonga prose → `outputs/limbu-prose-harvest/limbu-prose-corpus.jsonl`:
**10,285 unique lines, median 47 chars / 8 words** (≈ eval's 39/5), **0 eval
leakage** (NFC-normalized, deduped, excluded the 303 Gorkhapatra eval lines).
Sources: 9,710 textbooks (E-Pustakalaya/CDC) + 514 Sikkim Herald newspaper + 27
Wikipedia incubator + 25 pangloss + 8 UDHR. Archive.org yielded 0 (all English/
Devanagari *about* Limbu); academic mostly romanized/IPA. **License tiers (research
use only; clear before any redistribution):** textbooks CC-BY-NC-ND 3.0 (derivative
text — internal only); Sikkim Herald = govt bulletin, **silver** (legacy-decoded,
not gold); Wikipedia/Wiktionary CC-BY-SA; pangloss CC-BY-NC-SA; UDHR free.
More available later: offline Crúbadán (~340k words), empty textbook grades 2&5.
Note for recipe: eval lines run to 114 chars / 16 words → set `max_text_length` ~120
and consider a wider `rec_image_shape` than 320px for long newspaper lines.
**SHAPING HOLE (Hole 8, critical, must fix before GPU).** The line renderer
(`synthesis_resources.py:2354`) uses plain Pillow `draw.text` with **no
`layout_engine=LAYOUT_RAQM`**, and **`PIL.features.check('raqm')` is False** in this
env → no HarfBuzz complex shaping. Limbu stacks subjoined consonants (U+1929–192B) +
reorders vowel signs; basic layout has no GPOS/GSUB guarantee, so synthetic Sirijonga
crops may be mis-shaped vs the correctly-typeset real page — and prior "passed" render
audits never checked visual shaping (only font SHA / label coverage / degradation).
A basic-render vs real-crop visual diff looked *broadly similar* (Namdhinggo may
position marks acceptably without GPOS), but this is unverified-safe. Fix: render with
`LAYOUT_RAQM` (libraqm trivial on the Ubuntu G6) and visually verify a sample against
real crops before the full render+train. Add a shaping check to the render audit.
**SHAPING RESOLVED (2026-06-16) — locally, no GPU needed.** `brew install libraqm`
(+ fribidi) then rebuilt Pillow from source (`uv pip install --reinstall --no-binary
pillow pillow` with `PKG_CONFIG_PATH` to brew) → `PIL.features.check('raqm')` is now
**True**. Renderer (`synthesis_resources.py`) switched to `ImageFont.Layout.RAQM` when
available (falls back to BASIC). Finding: for **Namdhinggo, basic≈raqm** (a visual diff
of identical text matched) — so prior Limbu renders were NOT broken; but raqm is
**essential for Devanagari** (the other languages) and is now the default. Validated:
the 10,285-line prose corpus → synthesis-text manifest
(`data/synthesis-text/cdjk-limbu-prose-2026-06-16/limbu-prose-text.jsonl`) → rendered
with raqm produces correct Sirijonga (e.g. `'ᤁᤡᤖᤠᤋ᤻ ᤕᤠᤰᤌᤢᤱ ᤐᤠᤏ᤻'`). **Corrected
training-data pipeline (right corpus + correct shaping + PaddleOCR labels) is proven.**
**Dict fix (Hole 3, verified):** `data/dictionaries/limbu-sirijonga-v1.txt` is
missing 6 punctuation chars the real page uses — U+0027 `'`, U+0028 `(`, U+002A `*`,
U+002F `/`, U+2018/U+2019 `'` `'`. The recognizer can't emit them until added.
(The dict was audited against synthetic dictionary labels, not real prose.)
**v1 scope decisions (autonomy):** single Gorkhapatra page as honestly-scoped v1
proof-of-concept (expand editions before any paper claim); **clean-digital
recognizer first** (matches the clean-PDF eval; defer scan/photo-degradation +
multi-font robustness to v2). Eval is recognizer-only with oracle (PDF) line boxes
and machine-converted+human-verified labels — label the claim accordingly.
**Code change required:** `write_paddle_recognizer_recipe` (`training.py`) must emit
the PP-OCRv5 mobile skeleton (PPLCNetV3 + CTC/NRTR MultiHead, MultiScaleSampler,
LR 1e-4, `pretrained_model` → PP-OCRv5 mobile rec) instead of the hardcoded PP-OCRv3
config. The from-scratch PP-OCRv3 recipe generated earlier
(`runs/sirijonga-limbu-paddle-rec-fromscratch-2026-06-16`) is **superseded — discard**.
**Sequence (corrected): get the recipe + data right, then one GPU run.**
- [x] Update `training.py` to emit the PP-OCRv5 mobile recipe (PPLCNetV3 + CTC/NRTR,
NRTRLoss, `gtc_encode: NRTRLabelEncode`, anchored `max_text_length`) + added
`PADDLE_PPOCRV5_MOBILE_REC_PRETRAINED` and switched the function + CLI `--base-model`
defaults off the dead PP-OCRv3 base. Guard tests updated to pass the stopped base
explicitly; `test_training_benchmark.py` green (49 tests).
- [x] Built the right corpus + correct shaping: 10,285 prose lines → leakage-safe
split (train 9,209 / eval 1,076) → raqm render (**18,418 train + 1,076 eval images**).
- [x] Assembled the full PP-OCRv5 training package:
`runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/` (recipe yml w/ PP-OCRv5 pretrained
+ v2 dict; 18,418 train + 1,076 eval labels). **Ready to ship to GPU.**
- [~] G6 run IN PROGRESS (2026-06-16): instance i-0fa19dff7b233a353 running at
3.235.87.54 (NVIDIA L4 23GB, CUDA driver 13.0, Python 3.10); SSH key
`~/.ssh/ocrtech-g6-training-20260615.pem`, user `ubuntu`. Fresh instance — env
setup running (`~/setup.sh` → venv + paddlepaddle-gpu cu126 + PaddleOCR main +
requirements, logging `~/setup.log`). Bundle
`outputs/limbu-ppocrv5-bundle-2026-06-16/limbu-ppocrv5-training.tar.gz` (230MB,
19,501 files) transferring via scp. Next: extract → download PP-OCRv5 pretrained →
`tools/train.py -c paddle_recognizer.yml` (nohup, ~1-2h) → export → eval.
- [~] TRAINING LIVE (2026-06-16 ~18:12): PP-OCRv5 mobile fine-tune running on the L4,
no errors. ~0.94 s/iter, ~68 samples/s; by epoch 2 `norm_edit_dis` ≈ 0.13 and rising,
loss falling (CTC ~43 + NRTR ~2.9), GPU mem 13/23GB. PaddleOCR ETA ~7h24m for full
100 epochs (≈$6). **Decision: leave at 100 epochs, stop early on eval plateau**
(per-epoch checkpoints + best-metric tracking → no loss from early stop). Background
watcher polls eval metric → pulls me back on plateau / ~28-min check-in / done / error
to evaluate the best checkpoint against the real Limbu recognizer-eval manifest.
Launch cmd: `python ~/PaddleOCR/tools/train.py -c <recipe> -o
Global.pretrained_model=pretrain/PP-OCRv5_mobile_rec_pretrained` from `~/limbu-train`.
- [ ] Eval on real Limbu recognizer-eval manifest (post sign-off) + held-out
synthetic; record CER/norm_edit_dis. (Stock baseline deferred — known to fail on
Sirijonga.)
- [ ] Pass `audit-augmentation-ablation` with real verified evidence present.
### Phase 5 — Second language (Devanagari): converter SOLVED + real-eval STAGED
**Update (2026-06-17):** the verified-converter blocker is resolved.
- Built **`nepal-ttf2utf`** (public: github.com/Ampixa/nepal-ttf2utf, MIT) — legacy
ASCII-font → Unicode for Nepal scripts: Devanagari (preeti/kantipur/sagarmatha/pcs/
fontasy via tested `npttf2utf`) **+ nayanepal/Gorkhapatra extensions** (ƒ→र, †→्,
validated 97–99% clean Devanagari on real pages) **+ Limbu/Sirijonga** (SIL Namdhinggo
map). 9 tests green. Blog draft submitted to Ampixa (passed style check, queued).
- **Magar Devanagari real-eval STAGED:** `outputs/magar-real-eval-pack/` — 383 line crops
+ Unicode ground truth (via the converter), 13,561 chars. This is the first real
Gorkhapatra Devanagari-language eval set (couldn't exist before the converter).
- **Remaining for N≥2:** train a Devanagari PP-OCRv5 recognizer (harvest Devanagari
prose like Limbu → render w/ raqm → train on GPU) → eval on the Magar real-eval pack →
per-language CER. The converter/eval-data blocker is now gone; this is one GPU campaign.
### Phase 5 — Second language (Devanagari): (prior) blocked on a verified Preeti converter
Investigated (2026-06-16) to reach N≥2. The real Gorkhapatra Devanagari language pages
(Sanskrit/Magar/Wambule/Rana Tharu) have **no available Unicode ground truth**:
- `nayanepal`/`Gorkhapatra-Bold` fonts have **generic glyph names** → no derivable map.
- The web article body is published as an **image**, not text (only the Nepali title is
Unicode) — so no web cross-reference for the body.
- No verified Preeti/Devanagari-legacy converter in the migration or pip-installable here.
The spans **are** Preeti (`g]kfn`→नेपाल, `du/`→मगर, `dxfjL/ k\"g du/`→महावीर पून मगर), but a
hand-built Preeti map is only ~80% accurate (garbles e.g. `गूमँ७ाअाे` — digit/sign errors).
**Decision: do NOT build a Devanagari real-eval on noisy labels** (would conflate model
error with ~20% label noise — violates the real-validation discipline). N≥2 is a scoped
follow-up needing ONE verified component: a tested Preeti converter (or an expert/anchor-
validated table). Then all 4 Devanagari pages convert and the ready ~200k-row Devanagari
corpus trains a recognizer → languages #2–5 in one pass. **Status: N=1 (Limbu) done;
N≥2 pending verified converter.**
### Phase 5 — Generalize (hold until the Limbu loop closes)
Recon (2026-06-16): no `nayanepal`/Preeti/Devanagari legacy map exists in the repo
(only `data/mappings/limbu-legacy/Limbu.map`). Two paths to Devanagari-page references:
- **(A) nayanepal→Unicode map** — deterministic, bbox-aligned like Limbu, but requires
sourcing/deriving the real font encoding (embedded `nayanepal.ttf` files are saved
under the pdf-text `fonts/` dirs). Do NOT assume Preeti-compatibility — verify.
- **(B) article web-text cross-reference** — each page has an `article_url`
(`gorkhapatra.../news/<id>`) and the website serves Unicode; align web text to legacy
PDF spans for bbox-level line text. Likely cleaner than reverse-engineering the font.
- [ ] Resolve live-pack language→page ambiguity (2 languages per page) first.
- [ ] Decide path A vs B for Devanagari references; build the converter/aligner.
- [ ] Repeat triage → real eval → baseline → fine-tune per qualifying language.
- [ ] Decide single-multilingual vs per-script model shape.
- [ ] Per-language support table (native vs fallback, CER) — the deliverable.
---
## OVERNIGHT — multi-language push (2026-06-17, autonomous)
Mission: deliver **multiple languages with real-page CER** by morning. Strategy: one
**Devanagari** PP-OCRv5 recognizer evaluated on *each* Devanagari language's real
Gorkhapatra page (ground truth via the new `nepal-ttf2utf` converter) → Magar + Wambule;
plus the existing Limbu recognizer → **3 languages**.
**PRINT CORPUS FOUND (Naya Nepal archive).** Gorkhapatra's *Naya Nepal* publishes a
**daily page in a different language** — a scan of ~100 recent issues found **50 distinct
languages** (वाम्बुले, मगही, सुनुवार, थामी, राजवंशी, कुलुङ, बान्तावा, मैथिली, मगर, भोजपुरी,
माझी, शेर्पा, सन्थाली, याक्खा, धिमाल, गुरुङ, ल्होमी, **लिम्बू**, **तामाङ**, चाम्लिङ राई, **उर्दू**…).
Each is a real printed page (epaper PDF). **Almost all are Devanagari** → our ONE Devanagari
recognizer + `nepal-ttf2utf` scores ~47 of them. Genuinely non-Devanagari: **Limbu**
(Sirijonga, done), **Urdu** (Perso-Arabic — separate recognizer needed), and a few
Tibetan-sphere langs (likely Devanagari in print).
**9 languages staged for real-eval** (real Gorkhapatra pages → Unicode ground truth via the
converter): **Limbu** (Sirijonga) + **8 Devanagari** — Magar, Wambule, Darai, Magahi,
Sunuwar, Majhi, Bhojpuri, Thami (362–444 line crops each). 8 Devanagari packs shipped to the
G6 (`~/deva-train/real_eval/`); scored by the one recognizer once it plateaus.
State:
- **Converter enabler shipped:** `nepal-ttf2utf` (public, blogged). Unlocks Devanagari
real ground truth from legacy `nayanepal` pages (97–99% clean).
- **Real-eval packs staged (3 languages):** `outputs/{limbu,magar,wambule}-real-eval-pack/`
(Limbu 303 / Magar 383 / Wambule 383 line crops + Unicode labels). Magar & Wambule are
distinct Devanagari language pages (मगर भाषा / वाम्बुले राई).
- **Devanagari corpus:** 411,996 eval-matched prose lines (median 40 chars / 7 words ≈ eval
39/6; segmented from migration Devanagari sources, 0 eval leakage). Sampled 40k →
split train 35,937 / eval 4,063 → rendered **71,874 + 4,063** images (raqm, Noto Serif
Devanagari). Dict `devanagari-v1.txt` (141 chars, 75 Devanagari).
- **Recipe:** PP-OCRv5 mobile, **same-script `devanagari_PP-OCRv5` warm start** (verified
HTTP 200), LR 1e-4.
- **GPU:** ~4h budget on playaround (user-authorized start/stop). Capacity-retry poller
restarting the G6; bundle building. Plan: sync → train (stop-early on plateau, ~1–2h
like Limbu) → eval on Magar + Wambule packs → per-language CER → **stop instance**.
- **Honest caveat:** Devanagari renders use Noto Serif, eval is the `nayanepal` newspaper
face → a train/eval font gap (unlike font-matched Limbu), so Devanagari real CER will be
conservative. Labels are machine-converted (human-verification-pending), single page each.
## NON-DEVANAGARI NOVEL SCRIPTS — the real headline (2026-06-17)
User direction: *"those which aren't Devanagari will be our proper headline"* and
*"don't care we can synthesize. we will find other sources for validation"* — so
corpus/validation scarcity is **not** a blocker. With synthesis-from-font, the only
hard gates are **(1) the script is Unicode-encoded** (a target to OCR *into*) and
**(2) a usable font exists** (to render training data). Validation pages are sourced
separately (Sikkim Herald prints Kirat Rai / Sunuwar / Lepcha editions, etc.).
**Research: 7 sonnet agents** (Magar Akkha, Kirat Rai, Newa/Tirhuta, then Gurung
Khema+Sunuwar, Tibetan, Lepcha), primary-source-verified. Feasibility matrix:
| Script | Languages | Unicode | Font (for synthesis) | Existing OCR? | Verdict |
|---|---|---|---|---|---|
| **Limbu/Sirijonga** | Limbu | 4.0 | Namdhinggo (SIL) | none | ✅ **DONE — real CER 0.21** |
| **Kirat Rai** | Bantawa/Kiranti | 16.0 (U+16D40) | Kanchenjunga OFL + AKRS TECkit map | **none** | ✅ **novel** |
| **Sunuwar/Jenticha** | Sunuwar | 16.0 (U+11BC0) | **Noto Sans Sunuwar (Google Fonts)** | **none** | ✅ **novel — simplest** |
| **Lepcha (Róng)** | Lepcha (Ilam) | 5.1 (U+1C00) | Noto Sans Lepcha + Mingzat (SIL) | **none** | ✅ **novel** |
| **Tirhuta** | Maithili | 7.0 (U+11480) | Noto Sans Tirhuta | **none** | ✅ **novel** |
| **Gurung Khema** | Gurung | 16.0 (U+16100) | Noto (built from SFD source via fontforge — 58/58 covered) | **none** | ✅ **novel** |
| **Newa/Prachalit** | Newar | 9.0 (U+11400) | Noto Sans Newa | ⚠️ **exists** (3 papers, CER 2.6%) | ✅ feasible, not first |
| Tibetan | Sherpa/Tamang(Tamyig)… | 2.0 (U+0F00) | Noto/Jomolhari/BabelStone | ⚠️ **mature** (BDRC/Namsel/Monlam) | ❌ not novel; Nepal langs mostly print Devanagari |
| **Magar Akkha** | Magar | ❌ **not encoded** (2011 proposal shelved) | none | n/a | ❌ **blocked — no Unicode target** |
**Headline reframe:** not 1–2 non-Devanagari languages but **six first-ever
non-Devanagari OCR recognizers are feasible** — Limbu (done) + Kirat Rai + Sunuwar +
Lepcha + Tirhuta + Gurung Khema — all confirmed zero prior OCR, all Unicode-encoded,
all with a usable font. A *suite of first-ever OCR for endangered Himalayan scripts*.
Carve-outs are honest: Tibetan/Urdu are mature elsewhere (out); Magar Akkha is not in
Unicode (blocked until encoded); Newa already has OCR.
**Synthesis pipeline built (CPU, no GPU):**
- Fonts downloaded + verified (`data/fonts/himalayan-scripts/`): Noto Sans Sunuwar/
Lepcha/Tirhuta/Newa (from google/fonts), Kanchenjunga 2.001 + Mingzat 1.200 (SIL).
cmap coverage == assigned-character counts exactly (Sunuwar 44, Lepcha 74, KiratRai
58, Tirhuta 82, Newa 97); render proofs in `outputs/script-proofs/` show clean
glyphs, **no dotted-circle tofu**, via raqm shaping.
- Dicts generated from cmap by Unicode category (Python 3.14.2 ships Unicode 16.0.0,
so even the 2024 scripts classify): `data/dictionaries/{kiratrai,sunuwar,lepcha,
tirhuta,newa}-v1.txt`.
- New generic synthesizer `tools/synthesize_script_corpus.py`: classifies dict by
category (base L* / combining Mn,Mc / digits Nd / punct P*), generates structurally
valid syllables (combining mark always follows a base; **at most one mark per base**),
renders newspaper-length lines (~22–46 chars) with raqm + degradation profiles (clean/
scan/low_light/bright_uneven/rotate), writes PaddleOCR labels (`img\tlabel`). Linear
alphabets (Sunuwar, Kirat Rai) use mark-prob 0; abugidas (Lepcha/Tirhuta/Newa/Gurung
Khema) attach vowel signs. Train/eval de-duplicated by line text → no leakage.
- **Shaping validation (bug found + fixed):** an early 2-marks-per-base path made raqm
insert U+25CC dotted-circle placeholders for invalid mark stacks — glyphs absent from
the label, corrupting (image, label) pairs. Fix: cap at one mark **and** a `uharfbuzz`
`ShapeChecker` shapes every candidate line and rejects any producing a dotted-circle/
.notdef glyph in any render font (`shaping_validated=true`). All 6 final corpora are
provably dotted-circle-free. Per-script lines filtered: Kirat Rai/Sunuwar/Lepcha/
Gurung Khema 0, Tirhuta 426, **Newa 6,802** — Newa's combining marks have the strictest
valid-cluster rules, so the validator kept only valid clusters (linear scripts Kirat
Rai & Sunuwar have no marks, never affected).
- Corpora generated: `data/synth-corpora/{kiratrai,sunuwar,lepcha,tirhuta,newa,
gurungkhema}/` — **6 scripts**, 7,000 train + 900 eval images each. **Training-ready.**
**Honest scope:** synthetic text is structurally valid but not natural-language
(no real corpus exists) — it teaches the glyph→Unicode mapping, which is the core OCR
task; the Limbu precedent (synthetic-only train → real CER 0.21) shows this transfers.
Real-page validation is the user-sourced next step (Sikkim Herald editions, etc.).
GPU training of these is sequential and gated by budget — staged launch-ready, trained
as GPU frees after the Devanagari run.
### REAL-TEXT REALITY + corrected strategy (2026-06-17, after user pushback)
User correctly flagged: the staged novel-script corpora use **generated** structurally-
valid syllables, **not real language** — weaker than real prose rendered in the same
font (the same lesson Limbu taught: dictionary-words vs newspaper-prose mismatch hurt).
Also: these scripts aren't Gorkhapatra-gated — real text/images exist **elsewhere**.
A 5-agent harvest (codepoint-verified, non-Gorkhapatra) found:
| Script | Real Unicode text | Best real validation images |
|---|---|---|
| Newa | **668 lines / 35k cp** (nepal-lipi.com) | ✅ **Zenodo 6967421 — 683 manuscript pages + Unicode PAGE-XML GT, CC-BY 4.0, 503MB** |
| Lepcha | 498 lines / 65k cp (SIL Mingzat folk tale + combos + wiki) | 2 Wikimedia manuscript imgs (no GT); EAP281 (browser-gated) |
| Tirhuta | 53 lines (4 real wiki + 49 Aksharamukha Deva→Tirhuta) | LDCIL 21,412 Tirhuta sentences (paid); Varna Ratnakara MS |
| Sunuwar | 114 lines / 1.8k cp (r12a + GFonts specimen) | Sikkim Herald PDFs — **legacy font**, no public converter |
| Kirat Rai | 30 lines / 651 cp (SIL sample + r12a) | Sikkim Herald — **legacy AKRS; TECkit maps exist** (silnrsi/wsresources: Deva2Krai.map, kiratraifontnew.map) |
| Gurung Khema | **Superseded:** two official 2022 Herald editions now provide running Khema print; no rows admitted yet | 8 State Portal Herald pages, provenance/native review pending |
Harvested text saved under `data/real-text/<script>.txt`.
**Two paths that mirror the Limbu success (real text → real CER), beating random synth:**
1. **Sikkim Herald legacy conversion** = the eastern "Gorkhapatra." Sikkim's govt weekly
prints Lepcha/Limbu/Sunuwar/Kirat Rai editions, all legacy-font (AKRS/PUA), convertible
to Unicode via TECkit maps (Kirat Rai's exist). Yields **real training text AND real
validation crops+GT** — the proper Limbu playbook. Caveat: Herald PDFs use CID/PUA
fonts → extraction needs CIDToGIDMap handling (non-trivial, like the nayanepal work).
2. **Aksharamukha transliteration** of abundant Devanagari corpora → native script at
scale (e.g. Leipzig Newari 36k sentences = Devanagari → transliterate to Prachalit;
Maithili Devanagari → Tirhuta). Gives real-language content at volume. Plumbing: pip
pkg breaks on py3.14 (`ast.Str` removed); use the appspot API or a py3.10 sidecar venv.
**Per-script call:**
- **Newa** — best near-term *real-validated* non-Devanagari target (real text + the Zenodo
GT image set). Caveat: Newa OCR already exists (less "first-ever").
- **Lepcha / Kirat Rai / Sunuwar** — first-ever; pursue via Sikkim Herald conversion for
real text + real eval. Scale training text with transliteration where possible.
- **Tirhuta** — scale via Aksharamukha (Maithili Devanagari is abundant).
- **Gurung Khema** — official running print is now available in two 2022 State
Portal Herald editions. Keep random synthesis as augmentation only; do not use
the 749 mechanically eligible source lines until provenance adjudication and
native pixel-label, crop, split, and leakage gates pass.
**Corrected plan:** replace random-synth as the *primary* with real/transliterated text
(keep synth for glyph coverage + augmentation); acquire the Newa Zenodo real-eval set;
build the Sikkim-Herald legacy→Unicode converter for the eastern scripts. Training awaits
GPU re-auth (SSO expired at last teardown).
### SIKKIM HERALD CONVERTERS BUILT (2026-06-17) — real text + real eval, the Limbu playbook
The Sikkim Herald (Sikkim govt weekly) prints Limbu/Kirat Rai/Sunuwar/Lepcha editions in
legacy fonts. Built legacy→Unicode converters + real-eval packs per script (PDFs under
`data/sikkim-herald/<script>/`, converted text under `data/real-text/sikkim-herald/`):
| Script | Converter | Real text | Real eval | Open gap |
|---|---|---|---|---|
| **Kirat Rai** | `src/ocrtech/kiratrai_legacy.py` (SIL TECkit) | 887 lines | `data/kiratrai-real-eval-pack/` (70 clean) + train pkg `runs/kiratrai-ppocrv5-mobile/` | **6 bytes unmapped** (`f`…) → 778 lines pending |
| **Limbu** (2nd src) | reused `limbu_legacy.py` + Srijonga override map (byte 0x51→U+1922) | 2,269 lines / 46k cp | 2,175 clean | ~solved (0.018% unmapped) |
| **Sunuwar** | `src/ocrtech/sunuwar_legacy.py` — **font decoded from scratch by glyph-shape identity** (no public map existed) | 692 clean lines (**88.6%**) | buildable | **1 byte `\|`** (shape=shyele vs dist=u) → native reader |
| **Lepcha** | `src/ocrtech/lepcha_herald_legacy.py` — **Herald font decoded from scratch** (JG map was wrong font) | 132 clean lines | `data/lepcha-real-eval-pack/` (132 crops) | **2 bytes `]`,`%`** → ~349 lines pending |
| **Newa** | n/a (Unicode native) | 3,779 lines / 308k cp | `data/newa-real-eval-pack/` (3,180 manuscript crops, Zenodo) | manuscript HTR; OCR exists |
Method note: Sunuwar + Lepcha Herald fonts had **no public byte→Unicode map**; both were
reverse-engineered by rendering each legacy byte's glyph and matching it to the Unicode
font's glyph (Noto), **round-trip-validated against the printed ink**. Genuinely-ambiguous
bytes are flagged (CONFIRMED vs UNCERTAIN tiers), never guessed — closing them needs a
native reader or the source keyboard table. New tests: `tests/test_{kiratrai,sunuwar,
lepcha_herald}_legacy.py` (all green).
**nepal-ttf2utf extended (public lib):** ported Kirat Rai + Sunuwar + Lepcha converters
into `/Users/cdjk/github/llm/nepal-ttf2utf/` (2→5 scripts), strict-mode surfaces the
pending bytes, **35 tests pass**, honest coverage matrix. Working-tree only — push pending
user sign-off. (Also fixed a pre-existing `pyproject.toml` force-include build bug.)
**Blocked on user:** (1) GPU re-auth (`aws sso login --profile playaround`) to train; (2)
native-reader disambiguation of a few bytes (Sunuwar `|`, Lepcha `]`/`%`, Kirat Rai 6) to
reach ~100%; (3) sign-off to push nepal-ttf2utf.
## RESULT — Limbu/Sirijonga recognizer v1 (2026-06-16)
**First-ever Limbu/Sirijonga OCR recognizer trained and evaluated on a real page.**
- Model: PP-OCRv5 mobile fine-tune, 16 epochs (~1h, early-stopped on synthetic
plateau), trained **100% on synthetic** rendered prose (9,209 unique lines).
- **Real Gorkhapatra page (303 line crops): `norm_edit_dis` = 0.790 (≈ char
accuracy 79% / CER ≈ 0.21), exact-line acc = 6.6%.**
- Held-out synthetic eval: `norm_edit_dis` = 0.644 — *lower* than real because the
synthetic eval has longer lines (median 47 vs 39 chars) crushed by the 320px width
resize. So the synthetic "plateau" under-reported the model; the **real** page is the
truer measure.
- Baseline context: stock PaddleOCR / Tesseract have **no Limbu** → ~0 on Sirijonga.
This is from-zero on a brand-new script.
- Preserved: `outputs/limbu-sirijonga-recognizer-v1-2026-06-16/limbu-v1-model.tar.gz`
(best checkpoint + exported inference model + recipe). GPU instance stopped.
**Honesty scope (v1):** preliminary — real-eval labels are machine-converted +
human-verification-pending; single page (N=1); recognizer-only with oracle (PDF) line
boxes; single font; clean-digital. Proves the script is learnable and the pipeline
works end-to-end; not yet a paper-grade claim.
**v2 levers (clear next gains):** widen `rec_image_shape` beyond 320px (fixes long-line
squish — biggest lever per the synthetic-vs-real gap); human sign-off → claim-grade
labels; more Gorkhapatra editions (beat N=1); add degradation/multi-font for robustness;
run stock + PaddleOCR-VL baselines to quantify the "stock fails" bar.
## RESULT — Devanagari recognizer v1, 8 languages (2026-06-17)
**One PP-OCRv5 mobile Devanagari recognizer evaluated on 8 real Gorkhapatra
language pages** (Unicode ground truth via `nepal-ttf2utf`). Trained 100% on
synthetic (Noto Serif Devanagari renders of 35,937 eval-matched prose lines),
warm-started from `devanagari_PP-OCRv5`. **Early-stopped at epoch 9** by the
wall-clock budget cap (still improving — synthetic `norm_edit_dis` 0.737 and rising).
| Language (Devanagari page) | lines | norm_edit_dis | CER ≈ |
|---|---|---|---|
| Magahi | 425 | 0.655 | 0.35 |
| Bhojpuri | 362 | 0.626 | 0.37 |
| Magar | 383 | 0.622 | 0.38 |
| Sunuwar (Devanagari) | 444 | 0.607 | 0.39 |
| Darai | 427 | 0.606 | 0.39 |
| Majhi | 393 | 0.575 | 0.43 |
| Thami | 396 | 0.534 | 0.47 |
| Wambule | 383 | 0.522 | 0.48 |
| **mean** | 3,213 | **≈0.593** | **≈0.41** |
Artifact: `outputs/devanagari-recognizer-v1-2026-06-17/deva-model.tgz` (best
checkpoint + recipe + dict + eval results).
**Honest caveats:** epoch 9 only (budget-capped mid-improvement — more epochs would
lift this materially); **train/eval font gap** (Noto Serif Devanagari vs the
`nayanepal` newspaper face) — unlike font-matched Limbu (0.21), so these are
conservative; machine-converted labels (human-verification-pending); single page per
language. These are preliminary real-page numbers, not paper-grade.
**Budget/ops note (lesson):** the cap-watcher stopped training cleanly at 00:07 UTC
(DEADLINE-STOP), but defunct `train.py` processes lingered in `pgrep`, so the local
poller never detected the stop and the **GPU sat idle ~73 min** before I returned —
budget waste. Fix next time: poll GPU utilization or the `deva_train_stopped.sentinel`,
not just `pgrep`. Also: the staged `run_real_eval.sh` first failed silently (`python`
not on the non-interactive PATH; the venv is `/home/ubuntu/ocr-venv/bin/python`) — the
in-script `grep` swallowed the error. Fixed to use the venv interpreter + capture logs.
**AWS SSO expired** at teardown so the instance was stopped via OS `shutdown -h now`
(default behavior = stop, EBS preserved); **user should confirm instance state** =
`stopped` (not running/terminated) on next `aws sso login --profile playaround`.
## REAL same-font Devanagari training corpus (2026-06-18) — closes the font gap
**The font gap above is the single biggest lever on the 0.35–0.48 CER, and we now
have real same-font training data to close it.** `tools/build_gorkhapatra_devanagari_real.py`
harvests the `nayanepal` newspaper face (the *exact* eval face) straight from the
locally-downloaded Gorkhapatra epaper PDFs and converts it to Unicode via
`nepal_ttf2utf.convert_devanagari(font="nayanepal")` (Preeti base map + `ƒ`/`†` ext +
NFC), the same ~97–99% converter behind the eval GT.
- Output: `data/gorkhapatra-devanagari-real/` — `images/` + `labels.txt` (PaddleOCR
recognition format), `summary.json`.
- Scope reality: the task brief said "636 editions"; only **24 epaper PDFs are
physically on disk** (`_listings/probe_results.json` catalogues ~432 remote, but
only 24 were downloaded — 492 MB matches the 24, not 636). Harvested those 24.
- Design call: `nayanepal` is the body face of the *whole* paper, not just the
rotating "Naya Nepal" language page (verified per-page; the eval pages use it too).
So we harvest every `nayanepal` span on every page and only SKIP each edition's
native `NamdhinggoSILL` Limbu page (already harvested, not Devanagari). One span =
one printed line; cropped tight at 288 dpi from the span bbox.
- **Result: 79,364 unique real same-font Devanagari line crops, 2,813,427 Devanagari
chars** (median 42 chars/line), from 114,434 nayanepal spans seen. De-duplicated by
cosmetic-normalized text key across all editions (21,731 dups dropped). **Eval
leakage excluded against ALL 8 Devanagari eval packs** — 6 in
`outputs/multilang-eval/*/manifests/` (Thami, Bhojpuri, Majhi, Sunuwar, Magahi,
Darai) **plus Magar + Wambule** in `outputs/{magar,wambule}-real-eval-pack/manifests/`
(these two are NOT under multilang-eval — an easy-to-miss split; the loader globs
`*-real-eval-pack` roots too, also harmlessly catching Lepcha/Urdu non-Deva packs).
3,707 leakage keys; 155 lines dropped; **final corpus has 0 eval overlap**
(re-verified against the full 3,707-key set). Leakage matching uses a normalized key
(NFC + whitespace-collapse + smart-punct fold) because the eval manifests keep U+202F
etc. that the converter folds — raw exact-match would miss most overlaps.
- Conversion quality (honest): **5.9% span conversion-gap rate** — spans whose
converted text still carries a residual non-Devanagari glyph (top offenders
`‚` U+201A, `m`, `ì`, `Þ`, `Œ` — known nayanepal extension bytes the Preeti base
map doesn't cover) are DROPPED, not kept dirty. Kept lines are 100% Devanagari +
allow-set (digits/danda/punct), enforced on the final Unicode like the Limbu pack.
- Known GT artifact (preserved on purpose): ~18% of lines carry an intra-word kerning
space (e.g. `नार ासाथ` for `नारासाथ`). These spaces are **in the source PDF text
layer** (the Gorkhapatra typesetter inserts them) and are **also present in the
eval-pack GT** — so keeping them keeps train/eval GT conventions identical. Residual
NBSP/narrow-NBSP are folded to ASCII space; true mid-word spaces are left as-is.
- Per-language attribution: NOT recoverable per line — this is a **pooled** real
Devanagari corpus (Nepali general news + rotating minority-language pages), which is
exactly what the *shared* Devanagari recognizer wants. Fine for retraining.
- Next: warm-start the Devanagari recognizer on this real corpus (LOW LR ~2e-5,
moderate aug per the baked-in lessons), eval on the unchanged 8 held-out packs.
Expectation: meaningful CER drop from closing the Noto-Serif→nayanepal font gap.
## Tally — languages with real-page recognition (2026-06-17)
**9 languages with real Gorkhapatra-page CER:** Limbu/Sirijonga (own recognizer,
native Sirijonga script, CER **0.21**) + 8 Devanagari (one shared recognizer, CER
0.35–0.48). Plus **6 first-ever non-Devanagari native-script recognizers staged**
training-ready (Limbu done; Kirat Rai + Sunuwar launch-ready on the now-stopped GPU;
Lepcha + Tirhuta + Gurung Khema corpora local). Next GPU session: train Kirat Rai /
Sunuwar (recipes + corpora already staged) → first-ever own-script CER; resume
Devanagari past epoch 9 to lift the 8-language numbers.
## Triage matrix (Phase 1 result)
| Language | Gorkhapatra page | Text-layer form | Font | Convertible→Unicode | Native-script? | Status |
|----------|------------------|-----------------|------|---------------------|----------------|--------|
| Limbu/Sirijonga | 2083-02-12 p6 | legacy_font | NamdhinggoSILL | **yes** (`limbu_legacy`) | **yes (Sirijonga)** | converter exists, page disambiguated → **first target** |
| Sanskrit | 2083-03-01 p6/p7 | legacy_font | nayanepal (~85–89% ASCII) | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Magar) |
| Magar | 2083-03-01 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Sanskrit) |
| Wambule | 2083-02-31 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Rana Tharu) |
| Rana Tharu | 2083-02-31 p6/p7 | legacy_font | nayanepal | no converter yet | no (Devanagari) | alignment ambiguous (shares page with Wambule) |
### Strategic implications from triage
1. **The whole Gorkhapatra epaper text layer is legacy-font, not Unicode.** Real
references for *every* language must go through legacy→Unicode conversion. The
Limbu/Namdhinggo path is the template; Devanagari pages need a `nayanepal`→Unicode
converter (a Preeti-family legacy map — do **not** assume one exists; verify the
actual glyph map, or cross-reference the article web text which is served in
Unicode at `gorkhapatraonline.com/news/<id>`).
2. **Live-pack language alignment is unreliable** — 2 candidate languages resolve to
the same physical page. Disambiguation (which language is actually on the page) is
required before any Devanagari-language real eval. The Naya Nepal card → epaper
edition mapping picked overlapping editions.
3. **Limbu is the only clean first eval**: native script, existing converter,
already-disambiguated page (p6 Limbu vs p7 non-Limbu). Phase 2 builds on it and
the capability generalizes to `nayanepal` afterward.
---
## Log
- 2026-06-16: Goal set. Mapped real state (training infra complete but only the
dead mixed-PP-OCRv3 family trained; zero verified real references; only Limbu has
native-script evidence). Wrote this checklist. Starting Phase 1 triage.
- 2026-06-16: Phase 1 triage done. Ran PDF-native extractor on the live-pack
(8 pages). Finding: **all Gorkhapatra epaper text is legacy-font** (Namdhinggo for
Limbu, `nayanepal` ~85–89% ASCII for Devanagari pages) — no Unicode layer anywhere.
Live-pack alignment is ambiguous (Sanskrit≡Magar page, Wambule≡Rana Tharu page).
Confirmed no trained recognizer checkpoints exist. Limbu confirmed as first target.
Next: Phase 2 — wire Limbu PDF-native legacy→Unicode into the verified-reference
path (pre-fill + human sign-off), avoiding 298 from-scratch transcriptions.
- 2026-06-16: Phase 2 tooling shipped. Added
`prepare-gorkhapatra-language-page-pdf-native-references` (legacy→Unicode draft
with image-pixel bboxes + unmapped-codepoint flags) and a generic per-line
`review_note` pass-through in the verification bundle. Ran end-to-end on real Limbu
data: 303 pre-filled spans, bundle built (303 crops). Tests added and green; cli +
source_audit + limbu_legacy suites pass; `git diff --check` clean. Remaining for the
first real eval: human sign-off pass (light accept, not transcription) → `apply` →
claim-eligible verified Limbu reference. Then Phase 3 baselines + Phase 4 fine-tune.
- 2026-06-16: Phase 3 backbone shipped. Added
`prepare-gorkhapatra-language-page-recognizer-eval` (verified reference → tight line
crops + recognizer eval manifest). Smoke on Limbu draft: 303 crops, passes
`audit-manifest`. Tests added; `source_audit`/`limbu_legacy`/`cli` suites green;
`git diff --check` clean. User confirms an AWS G6 host is available for Phase 4, but
the local `aws` SSO token is expired — `aws sso login` needed before I can query or
launch it. Next unblocked work: Phase 4 launch-readiness (rendered split + recipe +
documented GPU launch command), pending AWS re-auth and the Limbu sign-off for the
real eval.
- 2026-06-16: AWS re-authed (playaround profile). Found host
`ocrtech-g6-training-20260615-v4` (i-0fa19dff7b233a353, g6.xlarge, us-east-1a);
start hit transient `InsufficientInstanceCapacity`, a background poller acquired it,
then I **stopped it** (no idle cost) per "research first, don't train on assumption".
- 2026-06-16: Ran a 4-agent research fan-out on the correct PaddleOCR recognizer
approach. **Overturned the from-scratch PP-OCRv3 assumption** → validated approach is
**fine-tune PP-OCRv5 mobile (devanagari multilingual config) from pretrained, reuse
backbone, reset head for the Limbu dict** (see "Validated training approach" above).
Harness is 2–3 gens stale; needs a `training.py` PP-OCRv5 recipe update + a fuller
~8,800-line rendered corpus before the GPU run. Instance stays stopped until ready.
- 2026-06-22: For the current Sirijanga Gorkhapatra page, the 31-row geometry/text
missed-line worklist has an oracle-reference ceiling. It improves line coverage
when inserted with perfect text (line F1 `0.933566`, misses 36), but append/geometry
ordering damages page OCR (`0.771993` codepoint CER with preserve/append,
`0.771825` with geometry, versus selected `conf070` at `0.739361`).
`candidate_document_order` is now the best diagnostic slotting policy: codepoint CER
`0.739024`, line F1 `0.933566`, misses 36, extras 2, reading-order accuracy
`0.775506`. It still fails admission because grapheme CER worsens to `0.778367`.
The next blocker is grapheme/text normalization on top of candidate-document-order
slotting, not just transcription. The non-oracle machine-text version under the same
order mode still fails at `0.763078` codepoint CER and `0.797416` grapheme CER, so
the machine candidate text remains rejected.
- 2026-06-22: The 31-row Sirijonga oracle ceiling was ablated by span group and single
row; no subset improves both codepoint and grapheme page CER under
candidate-document order. A diagnostic oracle reference-order projection over the
same rows does pass strongly (`0.102355` codepoint CER, `0.136823` grapheme CER,
line F1 `0.933566`, 0 effect-audit issues). This proves the remaining Gorkhapatra
Sirijonga blocker is deployable page reading order/layout assembly, not detector
recall or line recognizer text in isolation.
- 2026-06-22: Added `tools/analyze_page_reading_order_geometry.py` and profiled the
oracle reference-order ceiling. The page decomposes into 21 reference-order geometry
chunks and 20 large jumps, so the next deployable reading-order policy must reason
about article/block regions, not only columns.
- 2026-06-22: Added `tools/apply_page_reading_order_geometry_profile.py` and tested
the 21-chunk oracle-derived profile as a page-ordering mechanism. On the 31-row
oracle-text projection it reaches `0.095711` codepoint CER and `0.129700` grapheme
CER with line F1 `0.933566`; on the non-oracle machine-text projection it reaches
`0.143146` codepoint CER, `0.198112` grapheme CER, line F1 `0.933566`, recall
`0.881188`, misses 36, extras 2, and reading-order accuracy `1.0`, with the effect
audit safe. This is not deployable evidence because the profile is derived from
hidden eval alignment; it proves the next two-day goal should be non-oracle layout
profile calibration, not more detector or recognizer churn.
- 2026-06-22: Added `tools/build_page_reading_order_geometry_profile.py`, the
non-oracle bridge from geometry-only line-box review to the profile applier. It
rejects text-bearing geometry references, blocks pending packets unless explicitly
allowed, and blocks mixed multi-source profiles unless explicitly allowed. Applying
the current Sirijonga line-box packet as pending seed geometry gives 2 samples and
389 line boxes; the separate Gorkhapatra pending seed profile has 303 lines,
26 chunks, and `profile_claim_safe=false`. Applied to the 31-row machine-text
candidate projection, that Gorkhapatra seed profile reproduces the strong
profile-order score exactly: codepoint CER `0.143146`, grapheme CER `0.198112`,
line F1 `0.933566`, recall `0.881188`, misses 36, extras 2, reading-order accuracy
`1.0`, metrics SHA-256
`8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189`. This changes
the next task from "invent a layout profile path" to "complete/review the geometry
order packet and rebuild this profile without `--allow-pending`."
- 2026-06-22: Added `tools/audit_page_reading_order_geometry_profile.py` and the
`--require-profile-claim-safe` gate on `tools/apply_page_reading_order_geometry_profile.py`.
The current Gorkhapatra pending seed profile passes structural audit but has
`operational_profile_safe=false`; the required-safe audit fails, and the applier
refuses it under the new flag. Structural audit / required-safe audit SHA-256:
`9aecaa40b5ee509ad12e495777a857a76fea7d4ad68793e9055e721a163b8877` /
`694486b5a3f28bcdb3093c085814fdb76daf835be5599f26319daca80b70b3ce`. This keeps the
strong seed-profile diagnostic from being accidentally promoted before geometry
review is complete.
- 2026-06-22: Added `tools/admit_page_reading_order_profile_application.py`, the
promotion gate that binds profile audit, profile application, and page-effect audit.
On the real pending Gorkhapatra seed-profile projection it rejects admission with
`admitted_operational=false` even though `book_ocr_objective_safe=true`, because the
profile provenance is pending/non-claim-safe. Admission SHA-256:
`02bcd20dc1a5598f92b9a251dd5ae4669171ba8d9429df4eb9c06cf33bc354b6`. The operational
promotion sequence is now explicit: reviewed geometry profile -> required-safe
profile audit -> required-safe profile application -> page score/effect audit ->
profile-application admission.
- 2026-06-22: Extended `tools/apply_page_reading_order_geometry_profile.py` with
`--profile-map` for multi-page/profile-per-template routing, and extended
`tools/admit_page_reading_order_profile_application.py` to accept multiple
`--profile-audit` inputs. The current pending seed map
(`7e94909dacbe9be20d18a70ddbac5fcc0421b0c2a35c2055482b1f2859895397`) maps
Gorkhapatra and Sikkim to separate profiles; applying it to the Gorkhapatra
31-row machine-text projection reproduces metrics SHA-256
`8181c60e80a61ceb0129967a1032247aff8040f67b42b32902a869f3bb732189`.
- 2026-06-22: Added `tools/audit_page_reading_order_profile_map.py` and bound it into
profile-application admission. The current pending seed map passes structural audit
(`1090ec5254bad3e5d0ec492739488de5a65d684673d33cb36d50f1dde8c7d7d7`) but fails
required-safe audit (`bb4c07c4fcb7f2031ee085313d3a537eda01650b77f3af31ce23d605af136d06`)
because both mapped profiles are pending. Map-driven admission now rejects the
Gorkhapatra projection with SHA-256
`8764a46251c9e0bc1b2415de9089f5eb67032e9bf9eccf11167f54da8c18aaf6`. This closes the
obvious multi-page footgun: do not use one layout profile across a book unless a
profile map or template selector chooses it per page, and do not promote a map-driven
run without a safe profile-map audit.
- 2026-06-22: Integrated reading-order profile maps into `tools/parse_limbu_book_pages.py`.
The book wrapper now applies `--reading-order-profile-map` after optional line
assembly and before aggregate Markdown/review queues/`score-documents/` are written.
A sidecar test proves the profile map changes both `book.md` and the score projection
consistently. The wrapper also hash-binds supplied profile-map audits to the exact
map being applied and refuses `--require-reading-order-profile-map-safe` without an
audit, so stale map-audit artifacts cannot silently authorize a book run.
`tools/audit_limbu_book_output.py` now verifies the profile-map/application report,
assignments JSONL, optional map audit, and per-page profile source-document hashes.
This moves the layout-profile work from standalone diagnostics into the actual
multi-page book OCR wrapper and its normal bundle audit.
- 2026-06-23: Re-scoped the work after user pushback: stop treating eval harness work
as the main deliverable. The next target is real Limbu book OCR throughput and review,
with only thin guardrail audits. Re-ran the local source inventory at
`outputs/limbu-page-source-inventory-2026-06-23-realbook-pivot/`: 36 records,
3 real Devanagari-written Limbu CDC page candidates, 3 Devanagari text-only sources,
1 synthetic smoke pack, and `ready=True`. The 4TB host at `cdjk@100.110.148.43`
rejected SSH auth in this session, but the mirrored CDC PDFs are present locally.
Existing CDC page draft output has 3 real rendered CDC pages with unreviewed
curriculum references; the current multi-page CDC book audit is structurally clean
(`page_count=3`, `line_count=35`, `review_line_count=27`) but remains a book OCR
artifact audit, not an OCR-quality claim. Next work should be a real-page OCR/review
loop and reviewed-only training export, not more generic scoring infrastructure.
- 2026-06-23: Added operational PDF-book intake to
`tools/build_limbu_book_page_manifest.py`. The tool now accepts `--pdf` as an
alternative to `--image-dir`, renders selected PDF pages to PNG, and writes the same
page manifest shape consumed by capture prep and book OCR, with source PDF
path/hash, page number/index, rendered image hash, and render DPI recorded per row.
Real smoke on
`data/external-language-resources/cdjk-100.110.148.43/limbu-speech-toolkit/data/external/cdc_moecdc/mero_ganit_grade1_limbu.pdf`
rendered pages 8..10 at 220 DPI to
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-pages.jsonl`
(`pages=3`, no source-attention flags, 1870x2420 PNGs). That manifest then passed
through `tools/prepare_limbu_book_captures.py` with 3 pages and 0 failed capture
audits, producing
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/capture-prep/prepared-page-manifest.jsonl`.
This is the next real-book front door for CDC PDF pages: feed the prepared manifest
into `tools/parse_limbu_book_pages.py` and inspect layout/review packets.
- 2026-06-23: Ran that CDC PDF front door through the full operational book wrapper
using the current v4 router, with recognizers held fixed:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-square-nontext-guidance/`.
It emits editable book Markdown, page layout overlays, page triage, layout-gap
review, and a passing book audit over 3 real rendered CDC pages. Counts: 93
preserved lines, 21 book-line review rows, 3 page-layout review rows, 3 page-triage
rows, 9 suspicious layout-gap rows, and `book_audit_passed=True`. Visual overlay
inspection showed most high-priority empty detections were square page furniture,
especially 60x61 registration boxes, not missed body text. Updated
`tools/parse_limbu_book_pages.py` so square empty detections are guided as
`inspect_non_text_or_drop_empty_detection` instead of generic transcription work.
On the real rerun, action counts became 9 non-text inspect/drop, 1
transcribe/drop empty detection, and 11 low-confidence spotchecks, with audit still
passing. Remaining practical blocker on this artifact is the 9 layout-gap rows
(`7` possible-text ink, `2` sparse ink), which need review as missed text vs page
graphics before applying queues or exporting reviewed training rows.
- 2026-06-23: Inspected those CDC layout-gap rows against real crop/context evidence
instead of adding more eval machinery. The only remaining high-priority gap,
`gap-0001-cdc-maths-p0008-003`, was a wide dense box-border/illustration fragment,
not missed Limbu text. Updated `tools/run_limbu_book_ocr.py` so wide low-density
border/graphic bands are classified as `page_graphic_or_illustration` while the
existing compact missed-line proxy still stays `candidate_text_band`. Real rerun:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-gap-graphic-border-guidance/`.
Counts stayed stable at 3 pages and 93 extracted lines; wrapper book-line review
rows stayed 21 (`review_line_count=22` in the parse summary), layout-gap rows stayed
9, and `book_audit_passed=True`. The gap queue is now 6 page-graphic/illustration
rows and 3 rule/spacing rows, with 0 candidate-text gap rows; page triage accepts
all 3 pages and `operator_attention_pages=0`. This is a review-noise reduction on
real CDC pages, not an OCR-quality claim. Next useful work is to apply/resolve the
remaining line-review queue and export reviewed-only training rows, not to expand the
eval harness.
- 2026-06-23: Inspected the 21-row CDC book-line review queue via a crop contact sheet.
Three rows that were previously low-confidence text spotchecks were actually 60x60
page-corner registration marks with OCR hallucination text (`े?`). Updated
`tools/parse_limbu_book_pages.py` so low-confidence rows with the same square
non-text geometry are guided as
`inspect_non_text_or_drop_low_confidence_detection`, while text-shaped low-confidence
rows remain `spotcheck_low_confidence_text`. Real rerun:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/`.
Counts stayed stable at 3 pages, 93 extracted lines, 21 book-line review rows, 9
layout-gap rows, `operator_attention_pages=0`, and `book_audit_passed=True`;
standalone audit also passed at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/book-output-audit-lowconf-nontext-guidance/`.
The current line-review action counts are 9
`inspect_non_text_or_drop_empty_detection`, 3
`inspect_non_text_or_drop_low_confidence_detection`, 1
`transcribe_or_drop_empty_detection`, and 8 `spotcheck_low_confidence_text`. The
contact sheet lives at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/book-ocr/book-line-review-contact-sheet.png`.
This is the current CDC review packet; next work is human review/application/export,
not more harness expansion.
- 2026-06-23: Turned the obvious non-text rows from that packet into an auditable
cleaned projection, without pretending they are training labels. The proposal queue
at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops.tsv`
marks 12 rows as `drop`: 9 empty non-text detections and 3 low-confidence non-text
registration-corner hallucinations. It leaves 9 rows pending: the one
transcribe/drop row and 8 low-confidence text/graphic spotchecks. Applying it with
`tools/apply_limbu_book_line_reviews.py` produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/`
with `applied_count=12`, `dropped_count=12`, `skipped_count=9`, `line_count=69`,
and a passing standalone book-output audit. The apply tool also had a real chainable
output bug fixed: reviewed projections now write their own `output_dir` in
`book-run.json`, so `tools/audit_limbu_book_output.py` no longer fails on stale
source-output provenance. Running the reviewed-only training exporter on the cleaned
projection still correctly fails with `no training rows were exported`, because no
human accepted/corrected text labels exist yet.
- 2026-06-23: Prepared the human review station from the cleaned CDC
projection. Command:
`tools/prepare_limbu_book_unresolved_review_packet.py --book-output-dir .../review-applied-obvious-nontext-drops --book-line-review-queue .../manual-review-proposals/book-line-review-obvious-nontext-drops.tsv`.
Packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/pending-text-review-packet/`.
HTML review station:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops/pending-text-review-packet/book-unresolved-review-packet.html`.
The packet audit passes with 0 issues. Counts: 9 existing-line rows, 12 non-pending
source rows skipped, 9 crops, 9 page contexts, and 8 prefilled text rows. Review
guidance is now preserved in the packet as 8 `spotcheck_low_confidence_text` rows and
1 `transcribe_or_drop_empty_detection` row; `tools/prepare_limbu_book_unresolved_review_packet.py`
now maps book-line `review_priority`/`review_recommended_action` into the packet's
`review_category`/`suggested_review_action` fields. The packet TSV/export now also
preserves `page_context_path`, `page_context_sha256`, and `neighbor_context`, instead
of keeping that evidence only inside the HTML. After the HTML is reviewed and
exported as `book-unresolved-review-packet.reviewed.tsv`, run the existing
`tools/apply_limbu_book_reviewed_packet_to_training_pack.py` wrapper against the
cleaned projection.
- 2026-06-23: Inspected the 9-row packet visually with crop plus page context. One
remaining blank `transcribe_or_drop_empty_detection` row (`img-line-0013`) was
objectively another wavy illustration/rope segment, not visible text. Created v2
proposal queue
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv`
and applied it to
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/`.
Counts: `applied_count=13`, `dropped_count=13`, `skipped_count=8`,
`line_count=68`, standalone book-output audit passed, and reviewed-only export still
correctly refuses with `no training rows were exported`. The current review packet is
now
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0008-p0010/full-book-ocr-router-v4-lowconf-nontext-guidance/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/`;
it audits cleanly with 8 pending rows, 8 crops, 8 page contexts, all
`spotcheck_low_confidence_text`.
- 2026-06-23: Removed another operational bottleneck instead of expanding eval
machinery: `tools/run_limbu_book_ocr.py` now accepts `--pdf` directly. The underlying
manifest builder still prefers PyMuPDF, but now falls back to Poppler
(`pdfinfo` + `pdftoppm`) when running inside the Paddle Python 3.11 environment where
`fitz` is unavailable. The run summary records the source kind, source PDF hash,
PDF page window, DPI, and `pdf_renderer`. Focused tests cover image-dir behavior,
PyMuPDF rendering, and the Poppler fallback.
- 2026-06-23: Used that PDF front door to create the next larger real CDC review
target:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/`.
Command ran the Maths Limbu PDF pages 11..15 at 220 DPI through the current v4 router
with recognizers held fixed. It rendered 5 pages with Poppler, preserved 189 OCR
lines, wrote 58 pending book-line review rows, wrote 15 layout-gap rows, accepted all
5 pages in page triage, and passed the book-output audit. Review action counts are
22 `inspect_non_text_or_drop_empty_detection`, 17
`inspect_non_text_or_drop_low_confidence_detection`, 3
`transcribe_or_drop_empty_detection`, 2 `verify_non_limbu_or_drop`, and 14
`spotcheck_low_confidence_text`; gap actions are 6 page graphic/illustration and 9
rule/spacing rows. Prepared unresolved packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pending-text-review-packet/`.
Packet audit passes with 0 issues. It has 58 pending rows, 58 crops, 58 page contexts,
58 neighbor contexts, and 33 prefilled OCR text rows. This is a better label
production target than the earlier 8-row smoke, but still no training rows exist
until a non-machine reviewer accepts/corrects text or drops non-text rows.
- 2026-06-23: Reduced that larger packet's review burden with one conservative visual
drop pass, still without minting labels. Built a crop+page-context contact sheet at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pending-text-review-packet/book-unresolved-review-contact-sheet.jpg`.
The only automatic proposal marks repeated 60px page-corner registration marks/page
furniture as `drop`, including the top-right `े?` hallucinations; text-shaped labels
and ambiguous exercise graphics remain pending. Proposal queue:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-corner-registration-drops-v1.tsv`.
Applying it produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/`
with 20 dropped rows and 38 skipped/pending rows. The cleaned book-output audit
passes with 0 issues, and reviewed-only training export still correctly refuses with
`no training rows were exported`. Current cleaned packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/pending-text-review-packet/`.
Packet audit passes with 0 issues. It has 38 pending rows, all with crop,
page-context path/hash, and neighbor context; 28 rows have prefilled OCR text.
Remaining actions: 7 `inspect_non_text_or_drop_empty_detection`, 12
`inspect_non_text_or_drop_low_confidence_detection`, 3
`transcribe_or_drop_empty_detection`, 2 `verify_non_limbu_or_drop`, and 14
`spotcheck_low_confidence_text`.
- 2026-06-23: Made a second conservative visual drop pass on the 38-row packet after
generating a larger remaining-only sheet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-corner-registration-drops-v1/pending-text-review-packet/book-unresolved-review-remaining-contact-sheet.jpg`.
The v2 proposal keeps all text-shaped/glyph rows pending and drops only 13 additional
obvious non-text objects: arrows, blank answer rectangles, and standalone
illustrations. Proposal queue:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/manual-review-proposals/book-line-review-obvious-nontext-drops-v2.tsv`.
Applying v2 from the original 58-row book output produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/`
with 33 dropped rows and 25 skipped/pending rows. The cleaned book-output audit
passes with 0 issues, and reviewed-only training export still correctly refuses with
`no training rows were exported`. Current cleaned packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-review-packet/`.
Packet audit passes with 0 issues. It has 25 pending rows, all with crop,
page-context path/hash, neighbor context, and prefilled OCR text. Remaining actions:
13 `spotcheck_low_confidence_text`, 11
`inspect_non_text_or_drop_low_confidence_detection`, and 1 `verify_non_limbu_or_drop`.
- 2026-06-23: Added a small reviewer-context bridge instead of expanding the eval
harness. `tools/build_limbu_pdf_text_context.py` extracts `pdftotext -bbox-layout`
lines from legacy-font Limbu PDFs, uses word x-gaps to rejoin tiny legacy glyph
fragments before conversion, converts through `src/ocrtech/limbu_legacy.py`, and
writes JSONL/TSV/Markdown plus per-page raw/unicode text sidecars. It marks every row
as `pdf_text_layer_legacy_converted_review_context_not_gold`,
`is_gold_text=false`, and
`training_use=forbidden_without_human_alignment_review`. Running it on CDC Maths
PDF pages 11..15 with the rendered page manifest produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/`:
82 text-layer lines, 81 with converted Limbu codepoints, 0 unmapped-codepoint lines,
and rendered bboxes mapped to the same `cdc-maths-p0011`..`cdc-maths-p0015` page IDs.
This is useful for human review of the current 25-row unresolved packet, but it is
not an auto-label source.
- 2026-06-23: Connected that context directly to the current review station.
`tools/prepare_limbu_book_unresolved_review_packet.py` now accepts
`--pdf-text-context-jsonl` and attaches up to 3 nearest PDF text-layer rows to each
packet row using page ID plus rendered bbox y/overlap geometry. The packet TSV keeps
advisory columns (`pdf_text_context_*`) and the HTML shows a "PDF text-layer context"
panel with a warning that it is not gold and cannot be used as a label without human
alignment. Regenerated assisted packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-review-packet-with-pdf-context/`.
Counts: 25 packet rows, 25 matched PDF-context rows, 0 unmatched rows, 82 source
context rows, 3 context rows max per packet row. Packet audit passes with 0 issues.
- 2026-06-23: Added a source-text coverage check for missed-line recovery.
`tools/build_limbu_pdf_text_line_coverage.py` compares converted PDF text-layer
line boxes with the cleaned OCR document line boxes and emits review-only candidate
rows when a source text line has weak y coverage or weak x coverage from current OCR
boxes. On the same five CDC Maths pages, conservative thresholds
(`min_y_overlap_ratio=0.40`, `min_x_coverage_ratio=0.20`) produced
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pdf-text-line-coverage/`:
82 source PDF text lines, 68 covered, 1 no-Limbu/no-text skipped, and 13 candidate
rows. Candidate categories: 8 `pdf_text_line_missing_y_coverage` and 5
`pdf_text_line_partial_x_coverage`, mostly isolated Limbu numerals/page-structure
fragments. The derived missing-line review queue at
`.../pdf-text-missing-line-review/` has 13 rows, 13 page contexts, and audits cleanly
with 0 issues.
- 2026-06-23: Unified the existing 25 unresolved text rows and the 13 source-text
coverage candidates into the first combined review packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context/`.
It has 38 pending rows: 25 `existing_book_line`, 13 `candidate_missing_line`, 38
PDF-context matches, 0 unmatched rows, and a passing packet audit with 0 issues.
This packet is still not training data until a human reviewer accepts/corrects rows;
it is superseded by the strict no-prefill packet below as the current review target.
- 2026-06-23: Hardened the current review station against accidental label promotion.
`tools/prepare_limbu_book_unresolved_review_packet.py` no longer pre-fills pending
`corrected_text` from OCR/candidate text by default; `--prefill-corrected-text` is
now an explicit opt-in. The HTML still has a deliberate `Accept text` action that
copies OCR text into `corrected_text` before marking a row accepted, but a reviewer
must take that action or type a correction. Regenerated strict current packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/review-applied-obvious-nontext-drops-v2/pending-text-and-missing-line-review-packet-with-pdf-context-strict-v2/`.
It has 38 pending rows, 25 `existing_book_line`, 13 `candidate_missing_line`, 38
PDF-context matches, 0 unmatched rows, 0 pending prefilled corrected-text rows, and a
passing packet audit with 0 issues. This strict packet supersedes the earlier
prefilled combined packet as the current review target.
- 2026-06-23: Made the no-prefill policy an audit invariant. `tools/audit_limbu_book_unresolved_review_packet.py`
now counts pending rows with nonblank `corrected_text`, compares that count against
the packet report, and fails strict original-packet audits when
`pending_corrected_text_prefill_enabled=false` but pending corrected text is present.
The explicit `--prefill-corrected-text` packet path still audits cleanly when its
report declares the prefill. Re-auditing the real strict packet above still passes:
38 rows, 38 pending, 0 accepted/dropped, and
`pending_prefilled_corrected_text_count=0`.
- 2026-06-23: Hardened the PDF text-layer context boundary in the same packet audit.
`tools/audit_limbu_book_unresolved_review_packet.py` now validates that packet-level
`pdf_text_context_training_use` stays
`forbidden_without_human_alignment_review`, each embedded PDF context row keeps
`context_status=pdf_text_layer_legacy_converted_review_context_not_gold`, each
embedded context row keeps training use forbidden, and no embedded context row is
marked as gold. It also compares attached/unmatched PDF-context counts against the
packet report. Re-auditing the real strict packet still passes with 38 PDF-context
matches and 0 unmatched rows. This keeps PDF text useful for review while preventing
it from silently becoming training truth.
- 2026-06-23: Removed the remaining candidate-text fallback from missing-line
application. `tools/apply_candidate_line_reviews.py` now requires every accepted
candidate insertion to carry explicit nonblank `corrected_text`; it no longer inserts
`candidate_text` when reviewed text is blank. `tools/split_limbu_book_unresolved_review_packet.py`
now rejects accepted `candidate_missing_line` packet rows without explicit
`corrected_text` instead of warning that the applicator will use candidate text.
Focused tests cover direct candidate JSONL/TSV application, unified-packet splitting,
reviewed-packet application, and reviewed training handoff. The real strict packet
remains audit-clean because all rows are still pending and have blank
`corrected_text`.
- 2026-06-23: Closed the analogous existing-line split fallback. The unified packet
splitter no longer fills accepted existing-line `corrected_text` from OCR text; an
accepted existing line now needs explicit `corrected_text` or explicit
`split_corrected_texts` before it can be passed to
`tools/apply_limbu_book_line_reviews.py`. This keeps "accept current OCR" as an
explicit reviewer action in the HTML export rather than a status-only TSV edit.
Focused review-chain tests pass with 42 tests, and the real strict CDC packet still
audits with 0 issues.
- 2026-06-23: Hardened the downstream candidate-application audit for old or tampered
projections. `tools/audit_candidate_line_review_application.py` now inspects inserted
`candidate_line_review_insertion` metadata for the current review file and fails when
reviewed `corrected_text` is missing, `inserted_text` is missing, or the stored line
text no longer matches the reviewed/inserted text after normalization. Focused
review-chain tests now pass with 43 tests, and the real strict CDC packet still
audits with 0 issues.
- 2026-06-23: Pivoted from eval-harness hardening to actual recognizer training
input. Added `tools/export_limbu_pdf_text_training_pack.py`, which takes converted
PDF text-layer context rows and crops the corresponding full rendered PDF line from
the page image. This avoids training on current OCR guesses and avoids assigning a
full PDF line label to a partial OCR word crop. The export is explicitly silver, not
gold: rows carry `gt_status=pdf_text_layer_silver`, `claim_eligible=false`, and the
summary says it is not human-reviewed ground truth or CER/WER claim evidence.
Focused tests cover crop geometry, Paddle label emission, non-Limbu skipping, and
silver provenance.
- 2026-06-23: Materialized the first real Limbu PDF-text training pack from CDC Maths
pages 11..15:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-layer-silver-training-pack-v1/`.
It exports 81 labeled rendered-line crops from 82 PDF text-layer rows, skipping 1
no-Limbu row. Counts: 64 `sirijanga_limbu`, 17
`mixed_sirijanga_devanagari`, 1509 Sirijonga codepoints, 18 Devanagari codepoints,
and 55 dictionary entries. The pack manifest and label file both have 81 rows.
Because `ocrtech train-recognizer` needs image paths resolvable from repo root, a
root-path manifest was written at `.../manifests/train-root-paths.jsonl`.
- 2026-06-23: Split the silver PDF-text pack for training. The first attempt grouped
only by `sample_id` and failed leakage because duplicate crop/text signatures crossed
train/dev; that was the correct failure. Re-splitting with
`--group-by sample_signature --stratify-by slices --eval-ratio 0.15` produced a clean
split under `.../splits/`: 69 train rows, 12 dev rows, `leakage_passed=True`.
Generated a PP-OCRv5 mobile recognizer recipe at
`runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/paddle_recognizer.yml`
using the pack dictionary, 20 epochs, batch size 8, and learning rate `0.00005`.
- 2026-06-23: Started training in the only defensible way available now: a one-epoch
local CPU smoke on the silver PDF-text pack before any longer run. Command used
`/tmp/ocrtech-paddle311/bin/python` and `/tmp/PaddleOCR-v3.7.0/tools/train.py` with
`Global.use_gpu=False`, `Global.epoch_num=1`, `Train.loader.drop_last=False`, and
`num_workers=0`. It completed, loaded PP-OCRv5 pretrained weights, reported 9 train
iters and 2 dev iters, and saved `best_accuracy`, `latest`, and `iter_epoch_1`
checkpoints under
`runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/smoke-checkpoints/`.
The final log line records dev `norm_edit_dis=0.06734576621095845` and `acc=0.0`.
Treat this as a training-path proof, not a quality claim. Next work is a full
longer/GPU run, export/package, router integration, and frozen-page review-burden
comparison.
- 2026-06-23: Completed the generic PP-OCRv5 silver run and rejected it as the useful
candidate. The run lives at
`runs/limbu-cdc-maths-pdf-text-silver-ppocrv5-2026-06-23/`, trained for 20 local CPU
epochs, exported and packaged cleanly, and passed model audit. The final dev metric
was weak (`norm_edit_dis=0.0903799184964279`, `acc=0.0`), and a runtime smoke
prediction was visibly bad. The likely mistake is architectural, not just duration:
the 55-character pack dictionary changed the recognition head, so the run threw away
the useful existing Sirijonga head. Keep it as negative evidence; do not promote it to
the book router.
- 2026-06-23: Trained the better candidate by warm-starting from the existing Sirijonga
v4 checkpoint and dictionary instead of the generic pack dictionary. Recipe:
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/paddle_recognizer.yml`;
base checkpoint:
`outputs/limbu-first-router-v1/source-archives/limbu/checkpoints/best_accuracy.pdparams`;
dictionary: `data/dictionaries/limbu-sirijonga-v2.txt`; 20 local CPU epochs, batch
size 8, LR `0.00002`. Paddle loaded the base checkpoint without head-shape mismatch
warnings. Best dev checkpoint was epoch 14 with `acc=0.49999958333368055` and
`norm_edit_dis=0.7922192746296381`; final epoch drifted to
`acc=0.4166663194447338`, `norm_edit_dis=0.7741729695385723`, so the admitted
checkpoint is `full-checkpoints/best_accuracy`, not `latest`.
- 2026-06-23: Exported, packaged, audited, and smoke-tested that v4 warm-start
candidate. Inference export is under
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/export-best/`;
package/model card is
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/package-best/model-card.json`;
model audit passed at `package-best/model-audit/`. Runtime smoke on held-out crop
`cdc-maths-p0011-pdf-line-0001.png` returned exactly the silver label with average
confidence `0.9353305101394653`. A narrow package-level score over the 12-row silver
eval split is at
`runs/limbu-cdc-maths-pdf-text-silver-v4warm-ppocrv5-2026-06-23/eval-best/score/`:
codepoint CER `0.012618296529968454`, grapheme CER `0.023529411764705882`,
exact-line accuracy `0.75`, 12/12 matched predictions, and no GT codepoint gaps in
the 91-character dictionary. This is admission evidence only, not a benchmark claim:
the pack is PDF-text silver, only 12 scoring rows, not human verified, not unseen for
checkpoint selection, and has acknowledged train/dev text overlap. The immediate next
task is to run a frozen-page router comparison against current v4 and judge review
burden/regressions on real pages. Stop expanding eval-harness surface before that.
- 2026-06-23: Ran that frozen-page router comparison and rejected global promotion of
the warm-start recognizer. Created router variant
`outputs/limbu-router-v5-ppocr-devanagari-v4warm-sirijonga-silver-2026-06-23/model-card.json`
by keeping the current Devanagari primary and routing policy fixed while swapping
only the Sirijonga secondary to the warm-start package; `ocrtech audit-model` passed.
On CDC Maths pages 11..15, the v5 comparison run at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v5-v4warm-silver-cdc-compare/`
passed the book audit and preserved the same 189 lines as router v4, but review rows
rose from 58 to 68. PDF-text line coverage did not improve: both routes cover 68
PDF-text lines, leave 13 missing/partial candidates, and skip 1 no-Limbu row. A quick
geometry-aligned PDF-text check was only marginally better in aggregate for v5
(1730 -> 1708 codepoint edits over 1841 reference codepoints) and exposed several
long-line regressions.
- 2026-06-23: Also checked the same v5 router on the real-print Gorkhapatra two-page
packet. The run at
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-router-v5-v4warm-silver-compare/`
passed the book audit and preserved the same 692 lines, but review rows rose from
599 to 665. The usable Limbu page worsened from 230 to 296 review rows; the rejected
non-Limbu page stayed at 369 review rows. The comparison summary is
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/router-v5-v4warm-admission-comparison.json`.
Decision: keep router v4 as operational default. The warm-start recognizer is useful
evidence that PDF-text silver training can move crop recognition, but it should only
be used as a selective probe or further training input until it improves real-page
review burden.
- 2026-06-23: Moved back to the actual bottleneck: line preservation/layout with the
current v4 recognizer held fixed. Added
`tools/sweep_limbu_pdf_text_line_detector_params.py`, covered by
`tests/test_sweep_limbu_pdf_text_line_detector_params.py`, as a narrow detector-only
diagnostic against rendered PDF text-layer geometry. It is not an OCR-quality
harness; it only asks whether detected line boxes cover source text-layer lines. On
CDC Maths pages 11..15 the sweep at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/`
evaluated 216 simple morphology settings. Current defaults rank 154th: 68/81 Limbu
source lines covered, 13 missing/partial candidates, 189 detector boxes. The top
recall setting covers 76/81 with only 5 candidates but emits 335 boxes, so it is not
acceptable as a default without review-burden evidence.
- 2026-06-23: Ran the balanced detector candidate through the real book OCR path, not
just the geometry sweep. The derived manifest is
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-balanced-candidate-v1/book-pages-balanced-detector.jsonl`
and changes only detector parameters
(`horizontal_kernel=23`, `vertical_kernel=3`, `min_width=15`, `min_height=10`,
`min_area=50`, `max_height=100`, `detector_padding=2`). The OCR run at
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-balanced-detector-cdc-compare/`
passes the book audit with 214 lines and 60 review rows. PDF-text coverage improves
from 68 covered / 13 candidates under the current default to 73 covered / 8
candidates, and all missing-y rows become partial-x rows. The cost is real: +25 OCR
lines, +2 review rows, and 2 operator-attention pages. Decision: keep the current
default for now, but use this balanced config as the next real-page probe for pages
where missed lines matter more than slight review-row growth.
- 2026-06-23: Made that probe usable without hand-editing page manifests. The
one-command runner now accepts `--detector-profile current|balanced-pdf-text`.
`current` is the default and writes no detector overrides. `balanced-pdf-text` writes
a hash-bound profiled manifest under `<run>/profiled-page-manifest/book-pages.jsonl`
before capture prep, merges the balanced detector settings into per-page
`parse_kwargs`, records the profile and profiled manifest hashes in
`limbu-book-ocr-run.json`, and refuses to silently override conflicting row-level
detector settings. Focused runner tests now cover profile materialization and
conflict rejection. This is an operational OCR knob for real photographed/PDF pages,
not another eval harness.
- 2026-06-23: Ran the new `--detector-profile balanced-pdf-text` switch on the real
Gorkhapatra two-unique-page smoke with stable page IDs by reusing the current
baseline raw manifest. The run is
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-balanced-detector-profile-probe-stable-pageids/`;
it passes the book audit, records source manifest SHA
`b8bb7138a12c42c1a919339bbfad5165f5162ab768f4c8c00874f777b364384c`, and writes the
profiled manifest SHA
`b8f1be0a77ba7e86bf34cd15c3bd4592cbdbca252ca4c00ae20c30bf49e42010`. Compared with
`gorkhapatra-2unique-runner-v4-current-py311-priority-dashboard`, totals move from
692 lines / 599 review rows / 1 layout-gap row to 690 lines / 596 review rows /
5 layout-gap rows. On the usable Limbu page, lines move 323 -> 325, review rows
230 -> 231, Sirijonga-classified rows 286 -> 290, and other rows 6 -> 5. On the
rejected non-Limbu page, review rows fall 369 -> 365 but layout-gap warnings rise
0 -> 3. Decision stays conservative: `balanced-pdf-text` is safe as an explicit
line-preservation probe, but it is not proven better as the default.
- 2026-06-23: Added a review surface for detector-profile geometry deltas:
`tools/compare_limbu_book_line_geometry.py`, covered by
`tests/test_compare_limbu_book_line_geometry.py`. It compares two completed book OCR
runs by page/line bbox IoU, writes matched-line TSV plus added/removed-line TSV, and
copies crop plus full-page context images for unmatched lines. It also writes
`line-geometry-delta-contact-sheet.png` for fast visual triage. It is not a text
scorer or admission gate; it exists so layout-profile changes are visually reviewed
before promotion. Running it on current Gorkhapatra default vs the stable
`balanced-pdf-text` probe produced
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-balanced-detector-line-geometry-delta-review/`:
679 matched lines, 11 candidate-only lines, 13 baseline-only lines. On the usable
Limbu page specifically: 317 matched, 8 candidate-only, 6 baseline-only. A spot
check of candidate-only crop
`crops/added_in_candidate-gorkhapatra-unique-current-p0001-0094-img-line-0094.png`
confirms it is real visible page text, but still a fragment-level geometry change.
The regenerated report now records contact sheet SHA
`b164f9d3e01a64c74081272a6923921442b4a4023fa3142360fa2268e51ffeb6`. Visual pass:
the profile adds small real Sirijonga fragments on the usable page (`ᤛᤢᤱ`,
`ᤁᤡᤖᤠᤋ᤻`, `ᤂᤢᤏᤧ`, etc.), but it also removes larger baseline text/header detections.
Decision remains: use `balanced-pdf-text` only as an opt-in line-preservation probe;
do not make it the default from aggregate counts.
- 2026-06-23: Added a safer additive detector profile:
`--detector-profile current-plus-balanced-rescue`. Unlike `balanced-pdf-text`, it
keeps the current detector defaults as the primary pass and adds the balanced
PDF-text settings only as `image_line_rescue_detector_passes`, so baseline boxes are
not replaced. Focused runner tests verify this profile does not set replacement
`image_line_min_width`/`image_line_max_height` overrides and instead writes a rescue
pass. Real Gorkhapatra stable-page-ID run:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-2unique-current-plus-balanced-rescue-profile-probe-stable-pageids/`.
It passes the book audit with 703 lines / 606 review rows / 1 layout-gap row. Source
pass counts: usable page `primary=323`, `rescue-1=8`; rejected page `primary=369`,
`rescue-1=3`. Compared to current default, usable page lines move 323 -> 331,
review rows 230 -> 234, Sirijonga rows 286 -> 292, and the earlier
`possible_missed_line_gap` flag disappears while layout-gap count stays 1.
- 2026-06-23: Geometry-delta packet for current default vs
`current-plus-balanced-rescue`:
`outputs/limbu-live-book-smoke-2026-06-23/gorkhapatra-current-plus-balanced-rescue-line-geometry-delta-review/`.
It reports 692 matched default lines, 11 candidate-only rescue lines, and 0 removed
lines. Usable page: 323 matched, 8 added, 0 removed. Contact sheet SHA is
`be12597dc1f32ce446b0f122af530cd04b512dd9861db1b33ab9643f6d381544`. Visual read:
additions are mixed, but mostly small real Sirijonga fragments on the usable page;
since no baseline lines are lost and layout-gap warnings do not increase, this is the
best opt-in line-preservation profile so far. Still do not make it default until it
is checked on the actual next book/photo batch.
- 2026-06-23: Replayed `current-plus-balanced-rescue` on the CDC Maths pages 11..15
PDF-derived book run:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/book-ocr-router-v4-current-plus-balanced-rescue-cdc-compare/`.
It passes audit with 219 lines / 65 review rows / 14 layout-gap rows. Against the
current default it preserves all 189 baseline boxes and adds 30 rescue boxes; PDF-text
coverage improves from 68/81 to 73/81 covered source lines, matching the replacement
`balanced-pdf-text` run while avoiding its baseline-line deletion risk. Geometry
delta packet:
`outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/current-plus-balanced-rescue-line-geometry-delta-review/`
reports 189 matched, 30 added, and 0 removed lines, with contact sheet SHA
`838d1868b445f9ea8a11cf6e34db95fadcb9cb092098ff2321f2bec5680af22b`. This is enough
operational evidence to use additive rescue as the next opt-in probe, not enough to
change defaults or spend more time expanding generic eval harness.
- 2026-06-23: The one-command runner now writes detector-profile workflow guidance
directly into `limbu-book-ocr-run.json` and `README.md`. Each run records the profile
usage note and whether the profile is default/additive/replacement. Default `current`
runs also include `detector_profile_probe_command`, a ready-to-run command that
reruns the exact source page manifest with `--detector-profile
current-plus-balanced-rescue` into a sibling output directory, plus
`detector_profile_probe_compare_command` to generate the line-geometry delta packet
against that sibling rescue run. This is deliberately operator glue, not a
default-profile promotion.
- 2026-06-23: The generated OCR run README now links its "After review" section to
`docs/LIMBU_REVIEW_TO_TRAINING_RUNBOOK.md`, the current reviewed TSV -> audited final
book -> reviewed-only training pack handoff. This keeps new OCR runs connected to the
accepted/corrected-review blocker instead of leaving the operator at raw queue files.
- 2026-06-23: Fresh OCR runs with nonzero review rows and no page-triage
source-reject/reshoot blockers now also emit `browser_review_station_command` in
`limbu-book-ocr-run.json` and a `Browser review station:` README section. The command
creates a pending unresolved-review HTML/TSV packet from the run's focused review
queue via `tools/prepare_limbu_book_unresolved_review_packet.py`. The run now also
emits `browser_review_station_apply_command`, which expects the exported
`book-unresolved-review-packet.reviewed.tsv` in the browser review station directory
and applies it through `tools/apply_limbu_book_unresolved_review_packet.py` with
`--packet-dir` provenance. It also emits
`browser_review_station_training_pack_command`, which exports
`browser-review-station-applied/final-book` through
`tools/export_limbu_book_training_pack.py` after the reviewed packet has been
applied. Clean runs with 0 review rows do not emit empty browser-station commands,
and pages that need source rejection or reshoot must be handled in page triage before
line review. This closes the setup gap from completed OCR run to browser
review/application/training-pack handoff, while preserving the rule that no labels
exist until an exported reviewed TSV is audited and applied. The focused runner tests
now cover both sides: source-reject pages do not emit browser review commands, while a
reviewable low-confidence Limbu run executes the generated packet-builder target and
checks the generated apply/training-pack command paths.
- 2026-06-23: The browser review station now includes suspected missed-line layout
gaps instead of leaving them in a separate queue. When `layout_gap_review_row_count`
is nonzero, the generated command passes the layout-gap queue as
`--missing-line-review-queue`; `tools/prepare_limbu_book_unresolved_review_packet.py`
maps pending `gap_row_id` rows to `candidate_missing_line`, derives a page-wide bbox
from `from_y`/`to_y`, copies the gap crop, renders highlighted page context, and
attaches neighboring OCR-line context. The new tests prove both the packet-builder
conversion and the runner-generated command on a synthetic suspicious-gap page. This
is intentionally book-OCR operator glue, not another generic eval-harness layer.
- 2026-06-23: Proved the layout-gap browser station is not a dead-end artifact. The
suspicious-gap runner test now continues through reviewer TSV export/application
using the generated apply-command paths: it marks the generated
`candidate_missing_line` as accepted with explicit `corrected_text`, saves the TSV at
`browser_review_station_apply_command --review-packet`, applies it via
`tools/apply_limbu_book_unresolved_review_packet.py` using the original packet
directory as provenance, and verifies the final book projection contains the inserted
Unicode line with insertion metadata tied back to the original `gap_row_id`. This
closes the practical loop for missed-line recovery: detect suspicious gap ->
review/transcribe in browser -> audited final-book insertion.
- 2026-06-23: The Limbu recognizer now has a measured real-heavy fine-tune champion on
the frozen 255-line Gorkhapatra PDF-backed silver eval. Baseline combined 48,507-row
checkpoint:
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/`
scored codepoint CER `0.016498`, grapheme CER `0.038266`, exact-line accuracy
`0.662745`. Real-heavy low-LR fine-tune:
`runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/`
scored codepoint CER `0.011336`, grapheme CER `0.024736`, exact-line accuracy
`0.729412`. That is `51` fewer codepoint edits and `64` fewer grapheme edits on the
same image paths and identical GT text, with a `35.36%` relative grapheme error
reduction. Key artifacts: training audit
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-training-audit-2026-06-23/`,
model package
`runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json`,
package audit
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-package-audit-2026-06-23/`,
score
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-eval-2026-06-23/score/`,
failure analysis
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-best-eval-2026-06-23/score/failure-analysis.md`,
official paired comparison
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-vs-combined-best-comparison-2026-06-23/comparison.md`,
and path-keyed comparison
`outputs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-vs-combined-best-path-keyed-comparison-2026-06-23/comparison.md`.
Caveat remains strict: this is not human-ground-truth claim-safe because labels are
PDF text-layer silver and the checkpoint was selected on this eval surface. The
top remaining recognizer failures are omitted Limbu vowel/mark codepoints, quote or
ASCII punctuation mismatches, omitted spaces, and omitted sentence punctuation.
`tools/compare_recognizer_eval_runs.py` now falls back to unique `image_path`
pairing when sample IDs are rewritten; the real-heavy comparison pairs all 255
rows, audits cleanly, and keeps `claim_safe=false` because of silver labels,
selected-on-eval status, differing manifest files, and the older baseline's stale
row-level `split=train` metadata.
## 2026-07-15 — Newari four-writing-system support boundary
Newari / Nepal Bhasa is now represented by four explicit OCR profiles:
Newar/Prachalit, Bhujimol (Bhujinmol), Ranjana, and Devanagari. The runtime
alias resolver and codepoint-transcription router are in
`src/ocrtech/newari_scripts.py`; `languages/nepal-v1.json` records encoding,
input, output, font, and capability state; and
`configs/nepal-language-pack-pipeline-v1.json` exposes four recognizer refs to
the book-OCR runner. Runner metadata now preserves the selected writing-system
profile, source Unicode status, and output policy.
The encoding boundary is intentionally asymmetric. Newar/Prachalit uses native
Newa Unicode, Bhujimol is a distinct visual profile transcribed with Newa
codepoints and source-style metadata, Ranjana is unencoded and therefore uses
image input plus a Newa transcription/source-image policy, and Devanagari stays
an independent Unicode route. Shared transcription ranges never authorize
sharing a visual recognizer.
The adversarial audit
`outputs/newari-four-writing-system-support-audit-2026-07-15-r1/` reports
`PASS_PROFILE_SUPPORT_REGISTERED_NO_FOUR_SCRIPT_OCR_CLAIM`, four of four
profiles structurally registered, zero of four model-ready, and
`four_script_ocr_claim_ready=false`. Mutation tests reject a missing profile,
a fabricated Ranjana range, Ranjana route drift, premature Bhujimol artifact
admission, Ranjana-font de-quarantine, and a missing recognizer binding. The
local Nithya Ranjana DU font remains quarantined because neither its license nor
its Devanagari-to-Ranjana mapping is admitted. No training or inference ran in
this change.
## 2026-07-15 r36 — official Ranjana-font custody correction
The r1 font-quarantine paragraph above is superseded. The local Nithya Ranjana
DU binary is an exact match for Ek Type's official 1.000 release, and the NU
variant has been acquired from that same release. The release is SIL OFL 1.1.
The official README and the fonts' own cmap/GSUB tables agree on the mapping:
DU is Devanagari-based (78 Devanagari codepoints; `deva`/`dev2` shaping), while
NU is Newa-based (88 Newa codepoints; `newa` shaping). Both have zero
private-use codepoints.
The registry now records DU as a Ranjana reference-only font and NU as the
Newa-output-compatible Ranjana-style render candidate. This is a custody and
mapping result only. Ranjana remains unencoded, NU is not a Bhujimol font, no
synthetic training payload has been admitted, and a modern reference font does
not validate historical manuscript style. Qualified page decisions remain
0/339; no real pixels, labels, model, prediction, score, or codepoint CER were
created. The frozen evidence is
`outputs/newari-historical-font-custody-audit-2026-07-15-r36/`.
## 2026-07-15 r37 — CC0 Bhujimol reviewer reference
The exact Wikimedia Commons “Bhujimol script characters (Unicode)” SVG is now
frozen with its API metadata. Commons records it as `Z3lvs`'s own work under
CC0 1.0, dated 2026-05-24. The 181,286-byte file contains 111 vector paths and
no text nodes, font, explicit path ID/codepoint mapping, shaping rules, script,
external resource, or raster image.
This closes a reviewer-reference gap, not the Bhujimol font or OCR gap. The
chart cannot render arbitrary text or attest conjuncts and is excluded from
training. Bhujimol still has no admitted font, manuscript pixel-label set,
model, prediction, score, or codepoint CER. Qualified page decisions remain
0/339. Evidence:
`outputs/newari-bhujimol-reference-chart-audit-2026-07-15-r37/`.
## 2026-07-15 r38 — Bhujimol reference installed beside review dispatch
The exact r37 CC0 chart and custody evidence are now packaged as a separate,
hash-bound optional supplement for the r35 Bhujimol assignment. Its source
audit passes 12/12 checks and binds the unchanged 293-unit Bhujimol assignment
scope `7779fd85…e91d`, the full 339-unit review scope, and the exact r35 and r37
manifests. The packet contains no font, raster manuscript pixel, JSONL review
row, label, prediction, model, or metric.
The eight-file packet was installed in a separate directory on `omac`. All
seven manifested files passed there with supplement-manifest SHA-256
`b46da798e446617e27eee52d75cebc1817a137b2fd33f9d3a5fb8eabecd4f7e9`.
The complete r35 dispatch reverified unchanged at
`77e6e0c1fb324d3cf0a22b9e028a1c73a8dcad9f5dcd91efa11e8c404988007e`.
Qualified decisions remain 0/339 and audited real lines remain zero. Evidence:
`outputs/newari-bhujimol-review-reference-supplement-2026-07-15-r38/`.
## 2026-07-15 r39 — OFL Ranjana reference installed beside review dispatch
The exact official Nithya Ranjana NU 1.000 font is now packaged with its
upstream OFL 1.1, upstream README, and r36 custody evidence as a separate
optional supplement for the 46-unit r35 Ranjana assignment. The source audit
passes 18/18 checks. It binds the exact assignment and full-review scopes,
official release ZIP, font, license, current registry and pipeline policy, and
the frozen r35/r36 manifests.
The nine-file packet was installed in a separate directory on `omac`. All
eight manifested files passed there with manifest SHA-256
`a57bd60aeddaecb311d7939b75f930e990ff587ad6700f74d812e863a87b76b6`.
The complete r35 dispatch again reverified unchanged at
`77e6e0c1fb324d3cf0a22b9e028a1c73a8dcad9f5dcd91efa11e8c404988007e`.
NU maps Newa Unicode but is reference-only: Ranjana remains unencoded and no
historical hand, label, training payload, model, prediction, or metric is
admitted. Qualified decisions remain 0/339 and audited real lines remain zero.
Evidence: `outputs/newari-ranjana-review-font-supplement-2026-07-15-r39/`.
## 2026-07-15 r40 — bounded four-profile operational evidence
The unified support audit now binds four exact, separately frozen evidence
packets: the r31 Devanagari and r34 Newar/Prachalit one-line synthetic runtime
smokes, the r38 Bhujimol qualified-review reference supplement, and the r39
Ranjana qualified-review font supplement. All four evidence manifests and
reports pass exact hash, status, and claim-boundary checks.
This does not make the four profiles equivalent. Devanagari and
Newar/Prachalit have local diagnostic models with verified synthetic runtime
loads but no real-line quality evidence. Bhujimol and Ranjana have reviewer
orientation references but no admitted recognizer. A chart is not a Bhujimol
font, and a modern Newa-mapped Ranjana font is neither a source encoding nor
historical-manuscript validation. The audit reports four registered profiles,
two model-ready profiles, two runtime-smoke profiles, two reviewer-reference
profiles, zero qualified decisions, zero audited real lines, and
`four_script_ocr_claim_ready=false`. It performed no model load, prediction,
training, scoring, or GPU operation. Evidence:
`outputs/newari-four-writing-system-support-audit-2026-07-15-r40/`.
## 2026-07-15 r41 — official Newa-edition acquisition candidate
One exact State Portal record is now frozen as a metadata-only candidate for
the Devanagari-written Newari route: `SIKKIM HERALD-NEWA EDITION`, dated
13 April 2022, at the permission-covered Government of Sikkim Herald prefix.
The catalog binds the unchanged source-policy SHA-256, the exact title/date/URL,
five official HTML snapshots, and an eight-byte PDF Range probe whose
`Content-Range` reports 3,975,535 total bytes. It downloads no PDF body or page
image. The newer IPR portal currently lists zero Newar records and contributes
no PDF request or permission claim.
This is an acquisition candidate, not script or OCR evidence. The target route
is Devanagari, while source-script confirmation, embedded-font inspection, and
the permission-required third-party-material screen all remain pending a
remote all-page review. Admitted pixels, qualified decisions, labels, audited
real lines, models, and metrics remain zero. The 339-unit Bhujimol/Ranjana
qualified-review queue also remains unchanged at zero decisions. Evidence:
`outputs/newari-devanagari-sikkim-source-catalog-2026-07-15-r41/`.
## 2026-07-17 r46 — exact Newari runtime profile activation
The config-driven book runner now distinguishes the four declared Newari
writing-system profiles from the recognizers active in a particular run. With
the checked-in Newar config, the default preflight activates only the two
artifact-backed diagnostic routes, `devanagari` and `newa_prachalit`, while
recording `newa_bhujimol` and `newa_ranjana` as declared but unavailable. The
two intentionally null historical artifacts therefore no longer prevent an
otherwise valid Newar run.
`--writing-system PROFILE` provides an exact, repeatable selector using the
canonical alias resolver. An explicit Bhujimol or Ranjana request remains
fail-closed until that profile has an artifact, and
`--script-recognizer-artifact REF=PATH` can activate a separately supplied
artifact after normal preflight checks. Run and page metadata retain declared,
active, inactive, unavailable, and requested profile identities.
This is runtime activation plumbing, not new OCR evidence. The existing
Newar/Prachalit model remains a failed-fit diagnostic; the Devanagari export
remains diagnostic; Bhujimol and Ranjana still have no admitted recognizer;
and no real-line quality claim, CER, training run, or GPU operation was added.
The automatic artifact-backed activation described here is superseded by the
r54 execution-admission gate below.
## 2026-07-16 r48 — fail-closed Bhujimol/Ranjana review rendering
The combined r47 review UI and the per-profile r48 dispatch now require the
current frozen source image to load at its recorded dimensions before a
decision can be saved. Export additionally requires every review-unit render
to have passed that gate in the current browser session. Missing
files, dimension drift, invalid display-crop bounds, stale asynchronous image
loads, and immutable local-storage drift fail closed. The gate state is
intentionally not persisted across browser sessions.
The successor packets preserve the exact prior review scopes: 339 combined
units, split into 293 Bhujimol units and 46 Ranjana units. Their blank JSONL
SHA-256 values remain `110f240b…c101`, `7779fd85…e91d`, and
`30eeb57b…8d9b`, respectively. Both packet manifests verify. A 95-test Newari
acquisition, readiness, review, merger, freezer, supplement, and browser suite
passes, including real headless-Chrome tests for successful rendering, 404s,
wrong dimensions, programmatic save attempts, blocked export, and stale-load
isolation.
This is a reviewer-integrity improvement, not new recognition evidence. The
browser verifies decoded dimensions and crop bounds; it does not authenticate
the current file bytes or recompute the recorded SHA-256. The packets therefore state
`browser_pixel_hash_check_performed=false`. Qualified decisions, admitted
labels, and audited real lines remain zero, and no training, inference, score,
or CER was produced. Evidence:
`outputs/newari-qualified-page-review-ui-2026-07-16-r47/` and
`outputs/newari-qualified-page-review-dispatch-2026-07-16-r48/`.
## 2026-07-16 r51 — current-file source preflight becomes a freeze gate
A new read-only host-side preflight now binds the frozen 339 review units to
the source files currently present on `omacs-MacBook-Pro.local`. It validates
the exact four allowlisted roots, safe relative paths, regular non-symlink
files, streaming SHA-256, decoded dimensions, and unchanged device, inode,
size, modification time, and change time across each verification. The real
r49 run passed all 337 unique files: 291 Bhujimol and 46 Ranjana. It performed
no remote write, download, OCR, training, or scoring operation and copied no
source pixels into the repository.
The qualified-review freezer and split dispatch merger now require the passing
receipt. They validate its complete 337-file inventory, frozen worksheet and
profile scopes, verifier and tool hashes, exact observations, and receipt
immutability. The receipt scan must start at or after every review decision.
The freezer also independently reruns the complete read-only SSH verification
after the decisions and requires all 337 live observations to match the
receipt. The checked-in r49 baseline therefore cannot be reused after future
human review; the preflight must run again then. Missing, failed, stale,
partial, wrong-scope, live-mismatched, or swapped receipts fail closed.
The successor r50 combined UI and r51 dispatch retain byte-identical blank
review scopes (`110f240b…c101`, `7779fd85…e91d`, and `30eeb57b…8d9b`) while
documenting the enforced post-decision gate. Browser pixel hashing remains
false: the separate host-side receipt is the exact-file evidence. Qualified
decisions, admitted labels, and audited real lines remain zero, and no quality
or four-script OCR claim is ready. Evidence:
`outputs/newari-qualified-page-review-source-preflight-2026-07-16-r49/`,
`outputs/newari-qualified-page-review-ui-2026-07-16-r50/`, and
`outputs/newari-qualified-page-review-dispatch-2026-07-16-r51/`.
## 2026-07-16 r52 — bounded Ranjana reference-font render audit
The exact official Nithya Ranjana NU 1.000 font, OFL 1.1, and upstream README
are now frozen in a self-contained successor reference packet bound to the
then-current r51 Ranjana assignment. The r39 and r51 assignment scopes are
byte-identical: 46 units and zero decisions. The direct HarfBuzz/Pillow RAQM
audit uses only the exact font, with no system fallback.
The configured Ranjana transcription dictionary contains 76 Newa codepoints;
74 have font coverage, outlines, fallback-free shaping, and raster ink in the
bounded probes. The exact two gaps are `U+11446 NEWA SIGN NUKTA` and
`U+1144D NEWA COMMA`. The broader Newa dictionary is only 88/97 covered. A
direct-font probe of the exact 3,111-line CC-BY-4.0 Newa text corpus found
`.notdef` in 604 lines (1,927 glyphs), dotted-circle insertion in 21 lines, and
624 affected lines in the metadata-only gap inventory.
The 74-codepoint allowlist does not by itself authorize arbitrary strings.
The audit tool's exact-font sequence gate rejects empty or space-only text,
unsupported codepoints, `.notdef`, dotted-circle insertion, and blank raster
output. Passing that gate establishes mechanical rendering only, not
orthographic validity. Full-dictionary and full-repertoire conformance, corpus
synthesis, unrestricted rendering, historical Ranjana validation, qualified
decisions, labels, training, models, predictions, metrics, and the four-script
OCR claim all remain false. Human review is not needed for the digital
mechanics established here; it remains necessary for manuscript decisions,
labels, historical fidelity, and a qualified render suite. No remote
installation or outward action was performed. Evidence:
`outputs/newari-ranjana-render-conformance-audit-2026-07-16-r52/`.
## 2026-07-16 r53 — historical four-writing-system evidence boundary
The unified Newari support audit then bound the evidence successors for
all four declared writing-system profiles. It verifies the exact registry,
pipeline, and runtime-profile bytes; the r31 Devanagari and r34
Newar/Prachalit diagnostic runtime smokes; the r38 Bhujimol reference and r45
companion-text limit audit; the current zero-decision r51 review dispatch; the
bounded r52 Ranjana render mechanics and exact per-string validator; and the
r41/r42 Devanagari acquisition/review evidence. The packet is integrity-bound,
is written through a verified atomic staging directory, and fails closed on
claim promotion or evidence drift.
The four routes remain intentionally non-equivalent: 4/4 profiles are
registered, 2/4 have exact diagnostic models and one synthetic runtime smoke,
2/4 have qualified-review references, and 1/4 has a bounded reference-render
scope. There are zero exact Bhujimol companion-text prefill routes, zero
source-qualified real profiles, zero qualified decisions out of 339 review
units, and zero audited real lines. The Newar/Prachalit export is a known
failed fit; Bhujimol has no admitted font or machine-readable glyph map;
Ranjana has bounded coverage only; and the Devanagari source candidate still
needs script confirmation and third-party-material screening. Accordingly,
all four profiles remain claim-unready and
`four_script_ocr_claim_ready=false`.
This audit required no model load, inference, rendering, training, scoring,
GPU operation, remote command, or source download. Human review is not needed
to reproduce the current mechanical audit; it remains required for the
manuscript/source decisions and labels that the audit shows are absent.
Evidence:
`outputs/newari-four-writing-system-support-audit-2026-07-16-r53/`.
## 2026-07-16 r54 — fail-closed Newari runtime execution admission
The config-driven book runner no longer treats the mere presence of an
artifact as permission to run a Newari recognizer. With the checked-in config,
default Newari resolution activates zero recognizers: the Devanagari export is
diagnostic-only, the Newar/Prachalit export is both diagnostic and a known
failed fit, and Bhujimol and Ranjana have no admitted model or dataset.
Diagnostic execution now requires an exact writing-system selection plus an
explicit purpose flag. Devanagari requires `--writing-system Deva` with
`--allow-newari-diagnostic`. Prachalit requires `--writing-system Prachalit`
with the stronger `--allow-newari-known-failed-fit`; the generic diagnostic
flag cannot waive its failed-fit boundary. Bhujimol and Ranjana remain
non-runnable even if an artifact path is supplied. Artifact overrides can
relocate an already admitted model but cannot promote registry/config status.
Unknown or newly invented Newari execution statuses fail closed.
Activation and run metadata now record the exact execution policy, artifact
override state, active/inactive/unavailable/blocked routes, and opt-in kind.
`quality_claim_ready` and `claim_evidence_eligible` remain false: opting into a
diagnostic run does not turn its output into accuracy evidence. Preflight runs
before output directory creation or backend construction, so a default or
policy-blocked Newari request writes nothing and loads no model.
This milestone changed only local resolution, preflight, metadata, tests, and
documentation. Verification used fake artifact directories and backends; it
performed no OCR inference, model load, training, scoring, GPU operation,
remote command, or outward action.
## 2026-07-16 r55 — tracked Bhujimol evidence-conformance boundary
A new self-contained audit now connects the exact tracked Bhujimol evidence
chain without requiring ignored local assets or opening source pixels. It
verifies the frozen r37 chart, r38 reference supplement, r45 companion-text
limit, r49 source receipt, r51 dispatch, r53 global support boundary, current
registry and pipeline, 163-symbol output dictionary, and r54 runner. Every
packet manifest and primary report is pinned to its exact SHA-256.
The audit recomputes 293 blank r51 Bhujimol review units and 291 unique source
contracts, split as 217 `b14901560`, 73 `MW1NPL99`, and three Cambridge MS
Add.2137 units. Those contracts exactly equal the Bhujimol-filtered r49
observations, including paths, hashes, and decoded dimensions. r49 remains
baseline custody evidence only because it predates future review decisions; a
new preflight is still required after the final decision.
The chart is directly parsed as 111 SVG outlines with no text, IDs, font,
machine-readable mapping, shaping program, raster, script, or external
reference. The dictionary has 163 unique one-codepoint rows, including 76 Newa
codepoints, but is output vocabulary rather than a Bhujimol glyph map. The three
r45 companion-text routes still provide zero exact line-aligned transcriptions
and zero admissible prefills. Accompanying work-level text cannot determine the
source-image line alignment or replace script/content confirmation.
The r54 runner remains inactive for Bhujimol by default and rejects artifact
overrides as a means of promotion. The r55 writer is integrity-bound, replayed
against the exact inputs, atomically staged, and tested against packet, review,
contract, chart, vocabulary, configuration, and claim-boundary drift. It copies
no chart, text, pixels, font, review rows, labels, or model into the output.
Codepoint mapping, arbitrary rendering, qualified decisions, exact line labels,
training, inference, metrics, and the four-writing-system OCR claim remain
false. No model, GPU, remote host, download, or outward action was used.
## 2026-07-16 r56 — tracked Ranjana source and bounded-render conformance
A new tracked-only audit now gives Ranjana the current source-contract binding
previously established for Bhujimol. It verifies the exact r15 mixed-source
acquisition, r25 stable Ranjana reacquisition, r27 consolidation, r49 baseline
source receipt, r51 dispatch, and self-contained r52 bounded-render packet. It
also binds the current registry, pipeline, configured and full Newa
dictionaries, r49 producer, r25/r27 audit producers, r52 validator, r54 runner,
and its own tool bytes.
The audit recomputes 46 blank r51 Ranjana units and 46 unique source contracts.
Those contracts exactly equal the Ranjana-filtered r49 observations, including
paths, hashes, and decoded dimensions. The active sources are 21
`MW1NPL132` units from r25 and 25 `BDRC MW1NPL23` units from r15. r25 records a
stable reacquisition and explicitly invalidates r7 for current custody after
canvas 1 changed from the frozen r6/r7 hash. Every current r27/r51
`MW1NPL132` row points to r25; none uses the stale r7 bundle.
The r52 packet remains deliberately bounded. Its exact reference font covers
74/76 configured Newa codepoints and 88/97 full Newa codepoints, requires an
exact per-string sequence validator, and permits no undeclared fallback. The
3,111-line corpus is a render/glyph-coverage probe with 624 affected gap lines,
not transcription paired with any of the 46 manuscript images. It therefore
does not replace script/content review or establish line labels, orthographic
validity, historical style fidelity, unrestricted Ranjana rendering, or
manuscript OCR quality.
The writer uses canonical integrity, exact-input replay, strict manifests,
atomic staging, and no-clobber behavior. Tests reject packet drift, AppleDouble
sidecars, symlinks, stale-r7 promotion, source-bundle substitution, review
decisions, contract drift, render/style promotion, producer drift,
configuration promotion, and nested claim mutation. The r49 receipt remains
historical pre-review custody and must be replaced after the final decision.
Ranjana remains unencoded, non-runnable, source-unqualified, and without an
admitted model or OCR claim. No source pixels, rendering, model load, GPU,
remote command, download, or outward action was used.
## 2026-07-16 r57 — historical tracked-support aggregate
The tracked-only r57 audit combined the exact r55 Bhujimol and r56 Ranjana
focused packets with the then-current Devanagari and Newar/Prachalit evidence
boundary. It verified four registered profiles, zero default-active
recognizers, zero source-qualified profiles, zero admitted page- or
line-aligned transcriptions, zero admitted real lines or quality metrics, and
zero claim-ready profiles. It did not open source pixels, model binaries,
archives, or current review state.
r57 remains a valid historical aggregate snapshot. It predates the focused r58
Devanagari and r59 Newar/Prachalit packets, so r60 supersedes only its aggregate
index role; r55, r56, and r57 remain immutable evidence. Evidence:
`outputs/newari-tracked-support-conformance-audit-2026-07-16-r57/`.
## 2026-07-16 r58 — focused Devanagari evidence boundary
The focused Devanagari packet preserves the historical r42 sanitized metadata
receipt and binds the tracked r12 rejected claim/visual receipts, r31
synthetic-only runtime, r41 conditional catalog, r57 exclusion baseline, and
current runtime policy. It admits zero current page images, qualified review
decisions, labels, audited real lines, train/eval material, or CER. Source
script confirmation, third-party-material screening, current lawful PDF/pixel
custody, and qualified page review remain pending. The diagnostic runtime
opt-in is a separate execution policy and does not satisfy any source-admission
gate. Evidence:
`outputs/newari-devanagari-evidence-conformance-audit-2026-07-16-r58/`.
## 2026-07-16 r59 — focused Newar/Prachalit evidence boundary
The focused Prachalit packet binds the historical r34 one-line synthetic
runtime receipt, r33/r53 custody snapshots, the r57 exclusion boundary, and the
r58 sibling boundary. The recovered export remains a known failed fit and
requires its exact runtime opt-in. r59 did not replay current model custody,
open real source pixels, compare output with a source, admit a training run or
checkpoint, or compute a quality metric.
The Cambridge material remains historical review documentation: 635 blank rows
were previously reported, but current review state was not revalidated and
zero qualified decisions, labels, or audited real lines are admitted for this
support claim. The next gate is qualified Cambridge review followed by the
fail-closed post-review claim audit. Evidence:
`outputs/newari-prachalit-evidence-conformance-audit-2026-07-16-r59/`.
## 2026-07-16 r60 — current four-profile evidence index
The current tracked-only index exact-binds one focused packet for each explicit
Newari writing-system profile:
| Profile | Focused packet | Current runtime/evidence state | Next gate |
|---|---|---|---|
| Newar / Prachalit | r59 | exact known-failed-fit opt-in required | qualified Cambridge review and fail-closed post-review claim audit |
| Bhujimol / Bhujinmol | r55 | blocked; no admitted model or dataset | qualified review followed by a new exact source preflight |
| Ranjana | r56 | blocked; no admitted model or dataset | qualified review, new source preflight, and accepted-line transcription |
| Devanagari | r58 | exact diagnostic opt-in required | qualified page script/third-party review and current source custody |
r57 is retained as the historical aggregate baseline; r60 supersedes only its
index role. The review figures remain in different temporal scopes and are not
one current backlog: `frozen r55 review dispatch` for Bhujimol (293 units/291
contracts), `frozen r56 review dispatch` for Ranjana (46/46), a `historical r42
sanitized metadata receipt` for Devanagari with zero currently admitted review
units, and `historical Cambridge review documentation` for Prachalit with 635
blank rows and zero admitted qualified decisions. r60 did not revalidate
current review state.
Support here means explicit profile registration, exact evidence binding, and
fail-closed runtime routing—not demonstrated OCR quality. Default runtime
activation is zero. No profile has current model custody replayed, admitted
qualified decisions, page- or line-aligned transcriptions, labels, audited real
lines, quality metrics, a training run, a checkpoint, or claim-ready status.
No backend, inference, metric, training, GPU, remote command, download, or
outward action was used. Evidence:
`outputs/newari-four-profile-evidence-index-conformance-audit-2026-07-16-r60/`.
## 2026-07-16 r62 — strict recognizer-override front door
New operational book-OCR invocations now use
`tools/run_book_ocr_strict.py`. The front door exact-pins the byte-identical
r60 canonical runner, parses per-recognizer overrides without last-wins
behavior, and rejects malformed, duplicate, unknown, cross-language,
legacy/per-ref-colliding, or active-selection-unused overrides before
delegation. Accepted Devanagari and Prachalit requests retain their exact,
distinct diagnostic flags; Bhujimol and Ranjana remain non-runnable and cannot
be promoted by an artifact path. Accepted requests are delegated once with the
original parsed arguments unchanged.
The canonical `tools/run_book_ocr.py` SHA-256 remains
`ed23799e1887f88c84ca3b45dbd2e63519e39aba0c85e3a68f2bd47fb863d8aa`,
so r55-r60 exact replay remains intact. Direct invocation of the canonical
runner is outside the strict wrapper's additional override-intent guarantees.
This milestone establishes only fail-closed invocation validation: it does not
establish artifact byte identity or custody, Paddle semantic validity,
dictionary compatibility, model/data admission, OCR quality, script coverage,
or four-writing-system claim readiness. Verification used only synthetic tiny
artifact structures and fake delegation; it loaded no model, ran no backend or
inference, used no GPU or remote host, and took no outward action.
## 2026-07-16 r63 — two-phase strict artifact preflight
The strict front door now resolves language policy and override intent with no
operator override paths passed into canonical resolution. Newari overrides
require an explicit writing-system selection. Duplicate, malformed, unknown,
cross-language, colliding, inactive, unselected, and policy-blocked requests
therefore fail before override-derived filesystem probing. This preserves the
distinct Devanagari diagnostic and Prachalit known-failed-fit errors, while
Bhujimol and Ranjana remain non-runnable before their supplied paths are
inspected.
Only policy-authorized active overrides reach structural preflight. The exact
canonical relative-path and direct-before-`artifacts/` rules select the
candidate, but the strict layer requires a symlink-free directory chain and
three regular, non-symlink, non-empty Paddle files. It rejects missing paths,
non-directories, FIFOs and other special nodes, missing/empty/symlinked required
files, symlinked roots or ancestors, and simultaneous complete direct and
nested layouts. Structural issues are preflight errors before canonical
delegation, output creation, or backend construction. Accepted requests still
delegate their original parsed Namespace exactly once.
The front door also rejects a symlink or special-file canonical-runner leaf,
verifies that the opened node is the inspected regular file, hashes those
opened bytes, and executes those same bytes. The canonical runner itself
remains byte-identical at
`ed23799e1887f88c84ca3b45dbd2e63519e39aba0c85e3a68f2bd47fb863d8aa`,
preserving r55-r60 replay.
r63 proves only rejection ordering and one point-in-time filesystem-structure
snapshot. It does not read or hash artifact contents, establish byte identity
or custody, parse or validate Paddle semantics, prove dictionary compatibility,
prevent later path replacement, admit a model/data/source/label/metric, or
establish OCR quality, script coverage, or four-writing-system claim readiness.
Tests used synthetic tiny files, symlinks, FIFOs, and fake delegation only; no
model, backend, inference, training, scoring, GPU, remote host, download, or
outward action was used.
## 2026-07-16 r64 — exact diagnostic execution identity
The strict front door now closes the demonstrated static cross-profile
substitution gap for every active Newari Devanagari or Prachalit recognizer.
Configured artifacts and overrides must retain the exact r53-audited identity:
the three Paddle inference files and configured model dictionary are each
bound by exact size and SHA-256. The contract also pins recognizer status,
execution policy, script/profile/route, source-Unicode/output policy,
configured artifact declaration, required diagnostic opt-in, and the
`PP-OCRv5_mobile_rec` model name. Any future active Newari recognizer or changed
package bytes fail closed until a deliberate audit and allowlist update.
Policy and intent still take priority. Missing diagnostic opt-ins and blocked
Bhujimol/Ranjana requests fail before identity validation; exact semantic drift
fails before an operator override path is probed. Authorized paths are opened
component-by-component without following symlinks. Each regular file is opened
with no-follow/nonblocking flags, matched to the inspected inode, bounded-stream
hashed, and checked for stable metadata before close. Only after all active
identities pass does the wrapper delegate the original parsed Namespace once.
The canonical `tools/run_book_ocr.py` remains byte-identical at
`ed23799e1887f88c84ca3b45dbd2e63519e39aba0c85e3a68f2bd47fb863d8aa`,
preserving r55-r60 replay.
Focused verification used injected tiny packages for direct, nested, copied,
configured, override, mixed-profile, cross-swap, same-size mutation, wrong-size,
missing-contract, semantic-drift, symlink, FIFO, directory, stat/open leaf and
directory replacement, in-read mutation, short/error read, unsupported flag,
and predelegation failure cases. A guarded live local check also read and
matched all 15,682,997 runtime bytes across the two currently present ignored
packages; it did not construct a backend, run inference, or produce output.
This is a point-in-time
`exact diagnostic execution identity observed immediately before delegation`,
not continuous custody or proof of bytes later loaded: the unchanged canonical
runner and Paddle reopen pathnames, so a TOCTOU window remains. The external
dictionary hash binds configuration evidence, while the exact YAML also freezes
the decoder characters.
r64 is additive and does not retroactively change r60's honest
`current_model_custody_replayed=false` boundary. It does not establish package
origin/authenticity, rights or redistribution permission, semantic Paddle
validity, dependency identity, source/data/label/metric admission, OCR quality,
script correctness, or four-profile claim readiness. Devanagari remains
diagnostic-only, Prachalit remains a known failed fit, and Bhujimol/Ranjana
remain blocked. No inference, training, scoring, GPU, remote command, download,
or outward action was used.
## 2026-07-16 r65 — strict runtime conformance overlay
The tracked r65 packet exact-binds r60 as the current four-profile evidence
index, r53 as historical runtime-package identity authority, and the current
strict implementation that composes r63 policy/structural preflight with r64
exact predelegation identity. It also closes one equality chain across the
actual canonical-runner bytes, the strict wrapper's runner pin, and r60's
recorded runner identity. Evidence:
`outputs/newari-strict-runtime-conformance-audit-2026-07-16-r65/`.
The conformance audit independently exercised the real strict and identity
code with tiny temporary packages. It confirmed zero default activation,
distinct Devanagari diagnostic and Prachalit known-failed-fit opt-ins,
Bhujimol/Ranjana policy rejection before structural path validation, configured
and direct/nested relocated identity passes, cross-profile and mutated-byte
rejection, size/structure/symlink/FIFO failures, missing-contract and model-name
failures, and exactly one unchanged-Namespace delegation without output. The
two production four-file allowlists match both observed and expected r53 rows.
r65 consolidates only the tracked r63 structural-preflight and r64
predelegation-identity evidence. r60 remains the current four-profile evidence
index, r53 remains historical identity authority, and r53/r55-r60 stay
immutable. The audit did not probe or open an ignored model package and read
zero real model bytes, so `current_model_custody_replayed=false` remains the
current boundary. It does not establish present package availability,
backend-consumed identity, continuous custody, TOCTOU closure, Paddle semantic
validity, dictionary compatibility, provenance/authenticity, rights, source or
label admission, review progress, OCR quality, or four-script readiness.
Direct canonical-runner invocation remains outside the overlay's guarantees.
No backend, inference, metric, training, GPU, remote command, network request,
download, or outward action was used.
## 2026-07-16 r66 — crash-fail-closed integrity-packet publication
The tracked r66 packet exact-binds the frozen r45 Bhujimol companion-text
limit, r51 blank 339-unit review dispatch, and r65 strict-runtime overlay, and
introduces a reusable publisher for subsequent evidence packets. Evidence:
`outputs/newari-integrity-packet-publication-conformance-2026-07-16-r66/`.
Publication reserves the final directory without overwrite and writes a durable
`.INCOMPLETE` marker before any payload. Members and `SHA256SUMS` are created
through exclusive no-follow descriptors, fsynced, and verified before marker
removal becomes the commit point. Fault injection across the precommit
checkpoints produced zero reader acceptances. Under the explicit
quiescent/cooperating-writer recovery model, token-bound recovery removed an
exact owned incomplete packet and replay succeeded; recovery refused an
observed foreign entry and preserved its bytes. A crash before marker
initialization or during an AppleDouble guard remains reader-invalid but
requires manual handling. A caller failure after commit left a valid packet.
This closes future packet crash-validity ambiguity, not a script-support gate.
The next support-changing work still requires qualified review of 293 Bhujimol
and 46 Ranjana units, then a fresh exact source preflight and accepted-line
transcription. The current companion material has zero exact line alignments
and zero admissible prefills and therefore does not replace qualified review.
r66 creates no review decisions, labels, source/model custody, runtime model,
backend-consumed identity, metric, or four-script OCR claim. No source pixels,
models, backend, inference, training, GPU, remote command, network request,
download, or outward action were used.
## 2026-07-16 r67 — profile-scoped qualified-review handoff seal
The r51 split review can now be custodied one completed profile at a time with
`tools/seal_newari_qualified_page_review_profile.py`. The sealer derives the
selected profile from the exact r27 blank worksheet, verifies equality to the
frozen r51 assignment and manifest, and requires exact row order and identity,
immutable-field equality, complete script/content decisions, consistent line
estimates, UTC decision timestamps, and an exact single-profile v2 reviewer
attestation. Ranjana is bounded to 46 units/46 source contracts and assignment
scope `30eeb57b2d1d4de20f17b7f19167092b5ecc67a6d095cbf6d1a267f55e788d9b`;
Bhujimol is bounded to 293 units/291 source contracts and scope
`7779fd856d5780aac5311e38480e8fec293dcc8d0ae6a0177a076b6add98e91d`.
Passing inputs are published as an r66 integrity packet containing the exact
completed profile JSONL, exact scoped attestation, handoff report, progress
receipt, and closed manifest. Missing, extra, reordered, duplicate,
cross-profile, immutable-drifted, unexpected-field-injected, incomplete-decision,
bad-timestamp, inconsistent-line-count, wrong-scope, placeholder-qualification,
unused-reviewer, completed/attestation path-chain symlink, existing-output,
post-audit protected-field/input mutation, and precommit-crash cases fail
closed. The tracked r27/r51 metadata roots remain quiescent exact-byte contract
inputs rather than universal no-follow provenance. A valid 46-row Ranjana
packet was also confirmed unable to satisfy the unchanged 339-row merger by
itself.
The legacy sealer prints a digest obtained by reopening committed
`SHA256SUMS`. R69 later identified the short post-commit receipt race and
superseded this command for new seals while preserving it for compatibility.
Any legacy digest must still be retained separately and compared while r66's
strict packet verifier reports no closure errors immediately before use. The
closure is not a signature or malicious-writer authentication. The unchanged
merger revalidates decision semantics but does not verify sealed-packet custody
on the extracted paths.
This milestone adds no real decision: no synthetic completed review packet is
checked in. A seal reports profile decision-set completeness only and keeps
global qualified review, source preflight, decision/candidate admission,
pixels, audited lines, labels, models, inference, training, scoring, quality,
and four-script readiness false. The browser still does not hash the pixels it
displays. The next global gate remains both completed profile assignments plus
a combined v2 attestation and a fresh all-337-file preflight started after all
339 decisions, followed by the unchanged merger/freezer. Verification used
tracked metadata and temporary synthetic decisions only; it performed no
source-pixel read, remote command, network request, download, OCR, training, or
outward action.
## 2026-07-16 r68 — strict two-seal custody assembly
The r67 profile packets can now be joined without manually reopening unverified
member paths or rewriting the combined reviewer attestation. The new
`tools/assemble_newari_qualified_page_review_profile_handoffs.py` requires the
Bhujimol and Ranjana seal roots plus the two `SHA256SUMS` digests retained
separately at seal time. In each audit pass, each packet is opened through one
r66 component-wise no-follow directory descriptor; its manifest, exact four
payload members, and final stable file identities are captured through that
same descriptor. Publication recaptures both inputs before commit. The supplied
digest and every manifest/member hash must agree.
Captured bytes are independently replayed through the exact r67 validation
contract. The assembler pins r67 sealer `298ca19f...`, r66 publisher
`38345286...`, unchanged merger `4e1fd85a...`, freezer `a5b661ae...`, and full
preflight `10a90f57...`; it also rebinds exact r27/r51 metadata. Bhujimol must
cover 293 decisions/291 source contracts at scope `7779fd85...`, Ranjana must
cover 46/46 at `30eeb57b...`, and their identities must reconstruct the exact
339-unit canonical scope. Reports, progress receipts, immutable fields,
decisions, counters, attestations, seal chronology, and every non-admission
claim are recomputed rather than trusted.
The generated combined v2 attestation preserves distinct reviewer assertions.
A reviewer ID shared across profiles is combined only if qualification,
timestamp, and prediction-free assertions are identical; conflicts fail
closed. The atomic output retains the two original manifest/report/progress/
attestation evidence sets and emits exact `bhujimol-review.jsonl`,
`ranjana-review.jsonl`, and `combined-reviewer-attestation.json` inputs for the
unchanged merger. No synthetic completed assembly is checked in.
r68 is a custody bridge, not qualified-review completion. Even a valid 339-row
assembly keeps global review, source preflight and live recheck, decision/page
candidate admission, pixels, audited lines, labels, models, inference,
training, scoring, quality, and four-script readiness false. The next gate
remains one fresh all-337-file receipt started after the final decision and the
unchanged merger/freezer's independent SSH rerun. Verification uses temporary
synthetic decisions and a stubbed remote executor only for compatibility; the
assembler itself performs no source-pixel read, SSH, network request, download,
OCR, training, scoring, or outward action.
## 2026-07-16 r69 — receipt-safe profile seals with legacy compatibility
The original r67 packet publisher is crash-safe, but its CLI reopens the output
`SHA256SUMS` by pathname after commit to print the separately retained digest.
An `after_commit` writer can replace that manifest or the whole output
directory in the gap, causing the printed receipt to bind replacement bytes.
The r68 assembler detects malformed replacement packets, but a semantically
valid reclosed replacement can satisfy its retained-digest input. Because r67
was explicitly designed to create out-of-tree handoffs, repository absence of
a completed seal cannot prove that no legacy packet exists.
R69 therefore preserves the exact r67 sealer (`298ca19f...`) and r68 assembler
(`e2d5a8b0...`) for historical packets and adds
`tools/seal_newari_qualified_page_review_profile_v2.py` (`201acda9...`). The
v2 sealer binds its executing canonical path, the exact r67 semantic validator,
the unchanged freezer, and the r66 publisher. It derives `SHA256SUMS` from the
exact in-memory report/progress/review/attestation members before publication,
returns that digest only after commit, and never reopens the output to produce
the operator receipt. Its packet remains non-admitting and explicitly records
that the receipt is custody evidence rather than a signature or reviewer
authentication.
`tools/assemble_newari_qualified_page_review_profile_handoffs_v2.py`
(`5b9acb01...`) exact-replays legacy/legacy, both mixed-generation pairings,
and v2/v2 seals. It retains the original evidence bytes, derives the same three
unchanged merger inputs, and lists every old packet in
`legacy_receipt_profiles`; only v2/v2 sets
`all_profile_receipts_in_memory_bound=true`. Compatibility never upgrades a
legacy receipt claim, and original reviewed inputs should be resealed when
available. The old tools remain byte-identical.
Adversarial verification uses temporary synthetic decisions only. It covers
post-commit manifest replacement, complete output-directory replacement,
executed sealer/freezer substitution, dependency hash drift, exact successor
report replay, all four generation pairings, atomic outer publication, and
unchanged-merger compatibility with a stubbed remote executor. No real review
decision, source pixel, label, model, OCR, training, scoring, remote command,
network request, download, or outward action is admitted or performed. The
real gate remains 293 Bhujimol plus 46 Ranjana qualified decisions, followed by
one fresh all-337-file preflight and the unchanged merger/freezer.
# Limbu Book OCR Goal Progress
Living checklist for the active goal. Update often as work lands.
**Active goal:** build a usable Limbu book OCR system that accepts scanned or
photographed real Limbu book pages and outputs editable Unicode text with page and line
structure preserved. It must support both Sirijonga Limbu and Devanagari-written Limbu.
The current bottleneck is page layout, reading order, missed-line recovery, and review
burden with the current recognizers held fixed. Evaluation is a guardrail, not the
product: keep it to frozen real-book acceptance checks such as page CER/WER, line
detection precision/recall/F1, reading-order accuracy, blank/extra/missed rates, and
review-packet coverage. Do not expand a general eval harness unless it directly proves
one of those book OCR acceptance checks.
**Historical context:** the earlier recognizer-language program remains useful evidence,
but it is not the active thread goal. Any next work should make the book OCR loop more
true: cleaner source selection, stronger page preparation, better line preservation,
lower manual review burden, safer correction application, or measured improvement over
the current diagnostic baselines.
**2026-06-24 controlled recognizer ablation verdict:** after the targeted+Unicode bulk
run regressed, a five-cell ablation grid was trained on the AWS G6 L4 host and scored
against the current champion's exact frozen Gorkhapatra eval identity. The artifact
root is `outputs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`; the launcher
and scoring script are in `runs/limbu-ablation-grid-realheavy-targeted-v1-2026-06-24/`.
Results on `255` matched lines: champion reference remains codepoint CER
`0.011336032388663968`, grapheme CER `0.024735729386892176`, exact `0.7294117647058823`.
Control v2 replay at LR `1e-6` scored `0.011234817813765181` / `0.02452431289640592`
/ `0.7333333333333333`, a one-edit non-significant improvement and therefore only a
sanity check. v2 plus `256` capped targeted rows scored worse:
`0.01194331983805668` / `0.026004228329809725` / `0.7294117647058823`. v2 replay at
LR `1e-5` tied codepoint CER but worsened grapheme CER:
`0.011336032388663968` / `0.026004228329809725` / `0.7372549019607844`. Both v3
dictionary cells failed catastrophically, around `0.98` codepoint CER and `0.0`
exact-line accuracy, because expanding the dictionary from the v2 checkpoint resizes
recognition-head parameters. Do not promote any ablation checkpoint. Keep the
real-heavy v2 dictionary champion as the current recognizer. Next recognizer work
should not add dictionary characters by warm-starting directly from the v2 head, should
not chase Paddle `norm_edit_dis` alone, and should focus on real/book-page line
harvesting or a separate from-scratch/rehead training plan if full Unicode coverage is
actually required.
Started: 2026-06-16. Owner: Claude session.
---
## LIMBU OPERATIONAL OCR PIPELINE PIVOT (2026-06-21)
**2026-06-23 recognizer corpus correction:** do not describe the current Limbu
training inventory as only the `1,866` Gorkhapatra train rows. That count is only
the train split of
`outputs/limbu-gorkhapatra-pdf-backed-silver-2026-06-23-v3/`, a PDF-backed
silver real-domain pack with `255` held-out eval rows. The larger local Sirijonga
training inventory already includes `18,418` rendered Namdhinggo Limbu prose train
rows in `outputs/limbu-prose-rendered-train-2026-06-16/`, `28,155` augmented
rendered train rows in `data/limbu-aug-train/train.txt`, and the prior
`runs/limbu-sirijonga-ppocrv5-mobile-2026-06-16/` checkpoint trained from the
rendered prose pack. The `1,866`-row Gorkhapatra G6 run was therefore a narrow
real-domain fine-tune, not a full-corpus Limbu recognizer run. The corrected
training policy is: use synthetic/rendered rows for glyph, word, augmentation,
and font coverage; use Gorkhapatra PDF-backed rows for real-domain adaptation and
checkpoint selection; keep synthetic rows out of headline CER claims. The
dedicated builder `tools/build_limbu_combined_recognizer_pack.py` materializes
this as an auditable combined pack, rewrites legacy label paths to repo-root
paths, tags source components, and drops exact normalized train-text overlaps
against the held-out Gorkhapatra eval text before training.
**2026-06-23 combined corpus G6 launch:** the corrected combined training pack is
`outputs/limbu-combined-synth-real-gorkhapatra-v1-2026-06-23/`. It has `48,507`
train rows and `255` eval rows after dropping `13` exact normalized train-text
overlaps with the Gorkhapatra eval text. Train composition is `28,155`
augmented rendered rows, `18,418` rendered Namdhinggo prose rows, `1,853`
Gorkhapatra PDF-backed silver rows, and `81` CDC Sirijonga PDF-backed silver
rows. Eval is only the held-out Gorkhapatra PDF-backed silver real-print split.
Both manifests passed `ocrtech audit-manifest`. The generated recipe is
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-g6-l4-ppocrv5-2026-06-23/`
with pack hash
`1f77f7a0f947fc475afcef5568f08a1f0649a92dc8ee1a7eb0620956b734085b`, base
checkpoint
`runs/limbu-gorkhapatra-pdf-backed-silver-v4warm-20ep-g6-l4-ppocrv5-2026-06-23/checkpoints/best_accuracy.pdparams`,
batch size `64`, learning rate `0.00001`, `15` epochs, warmup `1`, GPU enabled,
and `train_drop_last=false`. The first generated run used Paddle's default
`main_indicator=acc`; that was wrong for the current CER-oriented goal because
it can overwrite a lower-edit-distance checkpoint with a higher exact-line-acc
checkpoint. `src/ocrtech/training.py` and the CLI now expose
`--main-indicator`, and the active corrected recipe is
`runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/`
with `main_indicator=norm_edit_dis`. The AWS G6 instance
`i-0fa19dff7b233a353` was restarted on 2026-06-23, current public IP
`35.175.116.142`, remote workspace `/opt/ocrtech`. An initial exact-acc run was
stopped after proving the selection-rule problem, then a norm-edit run initially
hit OOM because the old parent process was still holding GPU memory. After
clearing the GPU, the active corrected run was launched as remote PID `2404` via
`/opt/ocrtech/runs/limbu-combined-synth-real-gorkhapatra-v1-15ep-normedit-g6-l4-ppocrv5-2026-06-23/train-g6.sh`.
Startup sanity passed: zero missing label images, Paddle `3.3.1` sees one NVIDIA
L4, training dataloader has `758` iters, eval dataloader has `4` iters, and the
checkpoint loaded successfully from the previous Gorkhapatra best model. First
eval at global step `200` saved by norm-edit with `norm_edit_dis=0.8731627801677526`
and `acc=0.1686274443675512`.