# Limbu OCR Space Handoff Status date: 2026-06-30 The OCR-only Hugging Face Space payload is staged at `spaces/limbu-ocr/`. It exposes the current Limbu OCR work as an operational end-to-end v1: cropped-line recognition, Paddle-first full-page OCR, and Paddle-first book/PDF batch OCR for Sirijonga Limbu and Devanagari-written Limbu. The default page/book path does not depend on YOLO. It uses PaddleOCR's full-page text detector, orders detected text lines, routes each crop through the bundled Sirijonga/Devanagari Paddle recognizers, and returns Unicode text with crop/overlay evidence. Deterministic image-line OCR remains as a fallback. YOLO is retained only as a secondary diagnostic probe. ## Deployment Target Target Space: ```text https://huggingface.co/spaces/ampixa/limbu-ocr ``` The folder is configured as a Docker Space through `spaces/limbu-ocr/README.md`: ```yaml sdk: docker app_port: 7860 ``` The Space was uploaded with the Hugging Face token stored in `~/.fmw`. Uploaded commit for the 2026-06-29 Paddle-first book OCR update: ```text https://huggingface.co/spaces/ampixa/limbu-ocr/commit/bda8e8b5ebaf476e7a52cd56996f525fd7687d56 ``` Runtime URL: ```text https://ampixa-limbu-ocr.hf.space/ ``` The Space is currently private (`space_info.private=True`). Unauthenticated requests to the direct runtime domain return HTTP 404. An authenticated direct-domain probe with the token from `~/.fmw` returned HTTP 200 after the 2026-06-29 upload. As of the latest runtime check, Hugging Face CPU quota allows three active CPU Spaces. The intended active set is: - `ampixa/limbu-ocr`: running - `ampixa/limbu-translator`: running - `ampixa/limbu-tts`: running `ampixa/real-nepali-tts`, `ampixa/gurung-tts`, and `voidash/bantawa-tts-demo` are paused to preserve the Limbu OCR/MT/TTS set. Do not pause `ampixa/limbu-translator` or `ampixa/limbu-tts` to free OCR quota unless the user explicitly changes that priority. ## Payload - `spaces/limbu-ocr/app.py`: self-contained Gradio app. It deliberately does not import the repo package, so the Space can run from its own folder. Tabs: `Line OCR`, `Paddle Page OCR`, `Paddle Book OCR`, `Image-Line Page Fallback`, `Image-Line Book Fallback`, `YOLO Probe`, and `Runtime`. - `spaces/limbu-ocr/Dockerfile`: Python 3.11 CPU Docker image. - `spaces/limbu-ocr/requirements.txt`: pinned OCR runtime dependencies: `gradio==5.49.1`, `paddlepaddle==3.3.1`, `paddleocr==3.7.0`, `opencv-python-headless`, `pypdfium2`, and `ultralytics==8.3.0`. - `spaces/limbu-ocr/DEPLOY.md`: exact `uvx`-based Hugging Face upload command. - `spaces/limbu-ocr/models/limbu-sirijonga-real-heavy/`: current strongest Sirijonga line recognizer package. - `spaces/limbu-ocr/models/limbu-devanagari-ppocrv5/`: Devanagari-written Limbu recognizer package. - `spaces/limbu-ocr/models/line-detector/best.pt`: optional YOLO line detector for the probe tab only. ## Artifact Sources Sirijonga recognizer copied from: ```text runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/ ``` Devanagari recognizer copied from: ```text outputs/limbu-router-v4-ppocr-devanagari-sirijonga-confidence-guard-2026-06-22/components/limbu-devanagari-ppocrv5/ ``` Line detector copied from: ```text outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/best.pt ``` ## Verification Completed Syntax check: ```bash python3 -m py_compile spaces/limbu-ocr/app.py ``` Python 3.11 lightweight import and artifact check: ```bash uv run --python 3.11 \ --with gradio==5.49.1 \ --with pillow \ --with 'numpy>=1.26.0,<2.3' \ --with 'opencv-python-headless>=4.10.0.84,<4.13' \ --with 'pypdfium2>=4.30.0,<5' \ python - <<'PY' import importlib.util import sys from pathlib import Path app_path = Path("spaces/limbu-ocr/app.py").resolve() spec = importlib.util.spec_from_file_location("limbu_space_app", app_path) app = importlib.util.module_from_spec(spec) sys.modules[spec.name] = app spec.loader.exec_module(app) print(app.health()) PY ``` Observed result: ```json { "sirijonga_model": true, "devanagari_model": true, "line_detector": true, "paddle_device": "cpu", "artifact_check": "pass" } ``` Deterministic line-segmentation smoke: ```text synthetic_boxes 4 ``` Paddle-first runtime smoke against the actual shipped package versions (`paddlepaddle==3.3.1`, `paddleocr==3.7.0`): ```text PaddleOCR signature includes text_detection_model_dir and text_recognition_model_dir real_page data/limbu-gorkhapatra-language-page-pack-2026-06-16/inputs/gorkhapatra-language-page-0001-01.png paddle_real_detections 306 capped recognize_page_paddle max_lines=5 -> text_lines 4, gallery_items 5 ``` Focused repo tests for the operational book path: ```bash env PYTHONPATH=src uv run --python 3.11 --with pytest --with pillow \ --with 'numpy>=1.26.0,<2.3' \ --with 'opencv-python-headless>=4.10.0.84,<4.13' \ python -m pytest \ tests/test_image_lines.py \ tests/test_parse_limbu_book_pages.py \ tests/test_run_limbu_book_ocr.py -q ``` Observed result: ```text 49 passed, 1 skipped ``` During this verification, `tools/run_limbu_book_ocr.py` and `tools/prepare_limbu_book_captures.py` were hardened so the book runner forwards the new capture-audit fields (`min_prepared_laplacian_variance`, page-detection audit flags, and deskew audit flags) and old programmatic callers do not crash when those optional fields are absent. Package size check: ```bash du -sh spaces/limbu-ocr find spaces/limbu-ocr -type f -size +20M -print ``` Observed size: `21M`; no file over 20 MB. Docker was not built locally because Docker is not installed on this machine. Hugging Face runtime check: ```text stage=RUNNING hardware=cpu-basic sha=bda8e8b5ebaf476e7a52cd56996f525fd7687d56 authenticated https://ampixa-limbu-ocr.hf.space/ -> HTTP 200 unauthenticated https://ampixa-limbu-ocr.hf.space/ -> HTTP 404 because the Space is private deployed HTML contains `Paddle Page OCR`, `Paddle Book OCR`, `Image-Line Page Fallback`, and `/recognize_book_paddle` ``` ## 2026-06-30 Page Eval Status The current Space app now has a reproducible adapter into the existing page-level OCR harness: ```text tools/run_limbu_paddle_space_page_eval.py ``` Diagnostic command run against the Gorkhapatra Sirijonga page eval pack: ```bash env PYTHONPATH=src uv run --python 3.11 \ --with gradio==5.49.1 \ --with pillow \ --with 'numpy>=1.26.0,<2.3' \ --with 'opencv-python-headless>=4.10.0.84,<4.13' \ --with 'pypdfium2>=4.30.0,<5' \ --with paddlepaddle==3.3.1 \ --with paddleocr==3.7.0 \ --with rapidfuzz \ --with regex \ python tools/run_limbu_paddle_space_page_eval.py \ --out outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed \ --recognition-source seed \ --mode Sirijonga \ --seed-recognizer Sirijonga \ --reading-order Auto \ --max-lines 0 \ --force ``` Scored output: ```text outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed/score/ ``` Metrics on the one-page diagnostic pack: ```text scored pages: 1/1 reference lines / predicted lines / matched lines: 303 / 306 / 291 line detection precision / recall / F1: 0.950980 / 0.960396 / 0.955665 matched-line codepoint CER: 0.033035 matched-line grapheme CER: 0.063509 matched-line reading-order pair accuracy: 0.500581 page codepoint CER: 0.824390 page grapheme CER: 0.846778 ``` Order-only rescores of the same 306 Paddle detections did not fix page CER: ```text Rows LTR: page_cp_CER 0.815223, page_graph_CER 0.841809, order_acc 0.614314 Columns LTR: page_cp_CER 0.824390, page_graph_CER 0.846778, order_acc 0.500581 Columns RTL: page_cp_CER 0.796636, page_graph_CER 0.824416, order_acc 0.676146 ``` Read this correctly: the detector and recognizer are promising, but the current page OCR is not good enough as a book/page OCR system because reading order/layout reconstruction dominates the full-page CER. The Gorkhapatra reference remains machine-converted and unreviewed (`claim_evidence_eligible=false`), so this is diagnostic evidence only, not a benchmark claim. ### 2026-06-30 expanded eval hardening The adapter was hardened after the first diagnostic: it now emits one `Document` block per detected OCR line instead of one page-sized block. This is required for reading-order profile or strategy reordering to change `Document.text`. It also accepts explicit scorer metadata (`--script`, `--gt-status`, `--eval-pack-version`, `--checkpoint`, `--config`, `--dictionary`) so Devanagari CDC and Sirijonga Gorkhapatra runs do not silently inherit the wrong model metadata. New helper tools: ```text tools/build_limbu_gorkhapatra_silver_page_eval.py tools/apply_page_reading_order_strategy.py ``` The Gorkhapatra silver page pack was built from the existing PDF-backed page and line text-layer manifests: ```bash env PYTHONPATH=src uv run --python 3.11 --with regex \ python tools/build_limbu_gorkhapatra_silver_page_eval.py --force ``` Output: ```text outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/ manifests/eval.jsonl references/*.ref.json summary.json script-inventory.json ``` Pack contents: 7 full Gorkhapatra page images, 2,129 reference lines, 72 observed codepoints, 1,097 observed grapheme clusters. References are `pdf_text_layer_silver` and `claim_evidence_eligible=false`. The line bboxes are converted from rendered PDF `xyxy` boxes into the eval harness `xywh` format. CDC Devanagari page eval now runs through the same Paddle top-to-bottom path: ```text outputs/limbu-paddle-space-page-eval-2026-06-30/cdc-devanagari-paddle-seed/score/ ``` Metrics: ```text pages 3/3 page codepoint CER: 0.376880 page grapheme CER: 0.479263 line detection F1: 0.906250 matched-line codepoint CER: 0.057724 matched-line reading-order pair accuracy: 0.592308 ``` Expanded Gorkhapatra Sirijonga seven-page eval: ```text outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed/score/ ``` Raw `Auto` order metrics: ```text pages 7/7 page codepoint CER: 0.814444 page grapheme CER: 0.851347 line detection F1: 0.943981 matched-line codepoint CER: 0.027091 matched-line reading-order pair accuracy: 0.720352 ``` Same detections, no OCR rerun, reading-order-only rescoring: ```text Columns RTL: page_cp_CER 0.771116, page_graph_CER 0.803203, order_acc 0.777787 old geometry profile: page_cp_CER 0.736334, page_graph_CER 0.774886, order_acc 0.626211 Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, order_acc 0.993085 ``` Rows LTR is the best simple order on the seven-page silver pack, and the current Space UI defaults were changed accordingly for Paddle Page OCR and Paddle Book OCR: reading order default is now `Rows LTR`, and max lines default is now `0` (no cap). The old default capped dense Gorkhapatra pages at 180 lines even though the pages have roughly 300 detected lines. Do not misread this as solved page OCR. Reconstructing text from only bbox-matched Gorkhapatra lines yields about 2-3.5% codepoint CER per page, which means recognition on the target lines is good. The remaining full-page CER is dominated by page-scope and filtering: Paddle detects top/header/non-target text and the Limbu recognizer turns it into Sirijonga-looking garbage. The next real page-OCR fix is a target-region / language-script filtering policy, not another hard-coded column order. ### 2026-07-01 first non-oracle page filtering baseline `tools/filter_page_ocr_documents.py` and `tools/audit_page_ocr_line_filter.py` now support relative page-y filters: ```text --min-y-ratio --max-y-ratio ``` The Gorkhapatra failure mode was verified from existing seven-page detections: matched target lines start around `y_ratio >= 0.12`, while many unmatched newspaper header/date lines are above that. A conservative filter was applied to the existing `Rows LTR` documents, with no reference lookup: ```bash env PYTHONPATH=src uv run --python 3.11 --with regex \ python tools/filter_page_ocr_documents.py \ --eval-manifest outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/manifests/eval.jsonl \ --documents-dir outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-documents \ --out outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents \ --drop-empty \ --min-y-ratio 0.12 \ --max-y-ratio 0.96 \ --min-confidence 0.70 ``` The filter audit passes with 0 issues. It keeps 2,096 / 2,191 detected lines and removes 95 lines. Removal counts overlap by line and are: ```text low_confidence: 92 y_ratio_below_min: 71 empty_text: 8 y_ratio_above_max: 7 ``` Seven-page Gorkhapatra score progression: ```text raw Auto: page_cp_CER 0.814444, page_graph_CER 0.851347, line_F1 0.943981, order_acc 0.720352 Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, line_F1 0.943981, order_acc 0.993085 Rows LTR + y filter: page_cp_CER 0.425080, page_graph_CER 0.449335, line_F1 0.960868, order_acc 0.993078 Rows LTR + y + conf70: page_cp_CER 0.422770, page_graph_CER 0.445983, line_F1 0.964734, order_acc 0.993078 Rows LTR + y + conf80: page_cp_CER 0.422985, page_graph_CER 0.446007, line_F1 0.964929, order_acc 0.993068 ``` Use `conf70` as the current diagnostic baseline, not `conf80`: it gives the best measured page CER while being less aggressive. This still does not solve full-page OCR. It removes obvious page furniture, but the remaining gap needs a stronger target-region/language filtering policy or a page layout segmenter. ### 2026-07-01 reading-order and line-assembly diagnostic The next suspected bottleneck was tested against the same seven-page Gorkhapatra silver pack: variable-height headline boxes and fragmented line detections. Two changes were added and tested: - `tools/apply_page_reading_order_strategy.py` now supports `Rows LTR Top` and `Rows RTL Top`, which cluster rows by bbox top instead of bbox center. This is intentionally a separate mode, not a silent behavior change. - `tools/assemble_page_ocr_lines.py` now supports `--max-horizontal-overlap` so adjacent fragments with slightly overlapping detector boxes can be merged. The replay audit in `tools/audit_page_ocr_line_assembly.py` was updated for that parameter. Focused tests: ```bash env PYTHONPATH=src uv run --python 3.11 --with pytest \ pytest tests/test_assemble_page_ocr_lines.py \ tests/test_filter_page_ocr_documents.py \ tests/test_apply_page_reading_order_strategy.py -q ``` Result: `17 passed`. Evaluation results: ```text Baseline Rows LTR + y + conf70: score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score page_cp_CER 0.4227699, page_graph_CER 0.4459828, line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9930784 Rows LTR Top + same filter: score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-top-y012-y096-conf070-filtered-score page_cp_CER 0.4289698, page_graph_CER 0.4496008, line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9924359 Rows LTR + same filter + conservative assembly: documents: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score assembly params: y_tolerance=24, max_gap=32, max_horizontal_overlap=24, min_vertical_overlap_ratio=0.35, joiner=" " page_cp_CER 0.4226502, page_graph_CER 0.4457416, line_F1 0.9652050, matched_cp_CER 0.0277639, order_acc 0.9931425 Rows LTR Top + same filter + conservative assembly: score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-top-y012-y096-conf070-gap032-ov024-assembled-score page_cp_CER 0.4288860, page_graph_CER 0.4493596, line_F1 0.9654350, matched_cp_CER 0.0276650, order_acc 0.9924574 ``` Interpretation: simple top-anchored ordering is worse on this pack. Conservative line assembly correctly merges obvious fragments, but the score gain is negligible (`0.4227699 -> 0.4226502` CP CER) and matched-line CER worsens slightly because some merged predictions no longer correspond one-to-one with the PDF text-layer reference lines. Do not spend the next OCR block on another recognizer epoch or more fragment merging. The next real work is a non-oracle layout/target-region calibration stage that can choose the Limbu article region and its reading order before recognition export. The current old geometry-profile artifact is not a solution: it is non-claim-safe and scored much worse (`page_cp_CER 0.736334` on the same seven-page pack). ### 2026-07-03 expanded Gorkhapatra silver8 diagnostic Full provenance and commands are in `docs/LIMBU_GORKHAPATRA_EXPANDED_2026_07_03.md`. New pack: ```text outputs/limbu-gorkhapatra-pdf-backed-silver-2026-07-03-v1/ outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-07-03-v1/ ``` Counts: ```text 9 discovered Limbu dates 8 usable PDF-text-layer pages 2441 line rows 2430 usable silver labels known failure: 2082-11-12 no_namdhinggo_page_found ``` Expanded 8-page scores: ```text Rows LTR raw: page_cp_CER 0.4349285, page_graph_CER 0.4692741, line_F1 0.9418182, matched_cp_CER 0.0261180, order_acc 0.9931821 Rows LTR + y012/y096 + conf70: page_cp_CER 0.4143301, page_graph_CER 0.4380524, line_F1 0.9630089, matched_cp_CER 0.0260386, order_acc 0.9931762 Rows LTR + same filter + conservative assembly: page_cp_CER 0.4142255, page_graph_CER 0.4378626, line_F1 0.9633943, matched_cp_CER 0.0268101, order_acc 0.9932274 ``` Filter replay audit and assembly replay audit both pass with 0 issues. Treat the filtered Rows LTR score as the current comparable baseline. The assembly score is slightly lower but too small to matter and worsens matched-line CER, so do not make assembly the main line of work. ## Operational Contract Use `Line OCR` for already-cropped lines. Use `Paddle Page OCR` for one full page image. Use `Paddle Book OCR` for ordered page images or a PDF. The Paddle Book OCR output zip contains `book.txt`, `book.json`, annotated page overlays, and line crops; these artifacts are the evidence needed for review/correction and later training-pack export. The default recognizer mode should stay `Auto` unless testing one script-specific recognizer. Use the image-line fallback tabs only when comparing or when Paddle detection fails on a page. Do not use the `YOLO Probe` tab as the production OCR path. It exists for comparison and diagnostics only. ## Claim Boundary The line recognizer is the strongest measured component. The page/book path is now a usable draft extractor because it preserves each detected crop beside the recognized text, but it is not yet a claim-safe book OCR benchmark. It still needs real-page validation for line detection, reading order, page CER/WER, and review burden before making best-in-class claims.