Download ocr-tech/docs/LIMBU_OCR_SPACE_HANDOFF.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 19.3 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LIMBU_OCR_SPACE_HANDOFF.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/LIMBU_OCR_SPACE_HANDOFF.md
-
curl -L -o LIMBU_OCR_SPACE_HANDOFF.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LIMBU_OCR_SPACE_HANDOFF.md
Limbu OCR Space Handoff
Status date: 2026-06-30
The OCR-only Hugging Face Space payload is staged at spaces/limbu-ocr/. It exposes
the current Limbu OCR work as an operational end-to-end v1: cropped-line recognition,
Paddle-first full-page OCR, and Paddle-first book/PDF batch OCR for Sirijonga Limbu
and Devanagari-written Limbu. The default page/book path does not depend on YOLO. It
uses PaddleOCR's full-page text detector, orders detected text lines, routes each
crop through the bundled Sirijonga/Devanagari Paddle recognizers, and returns Unicode
text with crop/overlay evidence. Deterministic image-line OCR remains as a fallback.
YOLO is retained only as a secondary diagnostic probe.
Deployment Target
Target Space:
https://huggingface.co/spaces/ampixa/limbu-ocr
The folder is configured as a Docker Space through spaces/limbu-ocr/README.md:
sdk: docker
app_port: 7860
The Space was uploaded with the Hugging Face token stored in ~/.fmw.
Uploaded commit for the 2026-06-29 Paddle-first book OCR update:
https://huggingface.co/spaces/ampixa/limbu-ocr/commit/bda8e8b5ebaf476e7a52cd56996f525fd7687d56
Runtime URL:
https://ampixa-limbu-ocr.hf.space/
The Space is currently private (space_info.private=True). Unauthenticated requests
to the direct runtime domain return HTTP 404. An authenticated direct-domain probe
with the token from ~/.fmw returned HTTP 200 after the 2026-06-29 upload.
As of the latest runtime check, Hugging Face CPU quota allows three active CPU Spaces. The intended active set is:
ampixa/limbu-ocr: runningampixa/limbu-translator: runningampixa/limbu-tts: running
ampixa/real-nepali-tts, ampixa/gurung-tts, and voidash/bantawa-tts-demo are paused to preserve the Limbu OCR/MT/TTS set. Do not pause ampixa/limbu-translator or ampixa/limbu-tts to free OCR quota unless the user explicitly changes that priority.
Payload
spaces/limbu-ocr/app.py: self-contained Gradio app. It deliberately does not import the repo package, so the Space can run from its own folder. Tabs:Line OCR,Paddle Page OCR,Paddle Book OCR,Image-Line Page Fallback,Image-Line Book Fallback,YOLO Probe, andRuntime.spaces/limbu-ocr/Dockerfile: Python 3.11 CPU Docker image.spaces/limbu-ocr/requirements.txt: pinned OCR runtime dependencies:gradio==5.49.1,paddlepaddle==3.3.1,paddleocr==3.7.0,opencv-python-headless,pypdfium2, andultralytics==8.3.0.spaces/limbu-ocr/DEPLOY.md: exactuvx-based Hugging Face upload command.spaces/limbu-ocr/models/limbu-sirijonga-real-heavy/: current strongest Sirijonga line recognizer package.spaces/limbu-ocr/models/limbu-devanagari-ppocrv5/: Devanagari-written Limbu recognizer package.spaces/limbu-ocr/models/line-detector/best.pt: optional YOLO line detector for the probe tab only.
Artifact Sources
Sirijonga recognizer copied from:
runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/
Devanagari recognizer copied from:
outputs/limbu-router-v4-ppocr-devanagari-sirijonga-confidence-guard-2026-06-22/components/limbu-devanagari-ppocrv5/
Line detector copied from:
outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/best.pt
Verification Completed
Syntax check:
python3 -m py_compile spaces/limbu-ocr/app.py
Python 3.11 lightweight import and artifact check:
uv run --python 3.11 \
--with gradio==5.49.1 \
--with pillow \
--with 'numpy>=1.26.0,<2.3' \
--with 'opencv-python-headless>=4.10.0.84,<4.13' \
--with 'pypdfium2>=4.30.0,<5' \
python - <<'PY'
import importlib.util
import sys
from pathlib import Path
app_path = Path("spaces/limbu-ocr/app.py").resolve()
spec = importlib.util.spec_from_file_location("limbu_space_app", app_path)
app = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = app
spec.loader.exec_module(app)
print(app.health())
PY
Observed result:
{
"sirijonga_model": true,
"devanagari_model": true,
"line_detector": true,
"paddle_device": "cpu",
"artifact_check": "pass"
}
Deterministic line-segmentation smoke:
synthetic_boxes 4
Paddle-first runtime smoke against the actual shipped package versions
(paddlepaddle==3.3.1, paddleocr==3.7.0):
PaddleOCR signature includes text_detection_model_dir and text_recognition_model_dir
real_page data/limbu-gorkhapatra-language-page-pack-2026-06-16/inputs/gorkhapatra-language-page-0001-01.png
paddle_real_detections 306
capped recognize_page_paddle max_lines=5 -> text_lines 4, gallery_items 5
Focused repo tests for the operational book path:
env PYTHONPATH=src uv run --python 3.11 --with pytest --with pillow \
--with 'numpy>=1.26.0,<2.3' \
--with 'opencv-python-headless>=4.10.0.84,<4.13' \
python -m pytest \
tests/test_image_lines.py \
tests/test_parse_limbu_book_pages.py \
tests/test_run_limbu_book_ocr.py -q
Observed result:
49 passed, 1 skipped
During this verification, tools/run_limbu_book_ocr.py and
tools/prepare_limbu_book_captures.py were hardened so the book runner forwards the
new capture-audit fields (min_prepared_laplacian_variance, page-detection audit
flags, and deskew audit flags) and old programmatic callers do not crash when those
optional fields are absent.
Package size check:
du -sh spaces/limbu-ocr
find spaces/limbu-ocr -type f -size +20M -print
Observed size: 21M; no file over 20 MB.
Docker was not built locally because Docker is not installed on this machine.
Hugging Face runtime check:
stage=RUNNING
hardware=cpu-basic
sha=bda8e8b5ebaf476e7a52cd56996f525fd7687d56
authenticated https://ampixa-limbu-ocr.hf.space/ -> HTTP 200
unauthenticated https://ampixa-limbu-ocr.hf.space/ -> HTTP 404 because the Space is private
deployed HTML contains `Paddle Page OCR`, `Paddle Book OCR`, `Image-Line Page Fallback`,
and `/recognize_book_paddle`
2026-06-30 Page Eval Status
The current Space app now has a reproducible adapter into the existing page-level OCR harness:
tools/run_limbu_paddle_space_page_eval.py
Diagnostic command run against the Gorkhapatra Sirijonga page eval pack:
env PYTHONPATH=src uv run --python 3.11 \
--with gradio==5.49.1 \
--with pillow \
--with 'numpy>=1.26.0,<2.3' \
--with 'opencv-python-headless>=4.10.0.84,<4.13' \
--with 'pypdfium2>=4.30.0,<5' \
--with paddlepaddle==3.3.1 \
--with paddleocr==3.7.0 \
--with rapidfuzz \
--with regex \
python tools/run_limbu_paddle_space_page_eval.py \
--out outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed \
--recognition-source seed \
--mode Sirijonga \
--seed-recognizer Sirijonga \
--reading-order Auto \
--max-lines 0 \
--force
Scored output:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-paddle-seed/score/
Metrics on the one-page diagnostic pack:
scored pages: 1/1
reference lines / predicted lines / matched lines: 303 / 306 / 291
line detection precision / recall / F1: 0.950980 / 0.960396 / 0.955665
matched-line codepoint CER: 0.033035
matched-line grapheme CER: 0.063509
matched-line reading-order pair accuracy: 0.500581
page codepoint CER: 0.824390
page grapheme CER: 0.846778
Order-only rescores of the same 306 Paddle detections did not fix page CER:
Rows LTR: page_cp_CER 0.815223, page_graph_CER 0.841809, order_acc 0.614314
Columns LTR: page_cp_CER 0.824390, page_graph_CER 0.846778, order_acc 0.500581
Columns RTL: page_cp_CER 0.796636, page_graph_CER 0.824416, order_acc 0.676146
Read this correctly: the detector and recognizer are promising, but the current
page OCR is not good enough as a book/page OCR system because reading order/layout
reconstruction dominates the full-page CER. The Gorkhapatra reference remains
machine-converted and unreviewed (claim_evidence_eligible=false), so this is
diagnostic evidence only, not a benchmark claim.
2026-06-30 expanded eval hardening
The adapter was hardened after the first diagnostic: it now emits one Document
block per detected OCR line instead of one page-sized block. This is required for
reading-order profile or strategy reordering to change Document.text. It also accepts
explicit scorer metadata (--script, --gt-status, --eval-pack-version,
--checkpoint, --config, --dictionary) so Devanagari CDC and Sirijonga
Gorkhapatra runs do not silently inherit the wrong model metadata.
New helper tools:
tools/build_limbu_gorkhapatra_silver_page_eval.py
tools/apply_page_reading_order_strategy.py
The Gorkhapatra silver page pack was built from the existing PDF-backed page and line text-layer manifests:
env PYTHONPATH=src uv run --python 3.11 --with regex \
python tools/build_limbu_gorkhapatra_silver_page_eval.py --force
Output:
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/
manifests/eval.jsonl
references/*.ref.json
summary.json
script-inventory.json
Pack contents: 7 full Gorkhapatra page images, 2,129 reference lines, 72 observed
codepoints, 1,097 observed grapheme clusters. References are pdf_text_layer_silver
and claim_evidence_eligible=false. The line bboxes are converted from rendered
PDF xyxy boxes into the eval harness xywh format.
CDC Devanagari page eval now runs through the same Paddle top-to-bottom path:
outputs/limbu-paddle-space-page-eval-2026-06-30/cdc-devanagari-paddle-seed/score/
Metrics:
pages 3/3
page codepoint CER: 0.376880
page grapheme CER: 0.479263
line detection F1: 0.906250
matched-line codepoint CER: 0.057724
matched-line reading-order pair accuracy: 0.592308
Expanded Gorkhapatra Sirijonga seven-page eval:
outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed/score/
Raw Auto order metrics:
pages 7/7
page codepoint CER: 0.814444
page grapheme CER: 0.851347
line detection F1: 0.943981
matched-line codepoint CER: 0.027091
matched-line reading-order pair accuracy: 0.720352
Same detections, no OCR rerun, reading-order-only rescoring:
Columns RTL: page_cp_CER 0.771116, page_graph_CER 0.803203, order_acc 0.777787
old geometry profile: page_cp_CER 0.736334, page_graph_CER 0.774886, order_acc 0.626211
Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, order_acc 0.993085
Rows LTR is the best simple order on the seven-page silver pack, and the current
Space UI defaults were changed accordingly for Paddle Page OCR and Paddle Book OCR:
reading order default is now Rows LTR, and max lines default is now 0 (no cap).
The old default capped dense Gorkhapatra pages at 180 lines even though the pages have
roughly 300 detected lines.
Do not misread this as solved page OCR. Reconstructing text from only bbox-matched Gorkhapatra lines yields about 2-3.5% codepoint CER per page, which means recognition on the target lines is good. The remaining full-page CER is dominated by page-scope and filtering: Paddle detects top/header/non-target text and the Limbu recognizer turns it into Sirijonga-looking garbage. The next real page-OCR fix is a target-region / language-script filtering policy, not another hard-coded column order.
2026-07-01 first non-oracle page filtering baseline
tools/filter_page_ocr_documents.py and tools/audit_page_ocr_line_filter.py now
support relative page-y filters:
--min-y-ratio
--max-y-ratio
The Gorkhapatra failure mode was verified from existing seven-page detections: matched
target lines start around y_ratio >= 0.12, while many unmatched newspaper header/date
lines are above that. A conservative filter was applied to the existing Rows LTR
documents, with no reference lookup:
env PYTHONPATH=src uv run --python 3.11 --with regex \
python tools/filter_page_ocr_documents.py \
--eval-manifest outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-06-30/manifests/eval.jsonl \
--documents-dir outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-documents \
--out outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-documents \
--drop-empty \
--min-y-ratio 0.12 \
--max-y-ratio 0.96 \
--min-confidence 0.70
The filter audit passes with 0 issues. It keeps 2,096 / 2,191 detected lines and removes 95 lines. Removal counts overlap by line and are:
low_confidence: 92
y_ratio_below_min: 71
empty_text: 8
y_ratio_above_max: 7
Seven-page Gorkhapatra score progression:
raw Auto: page_cp_CER 0.814444, page_graph_CER 0.851347, line_F1 0.943981, order_acc 0.720352
Rows LTR: page_cp_CER 0.441968, page_graph_CER 0.475313, line_F1 0.943981, order_acc 0.993085
Rows LTR + y filter: page_cp_CER 0.425080, page_graph_CER 0.449335, line_F1 0.960868, order_acc 0.993078
Rows LTR + y + conf70: page_cp_CER 0.422770, page_graph_CER 0.445983, line_F1 0.964734, order_acc 0.993078
Rows LTR + y + conf80: page_cp_CER 0.422985, page_graph_CER 0.446007, line_F1 0.964929, order_acc 0.993068
Use conf70 as the current diagnostic baseline, not conf80: it gives the best
measured page CER while being less aggressive. This still does not solve full-page
OCR. It removes obvious page furniture, but the remaining gap needs a stronger
target-region/language filtering policy or a page layout segmenter.
2026-07-01 reading-order and line-assembly diagnostic
The next suspected bottleneck was tested against the same seven-page Gorkhapatra silver pack: variable-height headline boxes and fragmented line detections. Two changes were added and tested:
tools/apply_page_reading_order_strategy.pynow supportsRows LTR TopandRows RTL Top, which cluster rows by bbox top instead of bbox center. This is intentionally a separate mode, not a silent behavior change.tools/assemble_page_ocr_lines.pynow supports--max-horizontal-overlapso adjacent fragments with slightly overlapping detector boxes can be merged. The replay audit intools/audit_page_ocr_line_assembly.pywas updated for that parameter.
Focused tests:
env PYTHONPATH=src uv run --python 3.11 --with pytest \
pytest tests/test_assemble_page_ocr_lines.py \
tests/test_filter_page_ocr_documents.py \
tests/test_apply_page_reading_order_strategy.py -q
Result: 17 passed.
Evaluation results:
Baseline Rows LTR + y + conf70:
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-filtered-score
page_cp_CER 0.4227699, page_graph_CER 0.4459828,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9930784
Rows LTR Top + same filter:
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-top-y012-y096-conf070-filtered-score
page_cp_CER 0.4289698, page_graph_CER 0.4496008,
line_F1 0.9647337, matched_cp_CER 0.0270005, order_acc 0.9924359
Rows LTR + same filter + conservative assembly:
documents: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-documents
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-y012-y096-conf070-gap032-ov024-assembled-score
assembly params: y_tolerance=24, max_gap=32, max_horizontal_overlap=24,
min_vertical_overlap_ratio=0.35, joiner=" "
page_cp_CER 0.4226502, page_graph_CER 0.4457416,
line_F1 0.9652050, matched_cp_CER 0.0277639, order_acc 0.9931425
Rows LTR Top + same filter + conservative assembly:
score: outputs/limbu-paddle-space-page-eval-2026-06-30/gorkhapatra-sirijonga-silver7-paddle-seed-rows-ltr-top-y012-y096-conf070-gap032-ov024-assembled-score
page_cp_CER 0.4288860, page_graph_CER 0.4493596,
line_F1 0.9654350, matched_cp_CER 0.0276650, order_acc 0.9924574
Interpretation: simple top-anchored ordering is worse on this pack. Conservative
line assembly correctly merges obvious fragments, but the score gain is negligible
(0.4227699 -> 0.4226502 CP CER) and matched-line CER worsens slightly because some
merged predictions no longer correspond one-to-one with the PDF text-layer reference
lines. Do not spend the next OCR block on another recognizer epoch or more fragment
merging. The next real work is a non-oracle layout/target-region calibration stage
that can choose the Limbu article region and its reading order before recognition
export. The current old geometry-profile artifact is not a solution: it is
non-claim-safe and scored much worse (page_cp_CER 0.736334 on the same seven-page
pack).
2026-07-03 expanded Gorkhapatra silver8 diagnostic
Full provenance and commands are in
docs/LIMBU_GORKHAPATRA_EXPANDED_2026_07_03.md.
New pack:
outputs/limbu-gorkhapatra-pdf-backed-silver-2026-07-03-v1/
outputs/limbu-page-eval-drafts/gorkhapatra-limbu-sirijonga-silver-pages-2026-07-03-v1/
Counts:
9 discovered Limbu dates
8 usable PDF-text-layer pages
2441 line rows
2430 usable silver labels
known failure: 2082-11-12 no_namdhinggo_page_found
Expanded 8-page scores:
Rows LTR raw:
page_cp_CER 0.4349285, page_graph_CER 0.4692741,
line_F1 0.9418182, matched_cp_CER 0.0261180, order_acc 0.9931821
Rows LTR + y012/y096 + conf70:
page_cp_CER 0.4143301, page_graph_CER 0.4380524,
line_F1 0.9630089, matched_cp_CER 0.0260386, order_acc 0.9931762
Rows LTR + same filter + conservative assembly:
page_cp_CER 0.4142255, page_graph_CER 0.4378626,
line_F1 0.9633943, matched_cp_CER 0.0268101, order_acc 0.9932274
Filter replay audit and assembly replay audit both pass with 0 issues. Treat the filtered Rows LTR score as the current comparable baseline. The assembly score is slightly lower but too small to matter and worsens matched-line CER, so do not make assembly the main line of work.
Operational Contract
Use Line OCR for already-cropped lines. Use Paddle Page OCR for one full page
image. Use Paddle Book OCR for ordered page
images or a PDF. The Paddle Book OCR output zip contains book.txt, book.json,
annotated page overlays, and line crops; these artifacts are the evidence needed for
review/correction and later training-pack export. The default recognizer mode should
stay Auto unless testing one script-specific recognizer. Use the image-line fallback
tabs only when comparing or when Paddle detection fails on a page.
Do not use the YOLO Probe tab as the production OCR path. It exists for comparison
and diagnostics only.
Claim Boundary
The line recognizer is the strongest measured component. The page/book path is now a usable draft extractor because it preserves each detected crop beside the recognized text, but it is not yet a claim-safe book OCR benchmark. It still needs real-page validation for line detection, reading order, page CER/WER, and review burden before making best-in-class claims.