Download ocr-tech/README.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 8.94 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/README.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/README.md
-
curl -L -o README.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/README.md
ocr-tech
OCR for the languages of Nepal. The goal is a recognizer for every script in the Nepal language pack. Limbu (Sirijonga script) comes first, Newar (Nepal Bhasa) second, alongside Nepali in Devanagari. The repository holds the recognizers, the tools that build their training and evaluation data, the demo app, and the evidence behind every reported number.
Research state, decisions and open problems: RESEARCH.md. Read it before
changing anything.
Models
| Model | Reads | Status | Held-out result | Download |
|---|---|---|---|---|
| limbu-sirijonga-ft-v2 | Printed Limbu in Sirijonga (U+1900-U+194F) | Serves the Sirijonga route of the demo | 902 lines from held-out publication dates: 0.47% codepoint CER, 85.9% exact lines (previous model 1.38% / 63.8%); 3 held-out pages end to end: 2.58% page CER | release limbu-sirijonga-ft-v2 |
| devanagari-ft-v3 | Printed Nepali/Devanagari plus English letters and digits | Serves the Devanagari route of the demo since 2026-09-24 | Held-out Gorkhapatra editions: 0.57% CER on chunks and 2.57% on full lines (official PaddleOCR model: 5.13% and 7.69%); English 2.84% (official 10.62%) | release devanagari-ft-v3 |
The two models are internal releases: both are trained partly on Gorkhapatra material, which
is cleared for internal training only (docs/DATA_LICENSING.md).
How to read the numbers: CER is micro-averaged codepoint character error rate from
tools/eval_metrics.py (grapheme CER and exact-line accuracy are
reported alongside). Real-print references are PDF text-layer "silver" labels, converted
mechanically and not reviewed by a human, so they are engineering evidence, not claim-grade
benchmarks. PaddleOCR's norm_edit_dis is never reported as CER.
Other scripts:
- Newar in Devanagari: devanagari-ft-v3 scores 3.54% CER on the 1,139-line Newar test
split (official model 9.11%). That pack is
NOT_ADMITTED(leakage and provenance problems), so this is a diagnostic only. - Newa (Prachalit) script: the v1 export is a known failed fit (a 40-timestep CTC cap against longer labels); a retrain is planned.
- Tirhuta, Ol Chiki, Sunuwar, Kirat Rai, Lepcha, Gurung Khema: data built or staged;
training blocked on rights review or not yet launched. See
RESEARCH.md.
Demo
https://ampixa.thapa-ashish.com.np runs both recognizers on k2 behind a FastAPI worker
(POST /api/ocr/recognize with form fields file, mode = line|page, script =
auto|sirijonga|devanagari). The worker lives on k2 (~/ampixa-demo), not in this repo.
spaces/limbu-ocr/ is the Hugging Face Space app with the same routes
(model binaries are not committed; take them from the releases).
Using a released model
Both are line recognizers: give them one cropped text line. For pages, run a text detector and crop each line along its own angle.
# Limbu / Sirijonga (paddleocr 3.7.0, paddlex 3.7.1, paddlepaddle 3.3.1)
from paddleocr import TextRecognition
rec = TextRecognition(model_name="PP-OCRv5_mobile_rec",
model_dir="limbu-sirijonga-ft-v2/artifacts", device="cpu")
# Devanagari (paddlex 3.7.1 + onnxruntime 1.30.0)
from paddlex import create_model
rec = create_model(model_name="devanagari_PP-OCRv5_mobile_rec",
model_dir="devanagari-ft-v3/artifacts", engine="onnxruntime", device="cpu")
for res in rec.predict("line.png", batch_size=1):
print(res["rec_text"], res["rec_score"])
These exact calls were run on the unpacked release archives before publishing.
Known limitations
- Page cropping in the demo worker. It crops each detected line with an upright box.
On a scan tilted about 2°, that box takes in the neighbouring lines: 21.4% CER against 2.9%
with rotated crops on 43 hand-labelled lines from two government letters
(
tools/deva_rec_torch/page_probe/). Not fixed yet. - Handwriting. devanagari-ft-v3 is worse than the official model on handwritten words
(68.9% vs 40.8% CER on Nepali words). A handwriting fine-tune that may not regress print
is staged (
train.py --guard-dev). - Stamps, signatures, handwritten fill-ins. PP-DocLayoutV3 finds QR codes and emblems
but labelled neither of two Nepali stamps as
sealand has no signature or handwriting class. The recognizer turns those regions into junk text. - Reading order. Newspaper columns interleave under the demo's row-major ordering.
- Resolution. Page OCR at 144 dpi garbles small text; 288 dpi reads cleanly.
- No real citizenship certificate has been tested yet.
Repository layout
| Path | What |
|---|---|
src/ocrtech/ |
The ocrtech package and CLI: data prep, legacy-font converters (Limbu, Sunuwar, Kirat Rai, Lepcha, Gurung Khema), language registry, eval packs, benchmarks, claim gates |
tools/ |
Stand-alone builders and auditors (Gorkhapatra harvest and alignment, silver packs, synthesis, scoring) |
tools/deva_rec_torch/ |
Devanagari fine-tune pipeline: PyTorch/MPS training, ONNX export, served-path evaluation, recipes, results. Runbook |
spaces/limbu-ocr/ |
Hugging Face Space app (line, page and book OCR) |
languages/nepal-v1.json |
Nepal language target registry |
configs/ |
Pipeline, eval, admission and review configs |
data/dictionaries/, data/mappings/ |
Recognizer dictionaries and legacy-font maps (other data/ is not committed) |
paper/, paper-buildkit/ |
LimbuOCR paper draft, evidence, claim verification (paper-buildkit/verify_claims.py) |
docs/ |
Runbooks, inventories, licensing ledger, handoffs |
tests/ |
Test suite |
outputs/ and runs/ are ignored (a few records are force-added). The originals were lost
in the 2026-07-04 machine failure; backups are on Hugging Face ampixa/ocr-tech-artifacts
(private). See docs/RECOVERY_LEDGER.md.
Install and test
uv venv --python 3.11 && . .venv/bin/activate
uv pip install -e '.[test]' # other extras: eval, datasets, engines, surya, hf-recognizer, vlm
python -m pytest
On a clean clone the suite passes. About twenty tests skip because they audit
machine-local inputs that are not in git (ignored data/, outputs/, runs/, the sibling
nepal-ttf2utf checkout); each skip names the missing paths (tests/local_artifacts.py).
Four tests are marked xfail(strict=True) for a known bug in the Limbu book review wrappers
(they do not pass the original packet directory to the unresolved-packet applier).
The ocrtech command reference, engines, data adapters, baselines and claim-validation
workflow are in docs/CLI.md. Paddle work uses Python 3.11; silver-label tools
also need pymupdf and npttf2utf.
Compute runs on the remote hosts (k2 Mac Studio for MPS training and the demo, cdjk for data
builds, EC2 for CUDA), not on the orchestration laptop. See
docs/REMOTE_TRAINING.md.
Documentation map
| Topic | Document |
|---|---|
| Current state, results, mistakes to avoid | RESEARCH.md |
| Limbu pipeline and page harness | docs/LIMBU_FIRST_OCR_PIPELINE.md, docs/LIMBU_PAGE_OCR_HARNESS.md |
| Devanagari data sources and fine-tune | docs/DEVANAGARI_RESOURCES_INVENTORY.md, tools/deva_rec_torch/README.md |
| Data rights, per source | docs/DATA_LICENSING.md, docs/ATTRIBUTIONS.md |
| Model cards | docs/MODELS.md |
| Evaluation packs and claim gates | docs/EVAL_HARNESS_RUNBOOK.md, docs/EVAL_PACK.md, docs/PREFLIGHT.md, docs/VALIDATION.md |
| Paper | paper-buildkit/README.md, paper/AUTHOR_FACT_SHEET.md |
| June 2026 Tesseract/Surya/TrOCR baselines (superseded) | docs/HISTORY_2026-06_BASELINES.md |
| Recovery after the July machine failure | docs/RECOVERY_LEDGER.md |
References
- PaddleOCR (Apache-2.0), base of both recognizers
devanagari_PP-OCRv5_mobile_recand thetransformersPyTorch port used for fine-tuning- NVIDIA OCR-Synthetic-Multilingual-v1 (CC BY 4.0), English replay data