prosperious's picture
Add 1000 files (OCR_INDRA/.gitignore ...)
f9bc0c1 verified
|
Raw History Blame Contribute Delete
8.94 kB
# ocr-tech
OCR for the languages of Nepal. The goal is a recognizer for every script in the Nepal
language pack. **Limbu (Sirijonga script) comes first, Newar (Nepal Bhasa) second**, alongside
Nepali in Devanagari. The repository holds the recognizers, the tools that build their
training and evaluation data, the demo app, and the evidence behind every reported number.
Research state, decisions and open problems: [`RESEARCH.md`](RESEARCH.md). Read it before
changing anything.
## Models
| Model | Reads | Status | Held-out result | Download |
|---|---|---|---|---|
| **limbu-sirijonga-ft-v2** | Printed Limbu in Sirijonga (U+1900-U+194F) | Serves the Sirijonga route of the demo | 902 lines from held-out publication dates: **0.47% codepoint CER**, 85.9% exact lines (previous model 1.38% / 63.8%); 3 held-out pages end to end: 2.58% page CER | [release `limbu-sirijonga-ft-v2`](https://github.com/Ampixa/ocr-tech/releases/tag/limbu-sirijonga-ft-v2) |
| **devanagari-ft-v3** | Printed Nepali/Devanagari plus English letters and digits | Serves the Devanagari route of the demo since 2026-09-24 | Held-out Gorkhapatra editions: **0.57% CER** on chunks and 2.57% on full lines (official PaddleOCR model: 5.13% and 7.69%); English 2.84% (official 10.62%) | [release `devanagari-ft-v3`](https://github.com/Ampixa/ocr-tech/releases/tag/devanagari-ft-v3) |
The two models are internal releases: both are trained partly on Gorkhapatra material, which
is cleared for internal training only ([`docs/DATA_LICENSING.md`](docs/DATA_LICENSING.md)).
How to read the numbers: CER is micro-averaged codepoint character error rate from
[`tools/eval_metrics.py`](tools/eval_metrics.py) (grapheme CER and exact-line accuracy are
reported alongside). Real-print references are PDF text-layer "silver" labels, converted
mechanically and not reviewed by a human, so they are engineering evidence, not claim-grade
benchmarks. PaddleOCR's `norm_edit_dis` is never reported as CER.
Other scripts:
- **Newar in Devanagari**: devanagari-ft-v3 scores 3.54% CER on the 1,139-line Newar test
split (official model 9.11%). That pack is `NOT_ADMITTED` (leakage and provenance
problems), so this is a diagnostic only.
- **Newa (Prachalit) script**: the v1 export is a known failed fit (a 40-timestep CTC cap
against longer labels); a retrain is planned.
- **Tirhuta, Ol Chiki, Sunuwar, Kirat Rai, Lepcha, Gurung Khema**: data built or staged;
training blocked on rights review or not yet launched. See `RESEARCH.md`.
## Demo
`https://ampixa.thapa-ashish.com.np` runs both recognizers on k2 behind a FastAPI worker
(`POST /api/ocr/recognize` with form fields `file`, `mode` = `line`|`page`, `script` =
`auto`|`sirijonga`|`devanagari`). The worker lives on k2 (`~/ampixa-demo`), not in this repo.
[`spaces/limbu-ocr/`](spaces/limbu-ocr/) is the Hugging Face Space app with the same routes
(model binaries are not committed; take them from the releases).
## Using a released model
Both are line recognizers: give them one cropped text line. For pages, run a text detector
and crop each line along its own angle.
```python
# Limbu / Sirijonga (paddleocr 3.7.0, paddlex 3.7.1, paddlepaddle 3.3.1)
from paddleocr import TextRecognition
rec = TextRecognition(model_name="PP-OCRv5_mobile_rec",
model_dir="limbu-sirijonga-ft-v2/artifacts", device="cpu")
# Devanagari (paddlex 3.7.1 + onnxruntime 1.30.0)
from paddlex import create_model
rec = create_model(model_name="devanagari_PP-OCRv5_mobile_rec",
model_dir="devanagari-ft-v3/artifacts", engine="onnxruntime", device="cpu")
for res in rec.predict("line.png", batch_size=1):
print(res["rec_text"], res["rec_score"])
```
These exact calls were run on the unpacked release archives before publishing.
## Known limitations
- **Page cropping in the demo worker.** It crops each detected line with an upright box.
On a scan tilted about 2°, that box takes in the neighbouring lines: 21.4% CER against 2.9%
with rotated crops on 43 hand-labelled lines from two government letters
([`tools/deva_rec_torch/page_probe/`](tools/deva_rec_torch/page_probe/)). Not fixed yet.
- **Handwriting.** devanagari-ft-v3 is worse than the official model on handwritten words
(68.9% vs 40.8% CER on Nepali words). A handwriting fine-tune that may not regress print
is staged (`train.py --guard-dev`).
- **Stamps, signatures, handwritten fill-ins.** PP-DocLayoutV3 finds QR codes and emblems
but labelled neither of two Nepali stamps as `seal` and has no signature or handwriting
class. The recognizer turns those regions into junk text.
- **Reading order.** Newspaper columns interleave under the demo's row-major ordering.
- **Resolution.** Page OCR at 144 dpi garbles small text; 288 dpi reads cleanly.
- No real citizenship certificate has been tested yet.
## Repository layout
| Path | What |
|---|---|
| `src/ocrtech/` | The `ocrtech` package and CLI: data prep, legacy-font converters (Limbu, Sunuwar, Kirat Rai, Lepcha, Gurung Khema), language registry, eval packs, benchmarks, claim gates |
| `tools/` | Stand-alone builders and auditors (Gorkhapatra harvest and alignment, silver packs, synthesis, scoring) |
| `tools/deva_rec_torch/` | Devanagari fine-tune pipeline: PyTorch/MPS training, ONNX export, served-path evaluation, recipes, results. [Runbook](tools/deva_rec_torch/README.md) |
| `spaces/limbu-ocr/` | Hugging Face Space app (line, page and book OCR) |
| `languages/nepal-v1.json` | Nepal language target registry |
| `configs/` | Pipeline, eval, admission and review configs |
| `data/dictionaries/`, `data/mappings/` | Recognizer dictionaries and legacy-font maps (other `data/` is not committed) |
| `paper/`, `paper-buildkit/` | LimbuOCR paper draft, evidence, claim verification (`paper-buildkit/verify_claims.py`) |
| `docs/` | Runbooks, inventories, licensing ledger, handoffs |
| `tests/` | Test suite |
`outputs/` and `runs/` are ignored (a few records are force-added). The originals were lost
in the 2026-07-04 machine failure; backups are on Hugging Face `ampixa/ocr-tech-artifacts`
(private). See [`docs/RECOVERY_LEDGER.md`](docs/RECOVERY_LEDGER.md).
## Install and test
```bash
uv venv --python 3.11 && . .venv/bin/activate
uv pip install -e '.[test]' # other extras: eval, datasets, engines, surya, hf-recognizer, vlm
python -m pytest
```
On a clean clone the suite passes. About twenty tests skip because they audit
machine-local inputs that are not in git (ignored `data/`, `outputs/`, `runs/`, the sibling
`nepal-ttf2utf` checkout); each skip names the missing paths (`tests/local_artifacts.py`).
Four tests are marked `xfail(strict=True)` for a known bug in the Limbu book review wrappers
(they do not pass the original packet directory to the unresolved-packet applier).
The `ocrtech` command reference, engines, data adapters, baselines and claim-validation
workflow are in [`docs/CLI.md`](docs/CLI.md). Paddle work uses Python 3.11; silver-label tools
also need `pymupdf` and `npttf2utf`.
Compute runs on the remote hosts (k2 Mac Studio for MPS training and the demo, cdjk for data
builds, EC2 for CUDA), not on the orchestration laptop. See
[`docs/REMOTE_TRAINING.md`](docs/REMOTE_TRAINING.md).
## Documentation map
| Topic | Document |
|---|---|
| Current state, results, mistakes to avoid | [`RESEARCH.md`](RESEARCH.md) |
| Limbu pipeline and page harness | [`docs/LIMBU_FIRST_OCR_PIPELINE.md`](docs/LIMBU_FIRST_OCR_PIPELINE.md), [`docs/LIMBU_PAGE_OCR_HARNESS.md`](docs/LIMBU_PAGE_OCR_HARNESS.md) |
| Devanagari data sources and fine-tune | [`docs/DEVANAGARI_RESOURCES_INVENTORY.md`](docs/DEVANAGARI_RESOURCES_INVENTORY.md), [`tools/deva_rec_torch/README.md`](tools/deva_rec_torch/README.md) |
| Data rights, per source | [`docs/DATA_LICENSING.md`](docs/DATA_LICENSING.md), [`docs/ATTRIBUTIONS.md`](docs/ATTRIBUTIONS.md) |
| Model cards | [`docs/MODELS.md`](docs/MODELS.md) |
| Evaluation packs and claim gates | [`docs/EVAL_HARNESS_RUNBOOK.md`](docs/EVAL_HARNESS_RUNBOOK.md), [`docs/EVAL_PACK.md`](docs/EVAL_PACK.md), [`docs/PREFLIGHT.md`](docs/PREFLIGHT.md), [`docs/VALIDATION.md`](docs/VALIDATION.md) |
| Paper | [`paper-buildkit/README.md`](paper-buildkit/README.md), [`paper/AUTHOR_FACT_SHEET.md`](paper/AUTHOR_FACT_SHEET.md) |
| June 2026 Tesseract/Surya/TrOCR baselines (superseded) | [`docs/HISTORY_2026-06_BASELINES.md`](docs/HISTORY_2026-06_BASELINES.md) |
| Recovery after the July machine failure | [`docs/RECOVERY_LEDGER.md`](docs/RECOVERY_LEDGER.md) |
## References
- [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) (Apache-2.0), base of both recognizers
- [`devanagari_PP-OCRv5_mobile_rec`](https://huggingface.co/PaddlePaddle/devanagari_PP-OCRv5_mobile_rec_safetensors) and the `transformers` PyTorch port used for fine-tuning
- [NVIDIA OCR-Synthetic-Multilingual-v1](https://huggingface.co/datasets/nvidia/OCR-Synthetic-Multilingual-v1) (CC BY 4.0), English replay data