|
Download docs/NEPAL_MULTILINGUAL_OCR_GOAL.md from prosperious/ocr-tech-k2: direct link, hf CLI and curl.
- Browser
- Download file 13.8 kB
-
https://huggingface.co/prosperious/ocr-tech-k2/resolve/main/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
- Command line
-
hf download hf://prosperious/ocr-tech-k2/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
-
curl -L -o NEPAL_MULTILINGUAL_OCR_GOAL.md https://huggingface.co/prosperious/ocr-tech-k2/resolve/main/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
13.8 kB
| # Nepal Multilingual Document OCR Goal | |
| This is the research target after the steering decision on 2026-06-16. The | |
| project is now a PaddleOCR-centered multilingual document OCR system for | |
| languages used in Nepal, not a narrow Nepali/English OCR experiment. | |
| ## Goal | |
| Build and validate a PaddleOCR-based document OCR engine for at least ten | |
| languages spoken in Nepal, with Limbu first, that supports: | |
| - page/PDF/image parsing | |
| - text OCR | |
| - reading order | |
| - table reconstruction | |
| - figure and caption extraction | |
| - Markdown export | |
| - structured JSON export | |
| - selective PaddleOCR-VL/VLM fallback for hard pages | |
| The claim is valid only if the engine beats strong baselines on predefined real | |
| and held-out evaluations. Synthetic-only results are not enough. | |
| ## Research Claim Shape | |
| The paper should not claim "best OCR engine overall." The defensible claim is: | |
| > A script-aware PaddleOCR adaptation framework can substantially improve | |
| > document OCR and structured extraction for low-resource Nepal languages, | |
| > including scripts not covered well by stock OCR systems, while preserving | |
| > Nepali/English performance and using selective VLM fallback only when the | |
| > local pipeline is uncertain. | |
| The core contribution is the system and validation recipe: | |
| 1. Language/script registry for Nepal document OCR. | |
| 2. Gorkhapatra/Naya Nepal real-publication validation pack. | |
| 3. Unicode/font/renderability audit before synthesis. | |
| 4. Real-text synthetic data generation for lines, tables, captions, and pages. | |
| 5. PaddleOCR recognizer fine-tuning by script family. | |
| 6. PP-Structure-style document extraction. | |
| 7. Claim review that rejects weak SOTA statements. | |
| ## Primary Engine Direction | |
| Use PaddleOCR as the engine family: | |
| - **PP-OCRv6** for the modern small/local OCR baseline and fine-tuning target. | |
| - **PP-StructureV3** for layout, reading order, tables, and document structure. | |
| - **PaddleOCR-VL-1.6** as fallback/teacher for charts, figures, hard layouts, | |
| seals, and low-confidence pages. | |
| Do not route everything to a VLM and call that the engine. The local PaddleOCR | |
| path must be the default path; fallback usage must be measured and reported. | |
| ## First Ten Language Targets | |
| The first target set should be driven by actual Gorkhapatra/Naya Nepal | |
| availability plus local corpus availability. OCR support is tracked by script, | |
| Unicode, fonts, and evaluation data, not by language name alone. | |
| | Priority | Language | First script target | Why first | Notes | | |
| | --- | --- | --- | --- | --- | | |
| | 1 | Limbu/Yakthung | Limbu/Sirijonga, U+1900..U+194F | Has own Unicode block and local Limbu resources | First non-Devanagari research target | | |
| | 2 | Nepali | Devanagari | Core language and baseline | Must not regress | | |
| | 3 | English | Latin | Mixed documents and forms | Must not regress | | |
| | 4 | Tamang | Devanagari first; Tamyig audit later | Local NepTam corpus and dictionary | Start with Devanagari Tamang | | |
| | 5 | Magar | Devanagari first | Gorkhapatra item confirmed; local dictionary | Own/community script requires separate audit | | |
| | 6 | Wambule | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate | | |
| | 7 | Bantawa/Rai | Devanagari or Kirat Rai depending source | Gorkhapatra item confirmed; Rai resources on remote/local projects | Kirat Rai Unicode support is newer; font audit required | | |
| | 8 | Kulung | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate | | |
| | 9 | Yakkha | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate | | |
| | 10 | Tharu/Rana Tharu | Devanagari first | Gorkhapatra item confirmed | Multiple varieties; keep labels explicit | | |
| Additional Gorkhapatra-visible candidates include Sanskrit, Magahi, Bajjika, | |
| Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami, Sherpa, Achhami, Bote, | |
| Rajbanshi, Hayu, Santhali, Dhimal, Bhote, Uraon, Kumal, Gurung, Khash Jumli, | |
| and Meche. These should enter the registry after script/corpus/font checks. | |
| ## Source Findings So Far | |
| ### Gorkhapatra Naya Nepal | |
| The category page is useful for language-specific validation: | |
| - `https://gorkhapatraonline.com/categories/new-nepal` | |
| - `https://gorkhapatraonline.com/categories/new-nepal?page=2` | |
| Confirmed visible language-specific entries include Magar, Wambule, Rana Tharu, | |
| Magahi, Bajjika, Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami, | |
| Sherpa, Achhami, Bote, Rajbanshi, Hayu, Santhali, Kulung, Yakkha, Meche, Khash | |
| Jumli, Dhimal, Tharu, Bhote, Uraon, Bantawa, Kumal, Gurung, and others. | |
| The blog/category pages expose image URLs and article URLs. They are useful for | |
| real-world OCR samples, but they should not be trusted as clean text references | |
| without manual or tool-assisted verification. | |
| ### Gorkhapatra Epaper | |
| The epaper list endpoint exposes viewer links: | |
| ```text | |
| https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra | |
| ``` | |
| The visible links look like: | |
| ```text | |
| https://epaper.gorkhapatraonline.com/pdf?file=/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf | |
| ``` | |
| That `pdf?...` URL returns a PDF.js viewer shell. The real PDF is downloaded by | |
| fetching the file parameter directly: | |
| ```text | |
| https://epaper.gorkhapatraonline.com/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf | |
| ``` | |
| On `k2`, this was verified as a real PDF: | |
| ```text | |
| /Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf | |
| PDF document, version 1.6 | |
| size: 48M | |
| ``` | |
| The `k2` audit produced: | |
| ```text | |
| /Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/audit.txt | |
| /Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/new-nepal-language-items.txt | |
| /Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/epaper-assets.txt | |
| /Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf | |
| ``` | |
| `k2` does not currently expose `pdfinfo` or `pdftotext`. `pymupdf` was installed | |
| into `/Users/k2/ocr-tech/.venv` on 2026-06-16 as the immediate PDF audit tool. | |
| The sample epaper PDF opens successfully with PyMuPDF: | |
| ```text | |
| page_count: 26 | |
| total extracted text chars: 309924 | |
| preview image: outputs/gorkhapatra-source-audit/sample-page-1-preview.jpg | |
| audit: outputs/gorkhapatra-source-audit/sample-pymupdf-audit.json | |
| text: outputs/gorkhapatra-source-audit/sample-pymupdf-text.txt | |
| ``` | |
| The extracted text layer is not reliable reference text. The first-page preview | |
| contains mojibake/transliterated-looking output for Nepali content rather than | |
| clean Unicode Devanagari. Therefore the epaper PDFs are useful as real page | |
| images/layout sources, but their embedded text layer must be audited per file | |
| and must not be treated as ground truth without verification. | |
| ### Local And Remote Corpus Hints | |
| Local machine: | |
| - `~/github/llm/ltk/tamang`: NepTam train/test, NepTam80K, dictionary, TTS | |
| lexicon. | |
| - `~/github/llm/ltk/limbu`: Limbu dictionary, Pangloss IGT, Limbu G2P. | |
| - `~/github/llm/ltk/magar`: Western Magar dictionary and resource report. | |
| Remote `cdjk@100.110.148.43`: | |
| - `/Volumes/TRANSCEND` has about 2.0 TiB free. | |
| - `~/github/llm` contains additional language projects: | |
| `dhimal-toolkit`, `gurung-language-toolkit`, `limbu-speech-toolkit`, | |
| `magar-toolkit`, `rai-speech-toolkit`, `nllb-better`, `nepa-newa-text-frontend`, | |
| `new-place`, and `loksewa.zip`. | |
| - `nllb-better` has Rai-family generated/validation artifacts and a language | |
| registry that should be inspected before building the OCR registry. | |
| ## Data Plan | |
| ### Real Validation Data | |
| Use Gorkhapatra/Naya Nepal as the central real-publication validation source. | |
| Create a scraper that emits: | |
| - article URL | |
| - article title | |
| - language label extracted from title | |
| - article image URLs | |
| - epaper date | |
| - epaper PDF URL | |
| - direct PDF file URL | |
| - image/PDF hashes | |
| - source timestamp | |
| - license/terms note | |
| - manual-review status | |
| Then create a hand-labeled validation pack: | |
| - at least 10 languages | |
| - at least 20 pages or page crops per language for v1 if available | |
| - Limbu set first | |
| - no synthetic samples in the final real validation pack | |
| - references include text, line boxes, reading order, table cells, captions, and | |
| Markdown where applicable | |
| ### Synthetic Training Data | |
| Use real text when synthesizing: | |
| - Limbu dictionary/Pangloss/IGT text first. | |
| - Tamang NepTam and dictionary text. | |
| - Devanagari text from Gorkhapatra/Loksewa/government-like documents where | |
| licensing permits. | |
| - Rai-family text from verified local/remote corpora. | |
| Synthetic generation must produce: | |
| - line crops | |
| - word crops | |
| - table cell crops | |
| - captions | |
| - full pages | |
| - degraded scan/photo variants | |
| - PaddleOCR recognition labels | |
| - document JSON/Markdown references | |
| Every synthetic sample must carry: | |
| - language | |
| - script | |
| - Unicode ranges observed | |
| - font | |
| - source text hash | |
| - rendered image hash | |
| - template id | |
| - degradation id | |
| - train/eval split id | |
| ### Handwriting Synthesis | |
| Handwriting is not a v1 blocker. It can be a later ablation. | |
| Possible direction: | |
| - Use available fonts and handwritten-style fonts for cheap diversity. | |
| - Explore diffusion/DM-based handwriting generation only after print OCR is | |
| stable. | |
| - For Limbu, do not synthesize handwriting until we have real handwritten or | |
| community-reviewed glyph examples. Otherwise we risk generating plausible but | |
| wrong orthography. | |
| If we explore One-DM or related diffusion methods, it should be framed as | |
| optional augmentation, not the main claim. | |
| ## Validation Gates | |
| ### Gate 0: Source Audit | |
| Pass only if: | |
| - Gorkhapatra article image URLs are downloadable. | |
| - Epaper direct PDFs are downloadable. | |
| - PDF text/image extraction path is implemented and audited. | |
| - Every source artifact is hashed. | |
| - Source license/terms risk is recorded. | |
| ### Gate 1: Language Registry | |
| Pass only if every target language has: | |
| - language name | |
| - ISO code if known | |
| - script name | |
| - Unicode range | |
| - font list | |
| - corpus source | |
| - renderability audit | |
| - normalization policy | |
| - PaddleOCR dictionary path | |
| - evaluation availability | |
| ### Gate 2: Limbu First | |
| Pass only if: | |
| - Limbu Unicode rendering works with selected fonts. | |
| - Limbu PaddleOCR dictionary exists. | |
| - Limbu synthetic crop/page generator emits valid labels. | |
| - Limbu held-out real or manually reviewed eval exists. | |
| - Stock baselines are recorded before fine-tuning. | |
| ### Gate 3: PaddleOCR Baselines | |
| Run before any claim: | |
| - stock PP-OCRv6 | |
| - stock PP-OCRv5 if available | |
| - Tesseract where traineddata exists | |
| - Surya | |
| - GLM-OCR | |
| - PaddleOCR-VL-1.6 | |
| ### Gate 4: Fine-Tuned Candidate | |
| The candidate must beat baselines on: | |
| - CER/WER per language | |
| - strict and normalized Unicode CER | |
| - exact-line accuracy | |
| - reading order | |
| - table text and structure | |
| - figure/caption detection | |
| - Markdown reconstruction | |
| - JSON artifact completeness | |
| English and Nepali must not regress versus the chosen stock baseline. | |
| ### Gate 5: Claim Review | |
| The paper claim is allowed only if: | |
| - all public/real validation rows have complete artifacts | |
| - fallback usage is reported separately | |
| - synthetic-only improvements are not used as SOTA evidence | |
| - failures are included in error analysis | |
| - a reproducibility bundle can rerun the benchmark | |
| ## Immediate Implementation Steps | |
| 1. Add `ocrtech audit-gorkhapatra-source`. **Implemented:** the command audits | |
| Naya Nepal article images and Gorkhapatra epaper PDF links, resolves PDF.js | |
| viewer URLs to direct `/uploads/file/...pdf` URLs, optionally downloads and | |
| hashes assets, and records PyMuPDF text-layer metadata when available. | |
| 2. Add `ocrtech prepare-gorkhapatra-pack`. **Implemented:** the command turns a | |
| source-audit JSON file into a pending-reference validation manifest, copies | |
| audited article images, renders audited epaper PDF pages when PyMuPDF is | |
| available, writes placeholder reference JSON files, and marks samples as | |
| `reference_status=pending_manual_label` and | |
| `claim_evidence_eligible=false` until references are verified. | |
| 3. Add `languages/nepal-v1.json` registry with Limbu first and at least ten | |
| target languages. **Implemented:** the registry tracks Limbu/Sirijonga first, | |
| more than ten Nepal-language targets, English as a non-counted regression | |
| language, script/Unicode ranges, font candidates, corpus sources, Gorkhapatra | |
| labels, validation status, and PaddleOCR dictionary targets. The | |
| `ocrtech audit-language-registry` command validates the registry and writes | |
| JSON/Markdown audit artifacts. | |
| 4. Add renderability audit for Unicode ranges and fonts. **Implemented:** the | |
| `ocrtech audit-font-renderability` command reads TTF/OTF/TTC cmap tables, | |
| checks registered font candidates against each script's Unicode ranges, and | |
| writes JSON/Markdown evidence. Raster sample rendering is supported when | |
| Pillow is installed. Current cmap evidence verifies Limbu, Devanagari, | |
| Santhali/Ol Chiki, and Latin coverage when the registered runtime font assets | |
| are present. | |
| 5. Add Limbu synthetic line/page generator. | |
| 6. Add PP-OCRv6 stock baseline runner. | |
| 7. Build the first Limbu validation pack from Gorkhapatra/Naya Nepal plus local | |
| Limbu resources. | |
| ## External References | |
| - PaddleOCR docs: <https://paddlepaddle.github.io/PaddleOCR/main/en/index.html> | |
| - PaddleOCR repo: <https://github.com/PaddlePaddle/PaddleOCR> | |
| - PP-StructureV3 docs: | |
| <https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/pipeline_usage/PP-StructureV3.html> | |
| - PaddleOCR-VL-1.6 docs: | |
| <https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL-1.6.html> | |
| - Gorkhapatra Naya Nepal category: | |
| <https://gorkhapatraonline.com/categories/new-nepal> | |
| - Gorkhapatra Naya Nepal publication: | |
| <https://gorkhapatraonline.com/publications/naya-nepal> | |
| - Gorkhapatra epaper list example: | |
| <https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra> | |
| - Unicode Limbu chart: <https://www.unicode.org/charts/PDF/U1900.pdf> | |
| - Kirat Rai Unicode proposal: | |
| <https://www.unicode.org/L2/L2022/22043-kirat-rai.pdf> | |