Download docs/NEPAL_MULTILINGUAL_OCR_GOAL.md from prosperious/ocr-tech-k2: direct link, hf CLI and curl.
- Browser
- Download file 13.8 kB
-
https://huggingface.co/prosperious/ocr-tech-k2/resolve/main/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
- Command line
-
hf download hf://prosperious/ocr-tech-k2/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
-
curl -L -o NEPAL_MULTILINGUAL_OCR_GOAL.md https://huggingface.co/prosperious/ocr-tech-k2/resolve/main/docs/NEPAL_MULTILINGUAL_OCR_GOAL.md
Nepal Multilingual Document OCR Goal
This is the research target after the steering decision on 2026-06-16. The project is now a PaddleOCR-centered multilingual document OCR system for languages used in Nepal, not a narrow Nepali/English OCR experiment.
Goal
Build and validate a PaddleOCR-based document OCR engine for at least ten languages spoken in Nepal, with Limbu first, that supports:
- page/PDF/image parsing
- text OCR
- reading order
- table reconstruction
- figure and caption extraction
- Markdown export
- structured JSON export
- selective PaddleOCR-VL/VLM fallback for hard pages
The claim is valid only if the engine beats strong baselines on predefined real and held-out evaluations. Synthetic-only results are not enough.
Research Claim Shape
The paper should not claim "best OCR engine overall." The defensible claim is:
A script-aware PaddleOCR adaptation framework can substantially improve document OCR and structured extraction for low-resource Nepal languages, including scripts not covered well by stock OCR systems, while preserving Nepali/English performance and using selective VLM fallback only when the local pipeline is uncertain.
The core contribution is the system and validation recipe:
- Language/script registry for Nepal document OCR.
- Gorkhapatra/Naya Nepal real-publication validation pack.
- Unicode/font/renderability audit before synthesis.
- Real-text synthetic data generation for lines, tables, captions, and pages.
- PaddleOCR recognizer fine-tuning by script family.
- PP-Structure-style document extraction.
- Claim review that rejects weak SOTA statements.
Primary Engine Direction
Use PaddleOCR as the engine family:
- PP-OCRv6 for the modern small/local OCR baseline and fine-tuning target.
- PP-StructureV3 for layout, reading order, tables, and document structure.
- PaddleOCR-VL-1.6 as fallback/teacher for charts, figures, hard layouts, seals, and low-confidence pages.
Do not route everything to a VLM and call that the engine. The local PaddleOCR path must be the default path; fallback usage must be measured and reported.
First Ten Language Targets
The first target set should be driven by actual Gorkhapatra/Naya Nepal availability plus local corpus availability. OCR support is tracked by script, Unicode, fonts, and evaluation data, not by language name alone.
| Priority | Language | First script target | Why first | Notes |
|---|---|---|---|---|
| 1 | Limbu/Yakthung | Limbu/Sirijonga, U+1900..U+194F | Has own Unicode block and local Limbu resources | First non-Devanagari research target |
| 2 | Nepali | Devanagari | Core language and baseline | Must not regress |
| 3 | English | Latin | Mixed documents and forms | Must not regress |
| 4 | Tamang | Devanagari first; Tamyig audit later | Local NepTam corpus and dictionary | Start with Devanagari Tamang |
| 5 | Magar | Devanagari first | Gorkhapatra item confirmed; local dictionary | Own/community script requires separate audit |
| 6 | Wambule | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 7 | Bantawa/Rai | Devanagari or Kirat Rai depending source | Gorkhapatra item confirmed; Rai resources on remote/local projects | Kirat Rai Unicode support is newer; font audit required |
| 8 | Kulung | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 9 | Yakkha | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 10 | Tharu/Rana Tharu | Devanagari first | Gorkhapatra item confirmed | Multiple varieties; keep labels explicit |
Additional Gorkhapatra-visible candidates include Sanskrit, Magahi, Bajjika, Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami, Sherpa, Achhami, Bote, Rajbanshi, Hayu, Santhali, Dhimal, Bhote, Uraon, Kumal, Gurung, Khash Jumli, and Meche. These should enter the registry after script/corpus/font checks.
Source Findings So Far
Gorkhapatra Naya Nepal
The category page is useful for language-specific validation:
https://gorkhapatraonline.com/categories/new-nepalhttps://gorkhapatraonline.com/categories/new-nepal?page=2
Confirmed visible language-specific entries include Magar, Wambule, Rana Tharu, Magahi, Bajjika, Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami, Sherpa, Achhami, Bote, Rajbanshi, Hayu, Santhali, Kulung, Yakkha, Meche, Khash Jumli, Dhimal, Tharu, Bhote, Uraon, Bantawa, Kumal, Gurung, and others.
The blog/category pages expose image URLs and article URLs. They are useful for real-world OCR samples, but they should not be trusted as clean text references without manual or tool-assisted verification.
Gorkhapatra Epaper
The epaper list endpoint exposes viewer links:
https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra
The visible links look like:
https://epaper.gorkhapatraonline.com/pdf?file=/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf
That pdf?... URL returns a PDF.js viewer shell. The real PDF is downloaded by
fetching the file parameter directly:
https://epaper.gorkhapatraonline.com/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf
On k2, this was verified as a real PDF:
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf
PDF document, version 1.6
size: 48M
The k2 audit produced:
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/audit.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/new-nepal-language-items.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/epaper-assets.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf
k2 does not currently expose pdfinfo or pdftotext. pymupdf was installed
into /Users/k2/ocr-tech/.venv on 2026-06-16 as the immediate PDF audit tool.
The sample epaper PDF opens successfully with PyMuPDF:
page_count: 26
total extracted text chars: 309924
preview image: outputs/gorkhapatra-source-audit/sample-page-1-preview.jpg
audit: outputs/gorkhapatra-source-audit/sample-pymupdf-audit.json
text: outputs/gorkhapatra-source-audit/sample-pymupdf-text.txt
The extracted text layer is not reliable reference text. The first-page preview contains mojibake/transliterated-looking output for Nepali content rather than clean Unicode Devanagari. Therefore the epaper PDFs are useful as real page images/layout sources, but their embedded text layer must be audited per file and must not be treated as ground truth without verification.
Local And Remote Corpus Hints
Local machine:
~/github/llm/ltk/tamang: NepTam train/test, NepTam80K, dictionary, TTS lexicon.~/github/llm/ltk/limbu: Limbu dictionary, Pangloss IGT, Limbu G2P.~/github/llm/ltk/magar: Western Magar dictionary and resource report.
Remote cdjk@100.110.148.43:
/Volumes/TRANSCENDhas about 2.0 TiB free.~/github/llmcontains additional language projects:dhimal-toolkit,gurung-language-toolkit,limbu-speech-toolkit,magar-toolkit,rai-speech-toolkit,nllb-better,nepa-newa-text-frontend,new-place, andloksewa.zip.nllb-betterhas Rai-family generated/validation artifacts and a language registry that should be inspected before building the OCR registry.
Data Plan
Real Validation Data
Use Gorkhapatra/Naya Nepal as the central real-publication validation source.
Create a scraper that emits:
- article URL
- article title
- language label extracted from title
- article image URLs
- epaper date
- epaper PDF URL
- direct PDF file URL
- image/PDF hashes
- source timestamp
- license/terms note
- manual-review status
Then create a hand-labeled validation pack:
- at least 10 languages
- at least 20 pages or page crops per language for v1 if available
- Limbu set first
- no synthetic samples in the final real validation pack
- references include text, line boxes, reading order, table cells, captions, and Markdown where applicable
Synthetic Training Data
Use real text when synthesizing:
- Limbu dictionary/Pangloss/IGT text first.
- Tamang NepTam and dictionary text.
- Devanagari text from Gorkhapatra/Loksewa/government-like documents where licensing permits.
- Rai-family text from verified local/remote corpora.
Synthetic generation must produce:
- line crops
- word crops
- table cell crops
- captions
- full pages
- degraded scan/photo variants
- PaddleOCR recognition labels
- document JSON/Markdown references
Every synthetic sample must carry:
- language
- script
- Unicode ranges observed
- font
- source text hash
- rendered image hash
- template id
- degradation id
- train/eval split id
Handwriting Synthesis
Handwriting is not a v1 blocker. It can be a later ablation.
Possible direction:
- Use available fonts and handwritten-style fonts for cheap diversity.
- Explore diffusion/DM-based handwriting generation only after print OCR is stable.
- For Limbu, do not synthesize handwriting until we have real handwritten or community-reviewed glyph examples. Otherwise we risk generating plausible but wrong orthography.
If we explore One-DM or related diffusion methods, it should be framed as optional augmentation, not the main claim.
Validation Gates
Gate 0: Source Audit
Pass only if:
- Gorkhapatra article image URLs are downloadable.
- Epaper direct PDFs are downloadable.
- PDF text/image extraction path is implemented and audited.
- Every source artifact is hashed.
- Source license/terms risk is recorded.
Gate 1: Language Registry
Pass only if every target language has:
- language name
- ISO code if known
- script name
- Unicode range
- font list
- corpus source
- renderability audit
- normalization policy
- PaddleOCR dictionary path
- evaluation availability
Gate 2: Limbu First
Pass only if:
- Limbu Unicode rendering works with selected fonts.
- Limbu PaddleOCR dictionary exists.
- Limbu synthetic crop/page generator emits valid labels.
- Limbu held-out real or manually reviewed eval exists.
- Stock baselines are recorded before fine-tuning.
Gate 3: PaddleOCR Baselines
Run before any claim:
- stock PP-OCRv6
- stock PP-OCRv5 if available
- Tesseract where traineddata exists
- Surya
- GLM-OCR
- PaddleOCR-VL-1.6
Gate 4: Fine-Tuned Candidate
The candidate must beat baselines on:
- CER/WER per language
- strict and normalized Unicode CER
- exact-line accuracy
- reading order
- table text and structure
- figure/caption detection
- Markdown reconstruction
- JSON artifact completeness
English and Nepali must not regress versus the chosen stock baseline.
Gate 5: Claim Review
The paper claim is allowed only if:
- all public/real validation rows have complete artifacts
- fallback usage is reported separately
- synthetic-only improvements are not used as SOTA evidence
- failures are included in error analysis
- a reproducibility bundle can rerun the benchmark
Immediate Implementation Steps
- Add
ocrtech audit-gorkhapatra-source. Implemented: the command audits Naya Nepal article images and Gorkhapatra epaper PDF links, resolves PDF.js viewer URLs to direct/uploads/file/...pdfURLs, optionally downloads and hashes assets, and records PyMuPDF text-layer metadata when available. - Add
ocrtech prepare-gorkhapatra-pack. Implemented: the command turns a source-audit JSON file into a pending-reference validation manifest, copies audited article images, renders audited epaper PDF pages when PyMuPDF is available, writes placeholder reference JSON files, and marks samples asreference_status=pending_manual_labelandclaim_evidence_eligible=falseuntil references are verified. - Add
languages/nepal-v1.jsonregistry with Limbu first and at least ten target languages. Implemented: the registry tracks Limbu/Sirijonga first, more than ten Nepal-language targets, English as a non-counted regression language, script/Unicode ranges, font candidates, corpus sources, Gorkhapatra labels, validation status, and PaddleOCR dictionary targets. Theocrtech audit-language-registrycommand validates the registry and writes JSON/Markdown audit artifacts. - Add renderability audit for Unicode ranges and fonts. Implemented: the
ocrtech audit-font-renderabilitycommand reads TTF/OTF/TTC cmap tables, checks registered font candidates against each script's Unicode ranges, and writes JSON/Markdown evidence. Raster sample rendering is supported when Pillow is installed. Current cmap evidence verifies Limbu, Devanagari, Santhali/Ol Chiki, and Latin coverage when the registered runtime font assets are present. - Add Limbu synthetic line/page generator.
- Add PP-OCRv6 stock baseline runner.
- Build the first Limbu validation pack from Gorkhapatra/Naya Nepal plus local Limbu resources.
External References
- PaddleOCR docs: https://paddlepaddle.github.io/PaddleOCR/main/en/index.html
- PaddleOCR repo: https://github.com/PaddlePaddle/PaddleOCR
- PP-StructureV3 docs: https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/pipeline_usage/PP-StructureV3.html
- PaddleOCR-VL-1.6 docs: https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL-1.6.html
- Gorkhapatra Naya Nepal category: https://gorkhapatraonline.com/categories/new-nepal
- Gorkhapatra Naya Nepal publication: https://gorkhapatraonline.com/publications/naya-nepal
- Gorkhapatra epaper list example: https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra
- Unicode Limbu chart: https://www.unicode.org/charts/PDF/U1900.pdf
- Kirat Rai Unicode proposal: https://www.unicode.org/L2/L2022/22043-kirat-rai.pdf