|
Download ocr-tech/docs/CLAUDE_HANDOFF.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 17.8 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/CLAUDE_HANDOFF.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/CLAUDE_HANDOFF.md
-
curl -L -o CLAUDE_HANDOFF.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/CLAUDE_HANDOFF.md
17.8 kB
| # Claude Handoff: Nepal Multilingual OCR | |
| Goal: build and publish a research-grade PaddleOCR/PP-Structure OCR engine for | |
| at least ten languages spoken in Nepal, using Gorkhapatra/Naya Nepal pages as | |
| real validation evidence, PDF-native and legacy-font extraction for weak labels, | |
| and script-aware synthetic data for training and ablation. | |
| Updated: 2026-06-16 | |
| Repo: `Ampixa/ocr-tech` | |
| Visibility: private at handoff time | |
| Branch: `main` | |
| This handoff is for a follow-on Claude/Codex agent. The user is steering a | |
| research-grade OCR project, not a demo. The target is a PaddleOCR-centered | |
| document OCR engine whose claims are tied to real validation and reproducible | |
| training data, not synthetic-only benchmarks. | |
| ## Operating Rules | |
| - Use `uv run ...` for project commands. | |
| - Use `apply_patch` for manual edits. | |
| - Do not commit generated corpora, rendered images, model runs, or outputs | |
| unless explicitly requested; `data/`, `outputs/`, and `runs/` are ignored | |
| intentionally. | |
| - Do commit reproducibility configs, docs, tests, and source code changes. | |
| - GPG signing fails on this machine; use: | |
| ```bash | |
| git -c commit.gpgsign=false commit -m "<message>" | |
| ``` | |
| - Do not commit remote passwords or credentials. Hosts are documented below, | |
| but credentials must stay in the conversation/session, not tracked files. | |
| - Do not claim SOTA or model quality unless backed by a passed real validation | |
| audit and benchmark. | |
| - Synthetic OCR data is never claim evidence. It is training/ablation material. | |
| - Validate schemas before writing converters. Do not infer fields from file | |
| names. | |
| ## Current Research Goal | |
| Build a PaddleOCR/PP-OCR + PP-Structure-style document engine for at least ten | |
| languages spoken in Nepal. | |
| Core scope: | |
| - OCR recognition | |
| - reading order | |
| - tables | |
| - figures/captions | |
| - Markdown output | |
| - JSON output | |
| - optional VLM fallback for hard figures/pages | |
| Current research angle: | |
| > Low-resource Nepal language OCR adaptation using PDF-native weak labels, | |
| > legacy font conversion, real Gorkhapatra validation, and script-aware | |
| > synthetic robustness. | |
| The real claim should be about measurable OCR/structure improvement on held-out | |
| Gorkhapatra/Naya Nepal pages, not about "understanding everything." | |
| ## Repo State | |
| Important recent commits: | |
| ```text | |
| 22454e2 Document cdjk language corpus migration | |
| 1f98430 Record migrated synthesis text sweep | |
| ``` | |
| The repo was checked with: | |
| ```bash | |
| jq . configs/cdjk-language-corpus-migration-v1.json >/dev/null | |
| git diff --check | |
| uv run pytest tests/test_synthesis_resources.py -q | |
| ``` | |
| All passed before this handoff was written. | |
| At handoff time, the only unrelated visible untracked path was: | |
| ```text | |
| .playwright-mcp/ | |
| ``` | |
| That is browser scratch output and should not be committed. | |
| ## Most Important Existing Docs | |
| Read these before acting: | |
| - `docs/ACTIVE_RESEARCH_CHECKLIST.md` | |
| - `docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md` | |
| - `docs/CDJK_LANGUAGE_CORPUS_MIGRATION.md` | |
| - `docs/SYNTHESIS_RESOURCE_AUDIT.md` | |
| - `docs/FONT_ASSETS.md` | |
| - `docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md` | |
| - `docs/NEPAL_MULTILINGUAL_OCR_GOAL.md` | |
| ## What Has Been Built | |
| ### CLI Surface | |
| The repo already has many `ocrtech` commands. Most relevant now: | |
| ```text | |
| ocrtech audit-gorkhapatra-language-pages | |
| ocrtech prepare-gorkhapatra-language-page-pack | |
| ocrtech prepare-gorkhapatra-language-page-review | |
| ocrtech audit-gorkhapatra-language-page-review | |
| ocrtech extract-gorkhapatra-language-page-pdf-text | |
| ocrtech convert-limbu-legacy | |
| ocrtech audit-synthesis-resources | |
| ocrtech prepare-limbu-limdic-text | |
| ocrtech prepare-limbu-unicode-text | |
| ocrtech prepare-tamang-text | |
| ocrtech prepare-magar-text | |
| ocrtech prepare-toolkit-parallel-text | |
| ocrtech audit-synthesis-text-manifest | |
| ocrtech split-synthesis-text | |
| ocrtech render-synthesis-text-lines | |
| ocrtech render-synthesis-text-split | |
| ocrtech audit-rendered-synthesis-lines | |
| ocrtech audit-rendered-degradation-split | |
| ocrtech train-recognizer | |
| ``` | |
| ### Limbu PDF-Native Extraction | |
| Source rule for every Gorkhapatra/Naya Nepal language: | |
| `https://gorkhapatraonline.com/publications/naya-nepal` is only the discovery | |
| index for identifying which language/date was posted. Do not use the Naya Nepal | |
| blog/card image as OCR, eval, or training evidence. Fetch the full paper from | |
| the Gorkhapatra epaper date-range endpoint, for example | |
| `https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-16&pdfdaterange2=2026-06-23&slug=gorkhapatra`, | |
| then render the matching PDF page and extract/convert the text layer from that | |
| same PDF page. Record PDF URL/hash/page number, rendered image hash, text | |
| extraction provenance, and conversion map hash. Classify each language page as | |
| `unicode_native`, `legacy_font`, `devanagari_transcription`, `latin_notation`, or | |
| `no_text_layer`; manual transcription/native-reader work is only the fallback | |
| when the PDF has no usable text layer or the font cannot yet be decoded. | |
| The current successful Gorkhapatra Limbu path: | |
| ```bash | |
| uv run ocrtech extract-gorkhapatra-language-page-pdf-text \ | |
| data/limbu-gorkhapatra-language-page-pack-2026-06-16/manifests/gorkhapatra-language-page-pack.jsonl \ | |
| --candidate-language लिम्बू \ | |
| --out data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16 | |
| ``` | |
| Known result: | |
| ```text | |
| passed=True samples=2/2 spans=776 target_spans=304 fonts=10 warnings=0 | |
| sample_id=gorkhapatra-language-page-0001-01 | |
| pdf_page_number=6 | |
| target_font=NamdhinggoSILL | |
| target_spans=304 | |
| embedded_font=UFNWLJ+NamdhinggoSILL | |
| ``` | |
| Page conclusion: | |
| - page 6 is the Limbu/Sirijonga page | |
| - page 7 is the same-date non-Limbu page | |
| The PDF text layer exposes legacy ASCII-like Namdhinggo text, not Unicode | |
| Limbu. Example: | |
| ```text | |
| legacy: kfMG g' ;fK;SF[ yf]af]cf | |
| unicode: ᤐᤠ᤺ᤴ ᤏᤢ ᤛᤠᤵᤛᤰᤖᤧ ᤌᤥᤒᤥᤀᤠ | |
| ``` | |
| The converter is implemented in `src/ocrtech/limbu_legacy.py` using the vendored | |
| SIL map: | |
| ```text | |
| data/mappings/limbu-legacy/Limbu.map | |
| ``` | |
| ### Limbu Review Dashboard | |
| Generated dashboard path: | |
| ```text | |
| data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html | |
| ``` | |
| Correct local file URL: | |
| ```text | |
| file:///Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html | |
| ``` | |
| This is the correct browser UI for the Gorkhapatra PDF-native Limbu page. It | |
| shows the full rendered PDF image with bounded boxes; clicking a box selects the | |
| matching right-side row with converted text and the corresponding rendered span | |
| crop. Do not substitute the PDF-native verification-bundle HTML for this task; | |
| that was the wrong/broken UI. | |
| Local server used: | |
| ```bash | |
| uv run python -m http.server 8765 --bind 127.0.0.1 | |
| ``` | |
| Dashboard URL: | |
| ```text | |
| http://127.0.0.1:8765/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html | |
| ``` | |
| Important bug already fixed in the generated dashboard: clicking a bounding box | |
| must not re-render or scroll the image/page back to the top. The fix removed | |
| automatic `scrollIntoView` from bounding-box selection. | |
| If this dashboard is regenerated by code later, preserve this behavior: | |
| ```js | |
| function selectSpan(id){ | |
| selected=id; | |
| document.querySelectorAll('.box.selected,.span.selected').forEach(el => el.classList.remove('selected')); | |
| document.querySelectorAll(`[data-span="${id}"],[data-row="${id}"]`).forEach(el => el.classList.add('selected')); | |
| } | |
| ``` | |
| ### Migrated Corpus | |
| Source host: | |
| ```text | |
| cdjk@100.110.148.43 | |
| machine: cdjks-MacBook-Pro.local | |
| external volume: /Volumes/TRANSCEND | |
| ``` | |
| Do not write generated data back to that host. Treat it as a source for corpus | |
| and dictionaries unless the user explicitly asks otherwise. | |
| Tracked migration policy: | |
| ```text | |
| configs/cdjk-language-corpus-migration-v1.json | |
| ``` | |
| Ignored local copy: | |
| ```text | |
| data/external-language-resources/cdjk-100.110.148.43/ | |
| ``` | |
| Copied roots: | |
| ```text | |
| dhimal-toolkit | |
| gurung-language-toolkit | |
| limbu-speech-toolkit | |
| ltk | |
| magar-toolkit | |
| rai-speech-toolkit | |
| ``` | |
| Migration result: | |
| ```text | |
| 2307 text/resource files transferred | |
| 1.6G local size | |
| resource audit: passed=True roots=6/6 files=2308 warnings=0 | |
| ``` | |
| ### Prepared Synthesis Text | |
| Generated synthesis text is ignored under `data/synthesis-text/`, but the | |
| documented state is: | |
| ```text | |
| 20 manifests | |
| 228212 total samples | |
| 45258 duplicate rows removed/reported by converters | |
| 202560 Devanagari rows | |
| 10337 Mixed_Devanagari_Latin rows | |
| 6521 Latin rows | |
| 8772 true Unicode Limbu rows | |
| 14 Mixed_Limbu_Devanagari rows | |
| 3 Mixed_Limbu_Latin rows | |
| 5 Other rows | |
| ``` | |
| Important generated manifests: | |
| ```text | |
| data/synthesis-text/cdjk-limbu-limdic-2026-06-16/limbu-limdic-text.jsonl | |
| data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl | |
| data/synthesis-text/cdjk-tamang-2026-06-16/tamang-text.jsonl | |
| data/synthesis-text/cdjk-magar-2026-06-16/magar-text.jsonl | |
| data/synthesis-text/cdjk-gurung-2026-06-16/toolkit-parallel-text.jsonl | |
| data/synthesis-text/cdjk-dhimal-parallel-2026-06-16/toolkit-parallel-text.jsonl | |
| data/synthesis-text/cdjk-dhimal-mono-2026-06-16/toolkit-parallel-text.jsonl | |
| data/synthesis-text/cdjk-magar-parallel-extra-2026-06-16/toolkit-parallel-text.jsonl | |
| data/synthesis-text/cdjk-magar-mono-extra-2026-06-16/toolkit-parallel-text.jsonl | |
| data/synthesis-text/cdjk-rai-*-2026-06-16/toolkit-parallel-text.jsonl | |
| ``` | |
| All synthesis text audits passed with `claim_eligible=0`. | |
| Important interpretation: | |
| - True Unicode Limbu exists from Limdic and should render with Namdhinggo after | |
| filtering `script == Limbu`. | |
| - Most Gurung, Dhimal, Magar, Tamang, and Rai rows are Devanagari-transcribed | |
| language text, not native-script OCR. | |
| - Bantawa, Khaling, Koyu, and Thulung include substantial Latin phonetic data. | |
| These are useful for language resources but should not be called native-script | |
| OCR data. | |
| ### Synthetic Rendering | |
| Corrected Limbu-only smoke render: | |
| ```bash | |
| uv run ocrtech render-synthesis-text-lines \ | |
| data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl \ | |
| --out data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only \ | |
| --font-path data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Regular.ttf \ | |
| --limit 120 \ | |
| --degradation-profile clean \ | |
| --degradation-profile scan \ | |
| --degradation-profile low_light \ | |
| --degradation-profile uneven_light \ | |
| --degradation-profile camera_left \ | |
| --degradation-profile phone_photo | |
| ``` | |
| Audit: | |
| ```bash | |
| uv run ocrtech audit-rendered-synthesis-lines \ | |
| data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/manifests/rendered-line-manifest.jsonl \ | |
| --label-file data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/labels/train.txt \ | |
| --out outputs/cdjk-limbu-rendered-smoke-audit-2026-06-16-limbu-only | |
| ``` | |
| Result: | |
| ```text | |
| passed=True samples=120 claim_eligible=0 issues=0 warnings=0 | |
| degradation_counts: | |
| camera_left: 20 | |
| clean: 20 | |
| low_light: 20 | |
| phone_photo: 20 | |
| scan: 20 | |
| uneven_light: 20 | |
| ``` | |
| Important lesson: an earlier mixed-script render produced tofu boxes because | |
| Devanagari/Latin rows were rendered with a Limbu font. Always filter by script | |
| before choosing a font. | |
| ## Current Strategic Decision | |
| The user asked what next. The right order is: | |
| 1. Build Gorkhapatra validation for all target languages. | |
| 2. Create language/script/font matrix from actual Gorkhapatra PDF evidence. | |
| 3. Generate split-safe synthetic data per script/font bucket. | |
| 4. Research and execute PaddleOCR recognizer fine-tuning. | |
| 5. Keep PP-Structure/layout mostly stock until recognizer evidence says | |
| detection/layout fine-tuning is required. | |
| Do not scale synthetic data first without real validation. It will create a | |
| large training set but no credible paper claim. | |
| ## Immediate Next Task | |
| Build a reusable all-language Gorkhapatra validation generator. | |
| It should turn this manual Limbu success into a repeatable command/workflow: | |
| ```text | |
| Naya Nepal publication card | |
| -> date/language/article metadata | |
| -> matching epaper PDF | |
| -> candidate language-page pages | |
| -> rendered page image | |
| -> PDF-native spans/fonts/bboxes/codepoints | |
| -> optional legacy conversion by script/font | |
| -> review dashboard | |
| -> reviewed JSON/CSV annotations | |
| ``` | |
| Suggested CLI shape: | |
| ```text | |
| ocrtech prepare-gorkhapatra-language-validation | |
| ``` | |
| Likely arguments: | |
| ```text | |
| --language-label <Nepali label> | |
| --publication-pages <N> | |
| --epaper-url <date-range endpoint> | |
| --out <data/gorkhapatra-validation-...> | |
| --target-font <optional font name> | |
| --legacy-converter <optional converter id, e.g. limbu-namdhinggo> | |
| --render-pages | |
| --download-pdfs | |
| ``` | |
| Output should include: | |
| ```text | |
| validation.json | |
| validation.md | |
| samples/*.pdf-text.json | |
| samples/*.pdf-text.txt | |
| spans.csv | |
| fonts/ | |
| page-images/ | |
| span-renders/ | |
| dashboard.html | |
| review.csv | |
| review.jsonl | |
| ``` | |
| Dashboard requirements: | |
| - Full page image with clickable bbox overlay. | |
| - Right panel with selected span, PDF-rendered crop, raw extracted text, | |
| converted Unicode text when available, font, bbox, codepoints. | |
| - Review status controls: | |
| - `accept` | |
| - `needs_fix` | |
| - `bad_segment` | |
| - `drop` | |
| - `wrong_page` | |
| - `wrong_language` | |
| - Must preserve image/page scroll when clicking bbox. | |
| - Must not ask the user to transcribe text if PDF-native extraction/conversion | |
| already produces usable Unicode text. | |
| - Must save/export annotations as JSONL/CSV. | |
| ## Language Validation Targets | |
| At least ten target languages are desired. Current likely sources: | |
| - Limbu/Sirijonga | |
| - Tamang | |
| - Magar | |
| - Gurung | |
| - Dhimal | |
| - Newar/Newa | |
| - Bantawa/Rai | |
| - Chamling/Rai | |
| - Kulung/Rai | |
| - Athpariya/Rai | |
| - Khaling/Rai | |
| - Yakkha/Rai | |
| - Sunwar/Rai | |
| - Lohorung/Rai | |
| - Dumi/Rai | |
| - Thulung/Rai | |
| - Koyu/Rai | |
| But Gorkhapatra validation must decide actual script form: | |
| - Unicode native script | |
| - legacy font encoding | |
| - Devanagari transcription | |
| - Latin notation | |
| - unsupported/no text layer | |
| Do not assume native Unicode just because the language has a script. | |
| ## Training Research Direction | |
| Use primary sources before changing training code. Current understanding: | |
| - PaddleOCR/PP-OCR remains the primary engine family. | |
| - Start by fine-tuning recognizer, not layout/detection. | |
| - Use PP-Structure/PP-StructureV3-style parsing for document layout and | |
| structured output. | |
| - Consider PaddleOCR-VL only as fallback/teacher/comparator, not the default | |
| small local engine. | |
| Recognizer-first rationale: | |
| - The repo already has line/synthesis text and rendered line labels. | |
| - Recognition turns cropped text images into text. | |
| - Detection/layout claims require bbox/layout ground truth, which is not yet | |
| mature across languages. | |
| - Fine-tuning detection before measuring detection failures is premature. | |
| Training prerequisites before a serious run: | |
| - Split synthesis text with `split-synthesis-text`. | |
| - Exclude real validation/reference text hashes from synthetic training. | |
| - Render train/eval with font readiness and degradation audits. | |
| - Keep English/Nepali non-regression sets. | |
| - Package PaddleOCR recognizer training recipe. | |
| - Evaluate on Gorkhapatra real pages and held-out synthetic rows. | |
| ## Remote Machines | |
| Known hosts from the session: | |
| ```text | |
| k2@100.97.238.103 | |
| cdjk@100.110.148.43 | |
| cdjk@100.117.21.47 | |
| ``` | |
| Do not commit passwords. The user provided credentials in chat previously, but | |
| tracked docs must not contain them. | |
| Important storage findings: | |
| - `cdjk@100.110.148.43` has `/Volumes/TRANSCEND`, about `3.6Ti` total and | |
| `2.0Ti` available at inventory time. Use as source for corpus/dictionary | |
| resources unless instructed otherwise. | |
| - `cdjk@100.117.21.47` root was full at inspection time: | |
| ```text | |
| / 915G 865G 3.7G 100% | |
| /mnt/tts_cache_nvme 234G 150G 72G 68% | |
| ``` | |
| Do not dump large generated data to a nearly-full root filesystem. | |
| ## Source Files To Inspect Before Editing | |
| For Gorkhapatra flow: | |
| ```text | |
| src/ocrtech/source_audit.py | |
| src/ocrtech/cli.py | |
| src/ocrtech/limbu_legacy.py | |
| tests/test_source_audit.py | |
| tests/test_limbu_legacy.py | |
| ``` | |
| For synthesis: | |
| ```text | |
| src/ocrtech/synthesis_resources.py | |
| tests/test_synthesis_resources.py | |
| configs/cdjk-language-corpus-migration-v1.json | |
| ``` | |
| For fonts/language registry: | |
| ```text | |
| src/ocrtech/font_assets.py | |
| src/ocrtech/language_registry.py | |
| configs/namdhinggo-font-assets-v1.json | |
| configs/nepal-font-assets-v1.json | |
| languages/nepal-v1.json | |
| ``` | |
| For training: | |
| ```text | |
| src/ocrtech/training.py | |
| src/ocrtech/models.py | |
| docs/MODELS.md | |
| docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md | |
| ``` | |
| ## Known Pitfalls | |
| - The user is frustrated by fake OCR candidates and manual busywork. Do not ask | |
| them to validate bad OCR text when PDF-native text extraction can provide the | |
| answer. | |
| - For Limbu, PDF text was extractable but legacy encoded. The correct path was | |
| PDF-native text -> Namdhinggo legacy conversion -> Unicode display. | |
| - Dashboard bbox clicks must not reset scroll. | |
| - Do not treat Devanagari-transcribed language data as native script OCR. | |
| - Do not render mixed-script rows with a single script-specific font. | |
| - Do not collapse mixed schemas into one converter command by dropping | |
| validation. Split sources by schema instead. | |
| - Do not publish synthetic-only claims. | |
| ## Recommended Next Commit | |
| Implement the reusable validation generator and tests: | |
| 1. Refactor current generated dashboard logic into source code. | |
| 2. Add a CLI command that accepts a manifest/PDF-text extraction dir and emits | |
| dashboard + review CSV/JSONL. | |
| 3. Preserve current Limbu dashboard features: | |
| - full page image | |
| - bbox overlay | |
| - PDF-rendered span crop | |
| - converted Unicode text | |
| - raw legacy text | |
| - font metadata | |
| - stable scroll behavior | |
| 4. Add fixture tests for dashboard generation and annotation schema. | |
| 5. Document command in `docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md`. | |
| 6. Run focused tests: | |
| ```bash | |
| uv run pytest tests/test_source_audit.py tests/test_limbu_legacy.py -q | |
| git diff --check | |
| ``` | |
| Then commit with GPG signing disabled and push. | |