ocr-projects-archive / ocr-tech /docs /CLAUDE_HANDOFF.md
prosperious's picture
Add 1000 files (OCR_INDRA/.gitignore ...)
f9bc0c1 verified
|
Raw History Blame Contribute Delete
17.8 kB

Claude Handoff: Nepal Multilingual OCR

Goal: build and publish a research-grade PaddleOCR/PP-Structure OCR engine for at least ten languages spoken in Nepal, using Gorkhapatra/Naya Nepal pages as real validation evidence, PDF-native and legacy-font extraction for weak labels, and script-aware synthetic data for training and ablation.

Updated: 2026-06-16
Repo: Ampixa/ocr-tech
Visibility: private at handoff time
Branch: main

This handoff is for a follow-on Claude/Codex agent. The user is steering a research-grade OCR project, not a demo. The target is a PaddleOCR-centered document OCR engine whose claims are tied to real validation and reproducible training data, not synthetic-only benchmarks.

Operating Rules

  • Use uv run ... for project commands.
  • Use apply_patch for manual edits.
  • Do not commit generated corpora, rendered images, model runs, or outputs unless explicitly requested; data/, outputs/, and runs/ are ignored intentionally.
  • Do commit reproducibility configs, docs, tests, and source code changes.
  • GPG signing fails on this machine; use:
git -c commit.gpgsign=false commit -m "<message>"
  • Do not commit remote passwords or credentials. Hosts are documented below, but credentials must stay in the conversation/session, not tracked files.
  • Do not claim SOTA or model quality unless backed by a passed real validation audit and benchmark.
  • Synthetic OCR data is never claim evidence. It is training/ablation material.
  • Validate schemas before writing converters. Do not infer fields from file names.

Current Research Goal

Build a PaddleOCR/PP-OCR + PP-Structure-style document engine for at least ten languages spoken in Nepal.

Core scope:

  • OCR recognition
  • reading order
  • tables
  • figures/captions
  • Markdown output
  • JSON output
  • optional VLM fallback for hard figures/pages

Current research angle:

Low-resource Nepal language OCR adaptation using PDF-native weak labels, legacy font conversion, real Gorkhapatra validation, and script-aware synthetic robustness.

The real claim should be about measurable OCR/structure improvement on held-out Gorkhapatra/Naya Nepal pages, not about "understanding everything."

Repo State

Important recent commits:

22454e2 Document cdjk language corpus migration
1f98430 Record migrated synthesis text sweep

The repo was checked with:

jq . configs/cdjk-language-corpus-migration-v1.json >/dev/null
git diff --check
uv run pytest tests/test_synthesis_resources.py -q

All passed before this handoff was written.

At handoff time, the only unrelated visible untracked path was:

.playwright-mcp/

That is browser scratch output and should not be committed.

Most Important Existing Docs

Read these before acting:

  • docs/ACTIVE_RESEARCH_CHECKLIST.md
  • docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md
  • docs/CDJK_LANGUAGE_CORPUS_MIGRATION.md
  • docs/SYNTHESIS_RESOURCE_AUDIT.md
  • docs/FONT_ASSETS.md
  • docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md
  • docs/NEPAL_MULTILINGUAL_OCR_GOAL.md

What Has Been Built

CLI Surface

The repo already has many ocrtech commands. Most relevant now:

ocrtech audit-gorkhapatra-language-pages
ocrtech prepare-gorkhapatra-language-page-pack
ocrtech prepare-gorkhapatra-language-page-review
ocrtech audit-gorkhapatra-language-page-review
ocrtech extract-gorkhapatra-language-page-pdf-text
ocrtech convert-limbu-legacy
ocrtech audit-synthesis-resources
ocrtech prepare-limbu-limdic-text
ocrtech prepare-limbu-unicode-text
ocrtech prepare-tamang-text
ocrtech prepare-magar-text
ocrtech prepare-toolkit-parallel-text
ocrtech audit-synthesis-text-manifest
ocrtech split-synthesis-text
ocrtech render-synthesis-text-lines
ocrtech render-synthesis-text-split
ocrtech audit-rendered-synthesis-lines
ocrtech audit-rendered-degradation-split
ocrtech train-recognizer

Limbu PDF-Native Extraction

Source rule for every Gorkhapatra/Naya Nepal language: https://gorkhapatraonline.com/publications/naya-nepal is only the discovery index for identifying which language/date was posted. Do not use the Naya Nepal blog/card image as OCR, eval, or training evidence. Fetch the full paper from the Gorkhapatra epaper date-range endpoint, for example https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-16&pdfdaterange2=2026-06-23&slug=gorkhapatra, then render the matching PDF page and extract/convert the text layer from that same PDF page. Record PDF URL/hash/page number, rendered image hash, text extraction provenance, and conversion map hash. Classify each language page as unicode_native, legacy_font, devanagari_transcription, latin_notation, or no_text_layer; manual transcription/native-reader work is only the fallback when the PDF has no usable text layer or the font cannot yet be decoded.

The current successful Gorkhapatra Limbu path:

uv run ocrtech extract-gorkhapatra-language-page-pdf-text \
  data/limbu-gorkhapatra-language-page-pack-2026-06-16/manifests/gorkhapatra-language-page-pack.jsonl \
  --candidate-language लिम्बू \
  --out data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16

Known result:

passed=True samples=2/2 spans=776 target_spans=304 fonts=10 warnings=0
sample_id=gorkhapatra-language-page-0001-01
pdf_page_number=6
target_font=NamdhinggoSILL
target_spans=304
embedded_font=UFNWLJ+NamdhinggoSILL

Page conclusion:

  • page 6 is the Limbu/Sirijonga page
  • page 7 is the same-date non-Limbu page

The PDF text layer exposes legacy ASCII-like Namdhinggo text, not Unicode Limbu. Example:

legacy:  kfMG g' ;fK;SF[ yf]af]cf
unicode: ᤐᤠ᤺ᤴ ᤏᤢ ᤛᤠᤵᤛᤰᤖᤧ ᤌᤥᤒᤥᤀᤠ

The converter is implemented in src/ocrtech/limbu_legacy.py using the vendored SIL map:

data/mappings/limbu-legacy/Limbu.map

Limbu Review Dashboard

Generated dashboard path:

data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html

Correct local file URL:

file:///Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html

This is the correct browser UI for the Gorkhapatra PDF-native Limbu page. It shows the full rendered PDF image with bounded boxes; clicking a box selects the matching right-side row with converted text and the corresponding rendered span crop. Do not substitute the PDF-native verification-bundle HTML for this task; that was the wrong/broken UI.

Local server used:

uv run python -m http.server 8765 --bind 127.0.0.1

Dashboard URL:

http://127.0.0.1:8765/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html

Important bug already fixed in the generated dashboard: clicking a bounding box must not re-render or scroll the image/page back to the top. The fix removed automatic scrollIntoView from bounding-box selection.

If this dashboard is regenerated by code later, preserve this behavior:

function selectSpan(id){
  selected=id;
  document.querySelectorAll('.box.selected,.span.selected').forEach(el => el.classList.remove('selected'));
  document.querySelectorAll(`[data-span="${id}"],[data-row="${id}"]`).forEach(el => el.classList.add('selected'));
}

Migrated Corpus

Source host:

cdjk@100.110.148.43
machine: cdjks-MacBook-Pro.local
external volume: /Volumes/TRANSCEND

Do not write generated data back to that host. Treat it as a source for corpus and dictionaries unless the user explicitly asks otherwise.

Tracked migration policy:

configs/cdjk-language-corpus-migration-v1.json

Ignored local copy:

data/external-language-resources/cdjk-100.110.148.43/

Copied roots:

dhimal-toolkit
gurung-language-toolkit
limbu-speech-toolkit
ltk
magar-toolkit
rai-speech-toolkit

Migration result:

2307 text/resource files transferred
1.6G local size
resource audit: passed=True roots=6/6 files=2308 warnings=0

Prepared Synthesis Text

Generated synthesis text is ignored under data/synthesis-text/, but the documented state is:

20 manifests
228212 total samples
45258 duplicate rows removed/reported by converters
202560 Devanagari rows
10337 Mixed_Devanagari_Latin rows
6521 Latin rows
8772 true Unicode Limbu rows
14 Mixed_Limbu_Devanagari rows
3 Mixed_Limbu_Latin rows
5 Other rows

Important generated manifests:

data/synthesis-text/cdjk-limbu-limdic-2026-06-16/limbu-limdic-text.jsonl
data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl
data/synthesis-text/cdjk-tamang-2026-06-16/tamang-text.jsonl
data/synthesis-text/cdjk-magar-2026-06-16/magar-text.jsonl
data/synthesis-text/cdjk-gurung-2026-06-16/toolkit-parallel-text.jsonl
data/synthesis-text/cdjk-dhimal-parallel-2026-06-16/toolkit-parallel-text.jsonl
data/synthesis-text/cdjk-dhimal-mono-2026-06-16/toolkit-parallel-text.jsonl
data/synthesis-text/cdjk-magar-parallel-extra-2026-06-16/toolkit-parallel-text.jsonl
data/synthesis-text/cdjk-magar-mono-extra-2026-06-16/toolkit-parallel-text.jsonl
data/synthesis-text/cdjk-rai-*-2026-06-16/toolkit-parallel-text.jsonl

All synthesis text audits passed with claim_eligible=0.

Important interpretation:

  • True Unicode Limbu exists from Limdic and should render with Namdhinggo after filtering script == Limbu.
  • Most Gurung, Dhimal, Magar, Tamang, and Rai rows are Devanagari-transcribed language text, not native-script OCR.
  • Bantawa, Khaling, Koyu, and Thulung include substantial Latin phonetic data. These are useful for language resources but should not be called native-script OCR data.

Synthetic Rendering

Corrected Limbu-only smoke render:

uv run ocrtech render-synthesis-text-lines \
  data/synthesis-text/cdjk-limbu-limdic-2026-06-16/filtered/limbu-only.jsonl \
  --out data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only \
  --font-path data/fonts/namdhinggo/Namdhinggo-3.100/Namdhinggo-Regular.ttf \
  --limit 120 \
  --degradation-profile clean \
  --degradation-profile scan \
  --degradation-profile low_light \
  --degradation-profile uneven_light \
  --degradation-profile camera_left \
  --degradation-profile phone_photo

Audit:

uv run ocrtech audit-rendered-synthesis-lines \
  data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/manifests/rendered-line-manifest.jsonl \
  --label-file data/rendered-synthesis/cdjk-limbu-smoke-2026-06-16-limbu-only/labels/train.txt \
  --out outputs/cdjk-limbu-rendered-smoke-audit-2026-06-16-limbu-only

Result:

passed=True samples=120 claim_eligible=0 issues=0 warnings=0
degradation_counts:
  camera_left: 20
  clean: 20
  low_light: 20
  phone_photo: 20
  scan: 20
  uneven_light: 20

Important lesson: an earlier mixed-script render produced tofu boxes because Devanagari/Latin rows were rendered with a Limbu font. Always filter by script before choosing a font.

Current Strategic Decision

The user asked what next. The right order is:

  1. Build Gorkhapatra validation for all target languages.
  2. Create language/script/font matrix from actual Gorkhapatra PDF evidence.
  3. Generate split-safe synthetic data per script/font bucket.
  4. Research and execute PaddleOCR recognizer fine-tuning.
  5. Keep PP-Structure/layout mostly stock until recognizer evidence says detection/layout fine-tuning is required.

Do not scale synthetic data first without real validation. It will create a large training set but no credible paper claim.

Immediate Next Task

Build a reusable all-language Gorkhapatra validation generator.

It should turn this manual Limbu success into a repeatable command/workflow:

Naya Nepal publication card
  -> date/language/article metadata
  -> matching epaper PDF
  -> candidate language-page pages
  -> rendered page image
  -> PDF-native spans/fonts/bboxes/codepoints
  -> optional legacy conversion by script/font
  -> review dashboard
  -> reviewed JSON/CSV annotations

Suggested CLI shape:

ocrtech prepare-gorkhapatra-language-validation

Likely arguments:

--language-label <Nepali label>
--publication-pages <N>
--epaper-url <date-range endpoint>
--out <data/gorkhapatra-validation-...>
--target-font <optional font name>
--legacy-converter <optional converter id, e.g. limbu-namdhinggo>
--render-pages
--download-pdfs

Output should include:

validation.json
validation.md
samples/*.pdf-text.json
samples/*.pdf-text.txt
spans.csv
fonts/
page-images/
span-renders/
dashboard.html
review.csv
review.jsonl

Dashboard requirements:

  • Full page image with clickable bbox overlay.
  • Right panel with selected span, PDF-rendered crop, raw extracted text, converted Unicode text when available, font, bbox, codepoints.
  • Review status controls:
    • accept
    • needs_fix
    • bad_segment
    • drop
    • wrong_page
    • wrong_language
  • Must preserve image/page scroll when clicking bbox.
  • Must not ask the user to transcribe text if PDF-native extraction/conversion already produces usable Unicode text.
  • Must save/export annotations as JSONL/CSV.

Language Validation Targets

At least ten target languages are desired. Current likely sources:

  • Limbu/Sirijonga
  • Tamang
  • Magar
  • Gurung
  • Dhimal
  • Newar/Newa
  • Bantawa/Rai
  • Chamling/Rai
  • Kulung/Rai
  • Athpariya/Rai
  • Khaling/Rai
  • Yakkha/Rai
  • Sunwar/Rai
  • Lohorung/Rai
  • Dumi/Rai
  • Thulung/Rai
  • Koyu/Rai

But Gorkhapatra validation must decide actual script form:

  • Unicode native script
  • legacy font encoding
  • Devanagari transcription
  • Latin notation
  • unsupported/no text layer

Do not assume native Unicode just because the language has a script.

Training Research Direction

Use primary sources before changing training code. Current understanding:

  • PaddleOCR/PP-OCR remains the primary engine family.
  • Start by fine-tuning recognizer, not layout/detection.
  • Use PP-Structure/PP-StructureV3-style parsing for document layout and structured output.
  • Consider PaddleOCR-VL only as fallback/teacher/comparator, not the default small local engine.

Recognizer-first rationale:

  • The repo already has line/synthesis text and rendered line labels.
  • Recognition turns cropped text images into text.
  • Detection/layout claims require bbox/layout ground truth, which is not yet mature across languages.
  • Fine-tuning detection before measuring detection failures is premature.

Training prerequisites before a serious run:

  • Split synthesis text with split-synthesis-text.
  • Exclude real validation/reference text hashes from synthetic training.
  • Render train/eval with font readiness and degradation audits.
  • Keep English/Nepali non-regression sets.
  • Package PaddleOCR recognizer training recipe.
  • Evaluate on Gorkhapatra real pages and held-out synthetic rows.

Remote Machines

Known hosts from the session:

k2@100.97.238.103
cdjk@100.110.148.43
cdjk@100.117.21.47

Do not commit passwords. The user provided credentials in chat previously, but tracked docs must not contain them.

Important storage findings:

  • cdjk@100.110.148.43 has /Volumes/TRANSCEND, about 3.6Ti total and 2.0Ti available at inventory time. Use as source for corpus/dictionary resources unless instructed otherwise.
  • cdjk@100.117.21.47 root was full at inspection time:
/                 915G  865G  3.7G  100%
/mnt/tts_cache_nvme 234G 150G 72G   68%

Do not dump large generated data to a nearly-full root filesystem.

Source Files To Inspect Before Editing

For Gorkhapatra flow:

src/ocrtech/source_audit.py
src/ocrtech/cli.py
src/ocrtech/limbu_legacy.py
tests/test_source_audit.py
tests/test_limbu_legacy.py

For synthesis:

src/ocrtech/synthesis_resources.py
tests/test_synthesis_resources.py
configs/cdjk-language-corpus-migration-v1.json

For fonts/language registry:

src/ocrtech/font_assets.py
src/ocrtech/language_registry.py
configs/namdhinggo-font-assets-v1.json
configs/nepal-font-assets-v1.json
languages/nepal-v1.json

For training:

src/ocrtech/training.py
src/ocrtech/models.py
docs/MODELS.md
docs/MULTILINGUAL_PADDLEOCR_RESEARCH_DIRECTION.md

Known Pitfalls

  • The user is frustrated by fake OCR candidates and manual busywork. Do not ask them to validate bad OCR text when PDF-native text extraction can provide the answer.
  • For Limbu, PDF text was extractable but legacy encoded. The correct path was PDF-native text -> Namdhinggo legacy conversion -> Unicode display.
  • Dashboard bbox clicks must not reset scroll.
  • Do not treat Devanagari-transcribed language data as native script OCR.
  • Do not render mixed-script rows with a single script-specific font.
  • Do not collapse mixed schemas into one converter command by dropping validation. Split sources by schema instead.
  • Do not publish synthetic-only claims.

Recommended Next Commit

Implement the reusable validation generator and tests:

  1. Refactor current generated dashboard logic into source code.
  2. Add a CLI command that accepts a manifest/PDF-text extraction dir and emits dashboard + review CSV/JSONL.
  3. Preserve current Limbu dashboard features:
    • full page image
    • bbox overlay
    • PDF-rendered span crop
    • converted Unicode text
    • raw legacy text
    • font metadata
    • stable scroll behavior
  4. Add fixture tests for dashboard generation and annotation schema.
  5. Document command in docs/GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md.
  6. Run focused tests:
uv run pytest tests/test_source_audit.py tests/test_limbu_legacy.py -q
git diff --check

Then commit with GPG signing disabled and push.