|
Download ocr-tech/docs/ACTIVE_RESEARCH_CHECKLIST.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 22.4 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/ACTIVE_RESEARCH_CHECKLIST.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/ACTIVE_RESEARCH_CHECKLIST.md
-
curl -L -o ACTIVE_RESEARCH_CHECKLIST.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/ACTIVE_RESEARCH_CHECKLIST.md
22.4 kB
Active Research Checklist
This checklist records the current steering decision for the Nepal multilingual OCR paper track. Keep it updated when remote data, synthesis converters, validation packs, or claim gates change.
Current Goal
Build a PaddleOCR-centered document OCR engine for at least ten languages spoken in Nepal. Limbu/Sirijonga is the first non-Devanagari target. The engine must support OCR, reading order, tables, figures, captions, Markdown, and structured JSON. Synthetic data can train and ablate the system, but paper claims require real held-out validation, especially Gorkhapatra/Naya Nepal page evidence.
Non-Negotiable Gates
- Keep PaddleOCR/PP-OCR as the primary engine family.
- Use PP-Structure-style document extraction for layout, reading order, table reconstruction, figures, captions, Markdown, and JSON.
- Use PaddleOCR-VL or another VLM only as fallback, teacher, or hard-page comparator; report fallback rate separately.
- Do not call a language synthesis-ready until a text manifest passes
ocrtech audit-synthesis-text-manifest. - Do not call rendered OCR data training-ready until rendered outputs pass
ocrtech audit-rendered-synthesis-lines. - For split synthesis text, render with
render-synthesis-text-splitso train/eval labels and rendered audits are created together. - Do not use synthetic samples as claim evidence.
- Do not promote Gorkhapatra/Naya Nepal pages to claim evidence until the review audit and reference audit pass.
- Keep Nepali and English non-regression gates in every claim-grade run.
cdjk Remote Synthesis Path
Remote host verified read-only on 2026-06-16:
host: cdjk@100.110.148.43
machine: cdjks-MacBook-Pro.local
disk: /Volumes/TRANSCEND, 3.6Ti total, 2.0Ti available
Confirmed ~/github/llm leads:
/Users/cdjk/github/llm/limbu-speech-toolkit/Users/cdjk/github/llm/rai-speech-toolkit/Users/cdjk/github/llm/dhimal-toolkit/Users/cdjk/github/llm/gurung-language-toolkit/Users/cdjk/github/llm/magar-toolkit/Users/cdjk/github/llm/ltk/Users/cdjk/github/llm/ltk/limbu/Users/cdjk/github/llm/ltk/tamang/Users/cdjk/github/llm/ltk/magar/Users/cdjk/github/llm/nepa-newa-text-frontend/Users/cdjk/github/llm/nepali_text_frontend/Users/cdjk/github/llm/devanagari-OCR/Users/cdjk/github/llm/nllb-better
Confirmed /Volumes/TRANSCEND leads:
/Volumes/TRANSCEND/bible_brain_nepal/lif/LIFWBT_Limbu/Volumes/TRANSCEND/bible_brain_nepal/dhi/DHIDPI_Dhimal/Volumes/TRANSCEND/bible_brain_nepal/gvr/GVRWBT_Gurung/Volumes/TRANSCEND/bible_brain_nepal/new/NEWNCL_Newari/Volumes/TRANSCEND/bible_brain_nepal/taj/TAJWBT_Tamang__Eastern/Volumes/TRANSCEND/bible_brain_nepal/npi/NPIBIB_Nepali/Volumes/TRANSCEND/bible_brain_nepal/npi/NPIDPI_Nepali/Volumes/TRANSCEND/nepali_dataset_prep/Volumes/TRANSCEND/noiseremover_nepal18_timed_pipeline/bible_brain/Volumes/TRANSCEND/noiseremover_nepal18_timed_pipeline/speaker_cluster_limbu
Next remote steps:
- Selectively migrate text corpus/dictionary resources from
cdjk@100.110.148.43into ignored localdata/without writing generated corpus data back to the source machine. Tracked policy:configs/cdjk-language-corpus-migration-v1.json. - Run
ocrtech audit-synthesis-resourceson the migrated roots:passed=True roots=6/6 files=2308 warnings=0. - Prepare and audit first migrated synthesis-text manifests:
Limbu Limdic
26152rows, Limbu-only subset8772rows, Tamang46314rows, and Magar Western dictionary1182rows; all audits passed withclaim_eligible=0. - Render a corrected Limbu-only Namdhinggo smoke set with six degradation
profiles:
120images, rendered auditpassed=True,issues=0,warnings=0. - Convert the same local migration into full split train/eval synthesis manifests with exclusion sets and license policy before large-scale rendering.
- Inspect actual schemas for each candidate file before writing converters. Do not infer fields from filenames.
- Run explicit field-mapped conversion recipes for Gurung, Dhimal, and Rai dialect corpora discovered in the migration.
- Run explicit field-mapped conversion for extra Magar
mgp/source/mscoremonolingual rows andmgp/npilexicon/parallel rows. - Audit 20 migrated synthesis-text manifests after the second sweep:
228212total rows,202560Devanagari rows,8772true Unicode Limbu rows, and all manifest audits passed withclaim_eligible=0. - Add a combined index/report command for migrated synthesis-text manifests
so the aggregate counts do not depend on ad hoc
jqcommands. - Add an explicit-field converter for remote toolkit JSONL/TSV/CSV parallel corpora.
- Add
audit-synthesis-text-promotionso remote synthesis text must carry split policy and must not overlap excluded eval/reference text before rendering. - Add
split-synthesis-textso promoted train/eval synthesis text is grouped bytext_sha256, stamped with split policy, and promotion-audited. - Smoke-run the converter on real remote Magar toolkit JSONL and pass
audit-synthesis-text-manifestwith Magar/Nepali Devanagari gates. - Smoke-run that converter on selected Limbu, Rai, Dhimal, Gurung, and Magar remote toolkit files, then audit the emitted synthesis-text manifests.
- Promote remote toolkit smoke commands into full split synthesis-text
manifests that pass
split-synthesis-textwith exclusion sets and license policy appropriate to the run. - Prioritize remaining converters in this order: Limbu Pangloss XML/TSV, Bible Brain text manifests, Nepali prep TSV/transcript roots.
- Record license state per source. Bible Brain and derived text/audio roots
stay
pending_reviewuntil cleared. - Emit source path, source SHA-256, text SHA-256, language, script, Unicode
block counts, split policy, and
claim_evidence_eligible=falsefor every synthesis text row. - Deduplicate synthesis text against all real validation references before rendering.
- Render promoted train/eval synthesis text with verified fonts and
--require-fontbefore using labels for PaddleOCR training. - Verify fonts and Unicode renderability before rendering new scripts.
- Add Namdhinggo 3.100 as the primary Limbu/Sirijonga font candidate and keep Noto Sans Limbu as a fallback runtime font.
- Add
configs/namdhinggo-font-assets-v1.jsonso Namdhinggo 3.100 is reproducibly provisioned from the upstream SIL release ZIP with package and extracted-member SHA-256 checks. - Add
ocrtech inventory-fontsfor directory/ZIP font collection audits with SHA-256, cmap readability, script coverage, duplicate detection, and JSON/CSV/Markdown outputs. - Inventory current runtime font corpus:
outputs/font-inventory-current-2026-06-16has17font files,7cmap-readable fonts, and0duplicate SHA groups. - Inventory
/Users/cdjk/github/llm/ltk/fonts.zip: foundjanaki.ttfandmadan.ttf; both are cmap-readable but do not match registry Unicode ranges, so they stay legacy/conversion leads rather than OCR rendering fonts. - Probed
cdjk@100.110.148.43under~/github/llmfor font assets and copied direct font files into ignoreddata/fonts/remote-cdjk-2026-06-16. - Inventory remote-copied fonts:
outputs/font-inventory-remote-cdjk-2026-06-16has4readable fonts. Only Noto Sans Devanagari covers registered Devanagari targets; the 8-bit Devanagari font stays a legacy conversion lead. - Download and inventory Google Fonts OFL batch: Kanchenjunga Regular/Bold for Kirat Rai, Noto Sans Tirhuta for Tirhuta, Noto Serif Devanagari for print-style Devanagari, and Noto Sans Ol Chiki Variable for Santhali style diversity.
- Add
configs/nepal-font-assets-v1.jsonplusocrtech prepare-font-assetsso the Google Fonts batch is reproducibly provisioned with SHA-256 checks instead of manual curl commands. - Add
ocrtech audit-font-asset-readinessto require passing asset preparation, font inventory, and all-script renderability evidence before treating a runtime font corpus as synthesis-ready. - Connect rendered synthesis audits to font readiness evidence via
--font-readiness-report; rendered rows now fail training-bound audits when theirfont_sha256is not in the passed readiness manifest. - Reaudit the Namdhinggo-rendered Limbu/Sirijonga train/eval split against
Namdhinggo font readiness evidence:
train=150,eval=27,issues=0,warnings=0. - Add deterministic rendered-line degradation profiles for PaddleOCR
recognizer training:
clean,scan,low_light,uneven_light,glare_light,camera_left,camera_right,camera_top,camera_bottom, andphone_photo. Every generated image now records the selected profile, seed, and parameters in manifest metadata. - Add
ocrtech audit-rendered-degradation-splitto verify expected degradation profiles per text hash and reject train/evaltext_sha256overlap in augmented rendered bundles. - Generate and audit the full Namdhinggo augmented Limbu/Sirijonga rendered
split:
900train rows,162eval rows, six profiles per text hash,issues=0,warnings=0. - Materialize the Limbu/Sirijonga augmentation ablation spec:
configs/limbu-sirijonga-augmentation-ablation-v1.json, with clean, lighting, and camera arms plus training recipe directories for each arm. - Audit all ablation arms for profile balance and train/eval text-hash
isolation: clean
150/27, lighting450/81, camera600/108, allissues=0,warnings=0. - Include illumination and capture-angle robustness in the first
Sirijonga augmentation plan:
low_light,uneven_light,glare_light,camera_left,camera_right,camera_top,camera_bottom, andphone_photoare now deterministic profiles with recorded seeds and parameters. - Upgrade camera/phone-photo profiles from affine shear to deterministic
perspective transforms, and require
source_quad,output_quad, andperspective_coefficientsin rendered-manifest audits. - Tighten rendered-manifest audits so lighting profiles must record numeric brightness/contrast, gradient/blend, or glare metadata instead of relying on profile names alone.
- Add
ocrtech audit-augmentation-ablationso Limbu/Sirijonga synthetic ablation arms cannot be treated as promotion-ready without a passed real Gorkhapatra verified-reference review audit. - Run the current ablation promotion audit:
passed=False arms=3 real_evidence=0 verified_references=0 issues=1. All synthetic arms pass internally, but the intended blocker is missing real verified Gorkhapatra evidence. - Run a controlled Limbu/Sirijonga augmentation ablation: clean-only vs lighting-only vs camera-angle/phone-photo mix, evaluated on real Gorkhapatra/Naya Nepal pages and held-out synthetic rows without text-hash leakage.
- Promote coverage-proven Google Fonts candidates into
languages/nepal-v1.jsonand rerun all-script renderability:passed=True scripts=22 candidates=58 rendered=0. - Collect and inventory additional print-style Unicode Devanagari, Ol Chiki, Kirat Rai, Tibetan/Tamyig, and other Nepal-script fonts with license notes before using them for synthesis diversity.
Gorkhapatra Validation Path
- Start from
https://gorkhapatraonline.com/publications/naya-nepal. - Support paginated Naya Nepal publication discovery with
audit-gorkhapatra-language-pages --publication-pages N; Limbu appears on page 9 in the 2026-06-16 live scan. - Capture article publication dates and language labels.
- Query the epaper endpoint for the matching publication date:
https://epaper.gorkhapatraonline.com/epaper-list?...&slug=gorkhapatra. - Download the direct PDF from the viewer
file=path, not the PDF.js shell. - Search/render the language page, including
bhasa pristha/भाषा पृष्ठcandidates. - Extract PDF-native text spans, font names, bboxes, codepoints, and
embedded fonts before OCR/manual transcription. This is now implemented as
ocrtech extract-gorkhapatra-language-page-pdf-text. - Run the PDF-native extractor on the current Limbu pack:
passed=True samples=2/2 spans=776 target_spans=304 fonts=10 warnings=0. Page 6 (gorkhapatra-language-page-0001-01) has304NamdhinggoSILLspans and extracted embedded fontUFNWLJ+NamdhinggoSILL; page 7 has0target-font spans. - Compare the blog image, PDF page, and PDF font metadata to choose the exact page candidate. Current conclusion: page 6 is the Limbu/Sirijonga page; page 7 is the same-date non-Limbu page.
- Build or locate a Namdhinggo/Gorkhapatra legacy encoding converter to
Unicode Limbu (
U+1900..U+194F) so extracted PDF spans can become reference text instead of manual transcription burden. - Finalize only after manual review accepts exactly one candidate for the publication/language/date group.
- Add a candidate-language filter to the Gorkhapatra language-page review audit/finalize commands so Limbu can be gated independently from other accepted Naya Nepal languages on the same PDF.
- Run the filtered Limbu verified-reference audit:
passed=False samples=2 groups=1 accepted=1 rejected=1 issues=1 warnings=0. The only blocker is empty verified reference text forgorkhapatra-language-page-0001-01. - Add a transcription work-order generator for accepted Gorkhapatra language-page rows so manual labeling is explicit, auditable, and scoped by target language.
- Generate the Limbu work order:
passed=True samples=1 blocked=1 verified=0 warnings=0. The selected work item isgorkhapatra-language-page-0001-01, page 6 of the 2083-02-12 Gorkhapatra PDF. - Correct the work-order priority: for Gorkhapatra epapers, PDF-native extraction is upstream of OCR and manual transcription. The transcription dashboard is fallback/verification only, especially for Limbu/Sirijonga where OCR candidates are currently invalid.
- Add
prepare-gorkhapatra-language-page-assisted-referencesfor non-claim-ready machine OCR draft references. Drafts preservereference_status=draft,claim_evidence_eligible=false,machine_generated=true, andrequires_human_review=true. - Add
prepare-gorkhapatra-language-page-ocr-sidecarsso accepted Gorkhapatra rows can produce reproducible OCR sidecars before assisted reference drafting. - Run PaddleOCR sidecar generation for the current Limbu page on k2:
pages=1,lines=299,nonzero_bbox=299,average_confidence=0.5347613170992171. - Rerun the assisted-reference command on the current Limbu page:
passed=True samples=1 assisted=1 failed=0 skipped=3 warnings=0. The draft now contains298non-empty OCR text lines and remains non-claim-ready. - Add
prepare-gorkhapatra-language-page-verification-bundlefor line-level human correction sheets from assisted references. - Add
apply-gorkhapatra-language-page-verification-bundleto convert a completed line-review CSV into verified.ref.jsonfiles and an updated page-review CSV, while blockingpending,needs_context, unknown, or empty reviewed rows. - Tighten verified-reference application so claim-eligible references also require explicit table, figure, and caption review declarations. Checked absent is acceptable; silently inheriting empty draft arrays is not.
- Generate the current Limbu verification bundle:
passed=True samples=1 lines=298 crops=298 missing_bboxes=0 warnings=0. The CSV review sheet and line crops are ready for human correction. - Add an interactive HTML review dashboard to every verification bundle so
reviewers can scan line crops, ids, order, confidence, and OCR text, set
review_status, enter corrected text, add notes, and export a reviewed CSV that still must pass the audit/apply gates. - Add
audit-gorkhapatra-language-page-verification-csvfor dry-run status counts and blocker reports before writing verified references. - Add
split-gorkhapatra-language-page-verification-csvto produce reviewer-sized CSV/HTML batches while preserving line ids and order. - Add
merge-gorkhapatra-language-page-verification-batchesso reviewed batch CSVs can be recombined only when every original(sample_id, line_id)appears exactly once. - Add
assign-gorkhapatra-language-page-verification-batchesso split batches have an explicit reviewer/due-date ledger for paper-grade provenance. - Run the real Limbu split/merge smoke:
split passed=True samples=1 lines=298 batches=6 batch_size=50 warnings=0; untouched-batch mergepassed=True source_lines=298 merged_lines=298 batch_files=6 issues=0 warnings=0; merged audit still blocks withpending=298. - Run the current Limbu verification CSV audit:
passed=False samples=1 lines=298 ready=0 dropped=0 blocked=298 issues=298 warnings=0; all rows remainpending. - Write structured references with text, reading order, tables, figures, captions, and Markdown where present.
- Run
audit-gorkhapatra-language-page-reviewandaudit-referencesbefore using the pack in claim-grade validation.
Paper Evidence Checklist
- Every benchmark report records input hashes, output artifact hashes, model-card hashes, latency, memory, and fallback usage.
- Every public or real eval row has a parsed and audited reference.
- Claim review rejects stale artifacts, missing references, malformed structured references, and synthetic-only evidence.
- Error analysis includes failures by language, script, layout type, degradation, table presence, and fallback trigger reason.
- Paper tables separate OCR metrics, document-structure metrics, resource metrics, and fallback metrics.
Limbu/Sirijonga Current Evidence
- Built split-safe Limbu Unicode synthesis text:
outputs/sirijonga-limbu-split-2026-06-16/train.jsonlandoutputs/sirijonga-limbu-split-2026-06-16/eval.jsonl. - Split policy is
split_synthesis_text:text_sha256:seed=17:eval_ratio=0.15, with170train rows,30eval rows, and200text-hash groups. - Rendered the pure Limbu/Sirijonga subset with font provenance:
150train lines and27eval lines passed rendered-line audits using Noto Sans Limbu. - Rerendered the same split with Namdhinggo Regular; this is now the preferred Sirijonga training bundle.
- Reaudited the Namdhinggo-rendered split with
--font-readiness-report outputs/font-asset-readiness-namdhinggo-2026-06-16/font-asset-readiness.json; the training bundle is now tied to reproducible Namdhinggo asset provenance. - Regenerated the preferred Namdhinggo split with explicit degradation profiles after confirming the clean split remained reproducible. The degradation split audit confirms clean/degraded variants stay on the same split side.
- Add a Limbu/Sirijonga PaddleOCR dictionary covering
U+1900..U+194Fplus accepted punctuation. - Audit that dictionary against the Namdhinggo-rendered train/eval label
files:
76dictionary characters,68required Limbu-range characters,64observed label characters,0missing required,0missing labels. Label spaces are covered by PaddleOCRuse_space_char=true, not by a literal whitespace dictionary row. - Find or label real Gorkhapatra/Naya Nepal Sirijonga pages before any claim-grade evaluation.
- Found the real Limbu Naya Nepal item:
https://gorkhapatraonline.com/news/207555, date2083-02-12, article imagehttps://gorkhapatraonline.com/storage/media/360954/conversions/6-original.jpg. - Matched that date to Gorkhapatra epaper PDF
2026-05-26-03-43-57-2083-02-12.pdf, SHA-256afe1d3d2e64248745d94d511e5ca456abddd3ad828480baff87263029b987cd8. - Rendered same-date page candidates and visually disambiguated: page 6 is Limbu/Sirijonga, page 7 is Majhi/Devanagari.
- Created an accepted-row reviewer bundle for manual reference labeling:
data/limbu-gorkhapatra-language-page-reviewer-bundle-2026-06-16, with2images and2draft references. - Created a Limbu transcription work order:
data/limbu-gorkhapatra-language-page-transcription-work-order-2026-06-16. It confirms one accepted Limbu page is blocked only by the empty reference JSON, not by missing image/PDF provenance. - Created a Limbu assisted-reference draft run:
data/limbu-gorkhapatra-language-page-assisted-references-2026-06-16. It now contains PaddleOCR-assisted text suggestions from k2 and keeps the draft non-claim-ready. - Created a Limbu line-level verification bundle:
data/limbu-gorkhapatra-language-page-verification-bundle-2026-06-16, with298line crops, a pending review CSV, and an HTML review contact sheet. - Confirmed the correct PDF-native browser review surface:
file:///Users/cdjk/github/llm/ocr-tech/data/limbu-gorkhapatra-language-page-pdf-text-2026-06-16/gorkhapatra-language-page-pdf-text-dashboard.html. Use this full-page clickable PDF-span dashboard for verification. Do not use the PDF-native verification-bundle HTML as the review UI; that was identified as the wrong/broken interface. - Fill the verification CSV with explicit
accept,corrected, ordropline statuses, either directly or viasplit-gorkhapatra-language-page-verification-csvbatches. If batches are used, merge them withmerge-gorkhapatra-language-page-verification-batches. Then runaudit-gorkhapatra-language-page-verification-csv, then runapply-gorkhapatra-language-page-verification-bundleto produce verified references and an updated review CSV with--tables-reviewed,--figures-reviewed, and--captions-reviewedafter those structures are checked. - Fill verified text/layout references for the accepted Limbu page; strict review audit currently fails as intended until that reference is complete.
- Re-run the Limbu-only gate after transcription:
audit-gorkhapatra-language-page-review ... --candidate-language लिम्बू --require-verified-references.