|
Download ocr-tech/paper-buildkit/ARTIFACTS.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 20.1 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/paper-buildkit/ARTIFACTS.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/paper-buildkit/ARTIFACTS.md
-
curl -L -o ARTIFACTS.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/paper-buildkit/ARTIFACTS.md
20.1 kB
| # Limbu paper claim-to-artifact ledger | |
| “Verified” below means the value was read from a surviving artifact and is | |
| checked by `paper/verify_claims.py`. It does not make machine-converted silver | |
| references equivalent to human ground truth. | |
| ## A. Sirijonga real-print source and split | |
| **Verified values:** 16 source dates; 4,872 source line pairs; 38 cross-split | |
| exact-label rows removed; 4,834 retained rows; train/dev/test counts of | |
| 2,711/1,221/902 over 9/4/3 dates. | |
| **Artifact:** | |
| `outputs/limbu-sirijonga-silver-pack-v2-2026-07-05/pack-summary.json` | |
| **Identity:** pack content SHA-256 | |
| `e9c7851851c9613794cf0c0e5d72659186077ddebee650b7443c03969fe12e9a`. | |
| **Boundary:** `gt_status=pdf_text_layer_silver` and | |
| `claim_evidence_eligible=false`. Text is converted from a Namdhinggo legacy | |
| layer through `data/mappings/limbu-legacy/Limbu.map`, not transcribed by a | |
| human reviewer. | |
| ## B. ft-v2 training payload and configuration | |
| **Verified values:** 25,065 retained synthetic rows; 3,090 missing synthetic | |
| rows excluded; 2,711 real training lines replayed eight times; 46,753 training | |
| rows; 1,221 development rows; 40 configured epochs; batch size 64; learning | |
| rate $10^{-4}$; input 3x48x320; maximum text length 128. | |
| **Artifacts:** | |
| - `outputs/limbu-ft-v2-payload-2026-07-05/limbu-ft-v2-payload.tgz`, members | |
| `limbu-ft/payload-summary.json` and `limbu-ft/limbu-sirijonga-ft-v2.yml`; | |
| - `outputs/limbu-ft-v2-run-2026-07-06/checkpoints/config.yml`; and | |
| - `outputs/limbu-ft-v2-run-2026-07-06/train.log`. | |
| **Recorded history:** `RESEARCH.md` records approximately 8.5 L4 GPU-hours and | |
| best epoch 39. The configuration and selected checkpoint survive; the paper | |
| build does not repeat training. | |
| ## C. Date-disjoint line recognition | |
| **Verified values:** | |
| | System | Lines | Codepoint CER | Grapheme CER | Exact-line accuracy | | |
| |---|---:|---:|---:|---:| | |
| | Previous | 902 | 0.013787556730051128 | 0.028400954653937948 | 0.6374722838137472 | | |
| | ft-v2 | 902 | 0.004739472625955076 | 0.009785202863961813 | 0.8592017738359202 | | |
| **Artifacts:** | |
| - `outputs/limbu-ft-v2-run-2026-07-06/score-champion-frozen-test/metrics.json` | |
| - `outputs/limbu-ft-v2-run-2026-07-06/score-ftv2-frozen-test/metrics.json` | |
| **Derived:** codepoint CER relative reduction 65.625%; grapheme CER relative | |
| reduction 65.546%; exact-line accuracy gain 22.173 percentage points. | |
| **Boundary:** the split is frozen and date-separated for this diagnostic, but | |
| the labels are unreviewed silver. The previous recognizer's older training | |
| history may include some of these dates; ft-v2's dates are disjoint. | |
| ## D. Non-oracle page diagnostic on the line-test dates | |
| The page subset is deterministically selected by | |
| `paper/build_heldout_page_subset.py` from dates 2082-03-24, 2082-08-12, and | |
| 2082-12-25. It reuses already-produced document JSON and does not rerun OCR. | |
| **Verified ft-v2 values:** 3 pages; 907 reference lines; 904 predictions; 877 | |
| matches; detection F1 0.9685256764218665; page codepoint CER | |
| 0.025746508829373408; page grapheme CER 0.03360869073214892; matched-line | |
| codepoint CER 0.01012004871542075; matched-line exact accuracy | |
| 0.7616875712656784. | |
| **Verified previous-system values:** 3 pages; 907 reference lines; 903 | |
| predictions; 877 matches; detection F1 0.969060773480663; page codepoint CER | |
| 0.03957126465731956; page grapheme CER 0.06280411904492475; matched-line | |
| codepoint CER 0.025691584991010846; matched-line exact accuracy | |
| 0.45952109464082097. | |
| **Artifacts:** | |
| - `paper/evidence/limbu-page-test-dates/eval.jsonl` | |
| - `paper/evidence/limbu-previous-heldout-page-score/metrics.json` | |
| - `paper/evidence/limbu-ftv2-heldout-page-score/metrics.json` | |
| - each score directory's `page-eval-run.json`, `claim-safety.md`, line | |
| alignment, text pairs, and review files. | |
| **Boundary:** date-disjoint, non-oracle, and real-print, but only three pages; | |
| the references are silver, the subset is not a claim pack, and the scorer | |
| correctly reports `claim_safe=false`. | |
| ## E. Sixteen-page engineering gate | |
| **Verified ft-v2 values:** 16 pages; page codepoint CER 0.027222632149080803; | |
| page grapheme CER 0.034681104813771124; detection F1 | |
| 0.964986138207208; matched-line codepoint CER 0.0089806695569971; | |
| matched-line exact accuracy 0.7927218557139817. | |
| **Verified previous-system values under shared column-major scoring:** page | |
| codepoint CER 0.04328827626628151; page grapheme CER | |
| 0.06713527319549546; detection F1 0.9656872816930347; matched-line codepoint | |
| CER 0.026572279207232267; matched-line exact accuracy 0.4340425531914894. | |
| **Artifacts:** | |
| - `outputs/limbu-ftv2-page-gate-2026-07-06/score-v2/metrics.json` | |
| - `outputs/limbu-column-major-2026-07-05-score-v2/metrics.json` | |
| **Boundary:** 13 of the 16 dates participate in ft-v2 training or development. | |
| This is a deployment regression gate, not independent evaluation. | |
| ## F. Page-linearization correction | |
| **Verified values:** historical native serialization has codepoint CER | |
| 0.4134103646953546; shared column-major serialization has codepoint CER | |
| 0.04328827626628151. Both use 4,839 predictions and 4,700 matched lines from | |
| the earlier recognizer. | |
| **Artifacts:** | |
| - `outputs/limbu-native-regression-check-2026-07-05/metrics.json` | |
| - `outputs/limbu-column-major-2026-07-05-score-v2/metrics.json` | |
| **Boundary:** this is a metric-definition correction. The values are not two | |
| model runs and cannot be presented as model improvement. | |
| ## G. Source-pixel and representation verification | |
| `paper/LIMBU_SOURCE_VERIFICATION.md` records exact paths, hashes, commands, and | |
| bounded manual observations for: | |
| - a real six-column Gorkhapatra Sirijonga page used in the page test; and | |
| - CDC mathematics PDF page 8, whose pixels are visually Sirijonga while its | |
| curriculum reference contains Devanagari Unicode. | |
| The CDC finding excludes the historical 31-crop result from the verified | |
| Devanagari OCR tables. Its raw score directory was lost; only the packaged model | |
| and handoff ledger survive. The result is properly described as cross-script | |
| transcription evidence. | |
| ## H. Runtime artifact identity | |
| **Artifacts:** | |
| - `spaces/limbu-ocr/models/limbu-sirijonga-ft-v2/model-card.json` | |
| - `spaces/limbu-ocr/models/limbu-sirijonga-ft-v2/artifacts/` | |
| - `spaces/limbu-ocr/app.py` | |
| The model card binds the required inference JSON, parameters, YAML, and | |
| dictionary by SHA-256. The application points to ft-v2 and deva-v2 and exposes | |
| line, Paddle page, and Paddle book paths. This verifies the local payload, not | |
| the current network status of a hosted service. | |
| ## I. Figures | |
| `paper/build_figures.py` reads the recorded artifacts in this ledger and writes | |
| PDF, SVG, and 300-DPI PNG versions of: | |
| - `research-program`; | |
| - `system-architecture`; | |
| - `limbu-source-evidence`; | |
| - `limbu-recognizer-results`; | |
| - `limbu-page-results`; | |
| - `limbu-linearization-diagnostic`; and | |
| - `language-pack-status`. | |
| No publisher or curriculum pixels are embedded. The first two figures frame the | |
| delivered platform, human-review loop, scientific completion gates, and longer | |
| research horizon; the remaining figures are generated from recorded evidence. | |
| ## J. Local application packaging repair | |
| The application uses `models/limbu-sirijonga-ft-v2/artifacts` and | |
| `models/deva-v2/artifacts`. During the paper audit, its static artifact preflight | |
| was found to expect the obsolete Devanagari filename `nep_eng_dict.txt` even | |
| though the current model card binds `devanagari-v2.txt`. The check and Space | |
| documentation were corrected. `paper/verify_claims.py` now requires all eight | |
| model files named by the two current bundles to exist and requires the corrected | |
| dictionary path in `app.py`. | |
| ## K. AI-detector receipt | |
| The old detector receipt was bound to a superseded manuscript with the wrong | |
| scope and is invalid for this paper. `paper/AI_INTEGRITY_CHECK.md` remains red | |
| until the corrected PDF is built and checked on the requested remote host. | |
| ## L. Newar-Devanagari normalized pack splits | |
| **Verified values:** train/dev/test contain 4,096/1,528/1,139 rows. | |
| **Artifact:** | |
| `outputs/newar-pack-v1-normalized-2026-07-15/stats.json` | |
| (`train.total_lines`, `dev.total_lines`, and `test.total_lines`). | |
| **Boundary:** these are inventory counts for the recovered normalized pack, not | |
| evidence that its labels, provenance, or split isolation are claim-grade. The | |
| independent admission decision in section P applies to all three splits. | |
| ## M. Newar ft-v1 line diagnostic | |
| **Verified values:** 1,139 lines; codepoint CER | |
| 0.04655477596980231; grapheme CER 0.09569411931616656; exact-line accuracy | |
| 0.24495171202809482. | |
| **Artifact:** | |
| `outputs/newar-ft-v1-verify-omac-2026-07-07/metrics.json`, under `metrics`. | |
| **Boundary:** this is a diagnostic receipt, not claim-grade CER. The receipt | |
| records a host-local label path but no label-file hash, and the audited source | |
| pack is `NOT_ADMITTED`; the number therefore cannot override the boundary in | |
| section P. | |
| ## N. deva-v2 Newar line baseline | |
| **Verified values:** 1,139 lines; codepoint CER | |
| 0.062229425689364695; grapheme CER 0.13119812726111937; exact-line accuracy | |
| 0.15803336259877085. The verifier also requires its recorded codepoint and | |
| grapheme reference lengths to equal the ft-v1 receipt's reference lengths. | |
| **Artifact:** | |
| `outputs/deva-v2-reexport-2026-07-06/score-newar-test/metrics.json`, under | |
| `metrics`. | |
| **Boundary:** this is an internal diagnostic baseline. The two metric receipts | |
| name different absolute label paths and neither supplies a label hash, so their | |
| matching row and reference-length fields do not establish a cryptographically | |
| bound independent benchmark. | |
| ## O. Newar non-oracle page diagnostic | |
| **Verified values:** 19 pages; page codepoint CER | |
| 0.19918426853185692; page grapheme CER 0.27885891823188813; line-detection F1 | |
| 0.911629205640095; matched-line codepoint CER 0.09052417830487239. | |
| **Artifact:** | |
| `outputs/newar-page-eval-v2-2026-07-05/newar-ft-v1/score-v2/metrics.json`. | |
| **Boundary:** the same artifact records zero claim-eligible rows and classifies | |
| all 19 references as `nayanepal_font_mapping_silver_unreviewed`. These are page | |
| diagnostics against unreviewed silver, not claim-grade accuracy. Its leakage | |
| block also records `checked=false` with no training manifests supplied. | |
| ## P. Newar claim-boundary and leakage audit | |
| **Verified values:** train versus dev+test contains 18 exact-overlap left rows | |
| and 575 left rows sharing a substring of at least 16 codepoints, including the | |
| exact overlaps. | |
| **Artifact:** | |
| `outputs/newar-devanagari-normalized-claim-boundary-audit-2026-07-15-r12/claim-boundary-audit.json`. | |
| **Boundary:** the audit reports leakage status `FAIL`, overall status | |
| `NOT_ADMITTED`, `claim_grade=false`, `train_permitted=false`, | |
| `eval_permitted=false`, and `cer_permitted=false`. It also records missing | |
| row-level source provenance, Gorkhapatra ledger status `RISK`, absent qualified | |
| native review, and unreconstructable date/edition isolation. Sections L--O must | |
| not be promoted from diagnostics to publication claims. | |
| ## Q. Videha Tirhuta real-print silver | |
| **Verified value:** 10,696 kept lines; the crop archive also records 10,696 | |
| file entries and verified archive/label parity. | |
| **Artifact:** | |
| `outputs/videha-tirhuta-silver-v2-2026-07-17/summary.json`, under `totals`, | |
| `artifacts.crop_archive`, and `verification`. | |
| **Boundary:** this is mechanically converted silver substrate. The count and | |
| archive parity do not make the labels native-reviewed ground truth or establish | |
| recognizer accuracy. | |
| ## R. Tirhuta fine-tune payload | |
| **Verified values:** status `READY`; 9,765 real training rows; 844 real | |
| development rows. The progress receipt also says training was not launched. | |
| **Artifact:** | |
| `outputs/tirhuta-v2-ft-payload-2026-07-17/PROGRESS.md`. | |
| **Boundary:** `READY` is a payload-readiness state under the recorded Videha | |
| license caveat, not a trained-model or evaluation result. | |
| ## S. Tirhuta synthetic render archive | |
| **Verified value:** 48,960 PNG members. The verification receipt has | |
| `passed=true`, an equal expected count, and an empty issue list. | |
| **Artifact:** | |
| `outputs/tirhuta-synth-v1-prep-2026-07-17/audits/render-archive-verification.json`. | |
| **Boundary:** this is a synthetic pretraining inventory and archive-integrity | |
| result. It says nothing about real-print recognition quality. | |
| ## T. Ol Chiki real evaluation pack v3 | |
| **Verified values:** 895 candidate lines; 123 excluded blank renders; 772 clean | |
| evaluation lines. The verifier requires the clean and excluded counts to | |
| reconcile exactly to the candidate total. | |
| **Artifacts:** | |
| - `data/olchiki-real-eval-pack-v3/summary.json`; and | |
| - `docs/DATA_LICENSING.md` for the source-specific rights boundary. | |
| **Boundary:** the pack summary is an inventory artifact and contains no | |
| claim-eligibility field. It combines a public-marked issue with the | |
| September-2023 issue whose license is recorded as unspecified, so the full pack | |
| remains diagnostic rather than claim-grade. | |
| ## U. Ol Chiki v2 line diagnostic | |
| **Verified values:** 772 lines; codepoint CER 0.08263840048226666; exact-line | |
| accuracy 0.2033678756476684. | |
| **Artifact:** | |
| `outputs/olchiki-v2-run-2026-07-07/score-real-print-v3pack/metrics.json`, under | |
| `metrics`. | |
| **Boundary:** this is explicitly diagnostic-only. The mixed-rights evaluation | |
| boundary in section T and the recorded pending crop, label, punctuation, | |
| leakage, and asset-independence review prevent a claim-grade interpretation. | |
| ## V. Ol Chiki v3 fine-tune payload | |
| **Verified values:** 5,307 total images: 242 real training, 5,000 synthetic | |
| training, and 65 real development images. | |
| **Artifact:** | |
| `outputs/olchiki-v3-ft-payload-2026-07-17/olchiki-v3-ft-payload.tgz`, exact | |
| member `metadata/assembly-manifest.json`, under `payload_counts`. | |
| **Boundary:** the manifest records `training_launched=false`; these are staged | |
| payload counts, not v3 model or accuracy evidence. Its license boundary keeps | |
| the unspecified-license September issue evaluation-only. | |
| ## W. Sikkim Herald silver-pack yields | |
| **Verified values:** 9,743 clean Sunuwar lines; 3,699 clean Kirat Rai lines; | |
| zero clean Lepcha lines. | |
| **Artifacts:** | |
| - `outputs/sikkim-silver-packs-2026-07-16/sunuwar/summary.json`; | |
| - `outputs/sikkim-silver-packs-2026-07-16/kiratrai/summary.json`; and | |
| - `outputs/sikkim-silver-packs-2026-07-16/lepcha/summary.json`. | |
| **Boundary:** the summaries' readiness verdict is mechanical converter/archive | |
| readiness. The rows are silver, not native-reviewed labels; the separate legal | |
| gate in section X still controls training. | |
| ## X. Sunuwar and Kirat Rai fine-tune payloads | |
| **Verified values:** Sunuwar has 8,422 real training and 514 real development | |
| rows; Kirat Rai has 2,865 real training and 193 real development rows. | |
| **Artifacts:** | |
| - `outputs/sunuwar-v1-ft-payload-2026-07-17/PROGRESS.md`; and | |
| - `outputs/kiratrai-v1-ft-payload-2026-07-17/PROGRESS.md`. | |
| **Boundary:** both receipts report `BLOCKED-PENDING-LEGAL` and | |
| `Training launched: false`. These payloads are legal-blocked staging artifacts, | |
| not training or evaluation claims. | |
| ## Y. Lepcha real-evaluation anchor | |
| **Verified value:** 132 line crops, stored as `lines=132`. | |
| **Artifact:** `data/lepcha-real-eval-pack/README.json`. | |
| **Boundary:** the artifact encodes `claim_evidence_eligible=false` inside its | |
| `label_status` string rather than as a Boolean field. The labels are | |
| machine-converted and human verification is pending. | |
| ## Z. Lepcha zero-yield root cause | |
| **Verified artifact:** | |
| `outputs/lepcha-zero-yield-diagnosis-2026-07-17/REPORT.md` exists and records | |
| that the article bodies are rasterized as image XObjects. | |
| **Boundary:** this is a source-format diagnosis. It supports a raster/OCR or | |
| transcription route, not a converter yield, label, model, or accuracy claim. | |
| ## AA. Newa CTC-T200 preparation | |
| **Verified values:** exact and long-substring filtering removed 430 unique | |
| training rows and left 1,961; the patched train and evaluation paths both emit | |
| `T=200`. | |
| **Artifact:** `outputs/newa-ctc200-prep-2026-07-15/PROGRESS.md`. | |
| **Boundary:** the receipt states that no GPU launch is authorized. The sealed | |
| Cambridge evaluation labels still require native pixel-level review, so this is | |
| recipe and payload evidence rather than a trained-model result. | |
| ## AB. Gurung Khema mechanical eligibility audit | |
| **Verified values:** 759 mechanical candidates; 749 mechanically eligible; 10 | |
| mechanically rejected. | |
| **Artifact:** | |
| `outputs/sikkim-herald-gurung-khema-r13-adversarial-bundle-audit-2026-07-15-r13/adversarial-audit.json`. | |
| **Boundary:** both top-level and result status are | |
| `PASS_READY_NOT_ADMITTED`; admission is `NOT_ADMITTED`, `claim_grade=false`, | |
| and `labels_present=0`. “Eligible” means mechanically convertible, not reviewed | |
| or admitted training data. | |
| ## AC. Gorkhapatra monthly backfill | |
| **Verified values:** the exact glob | |
| `outputs/gorkhapatra-backfill-*/silver-line-packs/silver-pack-summary.json` | |
| matches 17 files, whose `total_kept_lines` sum to 515,586. | |
| **Artifacts:** those 17 monthly `silver-pack-summary.json` files, spanning the | |
| surviving March-2024 through July-2025 backfill directories. | |
| **Boundary:** this is converted silver inventory, not language-verified or | |
| claim-grade evidence. The summaries have no claim-status field, and | |
| Gorkhapatra training, numerical-publication, and redistribution rights remain | |
| `RISK` in `docs/DATA_LICENSING.md`. | |
| ## AD. Per-language silver rebuild receipts | |
| **Verified values:** the last line of batch 1 records `kept=122882`; the last | |
| line of batch 2 records `kept=184089`; their sum is 306,971. | |
| **Artifacts:** | |
| - `outputs/silver-lines-rebuild-b1-2026-07-05.log`; and | |
| - `outputs/silver-lines-rebuild-b2-2026-07-05.log`. | |
| **Boundary:** these terse completion logs establish generated silver-row | |
| counts only. They do not establish per-language native review, rights, split | |
| isolation, or accuracy. | |
| ## AE. Nepal language-pipeline registry | |
| **Verified value:** `languages` is a JSON array of length 50. | |
| **Artifact:** `configs/nepal-language-pack-pipeline-v1.json`. | |
| **Boundary:** a 50-language configuration describes registry breadth, not 50 | |
| trained, evaluated, rights-cleared, or claim-grade language systems. | |
| ## AF. Manuscript value and scope binding | |
| `paper/verify_claims.py` requires `paper/DRAFT.md` to contain the formatted | |
| values backed by sections L--AE, including the 50-language, 515,586, 306,971, | |
| T=200, payload, diagnostic, and silver-pack tokens. It rejects the known-wrong | |
| tokens `0.0441`, `299,611`, `0.0961`, and `5,330`, while retaining the existing | |
| Limbu boundary phrases `cross-script transcription`, `operational research | |
| platform`, and `review burden`. | |
| **Boundary:** this is a manuscript-consistency gate, not independent evidence. | |
| Every number remains subject to its source artifact and boundary above. | |
| ## AG. Language-pack status figure | |
| `paper/build_figures.py` writes non-empty PDF, SVG, and PNG versions of | |
| `paper/figures/language-pack-status`. `paper/verify_claims.py` requires all | |
| three alongside the six existing figure stems. | |
| **Boundary:** file presence and nonzero size verify build output only. The | |
| underlying values and status labels remain gated by their source checks and do | |
| not turn silver or diagnostic substrate into reviewed ground truth. | |
| ## Publication gate | |
| Before the diagnostic values become submission claims, the project needs: | |
| - a frozen, native-reviewed Sirijonga reference set; | |
| - at least 30 independent page units across source groups; | |
| - an explicit near-duplicate and train/evaluation leakage audit; | |
| - external baselines on identical inputs; | |
| - pack-wide resolution of the Sikkim Herald legal gate, including source-host | |
| eligibility, third-party screening, required attribution, and clearance of | |
| the `BLOCKED-PENDING-LEGAL` payloads; | |
| - qualified native pixel-label review, frozen separately for every script whose | |
| rows or metrics would appear as a claim; | |
| - resolution of the remaining Gorkhapatra redistribution question. The | |
| numerical-publication question was resolved by a recorded owner | |
| determination (2026-07-22, `docs/DATA_LICENSING.md`): derived metrics and | |
| counts may be published; pixel and crop redistribution remains withheld | |
| pending a formal license statement; and | |
| - a separate benchmark of visibly Devanagari-written Limbu pages. | |