Add 1000 files (ocr-tech/tests/test_audit_limbu_book_missed_line_review_coverage.py ...)
d82db76 verified |
Download ocr-tech/tools/deva_rec_torch/README.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 11.7 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/tools/deva_rec_torch/README.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/tools/deva_rec_torch/README.md
-
curl -L -o README.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/tools/deva_rec_torch/README.md
11.7 kB
| # Devanagari recognizer fine-tune (`deva_rec_torch`) | |
| Fine-tunes PaddleOCR's official `devanagari_PP-OCRv5_mobile_rec` for Nepali documents and | |
| serves the result through the unchanged PaddleX runtime. The model that came out of this | |
| pipeline, **devanagari-ft-v3**, has served the Devanagari route of the k2 OCR demo | |
| (`ampixa.thapa-ashish.com.np`) since 2026-09-24. Results and history are in | |
| [`RESEARCH.md`](../../RESEARCH.md) ("Devanagari recognizer fine-tune on k2"); every data | |
| source and its licence is in [`docs/DEVANAGARI_RESOURCES_INVENTORY.md`](../../docs/DEVANAGARI_RESOURCES_INVENTORY.md) | |
| and [`docs/DATA_LICENSING.md`](../../docs/DATA_LICENSING.md). | |
| ## Why PyTorch, not Paddle | |
| Paddle training on k2 (Apple M2 Ultra, macOS arm64) runs at about 2 images/s: the PPLCNetV3 | |
| depthwise convolutions have no fast CPU kernel there and the wheel has no MKLDNN. The | |
| `transformers` 5.17 PyTorch port of the same network (`PPOCRV5MobileRecForTextRecognition`) | |
| trains at 60-90 images/s on MPS. The pipeline is therefore: | |
| ``` | |
| PaddlePaddle/devanagari_PP-OCRv5_mobile_rec_safetensors | |
| -> train.py (CTC, MPS) -> HF checkpoint (model.safetensors) | |
| -> export_onnx.py -> inference.onnx + the official inference.yml | |
| -> paddlex.create_model(..., engine="onnxruntime") (same pre/post-processing as the official model) | |
| ``` | |
| The ONNX graph keeps the official I/O contract: input `x` `[N, 3, 48, W]` float32, output | |
| `fetch_name_0` `[N, T, 570]` probabilities (blank + 568 characters + space). | |
| ## Files | |
| | File | Purpose | | |
| |---|---| | |
| | `common.py` | PaddleX-exact preprocessing (`resized_width`, `padded_width`, BGR, `(x/255-0.5)/0.5`), CTC loss/greedy decode, dictionary loading | | |
| | `train.py` | Weighted multi-source CTC fine-tune; width-bucketed batches; frozen BatchNorm statistics; `--guard-dev` no-regression model selection | | |
| | `selection.py` | Guarded checkpoint selection (dependency-free; tested by `tests/test_deva_rec_selection.py`) | | |
| | `export_onnx.py` | HF checkpoint -> PaddleX-servable ONNX directory, verified against torch at widths 320/480/1200 | | |
| | `serve_predict.py` | Predicts eval sets through PaddleX `create_model` (the served path); engine `paddle_static` or `onnxruntime` | | |
| | `score.py` | Scores prediction TSVs with `tools/eval_metrics.py` (micro codepoint + grapheme CER, exact line) | | |
| | `torch_predict.py` | Fast torch-side predictions (equal pad widths per batch) | | |
| | `paddle_predict.py`, `parity.py` | Torch-vs-Paddle parity check of the port | | |
| | `extract_nvidia_en.py` | English line crops from one NVIDIA OCR-Synthetic-Multilingual-v1 HDF5 shard | | |
| | `recipes/cdjk/` | Exact data-build commands run on cdjk (`~/deva-ft-build`, `~/hw-data`), the OFL font selection, the `chunks-v4` build summary | | |
| | `recipes/k2/` | Exact environment setup, training runs v1b -> v2 -> v3 -> v4, eval-set registry and eval runners used on k2 (`~/ocr-deva-ft`), pinned venv requirements | | |
| | `results/` | Scored comparisons behind the published numbers (`final_comparison_v3.json`, `hw_baseline.json`) | | |
| | `page_probe/` | Page-level smoke test on two real government letters: layout probe, crop-strategy probe, hand-written GT, results | | |
| Data builders that live one level up, in `tools/`: | |
| `build_gorkhapatra_devanagari_chunks.py` (real print chunks from Gorkhapatra PDFs), | |
| `render_devanagari_form_synth.py` (synthetic form/office/real-text lines, OFL fonts only), | |
| `relabel_with_kra_fix.py` (corrects labels built before the क्र converter fix), | |
| `compose_dhcd_digit_strings.py` (handwritten digit strings from DHCD). | |
| ## Hosts and environments | |
| | Host | Path | Role | | |
| |---|---|---| | |
| | cdjk (Linux) | `~/deva-ft-build/` | Real-chunk and synthetic data builds (CPU) | | |
| | cdjk | `~/hw-data/` | Handwriting downloads and set preparation | | |
| | k2 (Mac Studio, MPS) | `~/ocr-deva-ft/` | Training (`.venv-torch`), served-path evaluation (`.venv-serve`), exports, predictions | | |
| | k2 | `~/ampixa-demo/models/ocr/space/models/` | Deployed models (not a git repo) | | |
| | Khalti T4 (`ampixa@202.51.0.7`, 2x Tesla T4, CUDA) | `~/ocr/` | Training home since 2026-10-01: copy of k2's OCR work plus a git clone of this repo; recipes in `recipes/t4/` | | |
| Do not train or build data on the 18 GB orchestration Mac. k2's GPU is shared with another | |
| project's ASR training: check `ps aux | grep -E "train|asr-node"` before an MPS run, and never | |
| stop another project's job. | |
| On the T4 box, `source recipes/t4/env.sh` (or run the `recipes/t4/*.sh` scripts, which source it): | |
| it sets every path under `~/ocr`, puts `tools/` on `PYTHONPATH` and pins `CUDA_VISIBLE_DEVICES=0`, | |
| because GPU 1 runs another project's job on that shared account. `recipes/t4/setup_env.sh` | |
| builds both venvs inside `~/ocr` (CUDA 12.6 torch 2.14.0; driver 565 supports CUDA <= 12.7); | |
| `recipes/t4/smoke_train.sh` checks CUDA CTC against CPU and runs a short guarded training. | |
| The SSH key for that box (`~/.ssh/khalti-t4`) lives on mbp2 only. | |
| Environments (Python 3.11): `recipes/k2/setup_torch.sh` builds `.venv-torch` and downloads the | |
| official safetensors and ONNX repos; pinned versions are in `recipes/k2/torch-requirements.txt` | |
| (torch 2.14.0, transformers 5.17.0, onnxruntime 1.30.0) and `recipes/k2/serve-requirements.txt` | |
| (paddlepaddle 3.3.1, paddlex 3.7.1, paddleocr 3.7.0, onnxruntime 1.30.0). The scripts import | |
| `eval_metrics` from `tools/`; on k2 the scripts and a copy of `eval_metrics.py` sit together in | |
| `~/ocr-deva-ft/torchft2/`. From a repo checkout, run with `PYTHONPATH=tools`. | |
| ## Pipeline | |
| ### 1. Data (cdjk) | |
| Label files everywhere use the PaddleOCR format `relative/image/path<TAB>text`. | |
| 1. **Real print chunks** (`recipes/cdjk/run_chunks_v4.sh`): 24 Gorkhapatra editions, | |
| split by edition (test 2082-03-24, 2082-08-12, 2082-12-25; dev 2081-12-12, 2083-01-11), | |
| labels converted from the `nayanepal` text layer with `npttf2utf==0.3.7` plus the | |
| `j|m -> s|` क्र fix. Result `chunks-v4`: 125,017 / 15,570 / 21,821 chunks (build stats in | |
| `recipes/cdjk/chunks-v4-summary.json`). Labels are PDF-text silver, not human-reviewed. | |
| 2. **Synthetic lines** (`recipes/cdjk/run_synth*.sh`): citizenship (ne/en), office and | |
| real-text subsets in 44 OFL train families; 4 held-out families (Sura, Khula, Dekko, Sarala) | |
| for test; 8 Latin families (`latin_train`) for English renders; Noto held out. | |
| 3. **English replay**: `extract_nvidia_en.py` on `en/validation/validation_000.h5` | |
| (CC BY 4.0; provenance per crop); `recipes/k2/build_v2_lists.py` removes English renders | |
| whose text leaks into the English dev/test lines. | |
| 4. **Label hygiene**: `recipes/k2/build_v3_lists.py` drops synthetic lines that carry the old | |
| क्र->व्र converter error; `relabel_with_kra_fix.py` corrects older line labels (E1b). | |
| 5. **Handwriting** (`recipes/cdjk/fetch.sh`, `fetch_iiit.sh`, `prep_hw_sets.py`, | |
| `recipes/k2/prep_iiit_k2.py`, `tools/compose_dhcd_digit_strings.py`): H1 Nepali words | |
| (CC0), H2 IIIT-HW-Dev Hindi words (**research/educational use only**), H3 IndicScan (CC BY | |
| 4.0, test only), H4 DHCD digit strings (CC BY 4.0; train strings from DHCD Train digits, | |
| test strings from DHCD Test digits). | |
| ### 2. Train (k2) | |
| ```bash | |
| cd ~/ocr-deva-ft/torchft2 | |
| ../.venv-torch/bin/python train.py \ | |
| --model-dir <init checkpoint> --out ../runs/<run> \ | |
| --train "name:root:labels:weight:aug" [...] \ | |
| --dev "name:root:labels" [...] \ | |
| --guard-dev real_dev --guard-dev synth_dev --guard-dev en_dev --guard-tol 0.002 \ | |
| --epochs 3 --batch-size 64 --lr 5e-5 --warmup-steps 300 --eval-every 1500 --max-len 60 | |
| ``` | |
| - `weight` is a per-epoch sampling multiplier of that source's size (0.35 = a random 35% each | |
| epoch, 4.0 = every line four times); `aug` is the probability of capture augmentation. | |
| - BatchNorm running statistics stay frozen: width-bucketed batches have very different | |
| padding fractions, and live statistics made evaluation unstable (v1b step 4000). | |
| - `--guard-dev`: a checkpoint can only become `best` while every guard set's codepoint CER is | |
| within `--guard-tol` of its step-0 value; eligible checkpoints are ranked by the mean CER of | |
| the other dev sets. Use it whenever a run adds a new domain (e.g. handwriting) on top of a | |
| deployed model. Without it the plain mean lets a large gain on the new domain hide a loss | |
| on print. | |
| - Lineage: `run_train_v1b.sh` (from the official safetensors) -> `run_train_v2.sh` -> | |
| `run_train_v3.sh` (on `chunks-v4`, the deployed model) -> `run_train_v4.sh` (handwriting + | |
| about 55% print replay per epoch, guarded; not run yet). | |
| - Launch long runs detached (`nohup ... &`) and poll the log; `train_log.jsonl` in the run | |
| directory records every evaluation. | |
| ### 3. Export | |
| ```bash | |
| ../.venv-torch/bin/python export_onnx.py ../runs/<run>/best \ | |
| ../hf/devanagari_PP-OCRv5_mobile_rec_onnx ../export/<run>-onnx | |
| ``` | |
| The export fails unless the ONNX output matches torch (max |diff| < 1e-3) at several widths. | |
| ### 4. Evaluate on the served path | |
| ```bash | |
| ~/ocr-deva-ft/run_serve_eval.sh ~/ocr-deva-ft/export/<run>-onnx <pred_name> # 12 frozen sets from evalsets.sh | |
| ``` | |
| Every model, including the official one, is scored through PaddleX `create_model` one image | |
| at a time, exactly like the worker's line path, and scored with `tools/eval_metrics.py`. Never | |
| report PaddleOCR's `norm_edit_dis` as CER. `run_hw_baseline.sh` does the same for the | |
| handwriting test sets H1-H3. | |
| A new model must beat the official model on the new domain and stay within 0.2 points of the | |
| deployed model on E1v4, E1b, E7, E3, E4 and E6 before it is deployed. | |
| ### 5. Deploy (k2 worker) | |
| Copy the export to `~/ampixa-demo/models/ocr/space/models/<name>/artifacts/` with a | |
| `MODEL-SOURCE.txt`, point `AMPIXA_OCR_DEVANAGARI_DIR` (or the worker default) at it, restart | |
| with `~/ampixa-demo/ops/restart.sh ocr`, and check `lsof -iTCP:8906` and `/health` for a stale | |
| duplicate process. The worker selects `onnxruntime` automatically when `inference.onnx` is | |
| present. Rollback: point the variable at `devanagari-ft-v2/artifacts` or the official | |
| `devanagari-ppocrv5/artifacts`. | |
| ## Traps this pipeline already hit | |
| - **Preprocessing parity.** PaddleX pads to `int(48 * max(320/48, w/h))` and caps the resize | |
| width at that value (floor for wide crops). A 1-px difference flipped low-confidence | |
| characters on about 40% of long lines. Batch only crops with equal pad widths: SVTR attention | |
| sees the padding. | |
| - **HF head applies softmax.** Take CTC logits from `model.head.head` (`common.logits`). | |
| - **Official Paddle recipe limits.** `max_text_length` 25, the NRTR branch rejects labels of | |
| 23+ characters, and `encode()` silently drops out-of-dictionary characters. `train.py` counts | |
| and reports every skipped label instead. | |
| - **Converter bug.** `npttf2utf` 0.3.7 decodes the `nayanepal` alternate क्र `j|m` as व्र | |
| (and प->फ); 3% of spans. v2 learned it (402 of 530 E7 predictions contained व्र); v3 does | |
| not (2 of 530). | |
| - **Training-time dev numbers are relative only.** `train.py` evaluates dev sets in batches | |
| of 64 padded to the widest crop, not at PaddleX pad widths. v3's selected step logged 8.9% | |
| codepoint CER on `real_dev`, while the served path gives 0.57% on held-out editions. Those | |
| numbers (and the `--guard-dev` comparison) rank checkpoints against step 0; report accuracy | |
| only from `run_serve_eval.sh`. Making the dev loader bucket by pad width is an open fix. | |
| - **Page crops.** The k2 worker crops each detected line with the axis-aligned box of the | |
| detector quad plus 12 px. On a 2°-tilted scan that swallows neighbouring lines: 21.4% CER | |
| versus 2.9% with rotated min-area crops plus 6 px on 43 hand-labelled lines | |
| (`page_probe/`). Line-level eval sets cannot see this; not fixed in the worker yet. | |