File size: 5,186 Bytes
f9bc0c1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | # Local Devanagari Recognizer Scoring
This is the local CPU path for scoring frozen PaddleOCR line-crop eval packs on
macOS arm64 without a GPU instance. The working route is:
1. export the training checkpoint `.tgz` to PaddleOCR inference format with a
pinned PaddleOCR source checkout, then
2. run `paddleocr._models.text_recognition.TextRecognition(model_dir=...)`, then
3. score with `tools/eval_metrics.py` through `tools/score_recognizer_pack.py`.
Direct `tools/infer_rec.py` checkpoint inference also works on CPU, but the
export route is preferred because it matches the production `TextRecognition`
consumption path used by `spaces/limbu-ocr/app.py`.
## Environment
Use Python 3.11. These commands keep uv and Paddle caches under `/tmp`, which is
important in restricted local/sandboxed runs where `~/.cache` is not writable.
```bash
export UV_CACHE_DIR=/tmp/ocrtech-uv-cache
export OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache
uv venv --python 3.11 --clear /tmp/ocrtech-paddle311
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
--python /tmp/ocrtech-paddle311/bin/python pip
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
--python /tmp/ocrtech-paddle311/bin/python \
--index-url https://www.paddlepaddle.org.cn/packages/stable/cpu/ \
paddlepaddle==3.3.1
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
--python /tmp/ocrtech-paddle311/bin/python \
--index-url https://pypi.org/simple \
paddleocr==3.7.0 paddlex==3.7.2 rapidfuzz==3.14.5 regex==2026.6.28
git clone --depth 1 --branch v3.7.0 \
https://github.com/PaddlePaddle/PaddleOCR.git \
/tmp/ocrtech-PaddleOCR-v3.7.0
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
--python /tmp/ocrtech-paddle311/bin/python \
--index-url https://pypi.org/simple \
-r /tmp/ocrtech-PaddleOCR-v3.7.0/requirements.txt
export PADDLEOCR_SOURCE=/tmp/ocrtech-PaddleOCR-v3.7.0
```
Verified locally:
- Python `3.11.13`
- `paddlepaddle==3.3.1`
- `paddleocr==3.7.0`
- `paddlex==3.7.2`
- PaddleOCR source tag `v3.7.0`, commit
`b03f46425e8ff4442b268ce449e3eef758146cd4`
- `numpy==2.3.5`, `opencv-python==5.0.0.93`,
`opencv-python-headless==5.0.0.93`,
`opencv-contrib-python==4.10.0.84`, `scikit-image==0.26.0`,
`lmdb==2.2.1`, `albumentations==2.0.8`
## Score A Pack
From the repo root:
```bash
mkdir -p /tmp/ocrtech-deva-v2-artifact
tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
-C /tmp/ocrtech-deva-v2-artifact
/tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \
--model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
--labels path/to/labels.txt \
--dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \
--paddleocr-source "$PADDLEOCR_SOURCE" \
--out /tmp/ocrtech-score-run
```
If the model argument is a checkpoint `.tgz`, the scorer exports it to
`/tmp/ocrtech-score-run/exported_model` and then runs `TextRecognition`. If the
model argument is already an exported directory containing `inference.json`,
`inference.pdiparams`, and `inference.yml`, the scorer skips export.
Outputs:
- `predictions.jsonl`: `image`, `gt`, `pred`, `confidence`
- `metrics.json`: route, inputs, and `tools/eval_metrics.py` metrics:
`cer_codepoint`, `cer_grapheme`, `exact_line_acc`
## Smoke Check
The Newar silver labels use relative image paths, so either run against the
original `labels.txt` in place or create an absolute-path 20-line smoke file:
```bash
mkdir -p /tmp/ocrtech-deva-v2-artifact
tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
-C /tmp/ocrtech-deva-v2-artifact
base="$PWD/outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/"
awk -v base="$base" 'BEGIN { FS="\t"; OFS="\t" } NR <= 20 { print base $1, $2 }' \
outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt \
> /tmp/ocrtech-newar-smoke20-labels.txt
/tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \
--model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
--labels /tmp/ocrtech-newar-smoke20-labels.txt \
--dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \
--paddleocr-source "$PADDLEOCR_SOURCE" \
--out /tmp/ocrtech-deva-v2-newar-smoke20-score
```
Observed smoke result on the first 20 rows of
`outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt`:
- codepoint CER: `0.1095679012345679`
- grapheme CER: `0.2512820512820513`
- exact-line accuracy: `0.1`
- `20` lines, `648` codepoints, `390` grapheme clusters
This is only a smoke check, not an official eval number.
## Gotchas
- The checkpoint archive stores a training checkpoint, not an exported inference
model. Export requires PaddleOCR source `tools/export_model.py`.
- The training config's `Global.character_dict_path` is relative to the original
training workspace. Pass the Devanagari dictionary explicitly.
- Under restricted macOS runs, Paddle export may try to write dy2static files to
`~/.cache/paddle`. Set `OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache`
or run with a writable `HOME`.
- Paddle emits `sysctl` and missing-`ccache` warnings on this machine. They did
not affect export or CPU inference.
|