|
Download ocr-tech/docs/LOCAL_RECOGNIZER_SCORING.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 5.19 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LOCAL_RECOGNIZER_SCORING.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/LOCAL_RECOGNIZER_SCORING.md
-
curl -L -o LOCAL_RECOGNIZER_SCORING.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LOCAL_RECOGNIZER_SCORING.md
5.19 kB
| # Local Devanagari Recognizer Scoring | |
| This is the local CPU path for scoring frozen PaddleOCR line-crop eval packs on | |
| macOS arm64 without a GPU instance. The working route is: | |
| 1. export the training checkpoint `.tgz` to PaddleOCR inference format with a | |
| pinned PaddleOCR source checkout, then | |
| 2. run `paddleocr._models.text_recognition.TextRecognition(model_dir=...)`, then | |
| 3. score with `tools/eval_metrics.py` through `tools/score_recognizer_pack.py`. | |
| Direct `tools/infer_rec.py` checkpoint inference also works on CPU, but the | |
| export route is preferred because it matches the production `TextRecognition` | |
| consumption path used by `spaces/limbu-ocr/app.py`. | |
| ## Environment | |
| Use Python 3.11. These commands keep uv and Paddle caches under `/tmp`, which is | |
| important in restricted local/sandboxed runs where `~/.cache` is not writable. | |
| ```bash | |
| export UV_CACHE_DIR=/tmp/ocrtech-uv-cache | |
| export OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache | |
| uv venv --python 3.11 --clear /tmp/ocrtech-paddle311 | |
| UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ | |
| --python /tmp/ocrtech-paddle311/bin/python pip | |
| UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ | |
| --python /tmp/ocrtech-paddle311/bin/python \ | |
| --index-url https://www.paddlepaddle.org.cn/packages/stable/cpu/ \ | |
| paddlepaddle==3.3.1 | |
| UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ | |
| --python /tmp/ocrtech-paddle311/bin/python \ | |
| --index-url https://pypi.org/simple \ | |
| paddleocr==3.7.0 paddlex==3.7.2 rapidfuzz==3.14.5 regex==2026.6.28 | |
| git clone --depth 1 --branch v3.7.0 \ | |
| https://github.com/PaddlePaddle/PaddleOCR.git \ | |
| /tmp/ocrtech-PaddleOCR-v3.7.0 | |
| UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ | |
| --python /tmp/ocrtech-paddle311/bin/python \ | |
| --index-url https://pypi.org/simple \ | |
| -r /tmp/ocrtech-PaddleOCR-v3.7.0/requirements.txt | |
| export PADDLEOCR_SOURCE=/tmp/ocrtech-PaddleOCR-v3.7.0 | |
| ``` | |
| Verified locally: | |
| - Python `3.11.13` | |
| - `paddlepaddle==3.3.1` | |
| - `paddleocr==3.7.0` | |
| - `paddlex==3.7.2` | |
| - PaddleOCR source tag `v3.7.0`, commit | |
| `b03f46425e8ff4442b268ce449e3eef758146cd4` | |
| - `numpy==2.3.5`, `opencv-python==5.0.0.93`, | |
| `opencv-python-headless==5.0.0.93`, | |
| `opencv-contrib-python==4.10.0.84`, `scikit-image==0.26.0`, | |
| `lmdb==2.2.1`, `albumentations==2.0.8` | |
| ## Score A Pack | |
| From the repo root: | |
| ```bash | |
| mkdir -p /tmp/ocrtech-deva-v2-artifact | |
| tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ | |
| -C /tmp/ocrtech-deva-v2-artifact | |
| /tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \ | |
| --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ | |
| --labels path/to/labels.txt \ | |
| --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \ | |
| --paddleocr-source "$PADDLEOCR_SOURCE" \ | |
| --out /tmp/ocrtech-score-run | |
| ``` | |
| If the model argument is a checkpoint `.tgz`, the scorer exports it to | |
| `/tmp/ocrtech-score-run/exported_model` and then runs `TextRecognition`. If the | |
| model argument is already an exported directory containing `inference.json`, | |
| `inference.pdiparams`, and `inference.yml`, the scorer skips export. | |
| Outputs: | |
| - `predictions.jsonl`: `image`, `gt`, `pred`, `confidence` | |
| - `metrics.json`: route, inputs, and `tools/eval_metrics.py` metrics: | |
| `cer_codepoint`, `cer_grapheme`, `exact_line_acc` | |
| ## Smoke Check | |
| The Newar silver labels use relative image paths, so either run against the | |
| original `labels.txt` in place or create an absolute-path 20-line smoke file: | |
| ```bash | |
| mkdir -p /tmp/ocrtech-deva-v2-artifact | |
| tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ | |
| -C /tmp/ocrtech-deva-v2-artifact | |
| base="$PWD/outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/" | |
| awk -v base="$base" 'BEGIN { FS="\t"; OFS="\t" } NR <= 20 { print base $1, $2 }' \ | |
| outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt \ | |
| > /tmp/ocrtech-newar-smoke20-labels.txt | |
| /tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \ | |
| --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ | |
| --labels /tmp/ocrtech-newar-smoke20-labels.txt \ | |
| --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \ | |
| --paddleocr-source "$PADDLEOCR_SOURCE" \ | |
| --out /tmp/ocrtech-deva-v2-newar-smoke20-score | |
| ``` | |
| Observed smoke result on the first 20 rows of | |
| `outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt`: | |
| - codepoint CER: `0.1095679012345679` | |
| - grapheme CER: `0.2512820512820513` | |
| - exact-line accuracy: `0.1` | |
| - `20` lines, `648` codepoints, `390` grapheme clusters | |
| This is only a smoke check, not an official eval number. | |
| ## Gotchas | |
| - The checkpoint archive stores a training checkpoint, not an exported inference | |
| model. Export requires PaddleOCR source `tools/export_model.py`. | |
| - The training config's `Global.character_dict_path` is relative to the original | |
| training workspace. Pass the Devanagari dictionary explicitly. | |
| - Under restricted macOS runs, Paddle export may try to write dy2static files to | |
| `~/.cache/paddle`. Set `OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache` | |
| or run with a writable `HOME`. | |
| - Paddle emits `sysctl` and missing-`ccache` warnings on this machine. They did | |
| not affect export or CPU inference. | |