# Local Devanagari Recognizer Scoring This is the local CPU path for scoring frozen PaddleOCR line-crop eval packs on macOS arm64 without a GPU instance. The working route is: 1. export the training checkpoint `.tgz` to PaddleOCR inference format with a pinned PaddleOCR source checkout, then 2. run `paddleocr._models.text_recognition.TextRecognition(model_dir=...)`, then 3. score with `tools/eval_metrics.py` through `tools/score_recognizer_pack.py`. Direct `tools/infer_rec.py` checkpoint inference also works on CPU, but the export route is preferred because it matches the production `TextRecognition` consumption path used by `spaces/limbu-ocr/app.py`. ## Environment Use Python 3.11. These commands keep uv and Paddle caches under `/tmp`, which is important in restricted local/sandboxed runs where `~/.cache` is not writable. ```bash export UV_CACHE_DIR=/tmp/ocrtech-uv-cache export OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache uv venv --python 3.11 --clear /tmp/ocrtech-paddle311 UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ --python /tmp/ocrtech-paddle311/bin/python pip UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ --python /tmp/ocrtech-paddle311/bin/python \ --index-url https://www.paddlepaddle.org.cn/packages/stable/cpu/ \ paddlepaddle==3.3.1 UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ --python /tmp/ocrtech-paddle311/bin/python \ --index-url https://pypi.org/simple \ paddleocr==3.7.0 paddlex==3.7.2 rapidfuzz==3.14.5 regex==2026.6.28 git clone --depth 1 --branch v3.7.0 \ https://github.com/PaddlePaddle/PaddleOCR.git \ /tmp/ocrtech-PaddleOCR-v3.7.0 UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \ --python /tmp/ocrtech-paddle311/bin/python \ --index-url https://pypi.org/simple \ -r /tmp/ocrtech-PaddleOCR-v3.7.0/requirements.txt export PADDLEOCR_SOURCE=/tmp/ocrtech-PaddleOCR-v3.7.0 ``` Verified locally: - Python `3.11.13` - `paddlepaddle==3.3.1` - `paddleocr==3.7.0` - `paddlex==3.7.2` - PaddleOCR source tag `v3.7.0`, commit `b03f46425e8ff4442b268ce449e3eef758146cd4` - `numpy==2.3.5`, `opencv-python==5.0.0.93`, `opencv-python-headless==5.0.0.93`, `opencv-contrib-python==4.10.0.84`, `scikit-image==0.26.0`, `lmdb==2.2.1`, `albumentations==2.0.8` ## Score A Pack From the repo root: ```bash mkdir -p /tmp/ocrtech-deva-v2-artifact tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ -C /tmp/ocrtech-deva-v2-artifact /tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \ --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ --labels path/to/labels.txt \ --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \ --paddleocr-source "$PADDLEOCR_SOURCE" \ --out /tmp/ocrtech-score-run ``` If the model argument is a checkpoint `.tgz`, the scorer exports it to `/tmp/ocrtech-score-run/exported_model` and then runs `TextRecognition`. If the model argument is already an exported directory containing `inference.json`, `inference.pdiparams`, and `inference.yml`, the scorer skips export. Outputs: - `predictions.jsonl`: `image`, `gt`, `pred`, `confidence` - `metrics.json`: route, inputs, and `tools/eval_metrics.py` metrics: `cer_codepoint`, `cer_grapheme`, `exact_line_acc` ## Smoke Check The Newar silver labels use relative image paths, so either run against the original `labels.txt` in place or create an absolute-path 20-line smoke file: ```bash mkdir -p /tmp/ocrtech-deva-v2-artifact tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ -C /tmp/ocrtech-deva-v2-artifact base="$PWD/outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/" awk -v base="$base" 'BEGIN { FS="\t"; OFS="\t" } NR <= 20 { print base $1, $2 }' \ outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt \ > /tmp/ocrtech-newar-smoke20-labels.txt /tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \ --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \ --labels /tmp/ocrtech-newar-smoke20-labels.txt \ --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \ --paddleocr-source "$PADDLEOCR_SOURCE" \ --out /tmp/ocrtech-deva-v2-newar-smoke20-score ``` Observed smoke result on the first 20 rows of `outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt`: - codepoint CER: `0.1095679012345679` - grapheme CER: `0.2512820512820513` - exact-line accuracy: `0.1` - `20` lines, `648` codepoints, `390` grapheme clusters This is only a smoke check, not an official eval number. ## Gotchas - The checkpoint archive stores a training checkpoint, not an exported inference model. Export requires PaddleOCR source `tools/export_model.py`. - The training config's `Global.character_dict_path` is relative to the original training workspace. Pass the Devanagari dictionary explicitly. - Under restricted macOS runs, Paddle export may try to write dy2static files to `~/.cache/paddle`. Set `OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache` or run with a writable `HOME`. - Paddle emits `sysctl` and missing-`ccache` warnings on this machine. They did not affect export or CPU inference.