File size: 5,186 Bytes
f9bc0c1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
# Local Devanagari Recognizer Scoring

This is the local CPU path for scoring frozen PaddleOCR line-crop eval packs on
macOS arm64 without a GPU instance. The working route is:

1. export the training checkpoint `.tgz` to PaddleOCR inference format with a
   pinned PaddleOCR source checkout, then
2. run `paddleocr._models.text_recognition.TextRecognition(model_dir=...)`, then
3. score with `tools/eval_metrics.py` through `tools/score_recognizer_pack.py`.

Direct `tools/infer_rec.py` checkpoint inference also works on CPU, but the
export route is preferred because it matches the production `TextRecognition`
consumption path used by `spaces/limbu-ocr/app.py`.

## Environment

Use Python 3.11. These commands keep uv and Paddle caches under `/tmp`, which is
important in restricted local/sandboxed runs where `~/.cache` is not writable.

```bash
export UV_CACHE_DIR=/tmp/ocrtech-uv-cache
export OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache

uv venv --python 3.11 --clear /tmp/ocrtech-paddle311
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
  --python /tmp/ocrtech-paddle311/bin/python pip
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
  --python /tmp/ocrtech-paddle311/bin/python \
  --index-url https://www.paddlepaddle.org.cn/packages/stable/cpu/ \
  paddlepaddle==3.3.1
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
  --python /tmp/ocrtech-paddle311/bin/python \
  --index-url https://pypi.org/simple \
  paddleocr==3.7.0 paddlex==3.7.2 rapidfuzz==3.14.5 regex==2026.6.28

git clone --depth 1 --branch v3.7.0 \
  https://github.com/PaddlePaddle/PaddleOCR.git \
  /tmp/ocrtech-PaddleOCR-v3.7.0
UV_CACHE_DIR=/tmp/ocrtech-uv-cache uv pip install \
  --python /tmp/ocrtech-paddle311/bin/python \
  --index-url https://pypi.org/simple \
  -r /tmp/ocrtech-PaddleOCR-v3.7.0/requirements.txt

export PADDLEOCR_SOURCE=/tmp/ocrtech-PaddleOCR-v3.7.0
```

Verified locally:

- Python `3.11.13`
- `paddlepaddle==3.3.1`
- `paddleocr==3.7.0`
- `paddlex==3.7.2`
- PaddleOCR source tag `v3.7.0`, commit
  `b03f46425e8ff4442b268ce449e3eef758146cd4`
- `numpy==2.3.5`, `opencv-python==5.0.0.93`,
  `opencv-python-headless==5.0.0.93`,
  `opencv-contrib-python==4.10.0.84`, `scikit-image==0.26.0`,
  `lmdb==2.2.1`, `albumentations==2.0.8`

## Score A Pack

From the repo root:

```bash
mkdir -p /tmp/ocrtech-deva-v2-artifact
tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
  -C /tmp/ocrtech-deva-v2-artifact

/tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \
  --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
  --labels path/to/labels.txt \
  --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \
  --paddleocr-source "$PADDLEOCR_SOURCE" \
  --out /tmp/ocrtech-score-run
```

If the model argument is a checkpoint `.tgz`, the scorer exports it to
`/tmp/ocrtech-score-run/exported_model` and then runs `TextRecognition`. If the
model argument is already an exported directory containing `inference.json`,
`inference.pdiparams`, and `inference.yml`, the scorer skips export.

Outputs:

- `predictions.jsonl`: `image`, `gt`, `pred`, `confidence`
- `metrics.json`: route, inputs, and `tools/eval_metrics.py` metrics:
  `cer_codepoint`, `cer_grapheme`, `exact_line_acc`

## Smoke Check

The Newar silver labels use relative image paths, so either run against the
original `labels.txt` in place or create an absolute-path 20-line smoke file:

```bash
mkdir -p /tmp/ocrtech-deva-v2-artifact
tar -xzf outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
  -C /tmp/ocrtech-deva-v2-artifact

base="$PWD/outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/"
awk -v base="$base" 'BEGIN { FS="\t"; OFS="\t" } NR <= 20 { print base $1, $2 }' \
  outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt \
  > /tmp/ocrtech-newar-smoke20-labels.txt

/tmp/ocrtech-paddle311/bin/python tools/score_recognizer_pack.py \
  --model outputs/devanagari-recognizer-v2-2026-06-18/deva-v2.tgz \
  --labels /tmp/ocrtech-newar-smoke20-labels.txt \
  --dictionary /tmp/ocrtech-deva-v2-artifact/data/dictionaries/devanagari-v2.txt \
  --paddleocr-source "$PADDLEOCR_SOURCE" \
  --out /tmp/ocrtech-deva-v2-newar-smoke20-score
```

Observed smoke result on the first 20 rows of
`outputs/language-pack-silver-lines-2026-07-04/label-1fefbafa97/labels.txt`:

- codepoint CER: `0.1095679012345679`
- grapheme CER: `0.2512820512820513`
- exact-line accuracy: `0.1`
- `20` lines, `648` codepoints, `390` grapheme clusters

This is only a smoke check, not an official eval number.

## Gotchas

- The checkpoint archive stores a training checkpoint, not an exported inference
  model. Export requires PaddleOCR source `tools/export_model.py`.
- The training config's `Global.character_dict_path` is relative to the original
  training workspace. Pass the Devanagari dictionary explicitly.
- Under restricted macOS runs, Paddle export may try to write dy2static files to
  `~/.cache/paddle`. Set `OCRTECH_PADDLE_CACHE_ROOT=/tmp/ocrtech-paddle-cache`
  or run with a writable `HOME`.
- Paddle emits `sysctl` and missing-`ccache` warnings on this machine. They did
  not affect export or CPU inference.