File size: 3,430 Bytes
aa861d5 5347b34 aa861d5 5347b34 aa861d5 e186a72 aa861d5 e186a72 aa861d5 5347b34 e186a72 5347b34 e186a72 aa861d5 5347b34 aa861d5 5347b34 e186a72 5347b34 aa861d5 5347b34 aa861d5 e186a72 5347b34 aa861d5 5347b34 aa861d5 5347b34 aa861d5 e186a72 5347b34 aa861d5 5347b34 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
tags:
- optical-character-recognition
- image-to-text
- kashmiri
- perso-arabic
- pytorch
language:
- ks
library_name: pytorch
---
# Koshur OCR v2
A compact Kashmiri Perso-Arabic **printed-line** OCR model from **Kash Labs AI**.
## v2.1 results — prefix beam + train-only character LM
- **Beam deduplicated validation CER:** 21.35% (greedy: 22.62%)
- **Beam deduplicated exact match:** 26.20% (greedy: 25.54%)
- **Held-out 720-row CER:** 21.24% (greedy: 22.44%)
- **Absolute held-out CER improvement:** 1.19 percentage points
- **Validation rows:** 920 deduplicated / 1,000 full
- **Best epoch:** 18
- **Training rows:** 100,000 synthetic sentence-line images
- **Beam evaluation:** https://wandb.ai/kash-ai-labs-kash-labs/koshur-pixel-ocr/runs/ctc-v2-prefix-beam-20260729T152530Z
The selected decoder uses width 10, top-12 token pruning, a trigram character LM trained only on the 100,000 training labels, LM weight 0.4, and token bonus 1.5. It was tuned on a fixed 200-row calibration subset and independently improved the remaining 720 rows. Across all 920 rows, deletions fell from 6,859 to 4,615.
## Architecture
`64×2048 grayscale line → CNN (48→96→160→192) → 2-layer bidirectional GRU (256 per direction) → 155-way CTC`
The model has **4,475,931 trainable parameters**, a 154-character inventory, and no temporal feature dropout in the selected v2 control checkpoint. This is a custom PyTorch checkpoint, not a Transformers-format model.
## Versioning
- `v1.0.0` — 10k temporal-dropout checkpoint, 28.13% deduplicated CER
- `v2.0.0` — 100k control checkpoint with greedy decoding, 22.62% deduplicated CER
- `v2.1.0` — same weights plus prefix-beam/trigram decoding, 21.35% deduplicated CER (current)
Use a Hub revision to pin a release.
## Files
- `best-model.pt` — epoch-18 state dictionary, character inventory, architecture metadata, and validation CER
- `model.py` — architecture, preprocessing, CTC decoding, and `recognize_line()`
- `ctc_prefix_beam.py` — tested prefix-beam implementation
- `char-trigram-lm.pkl` — compact train-only character language model
- `beam-eval.json` — calibration, held-out, and full evaluation metrics
- `config.json` — release, training, preprocessing, metrics, and checkpoint checksum
## Local CPU or Apple Silicon inference
```bash
pip install -r requirements.txt
```
```python
from PIL import Image
from model import recognize_line
text, diagnostics = recognize_line(Image.open("single-line.png"))
print(text)
print(diagnostics)
```
Prefix beam is the default. For the original fast decoder:
```python
text, diagnostics = recognize_line(Image.open("single-line.png"), decoder="greedy")
```
The input should be one tightly cropped, upright printed line. It is resized to height 64 and left-aligned on a width-2048 canvas. Inference runs on CPU and does not require a GPU.
## Limitations
- The reported score is an in-domain synthetic validation result, not real-world scan quality.
- Trained on synthetic printed lines, not handwriting or full pages.
- Camera photos should be rotated, deskewed, tightly cropped, and segmented into individual lines.
- Training targets exclude the established Kashmiri diacritic set; the model recognizes base text and does not restore those marks.
- Error rate remains substantial, with deletions the dominant residual error.
- Predictions require review before publication or archival use.
|