ScribeRec: a screen-text line recognizer for AMD's XDNA1 NPU
ScribeRec is ignite-scribe's text-line recognizer. It was designed and trained from scratch to run on the int8 engine of ignite-xdna on AMD's XDNA1 NPU (Phoenix and Hawk Point). It has 1.25 M parameters: a convolutional backbone that runs on the NPU, then a BiLSTM tail and CTC decoding on the CPU. It reads one line of screen text (UI, terminal, editor) at a time.
This is release-03, fine-tuned from release-01 on the geometry of terminal-line crops.
Files (r03/)
| File | What |
|---|---|
backbone_{320,640,960,1280}.onnx |
float backbone per input-width bucket (input height 80) |
backbone_{320,640,960,1280}_xint8.onnx |
AMD Quark XINT8 QDQ versions, calibrated per bucket; what the NPU runs |
backbone.onnx, backbone_xint8.onnx |
the 320 bucket under its old name |
tail.onnx |
BiLSTM tail and classifier (dynamic n and t) |
dict.txt |
the characters (436), PP-OCRv5's English dictionary |
MODEL.md |
the training notes: architecture, classes, preprocessing, recipe, results |
Classes: 438. Blank = 0; dict.txt = 1..436 in file order; space = 437. Preprocessing and decoding are in
MODEL.md and in ignite-scribe's scriberec_rec.py.
r03/npu/scriberec_{320,640,960,1280}.ignite are the compiled ignite-xdna NPU containers, built from the XINT8
backbones (ignite-compile --engine graph --task segment). They run through ignite-xdna's runtime, not ONNX
Runtime's VitisAI EP.
Results
| Test | ScribeRec r03 | PP-OCRv5 (CPU) | ScribeRec release-01 |
|---|---|---|---|
| Real Windows terminal, static text, character error rate | 0.22% | 0.48-0.54% | 1.40-1.55% |
| Live terminal UI full of box-drawing rules and spinners | 4.47% | 1.53% | 8.29% |
| Synthetic UI lines (seed 100), XINT8 | 0.154% |
- Real-window numbers: ignite-scribe's
tools/ocr_diag.py, with the window's UI Automation text as truth. All recognizers read the same crops. - On the NPU (Ryzen 7 8700G): bit-exact to the XINT8 ONNX in every bucket, at 10-17 ms a line (30 ms for a split long line).
- Weak spot: busy terminal UIs. It reads a space as '-' next to box-drawing rules, and it inserts glyphs that aren't in the dictionary (spinners, check marks).
Training data and fonts: read before use
- Text: public domain only, plus generators: PG19 (public-domain books) and the SQLite source.
- Fonts: the line images were rendered with Windows GDI and DirectWrite from the fonts installed on the
training PC, as whole lines of text, not single glyphs. No font file or glyph outline is in this model. By
family, of the 264 used:
- about 93 are Windows' own fonts (Segoe UI, Consolas, Cascadia, Calibri, Arial and others). They carry most of the training lines: the generator draws most lines from a fixed list of these.
- about 56 are openly licensed (OFL, Apache-2.0, Ubuntu Font Licence and similar: Noto, DejaVu, Liberation, Ubuntu Sans Mono and others).
- about 115 came with other software, mostly Microsoft Office: Monotype, ITC, URW, Font Bureau and Bigelow & Holmes faces such as Gill Sans MT, Bodoni MT and Agency FB; also an ASUS and an Adobe font. Their licences neither grant nor clearly forbid this use. They make up about 16% of the training lines, about 10 k lines per family.
- The publisher's reading: the model contains no font data, so publishing it does not distribute any font. This is not a settled legal question. Users who need certainty should take their own advice. A version trained only on Windows-supplied and openly licensed fonts can be made if needed.
Licence
AGPL-3.0-or-later, like ignite-scribe. dict.txt is from PaddleOCR (Apache-2.0); see r03/LICENSE.md.
manifest.json lists every file with its sha256.