| --- |
| license: mit |
| library_name: coreai |
| pipeline_tag: image-to-text |
| tags: |
| - core-ai |
| - on-device |
| - apple |
| - ocr |
| - document-understanding |
| - glm |
| base_model: zai-org/GLM-OCR |
| --- |
| |
| # GLM-OCR → Core AI |
|
|
| On-device document OCR, running entirely on Apple's **Core AI** (Neural Engine / GPU). |
| A port of [`zai-org/GLM-OCR`](https://huggingface.co/zai-org/GLM-OCR) (0.9B, **MIT**) — a small, |
| SOTA-quality document recognizer (OmniDocBench v1.5 **94.62**, #1 with its layout pipeline). |
| Prompt it with `Text Recognition:` / `Table Recognition:` / `Formula Recognition:` and get back |
| plain text (reading order), HTML tables (`<table>…`), or LaTeX. zh / en / fr / es / ru / de / **ja** / ko. |
|
|
| GLM-OCR is a small OCR variant of **GLM-4.V** (`Glm4v`): a CogViT vision tower + a 16-layer GLM text |
| decoder with sectioned 3D M-RoPE. This port reuses the shipped Qwen3-VL vision idiom and GLM text |
| decode — no R-SWA, no MoE, no MLA. |
|
|
| ## Bundles |
|
|
| | dir | what | precision | size | |
| |---|---|---|---| |
| | `vision/` | CogViT encoder → `image_embeds [N, 1536]` | fp16 | 829 MB | |
| | `decoder/` | GLM text decoder, S=1 pipelined, M-RoPE + image injection | int8hu (body int8 per-block-32 + untied head absmax) | 764 MB | |
| | `tokenizer/` | `tokenizer.json` etc. | — | — | |
|
|
| The decoder rides three static graph inputs — `image_embeds [682,1536]` f16, `rope_shift_start [1]`, |
| `rope_shift_amount [1]` — so the vision tower runs once, its output is injected at the image |
| placeholder positions (`V + slot`, row-major over the merged grid), and the text decodes on top. |
| `N` (visual-token count) is fixed at export by the chosen input resolution (here 682 = a 22×31 merged |
| grid); resize the page to that grid host-side. |
|
|
| ## Verified (M4 Max, GPU, Core AI pipelined engine) |
|
|
| - **End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference** — a |
| synthetic document read verbatim (*"Quarterly Report / On-device inference shipped across all |
| product lines this quarter…"*), **~375 tok/s** decode. |
| - Torch ladder vs HF: decoder logits cos **1.000020**, vision `image_embeds` cos **1.000061**, |
| full-VLM argmax **694/694**. |
| - Engine gate: vision `image_embeds` cos **0.9998**; decoder argmax exact over the sampled positions. |
| - int8hu vs fp16: **7 / 694** argmax flips, all at visual-token positions (0 in the text region), the |
| generation-driving position exact — i.e. the OCR text is preserved. |
|
|
| ## Run it |
|
|
| The decoder is a standard Core AI pipelined LLM bundle with three multimodal static inputs. Drive it |
| with the pipelined engine (S=1, `COREAI_CHUNK_THRESHOLD=1`; feed the prompt with the image |
| placeholders rewritten to `V+slot`, bind `image_embeds` from the vision tower, set |
| `rope_shift_start = img_start + N`, `rope_shift_amount = N − max(gh, gw)`). The full conversion recipe |
| and the host contract (with the exact static-input values) are in the |
| [Core AI model zoo](https://github.com/john-rocky/coreai-model-zoo) — |
| `conversion/export_glm_ocr_pipelined.py`, `zoo/glm-ocr.md`, `knowledge/glm-ocr-port.md`. |
|
|
| ## Scope / honesty |
|
|
| - This is the **recognition** model: per-prompt text / table / formula. Whole-page auto-structuring |
| (the 94.62 full-pipeline number) additionally needs a layout detector (PP-DocLayoutV3) that is not |
| part of this port. |
| - int4 is not shipped (weight-only int4 without QAT risks a quality cliff on a 0.9B model). |
|
|
| ## License |
|
|
| **MIT** (inherited from `zai-org/GLM-OCR`). *Community port — not affiliated with Apple or Z.ai.* |
|
|
