Unlimited-OCR β€” LiteRT-LM (OCR)

On a Galaxy S26 this file reads a page in 18–22 s on the CPU, engine creation included (2026-11-15). This repository is a conversion of baidu/Unlimited-OCR to LiteRT-LM (.litertlm), made with hf-to-litertlm for Google's LiteRT-LM (tested with 0.19.2, and one page with 0.19.1).

Unlimited-OCR (Baidu, 2026-06-22) is a 3.3B document-parsing model: a vision encoder (DeepEncoder: SAM ViT-B and CLIP-L, with a projector; 401M parameters) and a 12-layer DeepSeek-V2 text decoder (2.9B parameters; the first layer dense, then 11 mixture-of-experts layers of 64 routed experts, 6 per token, and 2 shared) with sliding-window attention: while it writes, each new token attends to the whole prompt and to the last 128 generated tokens. It writes a page as its layout blocks, each with a label (title, text, equation, table, …), a box and the block's content (in our tests, tables as HTML and formulas as LaTeX). This file reads one page per request, as one 1,024 Γ— 1,024 view of 273 image tokens: the original's base setting, one of the two single-page settings in its README and the view it takes of each page of a multi-page document.

File What is inside Size sha256
Unlimited-OCR_int8.litertlm text decoder with int8 weights in its linear layers (the position table stays float), fp32 activations declared for the GPU; int8 embedding table; vision encoder and projector with fp16 weights; the decoder's and the embedding's weights in LiteRT external-weights sections of the same file 3.80 GB 3ab6cbbc548c8b56c0885673a18231fd086bfbfbf38f70e1d3a1269523fa310c

Which backend. The file holds two models: a vision encoder that reads the picture and a text decoder that writes the text. Each can run on the CPU or the GPU.

  • Use the CPU on Android. On the Galaxy S26 a page took 18–22 s on the CPU against 25–27 s with the decoder on the GPU and the vision encoder on the CPU. On the Pixel 11 Pro, once the runtime's cache files were written, a page took 28–31 s on the CPU against 44 s on the GPU. On both phones the GPU missed 5 of our 18 single-page tests, against 3 near ties on the CPU (see Correctness).
  • iPhone: we have not run this file on an iPhone yet.
  • Mac (Apple M4 Max): the CPU passed 16 of our 18 single-page tests and the 3-page document (see Correctness). The GPU (--backend gpu) passed 15 of the 18 and the 3-page document. Once the runtime's cache files were written, a page took 4.4 s with the decoder on the GPU and the vision encoder on the CPU (--backend gpu --vision-backend cpu, the same text as both on the GPU), 4.7 s with both on the GPU and 6.6 s on the CPU (see Performance).
  • 8 GB Android phones: on a Pixel 8a the CPU read our one-line test like the original but took 97 s for it (decode 1.0 tok/s), and with the decoder on the GPU the phone's available memory fell to 0.29 GiB and our test harness stopped the run (see Known runtime issues).

How to use

ocr_unlimited.py pads your pages and runs the file with litert-lm:

pip install litert-lm pillow huggingface_hub
hf download litert-community/Unlimited-OCR --include "Unlimited-OCR_int8.litertlm" --include "ocr_unlimited.py" --include "samples/*" --local-dir uocr
python uocr/ocr_unlimited.py uocr/Unlimited-OCR_int8.litertlm uocr/samples/page_en.png
python uocr/ocr_unlimited.py uocr/Unlimited-OCR_int8.litertlm uocr/samples/table.png uocr/samples/formula_2.png

The first request on a device writes the runtime's cache files next to the model (1.6 to 4.4 GB depending on the backend and the device; see Known runtime issues) and takes the (cold) times in Performance; later requests are faster.

Why the padding: LiteRT-LM resizes every picture to 1,024 Γ— 1,024 without keeping its aspect ratio and without reading the EXIF orientation. So the script applies the EXIF orientation and then centres the page on a gray (127, 127, 127) square, as the original's code does in its base setting (it applies the EXIF orientation and pads the page to 1,024 Γ— 1,024 with the gray of its normalisation mean). Without the padding a non-square page is stretched to a square; our tests were not run that way.

The model takes no prompt of your choosing: the file's template writes the page and the model's instruction document parsing., the prompt of the original's single-page example. --backend gpu puts the decoder and the vision encoder on the GPU (--vision-backend cpu keeps the vision encoder on the CPU), --max-tokens caps each page's answer, --cache-dir moves the runtime's cache files and --no-cache writes none (see Known runtime issues). A page's prompt takes 277 of the 4,096 tokens the file holds, so a page's answer can be up to 3,819 tokens long (the original's longest answer in our tests, on the dense-equation page, was 2,132 tokens). Several pages are read one request each, in order, and their texts are printed one after the other, each after <PAGE>.

The output is the model's raw text with its tags: for each block <|det|>label [x1, y1, x2, y2]<|/det|> and then the block's content. The box is on a 0–999 scale of the page as you gave it, before the padding (checked on our drawn page, table, receipt and formula, where we know where the text was drawn).

From your own code, pad the page the same way (page_picture in the script), send it alone in one user message, start a new conversation for every page, and decode as the script does: the most likely token at each step (the file sets no sampler, so LiteRT-LM takes top-p with k = 1) with NoRepeatNgramConfig(35, 128), the original's no-repeat setting for one page.

Correctness

The bar: for each test case, the answer must match what the fp32 original model writes from the same page (transformers, greedy with the original's no-repeat rule for one page, one 1,024 Γ— 1,024 view per page, the original's own code). Match means, on the text without the block labels and boxes: character error rate ≀ 0.02 for text (≀ 0.05 for photos and pages), at least 18 of 20 table cells, the same LaTeX, every receipt field the original finds, and an end-of-sequence stop. For the 3-page document the file reads one page per request, and its three answers together must be within a character error rate of 0.05 of the original's own 3-page answer (its multi-page mode, Multi page parsing.). There are 19 cases. 14 are regions: a text line, a title, three paragraphs (plain, small, photographed), a Chinese paragraph, a table, two formulas, a drawn receipt, a receipt photo counted twice (once read as text, once checked for its fields), a calligraphy sample and a second receipt photo. 4 are pages: a drawn page, a dense-equation page, a handwritten derivation and a form photo. The last case is 3 consecutive pages of a MarathonDocBench document. The receipt photo, the dense-equation page, the handwritten derivation and the calligraphy sample are the example pictures of the Xiaomi-OCR-0 card; the second receipt photo and the form photo come from the transformers documentation.

Device Backend Regions that pass (of 14) Pages that pass (of 4) 3-page document
Mac (Apple M4 Max) CPU 13 3 pass
Mac (Apple M4 Max) GPU 12 3 pass
Pixel 11 Pro CPU 13 2 pass
Pixel 11 Pro GPU 11 2 pass
Galaxy S26 CPU 13 2 pass
Galaxy S26 GPU decoder, CPU vision encoder 11 2 pass

LiteRT-LM 0.19.2 on the phones (the PyPI Android wheel, 4 threads for the decoder on the CPU), one process per picture, so the 3-page document took three, one page per request. On the Galaxy S26 the GPU row runs the decoder on the GPU and the vision encoder on the CPU. On the Mac, the same with LiteRT-LM 0.19.2 from PyPI, without the runtime's cache files on the CPU and with a new cache directory on the GPU (written by the first case).

Each miss on the phones' CPU starts at a near tie: a token where the original's two best candidates are less than 0.1 logit (raw score) apart and the file writes the second. The table counts them as misses. Before we had any number from this file we fixed a rule for such splits: they do not count against the file when a second run of the original with another thread count (4 instead of 12) writes the original's token again, which it did in all three. The Pixel 11 Pro and the Galaxy S26 write the same text, byte for byte, in all 19 cases on the CPU.

  • The handwritten derivation. At the end of the first block's text the original writes ;, 0.0038 logit ahead of :. The file writes :, and the rest of the page takes another path (character error rate 0.42 against the original).
  • The dense-equation page. Where the original writes ^ {r}, its two best candidates for the brace, { and {\, are 0.031 logit apart. The file writes {\ and then tau (^ {\tau}), and its answer ends up 0.076 from the original's in character error rate (the bar is 0.05).
  • The second receipt photo. For the top edge of the second block's box the original writes 142, 0.067 logit ahead of 138. The file writes 138 and then reads the receipt differently (character error rate 0.16).

On the Mac's CPU the two misses also start where the original's two best candidates are less than 0.1 logit apart, both at a number in a box, but there the file writes a third number, so the rule does not apply: 188 for the bottom edge of the handwritten derivation's first box, where the original writes 190 (0.096 logit ahead of 191), and 139 for the top edge of the second receipt photo's second box, where the original writes 142 (0.067 logit ahead of 138). The two answers end 0.25 and 0.18 from the original's in character error rate. The Mac's CPU and the phones' CPU write different text in 12 of the 20 answers (the 3-page document counts as three); in 11 of them the first difference is a number in a box.

On the Mac's GPU each of the three misses starts where the file writes the original's second candidate, but 0.11 to 0.24 logit behind its first, so the rule does not apply: in the small paragraph (0.18), the handwritten derivation (0.24) and the second receipt photo (0.11). With the vision encoder on the CPU instead, the Mac writes the same text, byte for byte.

On the Pixel 11 Pro's GPU none of the five misses starts at a near tie: the original's margin at the first different token is 0.54 to 23.6 logits. On the text line the file leaves out the first block's <|det|> mark, which the original writes 23.6 logits ahead of its next candidate; on the other four the file writes another number in the first block's box, or another first label: 113 for the original's 95 (the small paragraph), 10 for 6 (the dense-equation page), 470 for 468 (the handwritten derivation) and text for header (the second receipt photo, 2.25 logits). The Galaxy S26's GPU decoder misses the same five cases, each from a token within the first 10 it writes, where the original's margin is also 0.54 to 23.6 logits; on the text line it leaves out the same <|det|> mark. The two phones' GPUs write the same text in 6 of the 20 answers. We have not found the cause.

Performance

One request = one process with a new Engine, timed inside the process from engine creation to the last token: engine creation, vision encode, prefill and decode. "Paragraph" = a drawn English paragraph (277 tokens in, 109 out). "Page" = the drawn English page of samples/page_en.png (277 tokens in, 262 out). First token = from the request to the first token of the answer (vision encode and prefill). Three requests per table cell (two for the Galaxy S26 GPU), shortest first, each started once the phone's CPU clocks were not capped (after a rest of 32 s to 5 min; one exception, marked); (cold) = the runtime's cache directory was empty. Bold = the quicker form on that device. LiteRT-LM 0.19.2 (the PyPI Android wheel), 4 threads for the decoder on the CPU, the runtime's cache beside the model.

Galaxy S26 (SM8850) Paragraph request (s) Page request (s) Engine (s) First token (s) Decode (tok/s) Peak (GiB)
CPU 11.5 / 12.2 / 16.8 (cold) 18.1 / 19.3 / 22.2 (cold) 1.39–7.32 4.2–4.8 19.5–20.6 5.5–6.5
GPU decoder, CPU vision encoder β€” 24.8 / 27.2 (cold) 6.39–9.05 3.5–3.6 17.2–17.5 3.3–3.8
Pixel 11 Pro (Tensor G6) Paragraph request (s) Page request (s) Engine (s) First token (s) Decode (tok/s) Peak (GiB)
CPU 21.7 (cold) / 34.8 (cold) / 39.9 (CPU capped) 27.8 / 30.9 / 50.8 (cold) 1.95–25.70 8.4–9.5 15.3–16.5 6.4–7.0
GPU 27.6 / 28.3 / 89.8 (cold) 43.8 / 44.1 / 99.8 (cold) 11.15–73.95 4.8–5.3 9.4–9.6 β€”

Peak is the process's peak resident memory (VmHWM). On the Galaxy S26 GPU row it counts only the part of the GPU's memory that the Adreno driver maps into the process. On the Pixel 11 Pro GPU, VmHWM reads only 0.4–1.0 GiB because it leaves the GPU's memory out, so that cell is empty. The Galaxy S26 was at Android thermal status 0 to 2, the Pixel 11 Pro at 1. The 39.9 s paragraph request on the Pixel 11 Pro CPU started while the phone still capped its CPU clocks (decode 10.5 tok/s); the row's other ranges leave it out. Back to back, as in our correctness tests, the Pixel 11 Pro capped its CPU clocks and decoded at 6.4–12.4 tok/s, while the Galaxy S26 kept 15.0–20.9 tok/s at thermal status up to 3 (severe). On the Pixel 11 Pro GPU the first request with an empty cache spends 68–74 s creating the engine (11–13 s afterwards) and writes a 2.8 GB weight cache.

Mac (Apple M4 Max) Paragraph request (s) Page request (s) Engine (s) First token (s) Decode (tok/s) Peak (GiB)
CPU 3.8 / 3.9 6.6 / 20.7 (cold) 0.54–9.78 1.3–3.0 32.3–57.0 2.2–2.5
GPU 3.2 / 3.3 4.7 / 7.4 (cold) 1.71–4.54 0.54–0.57 108.8–109.7 4.7–5.5
GPU decoder, CPU vision encoder 2.9 / 2.9 4.4 / 7.5 (cold) 0.84–4.02 1.05–1.07 103.3–109.1 2.4–3.1

On the Mac each row holds four requests made on 2026-11-16 (LiteRT-LM 0.19.2 from PyPI on macOS 27.0, 4 threads for the decoder on the CPU, a runtime cache directory for each row): the page once with an empty cache directory (cold) and once with the cache written, the paragraph twice with the cache written; the Engine, First token, Decode and Peak cells give the range over the four. Each request started right after the model file, and for a cached request its cache files, had been read in full (untimed). Peak there is the process's peak memory footprint as /usr/bin/time -l reports it, not VmHWM. The CPU's slowest decode (32.3 tok/s) is the cold page request.

On the Galaxy S26 and the Pixel 11 Pro, this file on the CPU is the fastest form we found that passes our tests (with the near-tie rule in Correctness). The GPU took longer per page on both phones and missed 5 tests, and a version with the decoder's routed experts in int4 was faster on the Pixel 11 Pro's CPU but missed the second receipt photo at a token that is not a near tie.

Conversion notes

  • Source: baidu/Unlimited-OCR at revision 8249d20b (3,336,106,240 parameters in the checkpoint: 401,372,160 in the vision encoder and projector, 2,934,734,080 in the text decoder).
  • Text decoder: 2 signatures, the graph's entry points: a prefill signature of 278 positions for the 277-token one-page prompt, and decode for one token at a time. The prefill signature takes the whole prompt in one call: with a 276-position one, LiteRT-LM split the 277-token prompt into two prefill calls and the answers came out garbled from the first token in our tests. A 4,096-token context. The embedding table in its own graph, for LiteRT-LM's fast_vlm path. The KV cache has 405 slots: the 277 prompt positions stay, and a ring holds the last 128 generated positions, the positions the original's attention reads; the caches are declared with LiteRT-LM's local cache type, its type for sliding-window and ring caches (declared global, a cache of 405 slots would cap the context at 405 tokens), so the runtime keeps the 4,096-token context. Each mixture-of-experts layer computes all 64 routed experts as two fully connected ops and keeps the 6 the router picks (the others get weight 0). Dynamic int8, channelwise, on the linear layers (the position table stays float) and on the embedding; fp32 activations declared for the GPU. The decoder's and the embedding's weights sit in TFLiteWeights sections of the same file. Toolchain: litert-torch 0.9.3, litert-converter 0.4.0, ai-edge-litert 2.2.0 and torch 2.13.0; quantized with ai-edge-quantizer 0.9.0.
  • Vision: the encoder and projector re-authored for a fixed 1,024 Γ— 1,024 picture, exported with litert-torch 0.9.3, litert-converter 0.4.0, ai-edge-litert 2.2.0 and torch 2.13.0, weights cast to fp16. On 4 pictures the fp32 export's output correlates at 0.9999999997 or higher with the checkpoint's own modules, and the fp16 copy's at 0.99999993 or higher with the same modules.
  • Template: the picture and document parsing., the original's single-page prompt; start token <|begin▁of▁sentence|>, stop token <|end▁of▁sentence|>. The tokenizer is the repository's own tokenizer.json.
  • Packing: litert-lm-builder 0.19.0; the decoder's and the embedding's weights then moved into external-weights sections with the externalize tool of hf-to-litertlm (LiteRT-LM 0.19.1 tools).
  • Machine-readable facts: litertlm_manifest.json.

Known runtime issues

  • Tested with LiteRT-LM 0.19.1 and 0.19.2. We ran this file with 0.19.1 (one page, on a Mac's CPU) and 0.19.2 (everything else), and not with 0.19.0 or earlier. The decoder's and the embedding's weights sit in external-weights sections: an Android build of v0.18.0 or v0.18.1 crashes (SIGSEGV) while it builds the GPU engine for a file whose weights sit in such sections (LiteRT-LM #3986).
  • One page per request. The graph is built for one page's fixed prompt (start token, 273 image tokens, 3 instruction tokens), so several pages are several requests and the model does not see the other pages. The original's multi-page mode (Multi page parsing.) reads a document's pages in one request and can join blocks that continue across pages. On our 3-page test document the original's answers page by page and its own 3-page answer carry the same text once the block labels, boxes and page marks are dropped.
  • One conversation per page. The graph is built for a page's prompt as the first turn of a conversation. From the second turn on, LiteRT-LM also puts the previous answer's stop token into the KV cache with a zero embedding, because this file holds the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.19.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). Start a new conversation for every page, as the script does.
  • Not the original's default single-page mode. For a single page the original's README offers two settings, and its example uses the tiled one (gundam: a 1,024 base size with 640 Γ— 640 crops). This file always reads one 1,024 Γ— 1,024 view per page (the base setting, also the original's view of each page of a multi-page document), and our bar compares against that mode.
  • All 64 experts on every token. LiteRT's MoE operation, which computes only the chosen experts, does not yet take this model's SiLU experts (LiteRT #11192 and #11188, both open on 2026-11-15). So the decoder computes all 64 and weights them by the router: the same answer, more work per token.
  • The runtime's cache files are large. The first run writes them next to the model: about 4.4 GB for the CPU and 2.8 GB with both on the GPU (about 7.2 GB if you run both); with the decoder on the GPU and the vision encoder on the CPU, 4.4 GB on the Mac and 1.6 GB on the Galaxy S26. Keep them: without the cache (--no-cache) one page peaked at an 8.5 GiB memory footprint on the Mac CPU instead of 2.2–2.5 GiB, and at 7.3 GiB on the Mac GPU instead of 4.7–5.5 GiB.
  • 8 GB Android phones. On a Pixel 8a (8 GB) the CPU read our one-line test like the original, at 1.0 token per second (97 s for the request). With the decoder on the GPU and the vision encoder on the CPU, the phone's available memory fell to 0.29 GiB while the runtime wrote its cache files, and our test harness stopped the process (it stops a run below 0.38 GiB).
  • The vision encoder on the Galaxy S26's GPU was not tried. On the Galaxy S26 we ran the GPU decoder with the vision encoder on the CPU. With our Unlimited-OCR-2 conversion, a file of about the same size (3.81 GB), the decoder and the vision encoder both on that phone's GPU took its available memory below 1 GB and our test harness stopped the process.
  • No NPU file. On a Galaxy S26, compiling this file for the NPU on the phone (LiteRT-LM with LiteRT's Qualcomm compiler plugin) refuses the decoder's int8 linear layers (int8 weights with fp32 activations) and its KV-cache writes, so the NPU would get the decoder in 186 separate pieces, with the CPU running the layers between them. We ship no NPU file.

License and changes

MIT, inherited from baidu/Unlimited-OCR. Changes from the original work: exported to the LiteRT-LM format for one 1,024 Γ— 1,024 page per request; a KV cache of 277 kept and 128 ring slots; each mixture-of-experts layer computed over all experts with the router's weights; the text decoder's linear layers and the embedding quantized to int8; the vision encoder and projector cast to fp16; a prompt template for one page. The sample pictures in samples/ were drawn with the fixture script of hf-to-litertlm (DejaVu fonts; sources and licenses in samples/SOURCES.md). Problems with these files: open a discussion in this repo.

Downloads last month
80
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for litert-community/Unlimited-OCR

Quantized
(41)
this model