aenawi/MonkeyOCRv2-B-GGUF

MonkeyOCRv2-B-Parsing, GGUF for llama.cpp

MonkeyOCRv2-B-Parsing (Apache-2.0), converted to GGUF so that an unmodified llama-server can run it.

File	What	Size	sha256
MonkeyOCRv2-B.q8_0.gguf	the text model (Qwen3, 0.6B), Q8_0	805 MB	4ad745d04e5c268c9dda399a25d22761ccdce1ea4f8d4092fcb75eac6256b718
MonkeyOCRv2-B.mmproj-f16.gguf	the vision part, F16	253 MB	f2fb03e3a0b631c86ece0caea58682e1d980258eade67339c7cac4465e42e569

Made from zenosai/MonkeyOCRv2-B-Parsing at commit e8871a0057c98378219b369f924a25ba10f392bf, with llama.cpp's own converter at build b11382. Tested with llama-server b11382 on Metal (Apple M2 Max), Vulkan (Intel Iris Xe) and CPU.

How it was converted

llama.cpp has no MonkeyOCRv2 architecture, but it has both of its parts:

• The text model is a plain Qwen3. Converted as one.
• The vision part is dots.ocr's vision encoder, line for line, except for the order of its 2D rotary table: MonkeyOCRv2 interleaves the height and width frequencies (h, w, h, w, …), dots.ocr puts all the height ones first. The rows of Q and K are reordered to dots.ocr's order, which leaves every attention score unchanged, and the part is converted as dots.ocr's vision encoder.
• llama.cpp wraps a dots.ocr image in <|img|> … <|endofimg|>. MonkeyOCRv2's own tokens for that are <|vision_start|> and <|vision_end|> (ids 151652, 151653), so those two are renamed in the GGUF's vocabulary.

How to run it

llama-server -m MonkeyOCRv2-B.q8_0.gguf --mmproj MonkeyOCRv2-B.mmproj-f16.gguf -c 16384

Use it as upstream's pipeline does: the system line You are a helpful assistant., greedy decoding, and every image scaled to at most ~1 megapixel (upstream's MOCR2_MAX_PIXELS, 1,003,520). First a layout pass over the page (Please output the categories and coordinates of the document elements in reading order.), then each block cut out and read with its label's prompt (text, table in OTSL, formula in LaTeX). See the upstream repository for the prompts and the pipeline.

Results

On 10 hand-checked scanned pages, mostly Arabic, at 160 DPI: 2.5% character error rate on the GPU (Q8_0 and F16 the same), 4.0% on the CPU. About 6 s a page on an M2 Max GPU, 41 s on an Intel Iris Xe.

License

Apache-2.0, as the original model. All credit for the model to its authors: MonkeyOCRv2 (paper).