---
datasets:
- zenosai/MonkeyDocv2
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
---
MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
[](https://arxiv.org/abs/2607.11562)
[](https://huggingface.co/collections/zenosai/monkeyocrv2)
[](https://modelscope.cn/datasets/zenosai/MonkeyDocv2)
[](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aopen+is%3Aissue)
[](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aissue+is%3Aclosed)
[](http://vlrlabmonkey.xyz:8891/)
## News
* `2026.07.11` 🚀 We release [MonkeyOCRv2](https://github.com/Yuliang-Liu/MonkeyOCRv2), including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.
## Introduction
MonkeyOCRv2 is a text-centric visual foundation model that unifies fine-grained text modeling, cross-task representation learning, and cross-lingual generalization in a single encoder. MonkeyOCRv2 generalizes effectively across a broad range of OCR and document intelligence tasks, including multilingual document parsing, document understanding, text recognition, formula recognition, document tampering detection, scene text detection, and overlapping text segmentation.
## Model Zoo
#### 1. Vision Encoder
#### 2. Document Parsing Model
| Model |
Link |
Total Params |
ViT |
LLM |
All |
Digit. |
Photo. |
Latin Avg. |
DE |
EN |
ES |
FR |
ID |
IT |
NL |
PT |
VI |
Non-Latin Avg. |
AR |
HI |
JP |
KO |
RU |
TH |
ZH |
ZH-T |
| MonkeyOCRv2-S-Parsing | HuggingFace ModelScope |
0.6B |
0.03B |
0.6B |
82.5 |
87.9 |
80.7 |
83.2 |
87.3 |
83.6 |
76.8 |
73.6 |
85.4 |
87.2 |
85.5 |
87.4 |
81.9 |
81.7 |
91.2 |
87.1 |
69.9 |
88.7 |
78.0 |
79.8 |
84.4 |
74.7 |
| MonkeyOCRv2-B-Parsing | HuggingFace ModelScope |
0.7B |
0.1B |
0.6B |
83.3 |
88.1 |
81.7 |
84.2 |
87.7 |
84.5 |
75.2 |
78.4 |
86.5 |
88.6 |
86.1 |
87.9 |
83.2 |
82.1 |
90.7 |
87.2 |
71.9 |
87.6 |
80.1 |
80.8 |
83.6 |
75.3 |
#### 3. Document Understanding Model
| Model |
Link |
Total Params |
Overall |
DocVQA |
InfoVQA |
DF |
KLC |
WTQ |
ChartQA |
DT-VQA |
OCRBench |
| MonkeyOCRv2-S-Und | HuggingFace ModelScope |
1.7B |
55.9 |
79.3 |
44.5 |
65.1 |
37.6 |
43.0 |
62.0 |
63.1 |
52.2 |
| MonkeyOCRv2-B-Und | HuggingFace ModelScope |
1.8B |
57.2 |
79.3 |
46.3 |
65.8 |
38.2 |
43.2 |
62.0 |
64.3 |
58.1 |
## Quick Start
### Vision Encoder
#### 1. Install
Install transformers and flash attention:
```bash
conda create -n MonkeyOCRv2 python=3.10
conda activate MonkeyOCRv2
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
pip install transformers==4.57.6
pip install flash-attn==2.7.4.post1 --no-build-isolation
pip install accelerate
pip install qwen_vl_utils
```
#### 2. Download Model Weights
Download our model from Huggingface.
```bash
python download_model.py -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS
```
You can also download our model from ModelScope.
```bash
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS
```
#### 3. Extract Image Feature
```bash
cd vision
# For MonkeyOCRv2-B and MonkeyOCRv2-S
python extract_feature.py
# For MonkeyOCRv2-AS
python extract_feature_vitae.py
```
### Document Parsing
#### 1. Install
Install vLLM following its [official guide](https://docs.vllm.ai/en/v0.11.2/getting_started/installation/gpu/):
```bash
conda create -n MonkeyOCRv2Parsing python=3.10
conda activate MonkeyOCRv2Parsing
pip install uv
uv pip install vllm==0.11.2 --torch-backend=auto -i https://pypi.tuna.tsinghua.edu.cn/simple requests
pip install -r parsing/requirements.txt
```
#### 2. Download Model Weights
Download our model from Huggingface.
```bash
python download_model.py -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
```
You can also download our model from ModelScope.
```bash
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
```
#### 3. Inference
Parse a single document or a directory containing PDFs or images:
```bash
cd parsing
python parse.py \
-i ../images_test/ar.JPEG \
-o output/test \
-m ../model_weight/MonkeyOCRv2-B-Parsing \
-g 500 \
--draw-layout \
--skip-processed
# Show help messages
python parse.py -h
```
#### 4. Web Demo
Start gradio web demo:
```bash
cd parsing
python demo/gradio_demo.py \
--model-path ../model_weight/MonkeyOCRv2-B-Parsing \
--output-dir output/demo_outputs
```
### Document Understanding
#### 1. Install
See install part of MonkeyOCRv2.
#### 2. Download Model Weights
Download our model from Huggingface.
```bash
python download_model.py -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und
```
You can also download our model from ModelScope.
```bash
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und
```
#### 3. Inference
```bash
cd understanding
python infer.py \
-m ../model_weight/MonkeyOCRv2-B-Und \
-i ../images_test/vqa.png \
-q 'What is the serving size?'
# Show help messages
python infer.py -h
```
## Visualization
Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
## Evaluation Results
#### 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of [OpenOCR](https://github.com/Topdu/OpenOCR/blob/main/docs/svtrv2.md).
| Model |
Overall |
Union14M-Benchmark |
Chinese Benchmarks |
Occlusion SceneText |
| Avg |
Artistic |
Context less |
Curve |
General |
Multi Oriented |
Multi Words |
Saliency |
Avg |
Scene |
Web |
Document |
Hand writing |
| ABINet |
73.7 |
75.7 | 71.7 | 74.7 | 80.4 | 79.8 | 69.0 | 76.8 | 77.6 |
70.3 | 66.6 | 63.2 | 98.2 | 53.1 |
75.0 |
| MAERec |
81.6 |
85.2 | 79.0 | 84.2 | 89.1 | 84.6 | 87.1 | 85.9 | 86.3 |
83.1 | 84.4 | 83.0 | 99.5 | 65.6 |
76.4 |
| CPPD |
80.4 |
81.9 | 76.5 | 82.9 | 86.2 | 83.5 | 78.7 | 81.9 | 83.5 |
81.7 | 82.7 | 82.4 | 99.4 | 62.3 |
79.6 |
| IGTR-AR |
81.0 |
84.9 | 77.0 | 82.4 | 90.4 | 84.4 | 91.2 | 84.0 | 84.7 |
81.7 | 82.0 | 81.7 | 99.5 | 63.8 |
76.3 |
| SMTR |
80.4 |
85.0 | 76.8 | 83.9 | 89.1 | 83.7 | 87.7 | 89.3 | 84.6 |
82.7 | 83.4 | 83.0 | 99.3 | 65.1 |
73.5 |
| SVTRv2 |
83.1 |
86.1 | 79.3 | 86.1 | 90.6 | 85.1 | 89.0 | 86.7 | 86.2 |
83.3 | 83.5 | 83.3 | 99.5 | 67.0 |
80.0 |
| |
| CRNN (ResNet) |
58.7 |
49.2 | 51.2 | 62.3 | 48.1 | 68.2 | 13.0 | 60.4 | 41.4 |
68.8 | 63.8 | 68.2 | 97.0 | 46.1 |
58.0 |
| CRNN (MonkeyOCRv2-S) |
67.3 |
65.2 | 63.7 | 73.0 | 71.1 | 74.5 | 28.6 | 72.1 | 73.4 |
74.2 | 73.0 | 74.9 | 96.9 | 51.8 |
62.4 |
| PARSeq (ViT) |
82.2 |
84.3 | 76.5 | 83.4 | 87.6 | 84.9 | 88.8 | 84.3 | 84.4 |
82.4 | 84.2 | 82.8 | 99.5 | 63.0 |
79.9 |
| PARSeq (MonkeyOCRv2-S) |
84.3 |
87.6 | 78.6 | 86.4 | 92.1 | 85.4 | 93.9 | 88.7 | 87.7 |
83.7 | 84.6 | 83.2 | 99.5 | 67.3 |
81.5 |
#### 2. Formula recognition results on [OmniDocBench 1.6](https://github.com/opendatalab/OmniDocBench), [MathWriting](https://arxiv.org/pdf/2404.10690), and [UniMER-Test](https://github.com/opendatalab/unimernet).
| Model |
Params |
Overall |
OmniDocBench 1.6 |
MathWriting |
SPE |
CPE |
HWE |
SCE |
| CDM |
ExpRate |
CDM |
ExpRate |
CDM |
ExpRate |
CDM |
ExpRate |
CDM |
ExpRate |
CDM |
ExpRate |
CDM |
ExpRate |
| Pix2tex |
25.5M |
53.8 |
23.3 |
69.4 | 27.0 |
0.4 | 0.0 |
96.2 | 72.4 |
64.9 | 7.1 |
24.5 | 0.6 |
67.6 | 32.8 |
| Texify |
312M |
67.3 |
40.4 |
76.5 | 46.4 |
26.6 | 2.0 |
98.5 | 91.0 |
70.4 | 28.2 |
52.7 | 23.6 |
79.3 | 51.3 |
| UniMERNet-B |
325M |
89.5 |
64.5 |
90.4 | 59.5 |
63.8 | 12.3 |
99.1 | 93.3 |
96.0 | 80.5 |
94.0 | 64.3 |
93.7 | 77.0 |
| UniMERNet-S |
202M |
89.8 |
64.0 |
90.1 | 59.1 |
65.9 | 12.7 |
99.1 | 93.4 |
95.9 | 77.7 |
93.7 | 63.9 |
94.1 | 76.9 |
| |
| UniMERNet-T (Swin) |
107M |
89.4 |
61.8 |
89.9 | 57.2 |
65.6 | 12.9 |
99.1 | 92.3 |
94.9 | 69.9 |
93.3 | 61.9 |
93.8 | 76.6 |
| UniMERNet-T (MonkeyOCRv2-S) |
110M |
90.9 |
66.4 |
90.8 | 61.1 |
70.8 | 16.2 |
99.2 | 93.8 |
96.1 | 79.2 |
94.3 | 69.5 |
94.0 | 78.6 |
#### 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of [MMOCR](https://github.com/open-mmlab/mmocr) and [DPText-DETR](https://github.com/ymy-k/DPText-DETR).
#### 4. Document tampering detection results on [DocTamper](https://github.com/qcf-568/DocTamper) benchmark.
| Method |
Params |
Overall |
DocTamper-Test |
DocTamper-FCD |
DocTamper-SCD |
| IoU |
F |
IoU |
P |
R |
F |
IoU |
P |
R |
F |
IoU |
P |
R |
F |
| PSCC-Net |
5M |
13.7 | 31.3 | 17.0 | 25.0 | 83.0 | 39.0 |
13.0 | 19.0 | 82.0 | 30.0 |
11.0 | 15.0 | 83.0 | 25.0 |
| UperNet |
67M |
49.3 | 54.0 | 70.0 | 66.0 | 60.0 | 62.0 |
30.0 | 57.0 | 35.0 | 43.0 |
48.0 | 57.0 | 58.0 | 57.0 |
| CAT-Net |
114M |
67.3 | 71.0 | 78.0 | 75.0 | 69.0 | 72.0 |
66.0 | 85.0 | 70.0 | 76.0 |
58.0 | 65.0 | 65.0 | 65.0 |
| Swin-UPer |
81M |
66.7 | 71.7 | 79.0 | 75.0 | 72.0 | 73.0 |
64.0 | 80.0 | 70.0 | 75.0 |
57.0 | 66.0 | 68.0 | 67.0 |
| SegFormer |
85M |
70.3 | 74.0 | 81.0 | 77.0 | 74.0 | 75.0 |
69.0 | 82.0 | 74.0 | 78.0 |
61.0 | 68.0 | 70.0 | 69.0 |
| Mask2Former |
69M |
69.7 | 78.0 | 84.0 | 82.0 | 83.0 | 82.0 |
66.0 | 81.0 | 75.0 | 78.0 |
59.0 | 70.0 | 79.0 | 74.0 |
| ConvNext |
122M |
69.7 | 75.3 | 84.0 | 81.0 | 78.0 | 79.0 |
62.0 | 76.0 | 71.0 | 74.0 |
63.0 | 71.0 | 74.0 | 73.0 |
| ConvNextV2 |
121M |
72.7 | 77.7 | 86.0 | 82.0 | 79.0 | 81.0 |
65.0 | 79.0 | 75.0 | 77.0 |
67.0 | 74.0 | 76.0 | 75.0 |
| InternImage |
128M |
73.3 | 77.7 | 84.0 | 81.0 | 77.0 | 79.0 |
72.0 | 83.0 | 79.0 | 81.0 |
64.0 | 73.0 | 74.0 | 73.0 |
| ASC-Former |
80M |
68.2 | 80.8 | 81.5 | 91.8 | 87.8 | 89.8 |
61.3 | 74.9 | 77.1 | 76.0 |
61.9 | 78.0 | 75.0 | 76.5 |
| DTD |
66M |
77.0 | 79.7 | 84.0 | 81.0 | 77.0 | 79.0 |
79.0 | 88.0 | 82.0 | 85.0 |
68.0 | 75.0 | 76.0 | 75.0 |
| |
| FFDN* (ViTAEv2) |
69M |
70.7 | 82.7 |
69.4 | 76.2 | 88.7 | 82.0 |
79.0 | 92.5 | 84.4 | 88.3 |
63.6 | 79.1 | 76.5 | 77.8 |
| FFDN (MonkeyOCRv2-AS) |
71M |
78.2 | 87.5 |
87.4 | 94.8 | 91.8 | 93.3 |
79.9 | 90.4 | 87.4 | 88.9 |
67.2 | 81.0 | 79.8 | 80.4 |
* denotes models trained with the ViTAEv2 pretrained by DeepSolo
#### 5. Overlapping text segmentation results on [MOT](https://github.com/willpat1213/MOTS) dataset.
| Model |
mIoUText |
IoUOcc |
IoUOccd |
IoUOv |
| Unet |
62.2 |
80.2 |
65.7 |
40.7 |
| Deeplab v3 |
67.9 |
83.2 |
71.2 |
49.3 |
| OCRNet |
65.8 |
81.0 |
68.5 |
47.8 |
| Segformer |
69.0 |
83.6 |
74.1 |
49.3 |
| MaskFormer |
68.4 |
83.5 |
70.3 |
51.4 |
| TexRNet |
68.9 |
84.2 |
73.2 |
49.3 |
| EAFormer |
69.1 |
83.8 |
74.2 |
50.5 |
| WASNet |
70.8 |
84.8 |
74.4 |
53.1 |
| |
| Mask2Former (ResNet) |
70.3 |
84.7 |
73.3 |
52.8 |
| Mask2Former (MonkeyOCRv2-AS) |
76.6 |
88.6 |
83.4 |
57.7 |
| MOTS (ResNet) |
72.6 |
85.2 |
77.5 |
54.9 |
| MOTS (MonkeyOCRv2-AS) |
76.9 |
88.6 |
82.6 |
59.4 |
#### 6. Document parsing results on [MDPBench](https://github.com/Yuliang-Liu/MultimodalOCR/tree/main/MDPBench), a comprehensive multilingual benchmark for real-world document parsing.
| Model |
Total Params |
ViT |
LLM |
All |
Digit. |
Photo. |
Latin Avg. |
DE |
EN |
ES |
FR |
ID |
IT |
NL |
PT |
VI |
Non-Latin Avg. |
AR |
HI |
JP |
KO |
RU |
TH |
ZH |
ZH-T |
|
Closed-source VLMs
|
| ChatGPT-5.2-2025-12-11 |
- |
- |
- |
68.6 |
85.6 |
63.0 |
75.2 |
70.8 |
79.4 |
71.4 |
60.0 |
77.7 |
78.5 |
71.6 |
85.0 |
82.1 |
61.1 |
64.9 |
63.4 |
55.8 |
65.4 |
60.7 |
63.8 |
56.3 |
58.7 |
| Claude-Sonnet-4.6 |
- |
- |
- |
73.1 |
85.0 |
69.3 |
79.2 |
79.8 |
80.6 |
72.8 |
66.5 |
82.3 |
83.3 |
76.7 |
88.0 |
83.1 |
66.2 |
67.8 |
71.7 |
63.4 |
64.3 |
70.8 |
65.2 |
61.3 |
65.1 |
| Doubao-2.0-pro |
- |
- |
- |
74.2 |
78.9 |
72.8 |
75.7 |
82.8 |
74.4 |
69.0 |
70.0 |
73.3 |
82.0 |
69.9 |
83.4 |
76.5 |
72.5 |
81.3 |
75.7 |
65.8 |
74.7 |
63.3 |
71.9 |
71.9 |
75.2 |
| Gemini-3-pro |
- |
- |
- |
86.4 |
90.4 |
85.1 |
88.4 |
91.2 |
90.6 |
83.4 |
82.7 |
91.5 |
91.6 |
87.7 |
91.4 |
85.9 |
84.1 |
89.4 |
90.4 |
74.8 |
85.5 |
84.9 |
80.6 |
85.1 |
82.1 |
|
Open-source VLMs
|
| InternVL-3.5-8B | 8.3B | 0.3B | 8B | 42.7 | 59.7 | 37.0 | 53.4 | 39.8 | 64.2 | 47.5 | 42.7 | 53.8 | 60.6 | 52.2 | 63.2 | 57.0 | 30.6 | 8.2 | 9.0 | 45.6 | 30.3 | 26.1 | 10.8 | 55.3 | 59.3 |
| MinerU-2.5 | 1.2B | 0.7B | 0.5B | 46.3 | 61.9 | 40.8 | 63.0 | 68.8 | 78.4 | 54.7 | 57.3 | 67.5 | 75.2 | 60.4 | 58.8 | 46.0 | 27.4 | 1.3 | 9.0 | 39.1 | 14.7 | 8.6 | 11.3 | 72.9 | 62.2 |
| DeepSeek-OCR | 3.4B | 0.4B | 3B | 51.8 | 80.7 | 42.2 | 54.5 | 55.0 | 58.3 | 44.1 | 43.2 | 60.9 | 69.3 | 52.4 | 53.0 | 54.1 | 48.9 | 56.9 | 52.2 | 49.1 | 28.2 | 36.2 | 49.4 | 59.7 | 59.2 |
| MonkeyOCR-pro-3B | 3.7B | 0.7B | 3B | 52.2 | 68.0 | 47.0 | 65.1 | 71.7 | 77.9 | 55.9 | 62.1 | 66.2 | 74.5 | 66.3 | 71.1 | 40.2 | 37.6 | 4.6 | 4.2 | 55.2 | 60.5 | 42.6 | 9.1 | 72.2 | 52.4 |
| Nanonets-OCR-s | 4.7B | 0.7B | 4B | 63.7 | 78.8 | 58.7 | 71.3 | 75.1 | 78.5 | 61.2 | 62.5 | 70.3 | 81.0 | 69.6 | 75.9 | 67.5 | 55.0 | 59.5 | 61.8 | 55.9 | 51.2 | 43.5 | 39.5 | 67.4 | 61.5 |
| Nanonets-OCR2-3B | 3.7B | 0.7B | 3B | 64.2 | 79.2 | 59.3 | 71.4 | 76.7 | 76.4 | 61.8 | 66.1 | 68.4 | 78.5 | 74.1 | 74.2 | 66.0 | 56.2 | 60.2 | 59.2 | 52.1 | 54.7 | 45.5 | 44.6 | 68.3 | 65.1 |
| Qwen3.5-Instruct-9B | 9.7B | 0.7B | 9B | 65.7 | 74.8 | 62.7 | 72.5 | 72.8 | 72.0 | 72.0 | 64.4 | 66.2 | 77.6 | 74.5 | 79.1 | 74.0 | 58.2 | 53.4 | 56.2 | 55.7 | 60.3 | 54.7 | 56.7 | 60.8 | 67.5 |
| GLM-OCR | 0.9B | 0.4B | 0.5B | 67.3 | 77.9 | 63.7 | 78.7 | 82.7 | 84.5 | 75.8 | 76.2 | 79.7 | 82.8 | 80.2 | 77.4 | 69.2 | 54.3 | 21.7 | 39.6 | 65.5 | 61.2 | 64.2 | 27.4 | 78.5 | 76.7 |
| Qwen3-VL-Instruct-8B | 8.3B | 0.3B | 8B | 68.3 | 78.4 | 65.0 | 73.6 | 73.7 | 71.4 | 69.3 | 66.2 | 68.5 | 79.1 | 78.3 | 82.2 | 73.4 | 62.5 | 63.1 | 58.4 | 59.9 | 61.9 | 57.9 | 62.0 | 62.6 | 73.8 |
| HunyuanOCR | 1B | 0.4B | 0.6B | 68.3 | 80.2 | 64.3 | 72.4 | 75.0 | 73.1 | 63.0 | 66.1 | 69.9 | 80.3 | 61.4 | 81.9 | 80.6 | 63.7 | 68.3 | 73.1 | 55.6 | 68.9 | 52.2 | 60.7 | 66.8 | 64.2 |
| PaddleOCR-VL | 0.9B | 0.6B | 0.3B | 69.6 | 87.6 | 63.6 | 72.1 | 78.2 | 79.3 | 62.9 | 66.0 | 77.4 | 78.4 | 67.9 | 72.0 | 66.6 | 66.7 | 65.8 | 68.4 | 59.9 | 77.8 | 56.9 | 57.8 | 78.2 | 68.5 |
| olmOCR2 | 7.7B | 0.7B | 7B | 70.4 | 79.9 | 67.2 | 76.7 | 75.7 | 77.3 | 72.5 | 68.9 | 70.6 | 81.0 | 72.0 | 88.0 | 84.0 | 63.3 | 59.0 | 60.8 | 59.4 | 70.6 | 65.8 | 59.2 | 68.6 | 63.4 |
| MinerU-2.5-Pro | 1.2B | 0.7B | 0.5B | 71.0 | 86.2 | 66.1 | 74.6 | 78.3 | 79.5 | 63.4 | 67.4 | 78.0 | 79.7 | 72.1 | 78.6 | 74.2 | 67.0 | 56.6 | 72.2 | 59.1 | 77.6 | 62.6 | 61.8 | 76.5 | 69.7 |
| PaddleOCR-VL-1.6 | 0.9B | 0.6B | 0.3B | 75.0 | 82.8 | 72.6 | 78.0 | 84.1 | 79.7 | 69.2 | 74.8 | 81.6 | 82.0 | 74.7 | 76.4 | 79.3 | 71.6 | 69.4 | 65.6 | 68.7 | 82.5 | 70.7 | 62.3 | 78.0 | 75.7 |
| HunyuanOCR-1.5 | 1B | 0.4B | 0.6B | 76.8 | 86.2 | 73.6 | 79.7 | 79.6 | 80.4 | 74.2 | 70.0 | 81.5 | 84.5 | 78.4 | 86.4 | 82.4 | 73.5 | 71.8 | 71.6 | 65.5 | 75.7 | 67.4 | 77.7 | 80.8 | 77.2 |
| Kimi-K2.5 | 1T | 0.4B | 1T | 77.5 | 85.0 | 75.0 | 81.6 | 85.9 | 86.2 | 72.7 | 71.0 | 80.6 | 86.6 | 77.4 | 87.6 | 86.2 | 72.9 | 75.8 | 74.5 | 72.5 | 70.9 | 61.8 | 67.0 | 81.7 | 78.6 |
| PaddleOCR-VL-1.5 | 0.9B | 0.6B | 0.3B | 78.3 | 87.4 | 75.2 | 81.2 | 84.8 | 83.0 | 75.7 | 78.1 | 83.9 | 85.2 | 80.6 | 80.2 | 78.9 | 74.9 | 71.3 | 67.7 | 69.5 | 86.0 | 76.0 | 68.4 | 84.8 | 75.7 |
| chandra-ocr-2 | 5.3B | 0.5B | 4.8B | 79.7 | 87.8 | 77.1 | 82.7 | 86.6 | 86.5 | 69.7 | 70.3 | 84.6 | 87.4 | 82.7 | 90.7 | 85.6 | 76.4 | 78.2 | 81.1 | 68.8 | 80.3 | 74.0 | 78.5 | 73.8 | 76.3 |
| dots.mocr | 3B | 1.2B | 1.8B | 80.5 | 90.5 | 77.2 | 81.7 | 82.6 | 87.4 | 71.3 | 70.1 | 84.5 | 89.3 | 83.2 | 86.8 | 79.9 | 79.2 | 83.3 | 83.6 | 75.0 | 78.7 | 71.2 | 77.9 | 84.6 | 79.6 |
| MonkeyOCRv2-S-Parsing🤗 | 0.6B | 0.03B | 0.6B | 82.5 | 87.9 | 80.7 | 83.2 | 87.3 | 83.6 | 76.8 | 73.6 | 85.4 | 87.2 | 85.5 | 87.4 | 81.9 | 81.7 | 91.2 | 87.1 | 69.9 | 88.7 | 78.0 | 79.8 | 84.4 | 74.7 |
| MonkeyOCRv2-B-Parsing🤗 | 0.7B | 0.1B | 0.6B | 83.3 | 88.1 | 81.7 | 84.2 | 87.7 | 84.5 | 75.2 | 78.4 | 86.5 | 88.6 | 86.1 | 87.9 | 83.2 | 82.1 | 90.7 | 87.2 | 71.9 | 87.6 | 80.1 | 80.8 | 83.6 | 75.3 |
#### 7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following [TextMonkey](https://arxiv.org/pdf/2403.04473) and [DT-VQA](https://arxiv.org/pdf/2405.06706).
| Model |
Params |
Overall |
DocVQA |
InfoVQA |
DF |
KLC |
WTQ |
ChartQA |
DT-VQA |
OCRBench |
| CLIP-B |
86M |
16.0 |
20.1 |
24.2 |
2.3 |
13.8 |
12.8 |
22.2 |
22.3 |
10.6 |
| SigLIP2-B |
93M |
24.9 |
27.0 |
23.5 |
3.1 |
16.7 |
17.4 |
35.0 |
41.5 |
35.1 |
| RADIOv2.5-B |
98M |
37.5 |
60.3 |
31.2 |
29.9 |
30.4 |
29.7 |
51.1 |
44.2 |
23.1 |
| OpenVision-B |
87M |
44.0 |
63.3 |
30.7 |
19.8 |
33.1 |
31.1 |
58.3 |
62.6 |
52.9 |
| DINOv3-B |
86M |
16.1 |
26.5 |
20.8 |
5.6 |
13.2 |
14.0 |
28.9 |
15.8 |
3.9 |
| SAM-B |
90M |
25.2 |
37.8 |
22.2 |
4.7 |
17.5 |
17.6 |
46.5 |
33.3 |
21.9 |
| SAM2-B |
69M |
22.3 |
32.5 |
21.9 |
2.7 |
15.8 |
16.6 |
40.2 |
30.3 |
18.4 |
| oCLIP |
24M |
12.4 |
14.8 |
19.5 |
1.4 |
7.4 |
11.4 |
17.9 |
19.2 |
7.4 |
| DiT |
86M |
8.9 |
11.3 |
20.9 |
0.9 |
5.2 |
9.9 |
12.0 |
9.2 |
1.9 |
| MonkeyOCRv2-S*🤗Link |
28M |
55.9 |
79.3 |
44.5 |
65.1 |
37.6 |
43.0 |
62.0 |
63.1 |
52.2 |
| MonkeyOCRv2-B*🤗Link |
113M |
57.2 |
79.3 |
46.3 |
65.8 |
38.2 |
43.2 |
62.0 |
64.3 |
58.1 |
## Expert Model Labeling Toolchain
We adopt a multi-expert labeling pipeline to obtain reliable annotations for documents. The pipeline includes the following steps:
1. **Structure Detection**
We use **dots.mocr** for document structure detection and reading-order prediction. The detected regions, including text blocks, tables, formulas, and other layout elements, are cropped from the original page image for subsequent recognition.
2. **Content Recognition**
Each cropped block is independently recognized by three expert models: **dots.mocr**, **PaddleOCR-VL**, and **Qwen3-VL**. These complementary models provide multiple annotations for the same block, reducing reliance on any single OCR system.
3. **Block-Level Agreement Filtering**
We compare the recognition results from the three expert models and filter out blocks with low agreement. For retained blocks, we select the prediction that has the highest average agreement with the other two predictions as the final block-level annotation.
4. **Page-Level Quality Control**
Pages containing any filtered block are discarded. In addition, we use **Qwen3** to verify whether the predicted reading order is reasonable, and **Qwen3-VL** to check whether document regions are missed during structure detection.
This multi-expert agreement strategy reduces model-specific annotation errors and improves the reliability of the generated annotations.
### References
- **dots.mocr**: https://github.com/rednote-hilab/dots.mocr
- **PaddleOCR-VL**: https://github.com/PaddlePaddle/PaddleOCR
- **Qwen3-VL**: https://github.com/QwenLM/Qwen3-VL
- **Qwen3**: https://github.com/QwenLM/Qwen3
## Copyright
We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. Note: This model is intended for academic research and non-commercial use only. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.