Buckets:

Speedodude
/

Real5-OmniDocBench-bucket

Files

xet

Speedodude/Real5-OmniDocBench-bucket / README.md

Speedodude

21 days ago

preview code

download

raw

24.6 kB

	---
	pretty_name: Real5-OmniDocBench
	license: apache-2.0
	viewer: false
	task_categories:
	- image-to-text
	size_categories:
	- 1K<n<10K
	language:
	- zh
	- en
	tags:
	- ocr
	- document-parsing
	- benchmark
	- multimodal
	- document
	- image
	---

	<div align="center">

	<h1>Real5-OmniDocBench</h1>

	<p><strong>A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild</strong></p>

	[![ECCV 2026](https://img.shields.io/badge/ECCV-2026-4b44ce.svg)](https://arxiv.org/abs/2603.04205)
	[![arXiv](https://img.shields.io/badge/arXiv-2603.04205-b31b1b.svg)](https://arxiv.org/abs/2603.04205)
	[![Dataset](https://img.shields.io/badge/Hugging%20Face-Dataset-ffd21e.svg?logo=huggingface)](https://huggingface.co/datasets/PaddlePaddle/Real5-OmniDocBench)
	[![Base Benchmark](https://img.shields.io/badge/Base-OmniDocBench%20v1.5-2f855a.svg)](https://github.com/opendatalab/OmniDocBench/tree/v1_5)
	[![License](https://img.shields.io/badge/License-Apache%202.0-2563eb.svg)](./LICENSE)

	[Leaderboard](#leaderboard) \| [Overview](#benchmark-overview) \| [Dataset](#dataset) \| [Evaluation](#evaluation) \| [Submit Results](#submit-results) \| [Citation](#citation)

	</div>

	Real5-OmniDocBench measures the robustness of document parsing systems under five physical acquisition conditions: Scanning, Warping, Screen-Photography, Illumination, and Skew. It reconstructs the same 1,355 pages from OmniDocBench v1.5 in every condition, producing 6,775 images in total. The one-to-one page correspondence and shared evaluation protocol isolate the effect of acquisition conditions from changes in document content.

	<div align="center">

	![Original pages and corresponding Real5-OmniDocBench reconstructions](./assets/real5-omnidocbench-sample.png)

	Original pages and their corresponding reconstructions under five physical acquisition conditions.

	</div>

	## News

	<!-- news:start -->
	- 2026-08-08: Added evaluation results for Kimi-K2.5、Kimi-K2.6、Doubao-Seed-2.1-Pro、MonkeyOCRv2-S-Parsing、MonkeyOCRv2-B-Parsing and OvisOCR2.
	- 2026-06-18: Real5-OmniDocBench was accepted to ECCV 2026. 🎉
	- 2026-05-28: Added evaluation results for PaddleOCR-VL-1.6 and MinerU2.5-Pro.

	<details>
	<summary><strong>Earlier updates</strong></summary>

	- 2026-03-05: Released the [paper](https://arxiv.org/abs/2603.04205) and added results for DeepSeek-OCR 2 and GLM-OCR.
	- 2026-01-28: Released the dataset and benchmark.

	</details>
	<!-- news:end -->

	## Leaderboard

	The leaderboard reports performance over all five acquisition conditions. All metrics follow OmniDocBench: Overall↑, TextEdit↓, FormulaCDM↑, TableTEDS↑, and Reading OrderEdit↓.

	Higher is better for ↑ metrics and lower is better for ↓ metrics. Best results in each column are shown in bold, and second-best results are <u>underlined</u>. Tables are sorted by Overall score in descending order.

	### 1. Overall

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Scanning↑ \| Warping↑ \| Screen-Photography↑ \| Illumination↑ \| Skew↑ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| PaddleOCR-VL-1.6 \| Specialized VLMs \| 0.9B \| 93.19 \| 94.74 \| 92.48 \| <u>92.78</u> \| 93.28 \| 92.66 \|
	\| <u>OvisOCR2</u> \| Specialized VLMs \| 0.9B \| <u>92.29</u> \| <u>93.77</u> \| <u>91.40</u> \| 93.09 \| <u>92.88</u> \| 90.33 \|
	\| PaddleOCR-VL-1.5 \| Specialized VLMs \| 0.9B \| 92.05 \| 93.43 \| 91.25 \| 91.76 \| 92.16 \| <u>91.66</u> \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 90.32 \| 92.67 \| 90.68 \| 91.75 \| 91.12 \| 85.39 \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 89.76 \| 90.08 \| 89.62 \| 89.58 \| 89.91 \| 89.61 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 89.24 \| 89.47 \| 88.90 \| 88.86 \| 89.53 \| 89.45 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 89.22 \| 89.49 \| 89.70 \| 88.40 \| 88.54 \| 89.97 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 89.09 \| 89.67 \| 88.86 \| 88.39 \| 89.66 \| 88.86 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 89.02 \| 88.85 \| 89.36 \| 88.99 \| 89.13 \| 88.79 \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 88.94 \| 92.11 \| 88.72 \| 91.29 \| 91.31 \| 81.26 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 88.90 \| 89.43 \| 89.99 \| 89.27 \| 89.27 \| 86.56 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 88.21 \| 89.25 \| 87.63 \| 87.11 \| 87.97 \| 89.07 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 87.90 \| 88.87 \| 88.17 \| 87.64 \| 86.75 \| 88.09 \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 86.92 \| 86.19 \| 87.77 \| 86.48 \| 87.25 \| 86.90 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 86.38 \| 86.87 \| 86.01 \| 87.18 \| 87.57 \| 84.27 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 85.61 \| 90.06 \| 83.76 \| 89.41 \| 89.57 \| 75.24 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 85.54 \| 92.11 \| 85.97 \| 82.54 \| 89.61 \| 77.47 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 84.19 \| 85.52 \| 83.56 \| 84.86 \| 85.01 \| 81.98 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 79.49 \| 86.94 \| 78.90 \| 82.44 \| 84.71 \| 64.47 \|
	\| GPT-5.2 \| General VLMs \| - \| 78.66 \| 84.43 \| 76.26 \| 76.75 \| 80.88 \| 75.00 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 78.29 \| 84.65 \| 77.27 \| 80.71 \| 83.16 \| 65.67 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 77.15 \| 84.64 \| 76.59 \| 80.24 \| 82.11 \| 62.18 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 76.95 \| 83.60 \| 73.73 \| 78.77 \| 80.51 \| 68.16 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 73.99 \| 86.17 \| 67.20 \| 75.31 \| 78.10 \| 63.01 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 73.01 \| 89.59 \| 66.53 \| 71.65 \| 76.02 \| 61.28 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 64.45 \| 84.68 \| 59.34 \| 66.89 \| 73.38 \| 37.98 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 61.78 \| 72.16 \| 60.35 \| 64.29 \| 67.29 \| 44.83 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 61.48 \| 83.39 \| 50.50 \| 69.76 \| 75.61 \| 28.16 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 60.10 \| 70.27 \| 58.98 \| 63.65 \| 66.31 \| 41.27 \|

	Overall is the mean score across all five scenarios.

	<details>
	<summary><strong>View per-scenario leaderboards</strong></summary>

	### 2. Scanning

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Text<sup>Edit</sup>↓ \| Formula<sup>CDM</sup>↑ \| Table<sup>TEDS</sup>↑ \| Reading Order<sup>Edit</sup>↓ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| PaddleOCR-VL-1.6 \| Specialized VLMs \| 0.9B \| 94.74 \| 0.035 \| 93.65 \| 94.12 \| <u>0.042</u> \|
	\| <u>OvisOCR2</u> \| Specialized VLMs \| 0.9B \| <u>93.77</u> \| 0.041 \| 92.13 \| <u>93.30</u> \| 0.039 \|
	\| PaddleOCR-VL-1.5 \| Specialized VLMs \| 0.9B \| 93.43 \| <u>0.037</u> \| <u>93.04</u> \| 90.97 \| 0.045 \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 92.67 \| 0.054 \| 91.10 \| 92.28 \| 0.061 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 92.11 \| 0.039 \| 90.35 \| 89.90 \| 0.048 \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 92.11 \| 0.040 \| 89.77 \| 90.57 \| 0.043 \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 90.08 \| 0.062 \| 89.02 \| 85.77 \| 0.072 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 90.06 \| 0.052 \| 88.22 \| 87.16 \| 0.050 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 89.67 \| 0.060 \| 88.67 \| 86.31 \| 0.079 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 89.59 \| 0.055 \| 88.55 \| 85.72 \| 0.056 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 89.49 \| 0.053 \| 86.29 \| 87.53 \| 0.051 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 89.47 \| 0.071 \| 88.16 \| 87.37 \| 0.078 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 89.43 \| 0.059 \| 89.01 \| 85.19 \| 0.066 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 89.25 \| 0.073 \| 87.44 \| 87.62 \| 0.098 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 88.87 \| 0.058 \| 86.32 \| 86.09 \| 0.053 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 88.85 \| 0.084 \| 86.56 \| 88.33 \| 0.093 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 86.94 \| 0.103 \| 86.29 \| 84.86 \| 0.141 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 86.87 \| 0.083 \| 83.27 \| 85.68 \| 0.081 \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 86.19 \| 0.110 \| 86.14 \| 83.41 \| 0.114 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 86.17 \| 0.078 \| 83.59 \| 82.69 \| 0.085 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 85.52 \| 0.106 \| 88.09 \| 79.11 \| 0.106 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 84.68 \| 0.094 \| 84.34 \| 79.06 \| 0.092 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 84.65 \| 0.100 \| 84.16 \| 79.81 \| 0.143 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 84.64 \| 0.123 \| 84.17 \| 82.13 \| 0.145 \|
	\| GPT-5.2 \| General VLMs \| - \| 84.43 \| 0.142 \| 85.68 \| 81.78 \| 0.109 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 83.60 \| 0.094 \| 79.76 \| 80.44 \| 0.091 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 83.39 \| 0.097 \| 76.25 \| 83.65 \| 0.090 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 72.16 \| 0.154 \| 64.58 \| 67.27 \| 0.130 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 70.27 \| 0.223 \| 77.03 \| 56.05 \| 0.238 \|

	---

	### 3. Warping

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Text<sup>Edit</sup>↓ \| Formula<sup>CDM</sup>↑ \| Table<sup>TEDS</sup>↑ \| Reading Order<sup>Edit</sup>↓ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| PaddleOCR-VL-1.6 \| Specialized VLMs \| 0.9B \| 92.48 \| 0.048 \| 91.63 \| 90.66 \| 0.061 \|
	\| <u>OvisOCR2</u> \| Specialized VLMs \| 0.9B \| <u>91.40</u> \| 0.058 \| 90.94 \| <u>89.08</u> \| 0.058 \|
	\| PaddleOCR-VL-1.5 \| Specialized VLMs \| 0.9B \| 91.25 \| 0.053 \| 90.94 \| 88.10 \| 0.063 \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 90.68 \| 0.071 \| 90.30 \| 88.78 \| 0.100 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 89.99 \| <u>0.051</u> \| 89.06 \| 85.95 \| 0.064 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 89.70 \| 0.058 \| 87.50 \| 87.39 \| 0.052 \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 89.62 \| 0.073 \| <u>91.18</u> \| 85.02 \| 0.079 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 89.36 \| 0.087 \| 87.74 \| 89.03 \| 0.095 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 88.90 \| 0.086 \| 88.10 \| 87.20 \| 0.087 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 88.86 \| 0.069 \| 89.77 \| 83.71 \| 0.084 \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 88.72 \| 0.100 \| 87.81 \| 88.36 \| 0.076 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 88.17 \| 0.069 \| 86.20 \| 85.17 \| <u>0.056</u> \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 87.77 \| 0.086 \| 88.85 \| 83.06 \| 0.102 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 87.63 \| 0.092 \| 86.50 \| 85.59 \| 0.109 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 86.01 \| 0.087 \| 85.03 \| 81.74 \| 0.093 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 85.97 \| 0.093 \| 85.45 \| 81.77 \| 0.092 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 83.76 \| 0.154 \| 85.92 \| 80.71 \| 0.104 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 83.56 \| 0.121 \| 86.24 \| 76.57 \| 0.124 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 78.90 \| 0.168 \| 79.55 \| 73.94 \| 0.212 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 77.27 \| 0.164 \| 79.08 \| 69.18 \| 0.211 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 76.59 \| 0.196 \| 78.85 \| 70.52 \| 0.221 \|
	\| GPT-5.2 \| General VLMs \| - \| 76.26 \| 0.239 \| 80.90 \| 71.80 \| 0.165 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 73.73 \| 0.202 \| 77.72 \| 63.65 \| 0.173 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 67.20 \| 0.328 \| 73.59 \| 60.80 \| 0.226 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 66.53 \| 0.293 \| 70.42 \| 58.44 \| 0.209 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 60.35 \| 0.316 \| 61.06 \| 51.58 \| 0.247 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 59.34 \| 0.376 \| 68.22 \| 47.40 \| 0.261 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 58.98 \| 0.349 \| 72.71 \| 39.08 \| 0.390 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 50.50 \| 0.383 \| 47.24 \| 42.52 \| 0.309 \|

	---

	### 4. Screen-Photography

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Text<sup>Edit</sup>↓ \| Formula<sup>CDM</sup>↑ \| Table<sup>TEDS</sup>↑ \| Reading Order<sup>Edit</sup>↓ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| OvisOCR2 \| Specialized VLMs \| 0.9B \| 93.09 \| 0.054 \| 91.80 \| 92.83 \| <u>0.046</u> \|
	\| <u>PaddleOCR-VL-1.6</u> \| Specialized VLMs \| 0.9B \| <u>92.78</u> \| 0.045 \| 90.64 \| <u>92.19</u> \| 0.054 \|
	\| PaddleOCR-VL-1.5 \| Specialized VLMs \| 0.9B \| 91.76 \| <u>0.050</u> \| <u>90.88</u> \| 89.38 \| 0.059 \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 91.75 \| 0.063 \| 89.83 \| 91.66 \| 0.070 \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 91.29 \| <u>0.050</u> \| 87.41 \| 91.44 \| 0.044 \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 89.58 \| 0.073 \| 90.10 \| 85.49 \| 0.077 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 89.41 \| 0.062 \| 87.55 \| 86.83 \| 0.053 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 89.27 \| 0.068 \| 88.72 \| 85.85 \| 0.071 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 88.99 \| 0.098 \| 87.47 \| 89.31 \| 0.102 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 88.86 \| 0.084 \| 87.33 \| 87.65 \| 0.087 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 88.40 \| 0.070 \| 87.07 \| 85.12 \| 0.050 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 88.39 \| 0.070 \| 87.41 \| 84.77 \| 0.078 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 87.64 \| 0.068 \| 84.96 \| 84.72 \| 0.049 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 87.18 \| 0.081 \| 85.34 \| 84.26 \| 0.079 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 87.11 \| 0.103 \| 85.30 \| 86.31 \| 0.117 \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 86.48 \| 0.100 \| 87.46 \| 82.00 \| 0.102 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 84.86 \| 0.112 \| 86.65 \| 79.09 \| 0.117 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 82.54 \| 0.103 \| 83.58 \| 74.36 \| 0.107 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 82.44 \| 0.124 \| 81.55 \| 78.13 \| 0.177 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 80.71 \| 0.122 \| 81.33 \| 73.04 \| 0.177 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 80.24 \| 0.148 \| 80.78 \| 74.74 \| 0.179 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 78.77 \| 0.139 \| 79.02 \| 71.17 \| 0.123 \|
	\| GPT-5.2 \| General VLMs \| - \| 76.75 \| 0.208 \| 79.27 \| 71.73 \| 0.148 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 75.31 \| 0.220 \| 77.68 \| 70.26 \| 0.169 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 71.65 \| 0.201 \| 73.49 \| 61.54 \| 0.157 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 69.76 \| 0.205 \| 61.80 \| 68.00 \| 0.177 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 66.89 \| 0.204 \| 73.26 \| 47.82 \| 0.165 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 64.29 \| 0.232 \| 58.66 \| 57.38 \| 0.195 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 63.65 \| 0.290 \| 72.73 \| 47.21 \| 0.325 \|

	---

	### 5. Illumination

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Text<sup>Edit</sup>↓ \| Formula<sup>CDM</sup>↑ \| Table<sup>TEDS</sup>↑ \| Reading Order<sup>Edit</sup>↓ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| PaddleOCR-VL-1.6 \| Specialized VLMs \| 0.9B \| 93.28 \| 0.042 \| 92.61 \| <u>91.48</u> \| <u>0.051</u> \|
	\| <u>OvisOCR2</u> \| Specialized VLMs \| 0.9B \| <u>92.88</u> \| <u>0.043</u> \| 91.18 \| 91.72 \| 0.040 \|
	\| PaddleOCR-VL-1.5 \| Specialized VLMs \| 0.9B \| 92.16 \| 0.046 \| <u>91.80</u> \| 89.33 \| <u>0.051</u> \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 91.31 \| 0.050 \| 88.15 \| 90.76 \| 0.052 \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 91.12 \| 0.059 \| 91.02 \| 88.20 \| 0.071 \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 89.91 \| 0.062 \| 89.76 \| 86.20 \| 0.072 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 89.66 \| 0.064 \| 89.83 \| 85.53 \| 0.077 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 89.61 \| 0.049 \| 86.66 \| 87.02 \| 0.055 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 89.57 \| 0.065 \| 88.36 \| 86.87 \| 0.062 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 89.53 \| 0.073 \| 87.78 \| 88.14 \| 0.080 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 89.27 \| 0.060 \| 87.81 \| 86.05 \| 0.070 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 89.13 \| 0.085 \| 87.29 \| 88.59 \| 0.091 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 88.54 \| 0.066 \| 86.72 \| 85.53 \| 0.059 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 87.97 \| 0.083 \| 86.13 \| 86.11 \| 0.103 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 87.57 \| 0.068 \| 85.07 \| 84.44 \| 0.076 \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 87.25 \| 0.087 \| 86.44 \| 84.03 \| 0.097 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 86.75 \| 0.074 \| 84.16 \| 83.49 \| 0.057 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 85.01 \| 0.099 \| 87.94 \| 76.96 \| 0.112 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 84.71 \| 0.120 \| 84.13 \| 82.02 \| 0.171 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 83.16 \| 0.118 \| 83.63 \| 77.62 \| 0.168 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 82.11 \| 0.144 \| 82.07 \| 78.67 \| 0.172 \|
	\| GPT-5.2 \| General VLMs \| - \| 80.88 \| 0.191 \| 84.41 \| 77.37 \| 0.134 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 80.51 \| 0.135 \| 80.72 \| 74.29 \| 0.123 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 78.10 \| 0.192 \| 81.71 \| 71.81 \| 0.156 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 76.02 \| 0.168 \| 77.83 \| 67.01 \| 0.122 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 75.61 \| 0.159 \| 70.04 \| 72.69 \| 0.133 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 73.38 \| 0.158 \| 77.75 \| 58.19 \| 0.126 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 67.29 \| 0.197 \| 61.42 \| 60.10 \| 0.173 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 66.31 \| 0.259 \| 74.80 \| 50.03 \| 0.337 \|

	---

	### 6. Skew

	\| Methods \| Model Type \| Parameters \| Overall↑ \| Text<sup>Edit</sup>↓ \| Formula<sup>CDM</sup>↑ \| Table<sup>TEDS</sup>↑ \| Reading Order<sup>Edit</sup>↓ \|
	\| --- \| --- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| PaddleOCR-VL-1.6 \| Specialized VLMs \| 0.9B \| 92.66 \| 0.045 \| 91.44 \| 91.04 \| 0.058 \|
	\| <u>PaddleOCR-VL-1.5</u> \| Specialized VLMs \| 0.9B \| <u>91.66</u> \| <u>0.047</u> \| <u>91.00</u> \| <u>88.69</u> \| 0.061 \|
	\| OvisOCR2 \| Specialized VLMs \| 0.9B \| 90.33 \| 0.048 \| 90.58 \| 85.23 \| 0.048 \|
	\| MonkeyOCRv2-B-Parsing \| Specialized VLMs \| 0.7B \| 89.97 \| 0.051 \| 88.10 \| 86.88 \| <u>0.051</u> \|
	\| Kimi-K2.6 \| General VLMs \| 1.1T \| 89.61 \| 0.060 \| 90.25 \| 84.27 \| 0.078 \|
	\| Gemini-3 Pro \| General VLMs \| - \| 89.45 \| 0.080 \| 88.33 \| 88.06 \| 0.092 \|
	\| Gemini-2.5 Pro \| General VLMs \| - \| 89.07 \| 0.077 \| 87.89 \| 86.99 \| 0.104 \|
	\| Kimi-K2.5 \| General VLMs \| 1.1T \| 88.86 \| 0.060 \| 89.62 \| 83.00 \| 0.080 \|
	\| Doubao-Seed-2.1-Pro \| General VLMs \| - \| 88.79 \| 0.080 \| 88.23 \| 86.11 \| 0.095 \|
	\| MonkeyOCRv2-S-Parsing \| Specialized VLMs \| 0.6B \| 88.09 \| 0.063 \| 84.54 \| 86.07 \| 0.055 \|
	\| Qwen2.5-VL-72B \| General VLMs \| 72B \| 86.90 \| 0.077 \| 87.26 \| 81.14 \| 0.091 \|
	\| Qwen3-VL-235B \| General VLMs \| 235B \| 86.56 \| 0.077 \| 83.96 \| 83.41 \| 0.091 \|
	\| GLM-OCR \| Specialized VLMs \| 0.9B \| 85.39 \| 0.099 \| 85.78 \| 80.28 \| 0.156 \|
	\| dots.ocr \| Specialized VLMs \| 3B \| 84.27 \| 0.087 \| 85.73 \| 75.74 \| 0.094 \|
	\| Nanonets-OCR-s \| Specialized VLMs \| 3B \| 81.98 \| 0.121 \| 85.78 \| 72.22 \| 0.133 \|
	\| MinerU2.5-pro \| Specialized VLMs \| 1.2B \| 81.26 \| 0.202 \| 83.92 \| 81.07 \| 0.107 \|
	\| PaddleOCR-VL \| Specialized VLMs \| 0.9B \| 77.47 \| 0.192 \| 78.81 \| 72.83 \| 0.193 \|
	\| MinerU2.5 \| Specialized VLMs \| 1.2B \| 75.24 \| 0.305 \| 81.78 \| 74.39 \| 0.151 \|
	\| GPT-5.2 \| General VLMs \| - \| 75.00 \| 0.257 \| 80.27 \| 70.47 \| 0.167 \|
	\| MinerU2-VLM \| Specialized VLMs \| 0.9B \| 68.16 \| 0.230 \| 74.45 \| 53.07 \| 0.191 \|
	\| MonkeyOCR-3B \| Specialized VLMs \| 3.7B \| 65.67 \| 0.248 \| 69.23 \| 52.59 \| 0.300 \|
	\| MonkeyOCR-pro-3B \| Specialized VLMs \| 3.7B \| 64.47 \| 0.251 \| 69.06 \| 49.42 \| 0.301 \|
	\| Deepseek-OCR \| Specialized VLMs \| 3B \| 63.01 \| 0.327 \| 73.27 \| 48.48 \| 0.231 \|
	\| MonkeyOCR-pro-1.2B \| Specialized VLMs \| 1.9B \| 62.18 \| 0.292 \| 66.25 \| 49.46 \| 0.317 \|
	\| Deepseek-OCR 2 \| Specialized VLMs \| 3B \| 61.28 \| 0.295 \| 66.16 \| 47.18 \| 0.221 \|
	\| Dolphin \| Specialized VLMs \| 322M \| 44.83 \| 0.500 \| 51.34 \| 33.22 \| 0.321 \|
	\| Marker-1.8.2 \| Pipeline Tools \| - \| 41.27 \| 0.536 \| 60.16 \| 17.23 \| 0.543 \|
	\| PP-StructureV3 \| Pipeline Tools \| - \| 37.98 \| 0.557 \| 44.37 \| 25.27 \| 0.417 \|
	\| Dolphin-1.5 \| Specialized VLMs \| 0.3B \| 28.16 \| 0.553 \| 25.60 \| 14.18 \| 0.419 \|

	</details>

	## Benchmark Overview

	Real5-OmniDocBench evaluates the same document content under controlled changes to the physical acquisition process. Except for the scanning subset, images were manually captured with handheld mobile devices.

	\| Scenario \| Acquisition condition \| Representative artifacts \|
	\|---\|---\|---\|
	\| Scanning \| Documents captured with scanning devices \| Scanner characteristics and clean planar capture \|
	\| Warping \| Curved or non-planar pages photographed by hand \| Page curvature, folding, and local deformation \|
	\| Screen-Photography \| Screens displaying documents photographed by hand \| Moiré patterns, reflections, and display artifacts \|
	\| Illumination \| Documents photographed under varied lighting \| Shadows, glare, and uneven exposure \|
	\| Skew \| Documents photographed from oblique viewpoints \| Perspective distortion and geometric skew \|

	This design provides:

	- Controlled comparison: every scenario contains the same 1,355 source pages.
	- Physical realism: acquisition artifacts are produced by real devices and environments rather than synthetic transformations.
	- Protocol compatibility: page identities, annotations, prediction format, and metrics follow [OmniDocBench v1.5](https://github.com/opendatalab/OmniDocBench/tree/v1_5).

	## Dataset

	The five scenario directories each contain 1,355 images. The complete download is approximately 16 GB.

	```text
	Real5-OmniDocBench/
	├── Real5-OmniDocBench-Scanning/
	├── Real5-OmniDocBench-Warping/
	├── Real5-OmniDocBench-Screen-Photography/
	├── Real5-OmniDocBench-Illumination/
	└── Real5-OmniDocBench-Skew/
	```

	Install the current Hugging Face Hub CLI and download the complete dataset:

	```bash
	pip install -U huggingface_hub
	hf download PaddlePaddle/Real5-OmniDocBench \
	--repo-type dataset \
	--local-dir ./Real5-OmniDocBench
	```

	To download a single condition, use a file pattern:

	```bash
	hf download PaddlePaddle/Real5-OmniDocBench \
	--repo-type dataset \
	--include "Real5-OmniDocBench-Warping/*" \
	--local-dir ./Real5-OmniDocBench
	```

	Download the matching OmniDocBench v1.5 ground-truth annotations separately:

	```bash
	hf download opendatalab/OmniDocBench OmniDocBench.json \
	--repo-type dataset \
	--revision v1_5 \
	--local-dir ./OmniDocBench-v1.5
	```

	## Evaluation

	Real5-OmniDocBench does not introduce a new prediction schema. It reuses the [OmniDocBench v1.5](https://github.com/opendatalab/OmniDocBench/tree/v1_5) annotation format and evaluation pipeline so that performance can be compared across acquisition conditions.

	1. Run the model independently on all 1,355 images in each scenario.
	2. Export predictions in the OmniDocBench end-to-end parsing format.
	3. Match each image to its corresponding OmniDocBench v1.5 ground-truth annotation.
	4. Apply the same preprocessing, evaluator, and metric settings to every scenario.

	Refer to the version-pinned [OmniDocBench v1.5 evaluation guide](https://github.com/opendatalab/OmniDocBench/tree/v1_5#evaluation) for environment setup, prediction formats, and evaluation commands.

	### Metrics

	\| Metric \| Direction \| Definition \|
	\|---\|---:\|---\|
	\| Overall \| ↑ \| `((1 - TextEdit) * 100 + TableTEDS + FormulaCDM) / 3` \|
	\| TextEdit \| ↓ \| Normalized edit distance for plain-text content \|
	\| FormulaCDM \| ↑ \| Character Detection Matching score for formulas \|
	\| TableTEDS \| ↑ \| Tree-Edit-Distance-based Similarity for table structure \|
	\| Reading OrderEdit \| ↓ \| Normalized edit distance for the reading-order sequence \|

	## Submit Results

	Model results can appear in the Hugging Face Hub leaderboard through the Hub evaluation-results workflow. Add an evaluation result file under `.eval_results/` in the model repository, set `evaluation_framework: real5-omnidocbench`, and follow the [Hugging Face evaluation results documentation](https://huggingface.co/docs/hub/eval-results) for the supported schema and submission process.

	## Citation

	If you use Real5-OmniDocBench in your research, please cite the following paper. Please also cite OmniDocBench when using its annotations or evaluation pipeline.

	```bibtex
	@misc{zhou2026real5omnidocbench,
	title = {Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild},
	author = {Changda Zhou and Ziyue Gao and Xueqing Wang and Tingquan Gao and Cheng Cui and Jing Tang and Yi Liu},
	year = {2026},
	eprint = {2603.04205},
	archivePrefix = {arXiv},
	primaryClass = {cs.CV},
	doi = {10.48550/arXiv.2603.04205},
	url = {https://arxiv.org/abs/2603.04205}
	}
	```

	## Acknowledgements

	Real5-OmniDocBench is built on [OmniDocBench v1.5](https://github.com/opendatalab/OmniDocBench/tree/v1_5) and adopts its annotations and evaluation protocol. We thank the OmniDocBench authors for making their benchmark and evaluation tools available to the community.

	## License

	The Real5-OmniDocBench dataset and repository materials are released under the [Apache License 2.0](./LICENSE). This benchmark inherits annotations and source-page correspondence from OmniDocBench v1.5; use of those materials remains subject to the applicable OmniDocBench and source-document terms.

Xet Storage Details

Size:: 24.6 kB
Xet hash:: d3b727a03aef6476b85200c2a7c79853d6355c068f5c26916619348b11b765ac

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.