Instructions to use zenosai/MonkeyOCRv2-B-Und with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zenosai/MonkeyOCRv2-B-Und with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="zenosai/MonkeyOCRv2-B-Und", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("zenosai/MonkeyOCRv2-B-Und", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use zenosai/MonkeyOCRv2-B-Und with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "zenosai/MonkeyOCRv2-B-Und" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Und", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/zenosai/MonkeyOCRv2-B-Und
- SGLang
How to use zenosai/MonkeyOCRv2-B-Und with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "zenosai/MonkeyOCRv2-B-Und" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Und", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "zenosai/MonkeyOCRv2-B-Und" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "zenosai/MonkeyOCRv2-B-Und", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use zenosai/MonkeyOCRv2-B-Und with Docker Model Runner:
docker model run hf.co/zenosai/MonkeyOCRv2-B-Und
Improve model card: Add transformers library tag, paper/code links, authors, and fix image URLs
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,14 +1,18 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
datasets:
|
| 4 |
- zenosai/MonkeyDocv2
|
|
|
|
| 5 |
pipeline_tag: image-text-to-text
|
|
|
|
| 6 |
---
|
| 7 |
-
<div align="center" xmlns="http://www.w3.org/1999/html">
|
| 8 |
-
<h2>
|
| 9 |
-
<b>MonkeyOCRv2: A Visual-Text Foundation Model for Document AI</b>
|
| 10 |
-
</h2>
|
| 11 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
[](https://arxiv.org/abs/2607.11562)
|
| 13 |
[](https://huggingface.co/collections/zenosai/monkeyocrv2)
|
| 14 |
[](https://modelscope.cn/datasets/zenosai/MonkeyDocv2)
|
|
@@ -16,7 +20,7 @@ pipeline_tag: image-text-to-text
|
|
| 16 |
[](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aissue+is%3Aclosed)
|
| 17 |
[](http://vlrlabmonkey.xyz:8891/)
|
| 18 |
|
| 19 |
-
<img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/
|
| 20 |
</div>
|
| 21 |
|
| 22 |
## News
|
|
@@ -308,7 +312,7 @@ python infer.py -h
|
|
| 308 |
Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
|
| 309 |
|
| 310 |
<p align="center">
|
| 311 |
-
<img src="https://
|
| 312 |
</p>
|
| 313 |
|
| 314 |
## Evaluation Results
|
|
@@ -447,6 +451,8 @@ Our model supports robust document parsing in real-world scenarios across 17 lan
|
|
| 447 |
<th>ExpRate</th>
|
| 448 |
<th>CDM</th>
|
| 449 |
<th>ExpRate</th>
|
|
|
|
|
|
|
| 450 |
</tr>
|
| 451 |
</thead>
|
| 452 |
<tbody>
|
|
@@ -533,7 +539,7 @@ Our model supports robust document parsing in real-world scenarios across 17 lan
|
|
| 533 |
#### 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of [MMOCR](https://github.com/open-mmlab/mmocr) and [DPText-DETR](https://github.com/ymy-k/DPText-DETR).
|
| 534 |
|
| 535 |
<p align="center">
|
| 536 |
-
<img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/
|
| 537 |
</p>
|
| 538 |
|
| 539 |
|
|
@@ -1189,4 +1195,4 @@ This multi-expert agreement strategy reduces model-specific annotation errors an
|
|
| 1189 |
- **Qwen3**: https://github.com/QwenLM/Qwen3
|
| 1190 |
|
| 1191 |
## Copyright
|
| 1192 |
-
We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. Note: This model is intended for academic research and non-commercial use only. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
datasets:
|
| 3 |
- zenosai/MonkeyDocv2
|
| 4 |
+
license: apache-2.0
|
| 5 |
pipeline_tag: image-text-to-text
|
| 6 |
+
library_name: transformers
|
| 7 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
+
# MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
|
| 10 |
+
|
| 11 |
+
* **Paper:** [MonkeyOCRv2: A Visual-Text Foundation Model for Document AI](https://huggingface.co/papers/2607.11562)
|
| 12 |
+
* **Code:** [GitHub Repository](https://github.com/Yuliang-Liu/MonkeyOCRv2)
|
| 13 |
+
* **Authors:** Yuliang Liu, Zhang Li, Ziyang Zhang, Shuo Zhang, Qiang Liu, Jiajun Song, Zidun Guo, Xinhan Wang, Handong Zheng, Yang Liu, Dongliang Luo, Zhiyin Ma, Jiarui Zhang, Xiang Bai
|
| 14 |
+
|
| 15 |
+
<div align="center" xmlns="http://www.w3.org/1999/html">
|
| 16 |
[](https://arxiv.org/abs/2607.11562)
|
| 17 |
[](https://huggingface.co/collections/zenosai/monkeyocrv2)
|
| 18 |
[](https://modelscope.cn/datasets/zenosai/MonkeyDocv2)
|
|
|
|
| 20 |
[](https://github.com/Yuliang-Liu/MonkeyOCRv2/issues?q=is%3Aissue+is%3Aclosed)
|
| 21 |
[](http://vlrlabmonkey.xyz:8891/)
|
| 22 |
|
| 23 |
+
<img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/main/asserts/overview.png" width="600"/>
|
| 24 |
</div>
|
| 25 |
|
| 26 |
## News
|
|
|
|
| 312 |
Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
|
| 313 |
|
| 314 |
<p align="center">
|
| 315 |
+
<img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/main/asserts/Visualization.gif?raw=true" width="600"/>
|
| 316 |
</p>
|
| 317 |
|
| 318 |
## Evaluation Results
|
|
|
|
| 451 |
<th>ExpRate</th>
|
| 452 |
<th>CDM</th>
|
| 453 |
<th>ExpRate</th>
|
| 454 |
+
<th>CDM</th>
|
| 455 |
+
<th>ExpRate</th>
|
| 456 |
</tr>
|
| 457 |
</thead>
|
| 458 |
<tbody>
|
|
|
|
| 539 |
#### 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of [MMOCR](https://github.com/open-mmlab/mmocr) and [DPText-DETR](https://github.com/ymy-k/DPText-DETR).
|
| 540 |
|
| 541 |
<p align="center">
|
| 542 |
+
<img src="https://raw.githubusercontent.com/Yuliang-Liu/MonkeyOCRv2/main/asserts/text_detection.png?raw=true" width="600"/>
|
| 543 |
</p>
|
| 544 |
|
| 545 |
|
|
|
|
| 1195 |
- **Qwen3**: https://github.com/QwenLM/Qwen3
|
| 1196 |
|
| 1197 |
## Copyright
|
| 1198 |
+
We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. Note: This model is intended for academic research and non-commercial use only. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.
|