Image-Text-to-Text
Transformers
Safetensors
multilingual
English
Chinese
hunyuan_vl
ocr
vision-language-model
document-parsing
text-spotting
information-extraction
text-image-translation
conversational
Eval Results
Instructions to use tencent/HunyuanOCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/HunyuanOCR with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="tencent/HunyuanOCR") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("tencent/HunyuanOCR") model = AutoModelForMultimodalLM.from_pretrained("tencent/HunyuanOCR", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/HunyuanOCR with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/HunyuanOCR" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/HunyuanOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/tencent/HunyuanOCR
- SGLang
How to use tencent/HunyuanOCR with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/HunyuanOCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/HunyuanOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/HunyuanOCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/HunyuanOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use tencent/HunyuanOCR with Docker Model Runner:
docker model run hf.co/tencent/HunyuanOCR
update README
Browse files- .gitattributes +25 -0
- README.md +61 -85
- assets/HyOCR_1_5_teaser.png +3 -0
- v1.0/README.md +35 -31
.gitattributes
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
*.arrow filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 3 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
|
@@ -8,6 +9,8 @@
|
|
| 8 |
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
| 11 |
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
|
@@ -33,3 +36,25 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.avro filter=lfs diff=lfs merge=lfs -text
|
| 4 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 5 |
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 6 |
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 9 |
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 10 |
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 11 |
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.lz4 filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.mds filter=lfs diff=lfs merge=lfs -text
|
| 14 |
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 15 |
*.model filter=lfs diff=lfs merge=lfs -text
|
| 16 |
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 36 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 37 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 38 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
# Audio files - uncompressed
|
| 40 |
+
*.pcm filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
*.sam filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
*.raw filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
# Audio files - compressed
|
| 44 |
+
*.aac filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
*.flac filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
*.mp3 filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
*.ogg filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
*.wav filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
# Image files - uncompressed
|
| 50 |
+
*.bmp filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
*.gif filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
*.tiff filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
# Image files - compressed
|
| 55 |
+
*.jpg filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
*.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
*.webp filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
# Video files - compressed
|
| 59 |
+
*.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
*.webm filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -3,16 +3,16 @@ license: other
|
|
| 3 |
license_name: tencent-hunyuan-community
|
| 4 |
license_link: https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE
|
| 5 |
language:
|
| 6 |
-
- multilingual
|
| 7 |
-
- en
|
| 8 |
-
- zh
|
| 9 |
tags:
|
| 10 |
-
- ocr
|
| 11 |
-
- vision-language-model
|
| 12 |
-
- document-parsing
|
| 13 |
-
- text-spotting
|
| 14 |
-
- information-extraction
|
| 15 |
-
- text-image-translation
|
| 16 |
pipeline_tag: image-text-to-text
|
| 17 |
library_name: transformers
|
| 18 |
---
|
|
@@ -20,17 +20,12 @@ library_name: transformers
|
|
| 20 |
# HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
|
| 21 |
|
| 22 |
<p align="center">
|
| 23 |
-
<img src="
|
| 24 |
</p>
|
| 25 |
|
| 26 |
-
🤗 [Model](https://huggingface.co/tencent/HunyuanOCR) | 💻 [GitHub](https://github.com/Tencent-Hunyuan/HunyuanOCR) | 📄 [Paper](https://arxiv.org/pdf/2607.04884)
|
| 27 |
|
| 28 |
-
> 📦 **Model layout.** This repository hosts **HunyuanOCR-1.5** at the root
|
| 29 |
-
> (target base weights). The **DFlash speculative-decoding draft** lives under
|
| 30 |
-
> [`dflash/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash), and
|
| 31 |
-
> the previous **HunyuanOCR-1.0** is archived under
|
| 32 |
-
> [`v1.0/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0)
|
| 33 |
-
> (load it with `subfolder="v1.0"`, or download the `v1.0/` directory directly).
|
| 34 |
|
| 35 |
---
|
| 36 |
|
|
@@ -50,20 +45,18 @@ Together, HunyuanOCR-1.5 achieves **both faster inference and broader OCR capabi
|
|
| 50 |
|
| 51 |
## ⚙️ Environment
|
| 52 |
|
| 53 |
-
Inference
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
|
|
|
| 60 |
|
| 61 |
-
|
| 62 |
-
[`inference/README.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/README.md)
|
| 63 |
-
for the selection guide and the full rationale.
|
| 64 |
|
| 65 |
-
**Common prerequisites:** Python 3.10+ (3.12 tested), an NVIDIA GPU, and
|
| 66 |
-
`huggingface_hub` for downloading the weights:
|
| 67 |
|
| 68 |
```bash
|
| 69 |
pip install -U "huggingface_hub[cli]"
|
|
@@ -77,10 +70,7 @@ The download contains both the base model and the `dflash/` draft model.
|
|
| 77 |
|
| 78 |
## 🧪 Inference
|
| 79 |
|
| 80 |
-
All
|
| 81 |
-
(`temperature=0.0`, `top_p=1.0`, `top_k=-1`, `repetition_penalty=1.08`) +
|
| 82 |
-
post-processing, so their outputs are directly comparable. Grab the toolkit from
|
| 83 |
-
GitHub first:
|
| 84 |
|
| 85 |
```bash
|
| 86 |
git clone https://github.com/Tencent-Hunyuan/HunyuanOCR.git
|
|
@@ -99,7 +89,7 @@ from transformers import AutoProcessor, HunYuanVLForConditionalGeneration
|
|
| 99 |
|
| 100 |
MODEL_ID = "tencent/HunyuanOCR"
|
| 101 |
|
| 102 |
-
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
|
| 103 |
model = HunYuanVLForConditionalGeneration.from_pretrained(
|
| 104 |
MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto",
|
| 105 |
trust_remote_code=True,
|
|
@@ -127,81 +117,54 @@ print(processor.batch_decode(gen, skip_special_tokens=True)[0])
|
|
| 127 |
```
|
| 128 |
|
| 129 |
For **multi-GPU batch inference** with sampling / early-stop / doc-parse
|
| 130 |
-
normalization strictly aligned to the vLLM client, use the shipped script
|
| 131 |
-
|
| 132 |
-
[`inference/
|
| 133 |
|
| 134 |
```bash
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
--
|
| 138 |
-
--
|
| 139 |
-
--output ./results/hf_out \
|
| 140 |
-
--gpu-ids 0,1,2,3,4,5,6,7 \
|
| 141 |
-
--max-new-tokens 8192 \
|
| 142 |
--merge
|
| 143 |
```
|
| 144 |
|
| 145 |
### B. vLLM (OpenAI-compatible)
|
| 146 |
|
| 147 |
-
|
| 148 |
-
|
| 149 |
|
| 150 |
-
**
|
| 151 |
-
supports `HunYuanVLForConditionalGeneration`; no nightly or patch required.
|
| 152 |
-
Install per
|
| 153 |
-
[`inference/vllm_0_18_1/requirements.txt`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/vllm_0_18_1/requirements.txt)
|
| 154 |
-
(core: `pip install "vllm==0.18.1"`), then:
|
| 155 |
|
| 156 |
```bash
|
| 157 |
-
MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/
|
| 158 |
curl -sf http://127.0.0.1:8000/v1/models # readiness check
|
| 159 |
```
|
| 160 |
|
| 161 |
-
**
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
```bash
|
| 166 |
-
uv pip install -U vllm --torch-backend=cu130 --extra-index-url https://wheels.vllm.ai/nightly
|
| 167 |
-
uv pip install runai-model-streamer
|
| 168 |
-
```
|
| 169 |
-
|
| 170 |
-
The DFlash draft lives under the `dflash/` subfolder of `tencent/HunyuanOCR`.
|
| 171 |
-
vLLM's `--speculative-config` does not accept an HF subfolder, so download the
|
| 172 |
-
draft weight into a flat local dir first:
|
| 173 |
-
|
| 174 |
-
```bash
|
| 175 |
-
huggingface-cli download tencent/HunyuanOCR dflash/model.safetensors --local-dir ./HunyuanOCR
|
| 176 |
-
cp -r ./HunyuanOCR/dflash ./hyocr_dflash
|
| 177 |
-
```
|
| 178 |
-
|
| 179 |
-
Then launch AR or DFlash:
|
| 180 |
|
| 181 |
```bash
|
| 182 |
-
|
| 183 |
-
MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/nightly/serve_ar.sh
|
| 184 |
-
|
| 185 |
-
# DFlash speculative decoding
|
| 186 |
-
MODEL_PATH=./HunyuanOCR DFLASH_PATH=./hyocr_dflash \
|
| 187 |
-
GPU=0 PORT=8001 NUM_SPEC_TOKENS=15 bash inference/nightly/serve_dflash.sh
|
| 188 |
```
|
| 189 |
|
| 190 |
-
**Client (either
|
| 191 |
-
prompt is locked to an official task type via `--task-type` (run
|
| 192 |
-
to see all); sampling and streaming tail-repetition early-stop
|
| 193 |
-
built in:
|
| 194 |
|
| 195 |
```bash
|
| 196 |
-
|
| 197 |
-
python inference/vllm_0_18_1/infer_vllm_client.py \
|
| 198 |
--host 127.0.0.1 --port 8000 \
|
| 199 |
--model tencent/HunyuanOCR \
|
| 200 |
--image /path/to/document.png \
|
| 201 |
--task-type doc_parse \
|
| 202 |
--max-tokens 32768
|
| 203 |
-
|
| 204 |
-
# add --no-
|
|
|
|
| 205 |
```
|
| 206 |
|
| 207 |
Available task types (`--task-type`): `doc_parse` (default), `structured_parse`, `spotting_json`, `spotting_hunyuan`, `layout`, `layout_parse`, `chart_parse`, `formula`, `table`, `doc_trans_en2zh`, `trans_other2en`, `trans_other2zh`.
|
|
@@ -210,7 +173,7 @@ For **batch** inference over a directory (same task types, multi-endpoint
|
|
| 210 |
concurrency, resumable):
|
| 211 |
|
| 212 |
```bash
|
| 213 |
-
python inference/
|
| 214 |
--image-dir /path/to/images \
|
| 215 |
--out-dir /path/to/output \
|
| 216 |
--ports 8000 \
|
|
@@ -303,3 +266,16 @@ Special thanks to the Hugging Face community for their Day-0 support.
|
|
| 303 |
## 📜 License
|
| 304 |
|
| 305 |
HunyuanOCR-1.5 is released under the same license as HunyuanOCR 1.0 — the **Tencent Hunyuan Community License Agreement**. See [`LICENSE`](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) for the full terms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
license_name: tencent-hunyuan-community
|
| 4 |
license_link: https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE
|
| 5 |
language:
|
| 6 |
+
- multilingual
|
| 7 |
+
- en
|
| 8 |
+
- zh
|
| 9 |
tags:
|
| 10 |
+
- ocr
|
| 11 |
+
- vision-language-model
|
| 12 |
+
- document-parsing
|
| 13 |
+
- text-spotting
|
| 14 |
+
- information-extraction
|
| 15 |
+
- text-image-translation
|
| 16 |
pipeline_tag: image-text-to-text
|
| 17 |
library_name: transformers
|
| 18 |
---
|
|
|
|
| 20 |
# HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
|
| 21 |
|
| 22 |
<p align="center">
|
| 23 |
+
<img src="assets/HyOCR_1_5_teaser.png" width="90%"/>
|
| 24 |
</p>
|
| 25 |
|
| 26 |
+
🤗 [HF Model](https://huggingface.co/tencent/HunyuanOCR) | 💻 [GitHub Repo](https://github.com/Tencent-Hunyuan/HunyuanOCR) | 📄 [Paper](https://arxiv.org/pdf/2607.04884)
|
| 27 |
|
| 28 |
+
> 📦 **Model layout.** This repository hosts **HunyuanOCR-1.5** checkpoint at the root (target base weights). The **DFlash speculative-decoding draft** lives under [`dflash/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash), and the previous **HunyuanOCR-1.0** is archived under [`v1.0/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0) (load it with `subfolder="v1.0"`, or download the `v1.0/` directory directly).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
---
|
| 31 |
|
|
|
|
| 45 |
|
| 46 |
## ⚙️ Environment
|
| 47 |
|
| 48 |
+
Inference now uses a **single unified environment** (built on `uv`, **requires CUDA 13**) that runs all three configurations from the same install: **vLLM AR, DFlash speculative decoding, and native transformers**. Accuracy alignment across the three has been verified.
|
| 49 |
|
| 50 |
+
```bash
|
| 51 |
+
pip install uv
|
| 52 |
+
uv venv --python 3.12 && source .venv/bin/activate
|
| 53 |
+
uv pip install "vllm>=0.25.1"
|
| 54 |
+
uv pip install --no-build-isolation --no-cache-dir "flash-attn==2.8.3"
|
| 55 |
+
```
|
| 56 |
|
| 57 |
+
The inference code lives on GitHub under [`inference/`](https://github.com/Tencent-Hunyuan/HunyuanOCR/tree/main/inference) (`inference/vLLM`, `inference/DFlash`, `inference/transformers`). See [`docs/inference/inference.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/docs/inference/inference.md) for the full setup and usage. If you lack CUDA 13 or only need one configuration, that document also points to the lighter per-configuration recipes in the archive.
|
|
|
|
|
|
|
| 58 |
|
| 59 |
+
**Common prerequisites:** Python 3.10+ (3.12 tested), an NVIDIA GPU, and `huggingface_hub` for downloading the weights:
|
|
|
|
| 60 |
|
| 61 |
```bash
|
| 62 |
pip install -U "huggingface_hub[cli]"
|
|
|
|
| 70 |
|
| 71 |
## 🧪 Inference
|
| 72 |
|
| 73 |
+
All configurations share the same weights and the same task-type prompts + sampling (`temperature=0.0`, `top_p=1.0`, `top_k=-1`, `repetition_penalty=1.08`) + post-processing, so their outputs are directly comparable. Grab the toolkit from GitHub first:
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
```bash
|
| 76 |
git clone https://github.com/Tencent-Hunyuan/HunyuanOCR.git
|
|
|
|
| 89 |
|
| 90 |
MODEL_ID = "tencent/HunyuanOCR"
|
| 91 |
|
| 92 |
+
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True, use_fast=False)
|
| 93 |
model = HunYuanVLForConditionalGeneration.from_pretrained(
|
| 94 |
MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto",
|
| 95 |
trust_remote_code=True,
|
|
|
|
| 117 |
```
|
| 118 |
|
| 119 |
For **multi-GPU batch inference** with sampling / early-stop / doc-parse
|
| 120 |
+
normalization strictly aligned to the vLLM client, use the shipped script
|
| 121 |
+
after installing the unified environment (see
|
| 122 |
+
[`docs/inference/inference.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/docs/inference/inference.md)):
|
| 123 |
|
| 124 |
```bash
|
| 125 |
+
python inference/transformers/infer_hf_8gpu.py \
|
| 126 |
+
--model ./HunyuanOCR --attn-implementation flash_attention_2 \
|
| 127 |
+
--input ./input.jsonl --output ./results/hf_out \
|
| 128 |
+
--gpu-ids 0,1,2,3,4,5,6,7 --max-new-tokens 32768 \
|
|
|
|
|
|
|
|
|
|
| 129 |
--merge
|
| 130 |
```
|
| 131 |
|
| 132 |
### B. vLLM (OpenAI-compatible)
|
| 133 |
|
| 134 |
+
The unified environment (installed as shown above) serves the model as `tencent/HunyuanOCR` with `-tp 1` and `--max-model-len 131072`, and supports both plain autoregressive (AR) decoding and DFlash speculative decoding from
|
| 135 |
+
the **same install**.
|
| 136 |
|
| 137 |
+
**AR (baseline).** Launch the vLLM server:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
|
| 139 |
```bash
|
| 140 |
+
MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/vLLM/serve.sh
|
| 141 |
curl -sf http://127.0.0.1:8000/v1/models # readiness check
|
| 142 |
```
|
| 143 |
|
| 144 |
+
**DFlash (speculative decoding).** The DFlash draft ships under the `dflash/`
|
| 145 |
+
subfolder of `tencent/HunyuanOCR`, so it is already inside `./HunyuanOCR` after
|
| 146 |
+
the `huggingface-cli download` above. `serve_DFlash.sh` defaults `DFLASH_PATH`
|
| 147 |
+
to `${MODEL_PATH}/dflash`, so no manual copy is needed:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
|
| 149 |
```bash
|
| 150 |
+
MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/DFlash/serve_DFlash.sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
```
|
| 152 |
|
| 153 |
+
**Client (either mode).** Send one image with the shipped client. The
|
| 154 |
+
prompt is locked to an official task type via `--task-type` (run
|
| 155 |
+
`--list-tasks` to see all); sampling and streaming tail-repetition early-stop
|
| 156 |
+
/ cleanup are built in:
|
| 157 |
|
| 158 |
```bash
|
| 159 |
+
python inference/vLLM/infer_vllm_client.py \
|
|
|
|
| 160 |
--host 127.0.0.1 --port 8000 \
|
| 161 |
--model tencent/HunyuanOCR \
|
| 162 |
--image /path/to/document.png \
|
| 163 |
--task-type doc_parse \
|
| 164 |
--max-tokens 32768
|
| 165 |
+
|
| 166 |
+
# add --no-stream to disable streaming + early-stop
|
| 167 |
+
# add --no-doc-postprocess to disable doc_parse markdown normalization
|
| 168 |
```
|
| 169 |
|
| 170 |
Available task types (`--task-type`): `doc_parse` (default), `structured_parse`, `spotting_json`, `spotting_hunyuan`, `layout`, `layout_parse`, `chart_parse`, `formula`, `table`, `doc_trans_en2zh`, `trans_other2en`, `trans_other2zh`.
|
|
|
|
| 173 |
concurrency, resumable):
|
| 174 |
|
| 175 |
```bash
|
| 176 |
+
python inference/vLLM/batch_infer.py \
|
| 177 |
--image-dir /path/to/images \
|
| 178 |
--out-dir /path/to/output \
|
| 179 |
--ports 8000 \
|
|
|
|
| 266 |
## 📜 License
|
| 267 |
|
| 268 |
HunyuanOCR-1.5 is released under the same license as HunyuanOCR 1.0 — the **Tencent Hunyuan Community License Agreement**. See [`LICENSE`](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) for the full terms.
|
| 269 |
+
|
| 270 |
+
---
|
| 271 |
+
|
| 272 |
+
## 📚 Citation
|
| 273 |
+
|
| 274 |
+
```bibtex
|
| 275 |
+
@article{HunyuanOCR_1_5_2026,
|
| 276 |
+
title = {{HunyuanOCR-1.5}: Making Lightweight {OCR} {VLMs} Faster and Better},
|
| 277 |
+
author = {Li, Gengluo and Wan, Xingyu and Peng, Shangpin and Wang, Weinong and Feng, Hao and Du, Yongkun and Wu, Binghong and Ruan, Zheng and Lu, Zhiqiong and Wu, Liang and Lyu, Pengyuan and Shen, Huawen and Lin, Zibin and Hu, Shijing and Yang, Jieneng and Wen, Hongbing and Yu, Guanghua and Liu, Hong and Wang, Bochao and Ma, Can and Hu, Han and Zhang, Chengquan and Zhou, Yu},
|
| 278 |
+
journal = {arXiv preprint arXiv:2607.04884},
|
| 279 |
+
year = {2026}
|
| 280 |
+
}
|
| 281 |
+
```
|
assets/HyOCR_1_5_teaser.png
ADDED
|
Git LFS Details
|
v1.0/README.md
CHANGED
|
@@ -1,25 +1,24 @@
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
language:
|
| 4 |
-
- multilingual
|
| 5 |
pipeline_tag: image-text-to-text
|
| 6 |
library_name: transformers
|
| 7 |
base_model:
|
| 8 |
-
- tencent/HunyuanOCR
|
| 9 |
tags:
|
| 10 |
-
- ocr
|
| 11 |
-
- hunyuan
|
| 12 |
-
- vision-language
|
| 13 |
-
- image-to-text
|
| 14 |
-
- 1B
|
| 15 |
-
- end-to-end
|
| 16 |
---
|
| 17 |
|
| 18 |
<p align="center">
|
| 19 |
-
|
| 20 |
</p>
|
| 21 |
|
| 22 |
-
|
| 23 |
<p align="center">
|
| 24 |
<a href="https://hunyuan.tencent.com/chat/HunyuanDefault?modelId=HY-OCR-1.0&mid=308&from=vision-zh"><b>🎯 Demo</b></a> |
|
| 25 |
<a href="https://huggingface.co/tencent/HunyuanOCR"><b>📥 Model Download</b></a> |
|
|
@@ -33,17 +32,18 @@ tags:
|
|
| 33 |
</p>
|
| 34 |
</h2>
|
| 35 |
|
| 36 |
-
|
| 37 |
## 📖 Introduction
|
| 38 |
-
**HunyuanOCR** stands as a leading end-to-end OCR expert VLM powered by Hunyuan's native multimodal architecture. With a remarkably lightweight 1B parameter design, it has achieved multiple state-of-the-art benchmarks across the industry. The model demonstrates mastery in **complex multilingual document parsing** while excelling in practical applications including **text spotting, open-field information extraction, video subtitle extraction, and photo translation**.
|
| 39 |
|
|
|
|
| 40 |
|
| 41 |
## 🚀 Quick Start with Transformers
|
| 42 |
|
| 43 |
### Installation
|
|
|
|
| 44 |
```bash
|
| 45 |
pip install git+https://github.com/huggingface/transformers@82a06db03535c49aa987719ed0746a76093b1ec4
|
| 46 |
```
|
|
|
|
| 47 |
> **Note**: We will merge it into the Transformers main branch later.
|
| 48 |
|
| 49 |
### Model Inference
|
|
@@ -60,16 +60,16 @@ def clean_repeated_substrings(text):
|
|
| 60 |
if n<8000:
|
| 61 |
return text
|
| 62 |
for length in range(2, n // 10 + 1):
|
| 63 |
-
candidate = text[-length:]
|
| 64 |
count = 0
|
| 65 |
i = n - length
|
| 66 |
-
|
| 67 |
while i >= 0 and text[i:i + length] == candidate:
|
| 68 |
count += 1
|
| 69 |
i -= length
|
| 70 |
|
| 71 |
if count >= 10:
|
| 72 |
-
return text[:n - length * (count - 1)]
|
| 73 |
|
| 74 |
return text
|
| 75 |
|
|
@@ -124,7 +124,6 @@ output_texts = clean_repeated_substrings(processor.batch_decode(
|
|
| 124 |
print(output_texts)
|
| 125 |
```
|
| 126 |
|
| 127 |
-
|
| 128 |
## 🚀 Quick Start with vLLM
|
| 129 |
|
| 130 |
Checkout [vLLM HunyuanOCR Usage Guide](https://docs.vllm.ai/projects/recipes/en/latest/Tencent-Hunyuan/HunyuanOCR.html).
|
|
@@ -139,6 +138,7 @@ uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
|
| 139 |
```
|
| 140 |
|
| 141 |
Note: We suggest to install [cuda-compat-12-9](https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/):
|
|
|
|
| 142 |
```bash
|
| 143 |
sudo dpkg -i cuda-compat-12-9_575.57.08-0ubuntu1_amd64.deb
|
| 144 |
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.9/compat:$LD_LIBRARY_PATH' >> ~/.bashrc
|
|
@@ -148,6 +148,7 @@ ls /usr/local/cuda-12.9/compat
|
|
| 148 |
```
|
| 149 |
|
| 150 |
### Model Deploy
|
|
|
|
| 151 |
```bash
|
| 152 |
vllm serve tencent/HunyuanOCR \
|
| 153 |
--no-enable-prefix-caching \
|
|
@@ -156,6 +157,7 @@ vllm serve tencent/HunyuanOCR \
|
|
| 156 |
```
|
| 157 |
|
| 158 |
### Model Inference
|
|
|
|
| 159 |
```python
|
| 160 |
from vllm import LLM, SamplingParams
|
| 161 |
from PIL import Image
|
|
@@ -167,16 +169,16 @@ def clean_repeated_substrings(text):
|
|
| 167 |
if n<8000:
|
| 168 |
return text
|
| 169 |
for length in range(2, n // 10 + 1):
|
| 170 |
-
candidate = text[-length:]
|
| 171 |
count = 0
|
| 172 |
i = n - length
|
| 173 |
-
|
| 174 |
while i >= 0 and text[i:i + length] == candidate:
|
| 175 |
count += 1
|
| 176 |
i -= length
|
| 177 |
|
| 178 |
if count >= 10:
|
| 179 |
-
return text[:n - length * (count - 1)]
|
| 180 |
|
| 181 |
return text
|
| 182 |
|
|
@@ -202,35 +204,37 @@ print(clean_repeated_substrings(output.outputs[0].text))
|
|
| 202 |
|
| 203 |
## 💬 Application-oriented Prompts
|
| 204 |
|
| 205 |
-
| Task
|
| 206 |
-
|------|---------|
|
| 207 |
-
| **Spotting**
|
| 208 |
-
| **Document Parsing**
|
| 209 |
-
| **General Parsing**
|
| 210 |
-
| **Information Extraction** | • 输出Key的值。<br><br>• 提取图片中的: ['key1','key2', ...] 的字段内容,并按照JSON格式返回。<br><br>• 提取图中的字幕
|
| 211 |
-
| **Translation**
|
| 212 |
|
| 213 |
## 🤝 Join Our Community
|
| 214 |
|
| 215 |
<div align="center">
|
| 216 |
|
| 217 |
-
|
|
| 218 |
-
| :---: | :---: |
|
| 219 |
| <img src="https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/assets/qrcode_for_hunyuanocr_wechat.jpg?raw=true" width="150"> | [Join HunyuanOCR Discord](https://discord.gg/XeD3p2MRDk) |
|
| 220 |
|
| 221 |
</div>
|
| 222 |
|
| 223 |
## 📚 Citation
|
|
|
|
| 224 |
```
|
| 225 |
@misc{hunyuanvisionteam2025hunyuanocrtechnicalreport,
|
| 226 |
-
title={HunyuanOCR Technical Report},
|
| 227 |
author={Hunyuan Vision Team and Pengyuan Lyu and Xingyu Wan and Gengluo Li and Shangpin Peng and Weinong Wang and Liang Wu and Huawen Shen and Yu Zhou and Canhui Tang and Qi Yang and Qiming Peng and Bin Luo and Hower Yang and Xinsong Zhang and Jinnian Zhang and Houwen Peng and Hongming Yang and Senhao Xie and Longsha Zhou and Ge Pei and Binghong Wu and Kan Wu and Jieneng Yang and Bochao Wang and Kai Liu and Jianchen Zhu and Jie Jiang and Linus and Han Hu and Chengquan Zhang},
|
| 228 |
year={2025},
|
| 229 |
journal={arXiv preprint arXiv:2511.19575},
|
| 230 |
-
url={https://arxiv.org/abs/2511.19575},
|
| 231 |
}
|
| 232 |
```
|
| 233 |
|
| 234 |
## 🙏 Acknowledgements
|
|
|
|
| 235 |
We would like to thank [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), [MinerU](https://github.com/opendatalab/MinerU), [MonkeyOCR](https://github.com/Yuliang-Liu/MonkeyOCR), [DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR), [dots.ocr](https://github.com/rednote-hilab/dots.ocr) for their valuable models and ideas.
|
| 236 |
-
We also appreciate the benchmarks: [OminiDocBench](https://github.com/opendatalab/OmniDocBench), [OCRBench](https://github.com/Yuliang-Liu/MultimodalOCR/tree/main/OCRBench), [DoTA](https://github.com/liangyupu/DIMTDA).
|
|
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
language:
|
| 4 |
+
- multilingual
|
| 5 |
pipeline_tag: image-text-to-text
|
| 6 |
library_name: transformers
|
| 7 |
base_model:
|
| 8 |
+
- tencent/HunyuanOCR
|
| 9 |
tags:
|
| 10 |
+
- ocr
|
| 11 |
+
- hunyuan
|
| 12 |
+
- vision-language
|
| 13 |
+
- image-to-text
|
| 14 |
+
- 1B
|
| 15 |
+
- end-to-end
|
| 16 |
---
|
| 17 |
|
| 18 |
<p align="center">
|
| 19 |
+
<img src="../assets/HyOCR_1_5_teaser.png" width="90%"/>
|
| 20 |
</p>
|
| 21 |
|
|
|
|
| 22 |
<p align="center">
|
| 23 |
<a href="https://hunyuan.tencent.com/chat/HunyuanDefault?modelId=HY-OCR-1.0&mid=308&from=vision-zh"><b>🎯 Demo</b></a> |
|
| 24 |
<a href="https://huggingface.co/tencent/HunyuanOCR"><b>📥 Model Download</b></a> |
|
|
|
|
| 32 |
</p>
|
| 33 |
</h2>
|
| 34 |
|
|
|
|
| 35 |
## 📖 Introduction
|
|
|
|
| 36 |
|
| 37 |
+
**HunyuanOCR** stands as a leading end-to-end OCR expert VLM powered by Hunyuan's native multimodal architecture. With a remarkably lightweight 1B parameter design, it has achieved multiple state-of-the-art benchmarks across the industry. The model demonstrates mastery in **complex multilingual document parsing** while excelling in practical applications including **text spotting, open-field information extraction, video subtitle extraction, and photo translation**.
|
| 38 |
|
| 39 |
## 🚀 Quick Start with Transformers
|
| 40 |
|
| 41 |
### Installation
|
| 42 |
+
|
| 43 |
```bash
|
| 44 |
pip install git+https://github.com/huggingface/transformers@82a06db03535c49aa987719ed0746a76093b1ec4
|
| 45 |
```
|
| 46 |
+
|
| 47 |
> **Note**: We will merge it into the Transformers main branch later.
|
| 48 |
|
| 49 |
### Model Inference
|
|
|
|
| 60 |
if n<8000:
|
| 61 |
return text
|
| 62 |
for length in range(2, n // 10 + 1):
|
| 63 |
+
candidate = text[-length:]
|
| 64 |
count = 0
|
| 65 |
i = n - length
|
| 66 |
+
|
| 67 |
while i >= 0 and text[i:i + length] == candidate:
|
| 68 |
count += 1
|
| 69 |
i -= length
|
| 70 |
|
| 71 |
if count >= 10:
|
| 72 |
+
return text[:n - length * (count - 1)]
|
| 73 |
|
| 74 |
return text
|
| 75 |
|
|
|
|
| 124 |
print(output_texts)
|
| 125 |
```
|
| 126 |
|
|
|
|
| 127 |
## 🚀 Quick Start with vLLM
|
| 128 |
|
| 129 |
Checkout [vLLM HunyuanOCR Usage Guide](https://docs.vllm.ai/projects/recipes/en/latest/Tencent-Hunyuan/HunyuanOCR.html).
|
|
|
|
| 138 |
```
|
| 139 |
|
| 140 |
Note: We suggest to install [cuda-compat-12-9](https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/):
|
| 141 |
+
|
| 142 |
```bash
|
| 143 |
sudo dpkg -i cuda-compat-12-9_575.57.08-0ubuntu1_amd64.deb
|
| 144 |
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.9/compat:$LD_LIBRARY_PATH' >> ~/.bashrc
|
|
|
|
| 148 |
```
|
| 149 |
|
| 150 |
### Model Deploy
|
| 151 |
+
|
| 152 |
```bash
|
| 153 |
vllm serve tencent/HunyuanOCR \
|
| 154 |
--no-enable-prefix-caching \
|
|
|
|
| 157 |
```
|
| 158 |
|
| 159 |
### Model Inference
|
| 160 |
+
|
| 161 |
```python
|
| 162 |
from vllm import LLM, SamplingParams
|
| 163 |
from PIL import Image
|
|
|
|
| 169 |
if n<8000:
|
| 170 |
return text
|
| 171 |
for length in range(2, n // 10 + 1):
|
| 172 |
+
candidate = text[-length:]
|
| 173 |
count = 0
|
| 174 |
i = n - length
|
| 175 |
+
|
| 176 |
while i >= 0 and text[i:i + length] == candidate:
|
| 177 |
count += 1
|
| 178 |
i -= length
|
| 179 |
|
| 180 |
if count >= 10:
|
| 181 |
+
return text[:n - length * (count - 1)]
|
| 182 |
|
| 183 |
return text
|
| 184 |
|
|
|
|
| 204 |
|
| 205 |
## 💬 Application-oriented Prompts
|
| 206 |
|
| 207 |
+
| Task | Prompt |
|
| 208 |
+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| 209 |
+
| **Spotting** | 检测并识别图片中的文字,将文本坐标格式化输出。 |
|
| 210 |
+
| **Document Parsing** | • 识别图片中的公式,用LaTeX格式表示。<br><br>• 把图中的表格解析为HTML。<br><br>• 解析图中的图表,对于流程图使用Mermaid格式表示,其他图表使用Markdown格式表示。<br><br>• 提取文档图片中正文的所有信息用markdown格式表示,其中页眉、页脚部分忽略,表格用html格式表达,文档中公式用latex格式表示,按照阅读顺序组织进行解析。 |
|
| 211 |
+
| **General Parsing** | • 提取图中的文字。 |
|
| 212 |
+
| **Information Extraction** | • 输出Key的值。<br><br>• 提取图片中的: ['key1','key2', ...] 的字段内容,并按照JSON格式返回。<br><br>• 提取图中的字幕 |
|
| 213 |
+
| **Translation** | 先提取文字,再将文字内容翻译为英文。若是文档,则其中页眉、页脚忽略。公式用latex格式表示,表格用html格式表示。 |
|
| 214 |
|
| 215 |
## 🤝 Join Our Community
|
| 216 |
|
| 217 |
<div align="center">
|
| 218 |
|
| 219 |
+
| Wechat Discussion Group | Discord Group |
|
| 220 |
+
| :------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------: |
|
| 221 |
| <img src="https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/assets/qrcode_for_hunyuanocr_wechat.jpg?raw=true" width="150"> | [Join HunyuanOCR Discord](https://discord.gg/XeD3p2MRDk) |
|
| 222 |
|
| 223 |
</div>
|
| 224 |
|
| 225 |
## 📚 Citation
|
| 226 |
+
|
| 227 |
```
|
| 228 |
@misc{hunyuanvisionteam2025hunyuanocrtechnicalreport,
|
| 229 |
+
title={HunyuanOCR Technical Report},
|
| 230 |
author={Hunyuan Vision Team and Pengyuan Lyu and Xingyu Wan and Gengluo Li and Shangpin Peng and Weinong Wang and Liang Wu and Huawen Shen and Yu Zhou and Canhui Tang and Qi Yang and Qiming Peng and Bin Luo and Hower Yang and Xinsong Zhang and Jinnian Zhang and Houwen Peng and Hongming Yang and Senhao Xie and Longsha Zhou and Ge Pei and Binghong Wu and Kan Wu and Jieneng Yang and Bochao Wang and Kai Liu and Jianchen Zhu and Jie Jiang and Linus and Han Hu and Chengquan Zhang},
|
| 231 |
year={2025},
|
| 232 |
journal={arXiv preprint arXiv:2511.19575},
|
| 233 |
+
url={https://arxiv.org/abs/2511.19575},
|
| 234 |
}
|
| 235 |
```
|
| 236 |
|
| 237 |
## 🙏 Acknowledgements
|
| 238 |
+
|
| 239 |
We would like to thank [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), [MinerU](https://github.com/opendatalab/MinerU), [MonkeyOCR](https://github.com/Yuliang-Liu/MonkeyOCR), [DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR), [dots.ocr](https://github.com/rednote-hilab/dots.ocr) for their valuable models and ideas.
|
| 240 |
+
We also appreciate the benchmarks: [OminiDocBench](https://github.com/opendatalab/OmniDocBench), [OCRBench](https://github.com/Yuliang-Liu/MultimodalOCR/tree/main/OCRBench), [DoTA](https://github.com/liangyupu/DIMTDA).
|