Instructions to use Evan-613/HunyuanOCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Evan-613/HunyuanOCR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Evan-613/HunyuanOCR")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Evan-613/HunyuanOCR")
model = AutoModelForMultimodalLM.from_pretrained("Evan-613/HunyuanOCR")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Evan-613/HunyuanOCR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Evan-613/HunyuanOCR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Evan-613/HunyuanOCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Evan-613/HunyuanOCR

SGLang

How to use Evan-613/HunyuanOCR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Evan-613/HunyuanOCR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Evan-613/HunyuanOCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Evan-613/HunyuanOCR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Evan-613/HunyuanOCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Evan-613/HunyuanOCR with Docker Model Runner:
```
docker model run hf.co/Evan-613/HunyuanOCR
```

HunyuanOCR / dflash /README.md

Evan-613

Duplicate from tencent/HunyuanOCR

b4b39c9 about 13 hours ago

preview code

Raw

History Blame Contribute Delete

5.51 kB

metadata

license: other
license_name: tencent-hunyuan-community
license_link: https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE
tags:
  - ocr
  - speculative-decoding
  - draft-model
  - dflash
  - block-diffusion
  - vision-language-model
base_model: tencent/HunyuanOCR
library_name: transformers

HunyuanOCR-1.5 · DFlash Draft · Preview

Speculative-decoding draft for tencent/HunyuanOCR

⚠️ This model is not usable standalone. It is a draft model used only for speculative decoding together with the target model tencent/HunyuanOCR.

📖 What is DFlash?

End-to-end OCR is often accompanied by long autoregressive decoding — the major bottleneck for dense documents, tables, formulas, and other long structured outputs.

HunyuanOCR-1.5 adopts a speculative-decoding framework based on DFlash:

A lightweight block-diffusion draft model (this repo) proposes multiple candidate tokens in parallel.
The target model (tencent/HunyuanOCR) verifies them in a single forward pass.
Accepted tokens are committed as-is, so the output distribution of the target model is preserved — DFlash is a lossless acceleration.

The result is significantly reduced decoding latency for long structured OCR outputs, without sacrificing accuracy.

Architecture: 5-layer Qwen3-style block-diffusion draft, predicting 16 masked tokens in a single block. The draft is bound to target-layer indices [1, 8, 15, 22] of the 24-layer HunyuanOCR-1.5 base.

⚙️ Environment

Python 3.10+
PyTorch 2.1+ (CUDA 12.1+)
transformers
vLLM nightly — required for real speculative-decoding speedup at deployment time. DFlash support is included in the nightly wheel; no separate patch is needed.

uv pip install -U vllm \
    --torch-backend=cu130 \
    --extra-index-url https://wheels.vllm.ai/nightly
uv pip install runai-model-streamer

💡 On CUDA 12.x, replace --torch-backend=cu130 with the matching tag (e.g. cu121, cu124).

🚀 How to use

A. transformers — draft-load check

Use the shipped script from the GitHub repo. It loads the draft, runs it alongside the target for one image, and verifies that the AR reference matches:

git clone -b develop https://github.com/Tencent-Hunyuan/HunyuanOCR.git
cd HunyuanOCR

python inference/infer_dflash.py \
    --model        tencent/HunyuanOCR \
    --dflash-model ./hunyuanocr_dflash \
    --image        /path/to/document.png \
    --num-spec-tokens 15

ℹ️ infer_dflash.py only verifies that the DFlash draft loads and produces a matching AR reference on a single image. Real speculative-decoding acceleration is only realized under vLLM, see below.

⬇️ Preparing the draft directory. The DFlash draft lives under the dflash/ subfolder of tencent/HunyuanOCR. Because vLLM's --speculative-config and trust_remote_code custom-code loading do not support HF subfolders, download that subfolder into a flat local directory first and point the draft path at it:
python -c "from huggingface_hub import snapshot_download; import shutil, os; \
d=snapshot_download('tencent/HunyuanOCR', allow_patterns=['dflash/*']); \
shutil.copytree(os.path.join(d,'dflash'), './hunyuanocr_dflash', dirs_exist_ok=True)"
Then use ./hunyuanocr_dflash as the draft path in the commands below.

B. vLLM speculative decoding

MODEL_PATH=tencent/HunyuanOCR \
DFLASH_PATH=./hunyuanocr_dflash \
GPU=0 PORT=8001 GPU_MEM_UTIL=0.9 \
NUM_SPEC_TOKENS=15 \
bash inference/serve_dflash.sh

Under the hood the launch script passes:

--speculative-config '{"method":"dflash","model":"./hunyuanocr_dflash","num_speculative_tokens":15}'

to the vLLM entrypoint. Send an OpenAI-compatible request with the shipped single-image client:

python inference/infer_vllm_client.py \
    --host 127.0.0.1 --port 8001 \
    --model tencent/HunyuanOCR \
    --image /path/to/document.png

C. llama.cpp (PC-side)

A DFlash-adapted llama.cpp fork is provided for CPU / consumer-GPU / laptop speculative decoding. See docs/llama_cpp.md in the GitHub repo for the full guide (GGUF conversion of both target + draft, llama-server launch, and a smoke-test client).

📦 Files in this repo

file	purpose
`model.safetensors`	draft weights (float32)
`config.json`	draft config; sets `auto_map` to `dflash.DFlashDraftModel`
`dflash.py`	`DFlashDraftModel` implementation (loaded via `trust_remote_code=True`)
`chat_template.jinja`, `tokenizer.json`, `tokenizer_config.json`, `processor_config.json`	tokenizer / processor, kept in sync with the target model

🔗 Related repositories

Target model (required): tencent/HunyuanOCR
GitHub — training & inference toolkit (branch develop): https://github.com/Tencent-Hunyuan/HunyuanOCR
HunyuanOCR-1.0 (previous generation, archived under v1.0/): tencent/HunyuanOCR/v1.0

📜 License

HunyuanOCR-1.5 (including the DFlash draft) is released under the same license as HunyuanOCR 1.0 — the Tencent Hunyuan Community License Agreement.