Update README.md

README.md

@@ -17,96 +17,81 @@ pipeline_tag: image-text-to-text
 library_name: transformers
 ---
 
-# HunyuanOCR-1.5  ·  Preview
+# HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
 
-<div align="center">
+<p align="center">
+  <img src="https://raw.githubusercontent.com/Tencent-Hunyuan/HunyuanOCR/main/assets/hyocr-1.5-head-img.png" width="90%"/>
+</p>
 
-**Towards Efficient and Effective E2E OCR**
+🤗 [Model](https://huggingface.co/tencent/HunyuanOCR) | 💻 [GitHub](https://github.com/Tencent-Hunyuan/HunyuanOCR) | 📄 [Paper](https://arxiv.org/pdf/2607.04884)
 
-</div>
-
-> 📦 **Model layout.** This repository now hosts **HunyuanOCR-1.5** at the
-> root (target base weights). The **DFlash speculative-decoding draft** lives
-> under [`dflash/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash),
-> and the previous **HunyuanOCR-1.0** is archived under
-> [`v1.0/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0).
-> To load HunyuanOCR-1.0, use `subfolder="v1.0"` (or download the `v1.0/`
-> directory directly).
+> 📦 **Model layout.** This repository hosts **HunyuanOCR-1.5** at the root
+> (target base weights). The **DFlash speculative-decoding draft** lives under
+> [`dflash/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash), and
+> the previous **HunyuanOCR-1.0** is archived under
+> [`v1.0/`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0)
+> (load it with `subfolder="v1.0"`, or download the `v1.0/` directory directly).
 
 ---
 
 ## 📖 Introduction
 
-**HunyuanOCR-1.5** is a lightweight, end-to-end OCR-specialized vision-language
-model. It targets a broad range of text-centric visual tasks and unifies
-**document parsing, text spotting, information extraction, and text-image
-translation** within a single end-to-end VLM.
-
-Building upon the validated lightweight architecture of **HunyuanOCR-1.0**,
-HunyuanOCR-1.5 does *not* redesign the backbone. Instead, it performs a
-systematic upgrade around two goals — **making the model faster and better**:
-
-- ⚡ **Faster — DFlash inference acceleration.**
-  A lightweight block-diffusion draft model drafts multiple candidate tokens in
-  parallel, verified by the target model in a single pass, significantly
-  reducing decoding latency of long structured OCR outputs (dense documents,
-  tables, formulas) while **preserving the target model's output distribution**.
-  Draft weights: [`tencent/HunyuanOCR/dflash`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash).
-
-- 💻 **PC-side deployment via llama.cpp.**
-  Beyond server-grade vLLM, HunyuanOCR-1.5 also supports **CPU / consumer-GPU /
-  laptop** deployment via [`llama.cpp`](https://github.com/ggml-org/llama.cpp)
-  with an OpenAI-compatible `llama-server`. A DFlash-adapted `llama.cpp` fork is
-  also provided so the same speculative-decoding acceleration is available on
-  PC.
-
-- 🧠 **Better — Agentic Data Flow + upgraded training recipe.**
-  An agent-driven data-construction system (**Agentic Data Flow**) translates
-  model weaknesses into executable data requirements, targeting long-tail
-  capabilities such as **low-resource OCR, ancient-script OCR, and multi-image
-  text-centric QA**. Pretraining Stage-3 is re-planned with **4K resolution** and
-  a **128K context window**; post-training refines SFT data and further explores
-  RL across different OCR tasks.
-
-Together, HunyuanOCR-1.5 achieves both faster inference and broader OCR
-capability coverage while retaining the deployment advantages of a lightweight
-end-to-end model.
+**HunyuanOCR-1.5** is a lightweight, end-to-end OCR-specialized vision-language model. It targets a broad range of text-centric visual tasks and unifies **document parsing, text spotting, information extraction, text-image translation** within a single end-to-end VLM.
+
+Building upon the validated lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does **not** redesign the model backbone. Instead, it performs a systematic upgrade around two goals — **making the model faster and better**:
+
+- ⚡ **Faster — DFlash inference acceleration.** End-to-end OCR is often accompanied by long autoregressive decoding, which becomes the major bottleneck for dense documents, tables, formulas, and other long structured outputs. HunyuanOCR-1.5 adapts a speculative-decoding framework based on **DFlash**: a lightweight block-diffusion draft model drafts multiple candidate tokens in parallel, which are then verified by the target model in a single pass. This significantly reduces the decoding latency of long structured outputs while **preserving the output distribution** of the target model.
+- 💻 **PC-side deployment via llama.cpp.** Beyond server-grade vLLM, HunyuanOCR-1.5 also supports **CPU / consumer-GPU / laptop** deployment through [`llama.cpp`](https://github.com/ggml-org/llama.cpp) with a GGUF-converted checkpoint and an OpenAI-compatible `llama-server`. A DFlash-adapted `llama.cpp` fork is provided as well, so the same speculative-decoding acceleration is available on PC.
+- 🧠 **Better — Agentic Data Flow + upgraded training recipe.** On the data side, we propose **Agentic Data Flow**, an agent-driven data-construction system that translates model weaknesses into executable data requirements. Agents deeply participate in material search, tool-based verification, sample cleaning, and data-pipeline development, and iterate in a closed loop with algorithm engineers. In HunyuanOCR-1.5, this system is used for targeted long-tail capabilities such as **low-resource OCR, ancient-script OCR, and multi-image text-centric QA**. On the training side, we systematically upgrade the recipe: pretraining Stage-3 is re-planned to incorporate the newly produced capability data, multi-image data, and historical OCR data, with maximum image resolution extended to **4K** and context window extended to **128K**; post-training refines the SFT data and further explores RL across different OCR tasks to amplify the gains from reinforcement learning.
+
+Together, HunyuanOCR-1.5 achieves **both faster inference and broader OCR capability coverage** while retaining the deployment advantages of a lightweight end-to-end model. The full SFT / DFlash training pipeline and the transformers / vLLM / llama.cpp inference stack are open-sourced in the [GitHub repo](https://github.com/Tencent-Hunyuan/HunyuanOCR).
 
 ---
 
 ## ⚙️ Environment
 
-- Python 3.10+
-- PyTorch 2.1+ (CUDA 12.1+)
-- **transformers** (ships `HunYuanVLForConditionalGeneration` + `AutoProcessor` for the HunyuanOCR-1.5 series)
-- **vLLM nightly** — for serving and DFlash speculative decoding
+Inference is split into **three self-contained, mutually exclusive setups** in the [GitHub repo](https://github.com/Tencent-Hunyuan/HunyuanOCR) under [`inference/`](https://github.com/Tencent-Hunyuan/HunyuanOCR/tree/main/inference). vLLM (AR / DFlash) and native transformers inference require different, incompatible `transformers` versions and **cannot share one environment** — this is a validated constraint, not a preference:
 
-### transformers
+| Setup | vLLM | DFlash accel. | transformers | CUDA | Best for |
+|---|:-:|:-:|:-:|---|---|
+| [`inference/vllm_0_18_1`](https://github.com/Tencent-Hunyuan/HunyuanOCR/tree/main/inference/vllm_0_18_1) | 0.18.1 (release) | ❌ | ❌ | 12.x | simplest setup, AR only |
+| [`inference/nightly`](https://github.com/Tencent-Hunyuan/HunyuanOCR/tree/main/inference/nightly) | nightly | ✅ | ❌ | 13 | AR + DFlash acceleration |
+| [`inference/transformers`](https://github.com/Tencent-Hunyuan/HunyuanOCR/tree/main/inference/transformers) | — | — | ✅ 5.13.0 | host driver | native HF inference |
+
+Each subfolder ships its own README and `requirements.txt`. See
+[`inference/README.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/README.md)
+for the selection guide and the full rationale.
+
+**Common prerequisites:** Python 3.10+ (3.12 tested), an NVIDIA GPU, and
+`huggingface_hub` for downloading the weights:
 
 ```bash
-pip install transformers torch pillow accelerate
-# for FlashAttention:
-pip install flash-attn --no-build-isolation
+pip install -U "huggingface_hub[cli]"
+# target base (1.5) — skip the archived 1.0 to save space
+huggingface-cli download tencent/HunyuanOCR --local-dir ./HunyuanOCR --exclude "v1.0/*"
 ```
 
-### vLLM serving
+The download contains both the base model and the `dflash/` draft model.
+
+---
+
+## 🧪 Inference
 
-We use a dedicated venv for inference to keep vLLM nightly isolated:
+All setups share the same weights and the same task-type prompts + sampling
+(`temperature=0.0`, `top_p=1.0`, `top_k=-1`, `repetition_penalty=1.08`) +
+post-processing, so their outputs are directly comparable. Grab the toolkit from
+GitHub first:
 
 ```bash
-uv pip install -U vllm \
-    --torch-backend=cu130 \
-    --extra-index-url https://wheels.vllm.ai/nightly
-uv pip install runai-model-streamer
+git clone https://github.com/Tencent-Hunyuan/HunyuanOCR.git
+cd HunyuanOCR
 ```
 
-> 💡 On CUDA 12.x, replace `--torch-backend=cu130` with the matching tag
-> (e.g. `cu121`, `cu124`).
----
-
-## 🚀 Quick start
+### A. HuggingFace transformers (native)
 
-### A. HuggingFace transformers
+The model ships the official `HunYuanVLForConditionalGeneration` + `AutoProcessor`
+integration (transformers **≥ 5.13.0**). The simplest path — weights are pulled
+from the Hub automatically:
 
 ```python
 import torch
@@ -124,7 +109,6 @@ prompt = (
     "提取文档图片中正文的所有信息用markdown格式表示，其中页眉、页脚部分忽略，"
     "表格用html格式表达，文档中公式用latex格式表示，按照阅读顺序组织进行解析。"
 )
-
 messages = [{
     "role": "user",
     "content": [
@@ -132,66 +116,161 @@ messages = [{
         {"type": "text",  "text":  prompt},
     ],
 }]
-
 inputs = processor.apply_chat_template(
     messages, add_generation_prompt=True, tokenize=True,
     return_dict=True, return_tensors="pt",
 ).to(model.device)
-
 with torch.inference_mode():
     out = model.generate(**inputs, max_new_tokens=8000, do_sample=False)
-
 gen = out[:, inputs["input_ids"].shape[1]:]
 print(processor.batch_decode(gen, skip_special_tokens=True)[0])
 ```
 
-Or use the ready-made single-image script from the repo:
+For **multi-GPU batch inference** with sampling / early-stop / doc-parse
+normalization strictly aligned to the vLLM client, use the shipped script in a
+dedicated `transformers==5.13.0` environment (see
+[`inference/transformers/README.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/transformers/README.md)):
 
 ```bash
-git clone -b develop https://github.com/Tencent-Hunyuan/HunyuanOCR.git
-cd HunyuanOCR
+# install per inference/transformers/requirements.txt, then:
+python inference/transformers/infer_hf_8gpu_hyocr15.py \
+    --model  ./HunyuanOCR \
+    --input  /path/to/bench.jsonl \
+    --output ./results/hf_out \
+    --gpu-ids 0,1,2,3,4,5,6,7 \
+    --max-new-tokens 8192 \
+    --merge
+```
 
-python inference/infer_base.py \
-    --model tencent/HunyuanOCR \
-    --image /path/to/document.png \
-    --max-new-tokens 8000
+### B. vLLM (OpenAI-compatible)
+
+Two mutually-exclusive vLLM setups. Both serve the model as `tencent/HunyuanOCR`
+with `-tp 1` and `--max-model-len 131072`.
+
+**B1 — vLLM 0.18.1 (release, AR only, simplest).** The release build natively
+supports `HunYuanVLForConditionalGeneration`; no nightly or patch required.
+Install per
+[`inference/vllm_0_18_1/requirements.txt`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/vllm_0_18_1/requirements.txt)
+(core: `pip install "vllm==0.18.1"`), then:
+
+```bash
+MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/vllm_0_18_1/serve.sh
+curl -sf http://127.0.0.1:8000/v1/models     # readiness check
+```
+
+**B2 — vLLM nightly (AR + DFlash speculative decoding).** Required for the real
+DFlash speedup. Install per
+[`inference/nightly/requirements.txt`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/inference/nightly/requirements.txt):
+
+```bash
+uv pip install -U vllm --torch-backend=cu130 --extra-index-url https://wheels.vllm.ai/nightly
+uv pip install runai-model-streamer
 ```
 
-### B. vLLM
+The DFlash draft lives under the `dflash/` subfolder of `tencent/HunyuanOCR`.
+vLLM's `--speculative-config` does not accept an HF subfolder, so download the
+draft weight into a flat local dir first:
 
 ```bash
-# Autoregressive baseline
-MODEL_PATH=tencent/HunyuanOCR \
-GPU=0 PORT=8000 GPU_MEM_UTIL=0.9 \
-bash inference/serve_ar.sh
+huggingface-cli download tencent/HunyuanOCR dflash/model.safetensors --local-dir ./HunyuanOCR
+cp -r ./HunyuanOCR/dflash ./hyocr_dflash
+```
+
+Then launch AR or DFlash:
+
+```bash
+# AR (autoregressive) baseline
+MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/nightly/serve_ar.sh
 
 # DFlash speculative decoding
-# The draft lives under the `dflash/` subfolder of tencent/HunyuanOCR;
-# download it into a flat local dir first (HF subfolder loading is
-# unsupported by vLLM's speculative-config):
-#   python -c "from huggingface_hub import snapshot_download; import shutil, os; \
-#   d=snapshot_download('tencent/HunyuanOCR', allow_patterns=['dflash/*']); \
-#   shutil.copytree(os.path.join(d,'dflash'), './hunyuanocr_dflash', dirs_exist_ok=True)"
-MODEL_PATH=tencent/HunyuanOCR \
-DFLASH_PATH=./hunyuanocr_dflash \
-GPU=0 PORT=8001 GPU_MEM_UTIL=0.9 NUM_SPEC_TOKENS=15 \
-bash inference/serve_dflash.sh
+MODEL_PATH=./HunyuanOCR DFLASH_PATH=./hyocr_dflash \
+GPU=0 PORT=8001 NUM_SPEC_TOKENS=15 bash inference/nightly/serve_dflash.sh
 ```
 
-Send one image with the shipped client (streaming + tail-repetition early-stop,
-matches internal bench sampling params):
+**Client (either vLLM setup).** Send one image with the shipped client. The
+prompt is locked to an official task type via `--task-type` (run `--list-tasks`
+to see all); sampling and streaming tail-repetition early-stop / cleanup are
+built in:
 
 ```bash
-python inference/infer_vllm_client.py \
+# use the client from the same setup folder, e.g. inference/vllm_0_18_1/ or inference/nightly/
+python inference/vllm_0_18_1/infer_vllm_client.py \
     --host 127.0.0.1 --port 8000 \
     --model tencent/HunyuanOCR \
-    --image /path/to/document.png
+    --image /path/to/document.png \
+    --task-type doc_parse \
+    --max-tokens 32768
+# add --no-stream to disable streaming + early-stop
+# add --no-doc-postprocess to disable doc_parse markdown normalization
+```
+
+Available task types (`--task-type`): `doc_parse` (default), `structured_parse`, `spotting_json`, `spotting_hunyuan`, `layout`, `layout_parse`, `chart_parse`, `formula`, `table`, `doc_trans_en2zh`, `trans_other2en`, `trans_other2zh`.
+
+For **batch** inference over a directory (same task types, multi-endpoint
+concurrency, resumable):
+
+```bash
+python inference/vllm_0_18_1/batch_infer.py \
+    --image-dir /path/to/images \
+    --out-dir /path/to/output \
+    --ports 8000 \
+    --task-type doc_parse \
+    --max-tokens 32768 \
+    --concurrency 16
+```
+
+Or hand-written with the OpenAI SDK:
+
+```python
+import base64
+from openai import OpenAI
+
+def data_url(p):  # Mime is fixed to image/jpeg
+    return f"data:image/jpeg;base64,{base64.b64encode(open(p,'rb').read()).decode()}"
+
+client = OpenAI(api_key="EMPTY", base_url="http://127.0.0.1:8000/v1")
+resp = client.chat.completions.create(
+    model="tencent/HunyuanOCR",
+    messages=[
+        {"role": "system", "content": ""},
+        {"role": "user", "content": [
+            {"type": "image_url", "image_url": {"url": data_url("/path/to/document.png")}},
+            {"type": "text", "text": "请提取图片中的文字内容。"},
+        ]},
+    ],
+    max_tokens=32768,
+    temperature=0.0, top_p=1.0,
+    extra_body={"top_k": -1, "repetition_penalty": 1.08, "skip_special_tokens": True},
+)
+print(resp.choices[0].message.content)
 ```
 
 ### C. PC-side deployment via llama.cpp
 
-See `docs/llama_cpp.md` in the GitHub repo for GGUF conversion, community
-`llama-server` launch, and the DFlash-adapted fork.
+For **CPU / consumer-GPU / laptop** environments, HunyuanOCR-1.5 can also be deployed through [`llama.cpp`](https://github.com/ggml-org/llama.cpp) after converting the checkpoint to GGUF. Both the community `llama.cpp` (HunyuanOCR base only) and a DFlash-adapted fork ([`wendadawen/llama.cpp @ dflash-adapt-hunyuanocr-hunyuanstyle`](https://github.com/wendadawen/llama.cpp/tree/dflash-adapt-hunyuanocr-hunyuanstyle)) are supported.
+
+Minimal build & serve (community, no DFlash):
+
+```bash
+# 1. Build
+git clone https://github.com/ggml-org/llama.cpp.git && cd llama.cpp
+cmake -B build -DLLAMA_BUILD_EXAMPLES=ON   # add -DGGML_CUDA=ON for NVIDIA GPU
+cmake --build ./build --config Release -j
+
+# 2. Convert HunyuanOCR to GGUF (base + mmproj)
+hf download tencent/HunyuanOCR --local-dir ./HunyuanOCR --exclude "v1.0/*"
+python3 convert_hf_to_gguf.py --outfile ./HunyuanOCR/hyocr-f16.gguf --outtype f16 ./HunyuanOCR
+python3 convert_hf_to_gguf.py --outfile ./HunyuanOCR/mmproj-hyocr-f16.gguf --outtype f16 --mmproj ./HunyuanOCR
+
+# 3. Serve (OpenAI-compatible)
+build/bin/llama-server \
+  --model ./HunyuanOCR/hyocr-f16.gguf \
+  --mmproj ./HunyuanOCR/mmproj-hyocr-f16.gguf \
+  --host 0.0.0.0 --port 8080 --alias HYVL \
+  --ctx-size 10240 --n-predict 4096
+```
+
+The DFlash-adapted variant and the full guide are in [`docs/llama_cpp.md`](https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/docs/llama_cpp.md) in the GitHub repo.
 
 ---
 
@@ -202,23 +281,26 @@ See `docs/llama_cpp.md` in the GitHub repo for GGUF conversion, community
 表格用html格式表达，文档中公式用latex格式表示，按照阅读顺序组织进行解析。
 ```
 
-The model also handles text spotting, information extraction, and text-image
-translation — pass a task-specific instruction as the text prompt.
+The model also handles text spotting, information extraction, and text-image translation — pass a task-specific instruction as the text prompt (or use `--task-type` with the shipped client).
 
 ---
 
-## 🔗 Related repositories
+## 🔗 Related resources
+
+- **GitHub — training & inference toolkit**: <https://github.com/Tencent-Hunyuan/HunyuanOCR>
+- **DFlash draft weights**: [`tencent/HunyuanOCR/dflash`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash)
+- **HunyuanOCR-1.0** (previous generation, archived under `v1.0/`): [`tencent/HunyuanOCR/v1.0`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0)
+
+---
+
+## 🙏 Acknowledgements
+
+We would like to thank [Qwen](https://github.com/QwenLM/Qwen3.6) and [DFlash](https://github.com/z-lab/dflash) for their valuable models and ideas.
 
-- **GitHub — training & inference toolkit** (branch `develop`):
-  <https://github.com/Tencent-Hunyuan/HunyuanOCR>
-- **DFlash draft weights** (required for speculative-decoding acceleration):
-  [`tencent/HunyuanOCR/dflash`](https://huggingface.co/tencent/HunyuanOCR/tree/main/dflash)
-- **HunyuanOCR-1.0** (previous generation, archived under `v1.0/`):
-  [`tencent/HunyuanOCR/v1.0`](https://huggingface.co/tencent/HunyuanOCR/tree/main/v1.0)
+Special thanks to the Hugging Face community for their Day-0 support.
 
 ---
 
 ## 📜 License
 
-HunyuanOCR-1.5 is released under the same license as HunyuanOCR 1.0 — the
-**Tencent Hunyuan Community License Agreement**.
+HunyuanOCR-1.5 is released under the same license as HunyuanOCR 1.0 — the **Tencent Hunyuan Community License Agreement**. See [`LICENSE`](https://huggingface.co/tencent/HunyuanOCR/blob/main/LICENSE) for the full terms.