Image-Text-to-Text
Transformers
Safetensors
English
Chinese
internvl_chat
feature-extraction
visual-language
paddleocr
document-parse
HPD-Parsing
speculative-decoding
P-MTP
eval results
conversational
custom_code
Instructions to use PaddlePaddle/HPD-Parsing with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PaddlePaddle/HPD-Parsing with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="PaddlePaddle/HPD-Parsing", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("PaddlePaddle/HPD-Parsing", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use PaddlePaddle/HPD-Parsing with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "PaddlePaddle/HPD-Parsing" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PaddlePaddle/HPD-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/PaddlePaddle/HPD-Parsing
- SGLang
How to use PaddlePaddle/HPD-Parsing with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "PaddlePaddle/HPD-Parsing" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PaddlePaddle/HPD-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "PaddlePaddle/HPD-Parsing" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "PaddlePaddle/HPD-Parsing", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use PaddlePaddle/HPD-Parsing with Docker Model Runner:
docker model run hf.co/PaddlePaddle/HPD-Parsing
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -71,7 +71,7 @@ We introduce **HPD-Parsing**, a lightweight (1B) and high-throughput document pa
|
|
| 71 |
**🔄 Staged Adaptation with Automated Difficulty-Aware Data Curation**: We develop a staged adaptation strategy that transfers conventional autoregressive document parsing capabilities to the proposed hierarchical parallel decoding paradigm while preserving parsing accuracy. The strategy is supported by an automated difficulty-aware data curation pipeline that integrates large-scale data collection, model-assisted annotation, difficulty estimation, and balanced sampling. By progressively adapting the model and emphasizing challenging samples, the training framework mitigates the accuracy degradation caused by the transition to parallel decoding with minimal manual annotation effort.
|
| 72 |
|
| 73 |
|
| 74 |
-
**⚡ State-of-the-Art Throughput with Competitive Parsing Accuracy**: HPD-Parsing achieves state-of-the-art inference efficiency on OmniDocBench v1.6, reaching a peak throughput of 4,752 Tokens Per Second (TPS). It delivers 1.
|
| 75 |
|
| 76 |
|
| 77 |
### **HPD-Parsing Architecture**
|
|
@@ -83,9 +83,7 @@ HPD-Parsing adopts **InternVL3.5-1B** as its backbone, applies dynamic tile-base
|
|
| 83 |
</div>
|
| 84 |
|
| 85 |
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
## Usage
|
| 89 |
|
| 90 |
HPD-Parsing runs on a **customized build of vLLM** (based on vLLM v0.17.1) that implements the dynamic request forking required by hierarchical parallel decoding and adapts P-MTP speculative decoding.
|
| 91 |
|
|
@@ -206,6 +204,26 @@ Key arguments:
|
|
| 206 |
|
| 207 |
> Reference implementation only. Use the vLLM path for production throughput — its paged KV cache enables true concurrent branches with zero-copy prefix sharing.
|
| 208 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
## Evaluation & Benchmark
|
| 210 |
|
| 211 |
Scripts under [`eval/`](./eval) reproduce both the throughput (TPS) and the OmniDocBench v1.6 accuracy numbers via a decoupled **infer -> convert -> evaluate** pipeline:
|
|
|
|
| 71 |
**🔄 Staged Adaptation with Automated Difficulty-Aware Data Curation**: We develop a staged adaptation strategy that transfers conventional autoregressive document parsing capabilities to the proposed hierarchical parallel decoding paradigm while preserving parsing accuracy. The strategy is supported by an automated difficulty-aware data curation pipeline that integrates large-scale data collection, model-assisted annotation, difficulty estimation, and balanced sampling. By progressively adapting the model and emphasizing challenging samples, the training framework mitigates the accuracy degradation caused by the transition to parallel decoding with minimal manual annotation effort.
|
| 72 |
|
| 73 |
|
| 74 |
+
**⚡ State-of-the-Art Throughput with Competitive Parsing Accuracy**: HPD-Parsing achieves state-of-the-art inference efficiency on OmniDocBench v1.6, reaching a peak throughput of 4,752 Tokens Per Second (TPS). It delivers $1.62\times$ the throughput of the fastest existing document parsing model and more than $3.06\times$ that of its autoregressive baseline, while maintaining competitive parsing accuracy. These results demonstrate that document parsing can be effectively executed through global layout coordination and localized parallel decoding rather than a single sequential generation trajectory.
|
| 75 |
|
| 76 |
|
| 77 |
### **HPD-Parsing Architecture**
|
|
|
|
| 83 |
</div>
|
| 84 |
|
| 85 |
|
| 86 |
+
## Inference with vLLM
|
|
|
|
|
|
|
| 87 |
|
| 88 |
HPD-Parsing runs on a **customized build of vLLM** (based on vLLM v0.17.1) that implements the dynamic request forking required by hierarchical parallel decoding and adapts P-MTP speculative decoding.
|
| 89 |
|
|
|
|
| 204 |
|
| 205 |
> Reference implementation only. Use the vLLM path for production throughput — its paged KV cache enables true concurrent branches with zero-copy prefix sharing.
|
| 206 |
|
| 207 |
+
## Performance
|
| 208 |
+
|
| 209 |
+
#### 1. OmniDocBench v1.6 — Accuracy
|
| 210 |
+
|
| 211 |
+
With only 1B parameters, HPD-Parsing establishes a new state of the art among end-to-end **unified** parsers. Best results in each column are in **bold**.
|
| 212 |
+
|
| 213 |
+
<div align="center">
|
| 214 |
+
<img src="https://raw.githubusercontent.com/weishu20/HPD-Parsing-images/master/images/Accuracy.png" width="800"/>
|
| 215 |
+
</div>
|
| 216 |
+
|
| 217 |
+
|
| 218 |
+
|
| 219 |
+
#### 2. OmniDocBench v1.6 — Efficiency
|
| 220 |
+
Throughput under batch size 512 on NVIDIA A800 80GB with vLLM, HPD-Parsing increases throughput from 1.02 to 2.68 PPS and from 1,554.8 to 4,752.1 TPS, corresponding to improvements $2.62\times$ and $3.06\times$, respectively. Despite processing approximately 4,800 input tokens per page, over four times that of DeepSeek-OCR-2, HPD-Parsing still achieves $1.31\times$ higher PPS and $1.62\times$ higher TPS, demonstrating strong inference efficiency under a considerably larger input-token budget.Its acceleration advantage grows with document length, reaching up to $18.04\times$ fewer decoding steps, $3.67\times$ higher request throughput, and $5.80\times$ lower single-request latency in the longest output-length bucket.
|
| 221 |
+
|
| 222 |
+
<div align="center">
|
| 223 |
+
<img src="https://raw.githubusercontent.com/weishu20/HPD-Parsing-images/master/images/Efficiency.png" width="800"/>
|
| 224 |
+
</div>
|
| 225 |
+
|
| 226 |
+
|
| 227 |
## Evaluation & Benchmark
|
| 228 |
|
| 229 |
Scripts under [`eval/`](./eval) reproduce both the throughput (TPS) and the OmniDocBench v1.6 accuracy numbers via a decoupled **infer -> convert -> evaluate** pipeline:
|