Instructions to use VietAlphaLabs/SenOCR-Zh with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use VietAlphaLabs/SenOCR-Zh with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="VietAlphaLabs/SenOCR-Zh", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("VietAlphaLabs/SenOCR-Zh", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("VietAlphaLabs/SenOCR-Zh", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use VietAlphaLabs/SenOCR-Zh with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "VietAlphaLabs/SenOCR-Zh"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VietAlphaLabs/SenOCR-Zh",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/VietAlphaLabs/SenOCR-Zh

SGLang

How to use VietAlphaLabs/SenOCR-Zh with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "VietAlphaLabs/SenOCR-Zh" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VietAlphaLabs/SenOCR-Zh",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "VietAlphaLabs/SenOCR-Zh" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VietAlphaLabs/SenOCR-Zh",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use VietAlphaLabs/SenOCR-Zh with Docker Model Runner:
```
docker model run hf.co/VietAlphaLabs/SenOCR-Zh
```

Research page · VietAlpha Lab

SenOCR-Zh is a Chinese-specialized document OCR model built on PaddleOCR-VL-1.6.

The model retains the approximately 0.96B-parameter PaddleOCR-VL-1.6 architecture while adapting the language decoder for Chinese text, tables, formulas, and historical CJK material. On our controlled 160-page Chinese/ZH evaluation group, SenOCR-Zh reaches an 85.19 Chinese document composite.

Fine-tuning updates 12.09M parameters, about 1.26% of the model, using decoder-only LoRA on a single NVIDIA A10G.

SenOCR-Zh is intended for Chinese document parsing, historical Chinese material, mixed text-table-formula pages, photographed documents, and Hán-Nôm digitization workflows.

Highlights

Chinese document parsing: 85.19 Chinese composite on the controlled 160-page ZH group, supported across 84.77 text (1 - NED), 81.57 table (TEDS) and 79.47 formula (CDM) rather than by a single task.
Strong on clean pages: 92.69 Chinese digital-document composite, against 82.69 on photographed documents.
Small and self-contained: 0.959B total parameters in a single merged checkpoint — no separate adapter branch is needed at inference.
Cheap to reproduce: decoder-only LoRA rank 32 / alpha 64 over 12.09M trainable parameters (~1.26% of the model), trained on 1 x NVIDIA A10G.
Multilingual context: 83.38 Vietnamese + English + Chinese composite, though the three-language score is supporting context rather than the optimization target.
Drop-in interface: identical to PaddleOCR-VL-1.6 — usable through transformers for element-level recognition, or as the recognition model inside the PaddleOCR-VL 1.6 page-parsing pipeline.

Inference examples

SenOCR-Zh follows the PaddleOCR-VL-1.6 model interface.

Transformers

For direct element recognition:

pip install "transformers>=5.0.0" torch pillow

from PIL import Image
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "VietAlphaLabs/SenOCR-Zh"
image_path = "document.png"
task = "ocr"  # "ocr" | "table" | "formula"

prompts = {
    "ocr": "OCR:",
    "table": "Table Recognition:",
    "formula": "Formula Recognition:",
}

device = "cuda" if torch.cuda.is_available() else "cpu"

model = (
    AutoModelForImageTextToText
    .from_pretrained(model_id, torch_dtype=torch.bfloat16)
    .to(device)
    .eval()
)
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open(image_path).convert("RGB")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": prompts[task]},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)

outputs = model.generate(**inputs, max_new_tokens=512)

text = processor.decode(
    outputs[0][inputs["input_ids"].shape[-1]:-1]
)

print(text)

PaddleOCR

For page-level document parsing, SenOCR-Zh can be used as the multimodal recognition model inside the PaddleOCR-VL 1.6 pipeline.

pip install -U "paddleocr[doc-parser]>=3.6.0" huggingface_hub

from pathlib import Path
from huggingface_hub import snapshot_download
from paddleocr import PaddleOCRVL

model_dir = snapshot_download(repo_id="VietAlphaLabs/SenOCR-Zh")

output_dir = Path("./output")
output_dir.mkdir(parents=True, exist_ok=True)

pipeline = PaddleOCRVL(
    pipeline_version="v1.6",
    vl_rec_model_dir=model_dir,
)

output = pipeline.predict("document.png")

for result in output:
    result.print()
    result.save_to_json(save_path=output_dir)
    result.save_to_markdown(save_path=output_dir)

See the PaddleOCR-VL documentation for PDF input, document restructuring, and optimized serving backends.

Download the model

hf download VietAlphaLabs/SenOCR-Zh --local-dir SenOCR-Zh/

Evaluation

SenOCR-Zh is evaluated on a controlled 480-page MDPBench slice covering Vietnamese, English, and Chinese, with 160 pages per language.

SenOCR-Zh has not yet been officially verified on the MDPBench leaderboard. The SenOCR-Zh scores below come from our project-controlled evaluation. External comparison scores come from the official MDPBench leaderboard.

Chinese

Model	Type	Chinese / ZH score
SenOCR-Zh	Specialized OCR, 0.959B	85.19*
Gemini-3-pro-preview	General VLM	85.10
PaddleOCR-VL-1.5	Specialized OCR	84.80
dots.mocr	Specialized OCR	84.60
MonkeyOCRv2-S-Parsing	Specialized OCR	84.40

* Project-controlled result; official MDPBench verification is pending.

Within this selected comparison, SenOCR-Zh is 0.09 points above Gemini-3-pro-preview, 0.39 above PaddleOCR-VL-1.5, 0.59 above dots.mocr, and 0.79 above MonkeyOCRv2-S-Parsing on Chinese/ZH.

Components

Chinese capability	Score	Evaluation support
Composite	85.19	160 pages
Text, `1 - NED`	84.77	156 pages / 3,016 instances
Table, TEDS	81.57	68 pages / 128 instances
Formula, CDM	79.47	32 pages / 88 instances

The 85.19 composite is supported across text, tables, and formulas rather than by a single task.

Document conditions

Chinese metric	Digital	Photographed
Composite	92.69	82.69
Text, `1 - NED`	91.19	82.64
Table, TEDS	98.72	75.86
Formula, CDM	95.45	74.24

Photographed pages are materially harder than clean digital pages, especially for tables and formulas.

Multilingual

The controlled Vietnamese + English + Chinese composite is 83.38.

This three-language score is supporting context rather than the primary optimization target. SenOCR-Zh is specialized for Chinese document parsing.

Training

SenOCR-Zh keeps the PaddleOCR-VL-1.6 architecture intact and concentrates adaptation in the language decoder.

Setting	Value
Base model	PaddlePaddle/PaddleOCR-VL-1.6
Total parameters	958.6M
Trainable parameters	12.09M
Trainable share	~1.26%
Decoder layers	18
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.0
Adapted projections	126
Peak learning rate	1e-4
Minimum learning rate	1e-5
Warmup	3%
Weight decay	0.01
Gradient clipping	1.0
Microbatch	8
Gradient accumulation	8
Effective batch size	64
Precision	BF16
Maximum sequence length	4,096
Training steps	2,367
Training passes	~3.0
Training GPU	1 x NVIDIA A10G

LoRA is applied to seven projections in each of the 18 decoder layers:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

The vision encoder, vision-language aligner, embeddings, and output head remain frozen.

Training interleaves three task prompts:

<image>OCR:
<image>Table Recognition:
<image>Formula Recognition:

Data

SenOCR-Zh is trained on a 50,451-record Chinese OCR corpus built from open, attributable sources.

Corpus lane	Records	Share
Chinese modern historical archives	18,000	35.7%
Broad Chinese OCR regions	14,000	27.7%
Hán-Nôm and Literary Chinese	12,000	23.8%
Chinese table synthesis	3,000	5.9%
Formula recognition	3,451	6.8%
Total	50,451	100%

The lanes draw on:

JIA244601/Handwritten-Historical-Archive-Image-Dataset-of-Modern-China_1840_1949 for modern Chinese historical material;
selected Chinese-oriented data from Yesianrohn/OCR-Data;
Aerbote88/nom-ocr-data for Hán-Nôm, rare characters, vertical reading order, marginalia, and IDS-preserved forms;
license-cleared Chinese text for synthetic table generation;
lukbl/LaTeX-OCR-dataset for formula recognition.

The data pipeline applies source-overlap checks, duplicate-page filtering, sequence-length audits, target-token analysis, annotation review, and manuscript-level train/test separation.

Dataset licenses vary by source and each retains its original terms.

Historical OCR

SenOCR-Zh is designed for historical Chinese and Hán-Nôm document workflows where OCR must recover both transcription and document structure.

Target material includes:

vertical Traditional Chinese;
historical letters and registers;
catalogs, yearbooks, and textbooks;
woodblock and degraded printed pages;
rare CJK characters;
marginalia and handwritten corrections;
Hán-Nôm documents containing Nôm-specific or unencoded forms;
historical mathematics and scientific material combining prose, tables, and formulas.

Historical evaluation is not yet complete. Planned benchmarks include AncientDoc, ChineseOCRBench, and a manuscript-level held-out Hán-Nôm evaluation.

Limitations

The main measured weakness is the performance gap between digital and photographed pages.

Other difficult cases include:

severe blur, skew, perspective distortion, or uneven illumination;
damaged or ambiguous rare characters;
bleed-through and faded ink;
handwritten corrections and marginalia;
complex reading order;
unencoded Nôm forms;
formula-heavy or structurally dense pages.

For scholarly, legal, historical, or otherwise high-stakes transcription, human review is recommended.

License

SenOCR-Zh is released under the Apache License 2.0.

The model is built on PaddleOCR-VL-1.6, which is also distributed under Apache 2.0.

Dataset licenses vary by source. The training source mix includes datasets under MIT and CC-BY-4.0 terms, and each source retains its original licensing terms.

Citation

@misc{vietalphalab2026senocrzh,
      title={SenOCR-Zh: A Chinese-Specialized Document OCR Model},
      author={VietAlpha Lab},
      year={2026},
      publisher={Hugging Face},
      url={https://huggingface.co/VietAlphaLabs/SenOCR-Zh},
}

References

Downloads last month: 6

Safetensors

Model size

1.0B params

Tensor type

F32

Model tree for VietAlphaLabs/SenOCR-Zh

Base model

PaddlePaddle/PaddleOCR-VL-1.6

Finetuned

(10)

this model

VietAlphaLabs
/

SenOCR-Zh