TongGuOCR

A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents.

Paper | Code and installation | Project page | Online demo | Hugging Face | ModelScope International

This repository contains the final Stage 3 OCR checkpoint, its matching processor and tokenizer, and the text-line detection checkpoint. The weights match the verified release uploaded to ModelScope International.

Model and Files

The OCR architecture is GlmOcrForConditionalGeneration, adapted from GLM-OCR with an expanded 84,083-token vocabulary. This is the final Stage 3 inference checkpoint. The full-page pipeline includes layout detection, block construction, crop refinement, and relation-token processing in the code repository.

Path	Purpose
model.safetensors	Final OCR checkpoint, 4,733,086,904 bytes
config.json, generation_config.json	Model and generation configuration
processor_config.json	Image/text processor configuration
tokenizer.json, tokenizer_config.json, chat_template.jinja	Matching tokenizer and prompt template
layout/finetune_infer.pth	Text-line detection checkpoint, 122,471,457 bytes
SHA256SUMS	SHA256 checksums of the packaged files

Download and Run

Install the inference environment following the code repository's README. Run the following inside the updated TongGuOCR code checkout. While a model repository is private, set HF_TOKEN or MODELSCOPE_API_TOKEN to a token with read access for the respective platform.

bash tools/download_weights.sh huggingface

If Hugging Face is unreachable, use ModelScope International:

python -m pip install modelscope
bash tools/download_weights.sh modelscope

The ModelScope repository is hosted on modelscope.ai. Both download paths install OCR files in weights/ocr/ and the detector checkpoint at weights/layout/finetune_infer.pth.

python -m tongguocr -i /path/to/page.jpg -o output/ --print-result

Loading the OCR checkpoint alone does not reproduce full-page pipeline results. The validated default uses bf16, greedy decoding, at most 8,192 output tokens, and an image budget of 1,048,576 pixels per recognition crop. See the code repository for environment setup and the optional vLLM backend.

Evaluation

Paper results:

Dataset	AR	CR	NED	RO-ED x100
MTHv2	97.93	98.33	2.05	1.51
M5HisDoc	93.76	94.46	6.15	3.49

Fresh inference can differ slightly by backend and batch settings. The code repository's evaluation/README.md records fresh results and the metric protocol. Training and evaluation datasets are not included in this model repository.

License and Attribution

TongGuOCR's own OCR adaptations, detection checkpoint and documentation use Apache-2.0, as selected by the model owner. The upstream GLM-OCR model retains MIT terms. See the GLM-OCR license and recorded source.

The HDC text-line detection checkpoint at layout/finetune_infer.pth is distributed under Apache-2.0 as selected by the model owner. This grant covers rights controlled by the TongGuOCR contributors; applicable third-party terms remain in force. See THIRD_PARTY_NOTICES.md.

Citation

@article{zhou2026tongguocr,
  title = {TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents},
  author = {Zhou, Zhongheng and Sun, Yi and He, Huiguo and Zhang, Yuyi and Zhang, Peirong and Fang, Yulin and Peng, Dezhi and Liao, Minghui and Jin, Lianwen},
  journal = {arXiv preprint arXiv:2608.07917},
  year = {2026}
}