README.md

metadata

license: mit
language:
  - zh
  - en
tags:
  - ncnn
  - ocr
  - tencent
  - hunyuan
  - vision-transformer
pipeline_tag: image-to-text

HunYuanOCR ncnn Models

Pre-converted ncnn model files for Tencent HunYuanOCR (1B OCR expert model).
0 custom layers, dynamic resolution, KV cache support.

📖 Source code & usage: infinity955/ncnn_ocr

---

hunyuanocr/ — Model Files

File	Size	Description
vision.ncnn.param	24 KB	ViT graph (27 layers, 1152-dim, patch=16, spatial_merge=2)
vision.ncnn.bin	1.75 GB	ViT weights
text_embed.ncnn.param	165 B	Text embedding lookup
text_embed.ncnn.bin	495 MB	Text + LM Head tied weights (120818 vocab × 1024-dim)
decoder.ncnn.param	37 KB	24-layer decoder with KV cache (GQA 16/8, XD-RoPE)
decoder.ncnn.bin	1.66 GB	Decoder weights
lm_head.ncnn.param	179 B	LM Head graph (shares weights with text_embed.bin)
pos_embed.bin	75.5 MB	Position embedding weights for ViT

Architecture

Image → ViT (vision.ncnn) → image_embeds [Lv, 1024]
Text  → BBPE → Text Embed (text_embed.ncnn) → text_embeds [Lt, 1024]
       → Inject image_embeds at vision token positions
       → Decoder (decoder.ncnn, 24 layers, KV cache) → hidden_states
       → LM Head (lm_head.ncnn) → logits → greedy sample → EOS

Input / Output

Module	Input	Output
Vision Encoder	[1,3,H,W] + [1,1152,gh,gw]	[Lv, 1024]
Text Embed	[1,N] int64 token IDs	[N, 1024]
Decoder	embeds + mask + XD-RoPE cos/sin	[N, 1024]
LM Head	[1, 1024]	[1, 120818]

Download

# Clone entire repo (all models)
huggingface-cli download infinity955/ncnn_ocr --local-dir ./models

# Or download specific files
huggingface-cli download infinity955/ncnn_ocr hunyuanocr/vision.ncnn.bin hunyuanocr/vision.ncnn.param

Usage

Place under models/ and run with ncnn_ocr:

./hunyuanocr_ncnn --model . --image test.jpg
# Output: 顺利上岸(231,392),(764,632)LAMAR(446,850),(559,878)05483(934,835),(997,870)

Conversion Notes

• Converted from PyTorch via torch.jit.script → pnnx → ncnn
• Decoder has KV cache baked in (SDPA 7=1 flag)
• LM Head weights tied with Text Embed (shared .bin)
• 100% text match vs HuggingFace PyTorch inference

References

• Tencent/ncnn — neural network inference framework
• futz12/ncnn_llm — reference GLM-OCR implementation
• Tencent/HunyuanOCR — original PyTorch model