README.md

---
license: mit
language:
- zh
- en
tags:
- ncnn
- ocr
- tencent
- hunyuan
- vision-transformer
pipeline_tag: image-to-text
---

# HunYuanOCR ncnn Models

Pre-converted ncnn model files for [Tencent HunYuanOCR](https://huggingface.co/tencent/HunyuanOCR) (1B OCR expert model).  
**0 custom layers**, dynamic resolution, KV cache support.

📖 Source code & usage: [infinity955/ncnn_ocr](https://github.com/infinity955/ncnn_ocr)

---

## hunyuanocr/ — Model Files

| File | Size | Description |
|------|------|-------------|
| `vision.ncnn.param` | 24 KB | ViT graph (27 layers, 1152-dim, patch=16, spatial_merge=2) |
| `vision.ncnn.bin` | 1.75 GB | ViT weights |
| `text_embed.ncnn.param` | 165 B | Text embedding lookup |
| `text_embed.ncnn.bin` | 495 MB | Text + LM Head tied weights (120818 vocab × 1024-dim) |
| `decoder.ncnn.param` | 37 KB | 24-layer decoder with KV cache (GQA 16/8, XD-RoPE) |
| `decoder.ncnn.bin` | 1.66 GB | Decoder weights |
| `lm_head.ncnn.param` | 179 B | LM Head graph (shares weights with text_embed.bin) |
| `pos_embed.bin` | 75.5 MB | Position embedding weights for ViT |

### Architecture

```
Image → ViT (vision.ncnn) → image_embeds [Lv, 1024]
Text  → BBPE → Text Embed (text_embed.ncnn) → text_embeds [Lt, 1024]
       → Inject image_embeds at vision token positions
       → Decoder (decoder.ncnn, 24 layers, KV cache) → hidden_states
       → LM Head (lm_head.ncnn) → logits → greedy sample → EOS
```

### Input / Output

| Module | Input | Output |
|--------|-------|--------|
| Vision Encoder | `[1,3,H,W]` + `[1,1152,gh,gw]` | `[Lv, 1024]` |
| Text Embed | `[1,N]` int64 token IDs | `[N, 1024]` |
| Decoder | embeds + mask + XD-RoPE cos/sin | `[N, 1024]` |
| LM Head | `[1, 1024]` | `[1, 120818]` |

### Download

```bash
# Clone entire repo (all models)
huggingface-cli download infinity955/ncnn_ocr --local-dir ./models

# Or download specific files
huggingface-cli download infinity955/ncnn_ocr hunyuanocr/vision.ncnn.bin hunyuanocr/vision.ncnn.param
```

### Usage

Place under `models/` and run with [ncnn_ocr](https://github.com/infinity955/ncnn_ocr):

```bash
./hunyuanocr_ncnn --model . --image test.jpg
# Output: 顺利上岸(231,392),(764,632)LAMAR(446,850),(559,878)05483(934,835),(997,870)
```

### Conversion Notes

- Converted from PyTorch via `torch.jit.script → pnnx → ncnn`
- Decoder has KV cache baked in (SDPA `7=1` flag)
- LM Head weights tied with Text Embed (shared `.bin`)
- 100% text match vs HuggingFace PyTorch inference

### References

- [Tencent/ncnn](https://github.com/Tencent/ncnn) — neural network inference framework
- [futz12/ncnn_llm](https://github.com/futz12/ncnn_llm) — reference GLM-OCR implementation
- [Tencent/HunyuanOCR](https://huggingface.co/tencent/HunyuanOCR) — original PyTorch model