infinity955 commited on
Commit
0dfecca
Β·
verified Β·
1 Parent(s): fb335c4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +83 -0
README.md CHANGED
@@ -1,3 +1,86 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - zh
5
+ - en
6
+ tags:
7
+ - ncnn
8
+ - ocr
9
+ - tencent
10
+ - hunyuan
11
+ - vision-transformer
12
+ pipeline_tag: image-to-text
13
  ---
14
+
15
+ # HunYuanOCR ncnn Models
16
+
17
+ Pre-converted ncnn model files for [Tencent HunYuanOCR](https://huggingface.co/tencent/HunyuanOCR) (1B OCR expert model).
18
+ **0 custom layers**, dynamic resolution, KV cache support.
19
+
20
+ πŸ“– Source code & usage: [infinity955/ncnn_ocr](https://github.com/infinity955/ncnn_ocr)
21
+
22
+ ---
23
+
24
+ ## hunyuanocr/ β€” Model Files
25
+
26
+ | File | Size | Description |
27
+ |------|------|-------------|
28
+ | `vision.ncnn.param` | 24 KB | ViT graph (27 layers, 1152-dim, patch=16, spatial_merge=2) |
29
+ | `vision.ncnn.bin` | 1.75 GB | ViT weights |
30
+ | `text_embed.ncnn.param` | 165 B | Text embedding lookup |
31
+ | `text_embed.ncnn.bin` | 495 MB | Text + LM Head tied weights (120818 vocab Γ— 1024-dim) |
32
+ | `decoder.ncnn.param` | 37 KB | 24-layer decoder with KV cache (GQA 16/8, XD-RoPE) |
33
+ | `decoder.ncnn.bin` | 1.66 GB | Decoder weights |
34
+ | `lm_head.ncnn.param` | 179 B | LM Head graph (shares weights with text_embed.bin) |
35
+ | `pos_embed.bin` | 75.5 MB | Position embedding weights for ViT |
36
+
37
+ ### Architecture
38
+
39
+ ```
40
+ Image β†’ ViT (vision.ncnn) β†’ image_embeds [Lv, 1024]
41
+ Text β†’ BBPE β†’ Text Embed (text_embed.ncnn) β†’ text_embeds [Lt, 1024]
42
+ β†’ Inject image_embeds at vision token positions
43
+ β†’ Decoder (decoder.ncnn, 24 layers, KV cache) β†’ hidden_states
44
+ β†’ LM Head (lm_head.ncnn) β†’ logits β†’ greedy sample β†’ EOS
45
+ ```
46
+
47
+ ### Input / Output
48
+
49
+ | Module | Input | Output |
50
+ |--------|-------|--------|
51
+ | Vision Encoder | `[1,3,H,W]` + `[1,1152,gh,gw]` | `[Lv, 1024]` |
52
+ | Text Embed | `[1,N]` int64 token IDs | `[N, 1024]` |
53
+ | Decoder | embeds + mask + XD-RoPE cos/sin | `[N, 1024]` |
54
+ | LM Head | `[1, 1024]` | `[1, 120818]` |
55
+
56
+ ### Download
57
+
58
+ ```bash
59
+ # Clone entire repo (all models)
60
+ huggingface-cli download infinity955/ncnn_ocr --local-dir ./models
61
+
62
+ # Or download specific files
63
+ huggingface-cli download infinity955/ncnn_ocr hunyuanocr/vision.ncnn.bin hunyuanocr/vision.ncnn.param
64
+ ```
65
+
66
+ ### Usage
67
+
68
+ Place under `models/` and run with [ncnn_ocr](https://github.com/infinity955/ncnn_ocr):
69
+
70
+ ```bash
71
+ ./hunyuanocr_ncnn --model . --image test.jpg
72
+ # Output: 鑺利上岸(231,392),(764,632)LAMAR(446,850),(559,878)05483(934,835),(997,870)
73
+ ```
74
+
75
+ ### Conversion Notes
76
+
77
+ - Converted from PyTorch via `torch.jit.script β†’ pnnx β†’ ncnn`
78
+ - Decoder has KV cache baked in (SDPA `7=1` flag)
79
+ - LM Head weights tied with Text Embed (shared `.bin`)
80
+ - 100% text match vs HuggingFace PyTorch inference
81
+
82
+ ### References
83
+
84
+ - [Tencent/ncnn](https://github.com/Tencent/ncnn) β€” neural network inference framework
85
+ - [futz12/ncnn_llm](https://github.com/futz12/ncnn_llm) β€” reference GLM-OCR implementation
86
+ - [Tencent/HunyuanOCR](https://huggingface.co/tencent/HunyuanOCR) β€” original PyTorch model