Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,86 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- zh
|
| 5 |
+
- en
|
| 6 |
+
tags:
|
| 7 |
+
- ncnn
|
| 8 |
+
- ocr
|
| 9 |
+
- tencent
|
| 10 |
+
- hunyuan
|
| 11 |
+
- vision-transformer
|
| 12 |
+
pipeline_tag: image-to-text
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
# HunYuanOCR ncnn Models
|
| 16 |
+
|
| 17 |
+
Pre-converted ncnn model files for [Tencent HunYuanOCR](https://huggingface.co/tencent/HunyuanOCR) (1B OCR expert model).
|
| 18 |
+
**0 custom layers**, dynamic resolution, KV cache support.
|
| 19 |
+
|
| 20 |
+
π Source code & usage: [infinity955/ncnn_ocr](https://github.com/infinity955/ncnn_ocr)
|
| 21 |
+
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
## hunyuanocr/ β Model Files
|
| 25 |
+
|
| 26 |
+
| File | Size | Description |
|
| 27 |
+
|------|------|-------------|
|
| 28 |
+
| `vision.ncnn.param` | 24 KB | ViT graph (27 layers, 1152-dim, patch=16, spatial_merge=2) |
|
| 29 |
+
| `vision.ncnn.bin` | 1.75 GB | ViT weights |
|
| 30 |
+
| `text_embed.ncnn.param` | 165 B | Text embedding lookup |
|
| 31 |
+
| `text_embed.ncnn.bin` | 495 MB | Text + LM Head tied weights (120818 vocab Γ 1024-dim) |
|
| 32 |
+
| `decoder.ncnn.param` | 37 KB | 24-layer decoder with KV cache (GQA 16/8, XD-RoPE) |
|
| 33 |
+
| `decoder.ncnn.bin` | 1.66 GB | Decoder weights |
|
| 34 |
+
| `lm_head.ncnn.param` | 179 B | LM Head graph (shares weights with text_embed.bin) |
|
| 35 |
+
| `pos_embed.bin` | 75.5 MB | Position embedding weights for ViT |
|
| 36 |
+
|
| 37 |
+
### Architecture
|
| 38 |
+
|
| 39 |
+
```
|
| 40 |
+
Image β ViT (vision.ncnn) β image_embeds [Lv, 1024]
|
| 41 |
+
Text β BBPE β Text Embed (text_embed.ncnn) β text_embeds [Lt, 1024]
|
| 42 |
+
β Inject image_embeds at vision token positions
|
| 43 |
+
β Decoder (decoder.ncnn, 24 layers, KV cache) β hidden_states
|
| 44 |
+
β LM Head (lm_head.ncnn) β logits β greedy sample β EOS
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
### Input / Output
|
| 48 |
+
|
| 49 |
+
| Module | Input | Output |
|
| 50 |
+
|--------|-------|--------|
|
| 51 |
+
| Vision Encoder | `[1,3,H,W]` + `[1,1152,gh,gw]` | `[Lv, 1024]` |
|
| 52 |
+
| Text Embed | `[1,N]` int64 token IDs | `[N, 1024]` |
|
| 53 |
+
| Decoder | embeds + mask + XD-RoPE cos/sin | `[N, 1024]` |
|
| 54 |
+
| LM Head | `[1, 1024]` | `[1, 120818]` |
|
| 55 |
+
|
| 56 |
+
### Download
|
| 57 |
+
|
| 58 |
+
```bash
|
| 59 |
+
# Clone entire repo (all models)
|
| 60 |
+
huggingface-cli download infinity955/ncnn_ocr --local-dir ./models
|
| 61 |
+
|
| 62 |
+
# Or download specific files
|
| 63 |
+
huggingface-cli download infinity955/ncnn_ocr hunyuanocr/vision.ncnn.bin hunyuanocr/vision.ncnn.param
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
### Usage
|
| 67 |
+
|
| 68 |
+
Place under `models/` and run with [ncnn_ocr](https://github.com/infinity955/ncnn_ocr):
|
| 69 |
+
|
| 70 |
+
```bash
|
| 71 |
+
./hunyuanocr_ncnn --model . --image test.jpg
|
| 72 |
+
# Output: ι‘Ίε©δΈε²Έ(231,392),(764,632)LAMAR(446,850),(559,878)05483(934,835),(997,870)
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
### Conversion Notes
|
| 76 |
+
|
| 77 |
+
- Converted from PyTorch via `torch.jit.script β pnnx β ncnn`
|
| 78 |
+
- Decoder has KV cache baked in (SDPA `7=1` flag)
|
| 79 |
+
- LM Head weights tied with Text Embed (shared `.bin`)
|
| 80 |
+
- 100% text match vs HuggingFace PyTorch inference
|
| 81 |
+
|
| 82 |
+
### References
|
| 83 |
+
|
| 84 |
+
- [Tencent/ncnn](https://github.com/Tencent/ncnn) β neural network inference framework
|
| 85 |
+
- [futz12/ncnn_llm](https://github.com/futz12/ncnn_llm) β reference GLM-OCR implementation
|
| 86 |
+
- [Tencent/HunyuanOCR](https://huggingface.co/tencent/HunyuanOCR) β original PyTorch model
|