jasonni2 commited on
Commit
053cc2a
Β·
verified Β·
1 Parent(s): 1e1b9bd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +152 -0
README.md CHANGED
@@ -1,3 +1,155 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ - zh
6
+ - ja
7
+ - ko
8
+ pipeline_tag: image-to-text
9
+ tags:
10
+ - mlx
11
+ - mlx-weights
12
+ - paddlepaddle-ocr
13
+ - ppocrv5
14
+ - ppocrv6
15
+ - ppdoclayoutv3
16
+ - pp-structure
17
+ - apple-silicon
18
  ---
19
+
20
+ # PP-Structure / PP-OCR Models β€” MLX
21
+
22
+ This repository is an **[MLX](https://github.com/ml-explore/mlx)** conversion of the
23
+ PaddlePaddle **PP-Structure** and **PP-OCR** model families. Every subdirectory holds a
24
+ converted MLX weights file (`model.mlx.safetensors`) alongside the original source weights
25
+ and configs, so the models can run natively and efficiently on Apple Silicon (M-series).
26
+
27
+ The models are converted from the official PaddlePaddle / Hugging Face
28
+ [`transformers`](https://github.com/huggingface/transformers) safetensors checkpoints. They
29
+ cover the full document-intelligence pipeline: layout analysis, text detection &
30
+ recognition, orientation & rectification, table recognition, and formula recognition.
31
+
32
+ > See each subdirectory's own `README.md` for model-specific details, accuracy metrics, and
33
+ > the original PaddlePaddle usage examples.
34
+
35
+ ## Repository layout
36
+
37
+ Models are grouped by pipeline stage. Below, `↳` links each folder to its source model.
38
+
39
+ ### Document layout analysis
40
+
41
+ | Folder | Model | Description |
42
+ | --- | --- | --- |
43
+ | [`doclayoutv3/`](./doclayoutv3) | PP-DocLayoutV3 | RT-DETR-style detector (HGNetV2-L backbone) for 25 document layout regions (title, text, figure, table, formula, …). |
44
+
45
+ ### Text detection (PP-OCRv5 / v6)
46
+
47
+ | Folder | Model | Description |
48
+ | --- | --- | --- |
49
+ | [`det/`](./det) | PP-OCRv5_mobile_det | Legacy mobile text-line detector (LCNetV3 backbone, scale 0.75). |
50
+ | [`det_v6_medium/`](./det_v6_medium) | PP-OCRv6_medium_det | Largest v6 detector β€” LCNetV4 backbone + RepLKFPN neck, 15.5M params. |
51
+ | [`det_v6_small/`](./det_v6_small) | PP-OCRv6_small_det | Mid-tier v6 detector, 2.48M params. |
52
+ | [`det_v6_tiny/`](./det_v6_tiny) | PP-OCRv6_tiny_det | Smallest v6 detector, 0.43M params. |
53
+
54
+ ### Text recognition (PP-OCRv5 / v6)
55
+
56
+ | Folder | Model | Description |
57
+ | --- | --- | --- |
58
+ | [`rec/`](./rec) | PP-OCRv5_mobile_rec | Legacy mobile recognizer (LCNetV3 backbone). |
59
+ | [`en_rec/`](./en_rec) | PP-OCRv5_mobile_rec (EN) | English-dictionary variant of the mobile recognizer. |
60
+ | [`server_rec/`](./server_rec) | PP-OCRv5_server_rec | Server-grade recognizer for ZH/EN/JA + handwriting, vertical text, pinyin, rare characters. |
61
+ | [`rec_v6_medium/`](./rec_v6_medium) | PP-OCRv6_medium_rec | Largest v6 recognizer β€” LCNetV4 + EncoderWithLightSVTR, CTC+NRTR heads, 50 languages, 19M params. |
62
+ | [`rec_v6_small/`](./rec_v6_small) | PP-OCRv6_small_rec | Mid-tier v6 recognizer, 5.2M params, 50 languages. |
63
+ | [`rec_v6_tiny/`](./rec_v6_tiny) | PP-OCRv6_tiny_rec | Smallest v6 recognizer, 1.1M params, 49 languages. |
64
+
65
+ ### Orientation & rectification
66
+
67
+ | Folder | Model | Description |
68
+ | --- | --- | --- |
69
+ | [`ori/`](./ori) | PP-LCNet_x1_0_doc_ori | Document image orientation classifier (0Β°/90Β°/180Β°/270Β°), 99.06% avg accuracy. |
70
+ | [`uvdoc/`](./uvdoc) | UVDoc | Document image unwarping / geometric rectification (CER 0.179 on DocUNet benchmark). |
71
+
72
+ ### Table recognition
73
+
74
+ | Folder | Model | Description |
75
+ | --- | --- | --- |
76
+ | [`table_cls/`](./table_cls) | PP-LCNet_x1_0_table_cls | Wired vs. wireless table classifier, 94.2% Top-1. |
77
+ | [`table_structure/`](./table_structure) | SLANet | Legacy table-structure recognition (LCNet backbone, scale 1). |
78
+ | [`table_wired/`](./table_wired) | SLANeXt_wired | Wired-table structure recognition, 69.65% accuracy, 351M. |
79
+ | [`table_wireless/`](./table_wireless) | SLANeXt_wireless | Wireless-table structure recognition, 69.65% accuracy, 351M. |
80
+ | [`table_cell_wired/`](./table_cell_wired) | RT-DETR-L_wired_table_cell_det | Wired-table cell detector (RT-DETR-L), 82.7% Top-1, 124M. |
81
+ | [`table_cell_wireless/`](./table_cell_wireless) | RT-DETR-L_wireless_table_cell_det | Wireless-table cell detector (RT-DETR-L), 82.7% Top-1, 124M. |
82
+
83
+ ### Formula recognition
84
+
85
+ | Folder | Model | Description |
86
+ | --- | --- | --- |
87
+ | [`formula/`](./formula) | PP-FormulaNet_plus-L | Encoder-decoder vision-language model that converts formula images to LaTeX (~182M params, 50k-token vocabulary). |
88
+
89
+ ## Pipeline
90
+
91
+ These modules compose into the standard PP-Structure document pipeline:
92
+
93
+ ```
94
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
95
+ page image β†’ β”‚ doc ori β”‚ (optional) orient the page
96
+ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
97
+ β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”
98
+ β”‚ uvdoc β”‚ (optional) dewarp the page
99
+ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
100
+ β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”
101
+ β”‚ doclayoutv3β”‚ detect layout regions
102
+ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
103
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
104
+ β–Ό β–Ό β–Ό
105
+ text branch table formula
106
+ β”Œβ”€οΏ½οΏ½οΏ½β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”
107
+ β”‚ det β”‚ β”‚ cls β”‚ β”‚formulaβ”‚
108
+ β””β”€β”€β”¬β”€β”€β”€β”˜ β””β”€β”€β”€β”¬β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”˜
109
+ β”‚ β”Œβ”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”
110
+ β–Ό β–Ό β–Ό
111
+ β”Œβ”€β”€β”€β”€β” cell det structure
112
+ β”‚rec β”‚ (wired/ (wired/
113
+ β””β”€β”€β”€β”€β”˜ wireless) wireless)
114
+ ```
115
+
116
+ For the OCR sub-pipeline, PP-OCRv6 pairs `det_v6_*` with the matching `rec_v6_*` tier
117
+ (e.g. `det_v6_medium` + `rec_v6_medium`), selectable across medium / small / tiny for
118
+ server-to-edge trade-offs.
119
+
120
+ ## Loading the MLX weights
121
+
122
+ Each folder follows the same convention β€” the MLX weights live in `model.mlx.safetensors`
123
+ and the architecture in `config.json`:
124
+
125
+ ```
126
+ <model>/
127
+ β”œβ”€β”€ model.mlx.safetensors # MLX-converted weights (load with mlx.nn / mlx-vlm)
128
+ β”œβ”€β”€ model.safetensors # original source weights
129
+ β”œβ”€β”€ config.json # architecture config
130
+ └── preprocessor_config.json (or processor_config.json)
131
+ ```
132
+
133
+ Load with MLX (Python):
134
+
135
+ ```python
136
+ import mlx.core as mx
137
+ from mlx.utils import tree_unflatten
138
+
139
+ weights = mx.load("det_v6_medium/model.mlx.safetensors")
140
+ params = tree_unflatten(list(weights.items()))
141
+ ```
142
+
143
+ > These are weight conversions only. A matching MLX model implementation (e.g. via
144
+ > [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) or a custom MLX module) is required to run
145
+ > inference. Refer to each subdirectory's `config.json` for the exact architecture.
146
+
147
+ ## Model sources
148
+
149
+ Original checkpoints and documentation from the
150
+ [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) project and the
151
+ [PaddlePaddle](https://huggingface.co/PaddlePaddle) Hugging Face organization.
152
+
153
+ ## License
154
+
155
+ Apache 2.0. See the [LICENSE](./LICENSE) of the upstream PaddleOCR project for details.