Hugging Face's logo

mlboydaisuke
/
MinerU2.5-Pro-CoreAI

coreai-aimodel
Model card Files
xet
 Community
mlboydaisuke commited on
Commit
d834e4f
·
verified ·
1 Parent(s): f8fd792

MinerU2.5-Pro -> Core AI: vision (fp16) + decoder (int8lin S=1) + tokenizer

Browse files
Files changed (15) hide show
  1. .gitattributes +4 -0
  2. README.md +92 -0
  3. decoder/metadata.json +28 -0
  4. decoder/mineru_decode_int8lin_s1.aimodel/main.hash +1 -0
  5. decoder/mineru_decode_int8lin_s1.aimodel/main.mlirb +3 -0
  6. decoder/mineru_decode_int8lin_s1.aimodel/metadata.json +3 -0
  7. decoder/tokenizer/chat_template.jinja +7 -0
  8. decoder/tokenizer/tokenizer.json +3 -0
  9. decoder/tokenizer/tokenizer_config.json +297 -0
  10. tokenizer/chat_template.jinja +7 -0
  11. tokenizer/tokenizer.json +3 -0
  12. tokenizer/tokenizer_config.json +297 -0
  13. vision/mineru_vision.aimodel/main.hash +1 -0
  14. vision/mineru_vision.aimodel/main.mlirb +3 -0
  15. vision/mineru_vision.aimodel/metadata.json +3 -0
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ decoder/mineru_decode_int8lin_s1.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
37
+ decoder/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
39
+ vision/mineru_vision.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,92 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MinerU2.5-Pro (1.2B) — Core AI whole-page document parsing
2
+
3
+ **On-device whole-page document structuring on Core AI.** A port of
4
+ [`opendatalab/MinerU2.5-Pro-2605-1.2B`](https://huggingface.co/opendatalab/MinerU2.5-Pro-2605-1.2B)
5
+ (**Apache-2.0**, 1.2B) — a small, SOTA-quality document parser (OmniDocBench v1.6 **95.69**). Unlike a
6
+ per-prompt recognizer, MinerU2.5 does the *whole page in one model*: a **layout** pass finds the
7
+ blocks (title / text / table / formula / figure) and their reading order, then a **recognition** pass
8
+ reads each region into plain text, HTML tables (`<table>…`), or LaTeX — assembled into structured
9
+ Markdown / JSON. This is the piece the zoo's per-region OCRs ([Unlimited-OCR](unlimited-ocr.md),
10
+ [GLM-OCR](glm-ocr.md)) leave to a separate layout detector; MinerU folds it into the same weights.
11
+
12
+ MinerU2.5 is **stock Qwen2-VL** (`Qwen2VLForConditionalGeneration`): a Qwen2-VL ViT vision tower + a
13
+ Qwen2-0.5B text decoder, no custom code. This port is the shipped [Qwen3-VL](qwen3-vl.md) /
14
+ [GLM-OCR](glm-ocr.md) vision idiom (the `image_embeds` + rope-shift static-input hook) with a Qwen2
15
+ text decode — **no deepstack, no MoE, no MLA**. The vision tower runs once; its `image_embeds` are
16
+ injected at the image-placeholder positions (`V + slot`, row-major over the merged grid) and the text
17
+ decodes on top. The host runs the model twice per page (layout prompt, then recognition prompts).
18
+
19
+ **⬇️ Converted `.aimodel` bundles:
20
+ [mlboydaisuke/MinerU2.5-Pro-CoreAI](https://huggingface.co/mlboydaisuke/MinerU2.5-Pro-CoreAI)** —
21
+ `vision/` (Qwen2-VL ViT, fp16) + `decoder/` (Qwen2 decoder, S=1 pipelined, int8lin) + `tokenizer/`.
22
+ Apache-2.0.
23
+
24
+ ## Architecture
25
+
26
+ - **Vision (Qwen2-VL ViT)**: embed 1280 / 32 L / 16 heads (head 80) / patch 14 / temporal 2 /
27
+ spatial-merge 2, out 896. **LayerNorm** blocks, fused qkv (bias), **no q/k-norm**, non-gated
28
+ `fc1 → quick_gelu → fc2` MLP, and the standard Qwen2-VL `PatchMerger`
29
+ (`ln_q → view(merge²) → Linear → GELU → Linear`). **No deepstack, no learned pos-embed** — just
30
+ baked 2D-rope constants. Exported as one fp16 `.aimodel`; `N` (visual tokens) is fixed by the
31
+ export grid.
32
+ - **Decoder (Qwen2-0.5B)**: hidden 896 / 24 L / GQA 14-2 / head_dim 64 / vocab 151936, `tie_word_embeddings`.
33
+ Separate q/k/v **with bias**, **no q/k-norm**, standard 2-norm block, silu SwiGLU, **sectioned
34
+ M-RoPE `[8,12,12]`** applied split-half (Qwen standard `rotate_half` — not GLM's interleave). Driven
35
+ on the pipelined-engine S=1 contract: `input_ids [1,1]`, `position_ids`, static `image_embeds [N,896]`
36
+ + `rope_shift_start` + `rope_shift_amount`. Zero embeds + `shift_start = 1<<30` → a plain Qwen2 text
37
+ decoder.
38
+
39
+ ## Verified (M4 Max, GPU, Core AI pipelined engine)
40
+
41
+ - **End-to-end real generation on the engine: the sample page read verbatim** — GPU vision `.aimodel`
42
+ → `image_embeds` → AOT-compiled (h16c) int8lin S=1 decoder, autoregressive greedy: title +
43
+ paragraph + the **full HTML-less table reconstructed row-by-row** (*"Quarterly Report / On-device
44
+ inference shipped across all product lines this quarter… / Whisper 809M 0.18 s/token / …"*),
45
+ matching the fp32 reference. **211.7 tok/s** decode (int8lin S=1, AOT h16c GPU).
46
+ - Torch ladder vs HF fp32: text-only + vision + full-VLM argmax **706/706 exact**, max logit diff
47
+ **0.0001**, the generation-driving token bit-identical.
48
+ - Engine gate (Mac GPU): vision `image_embeds` cos **1.0002**; AOT int8lin decoder teacher-forced over
49
+ 706 positions — **text region 24/24 exact**, generated token exact.
50
+ - **int8lin vs fp16: 13 / 706 argmax flips, all at visual-token positions (0 in the text region)** —
51
+ the OCR text is preserved. Greedy generation byte-identical to fp16.
52
+
53
+ ## Pipeline (host side, 2-stage whole-page)
54
+
55
+ ```
56
+ page image
57
+ ├─ layout stage: resize → vision .aimodel → image_embeds → decoder ["…<image>×N, "Layout Detection:"]
58
+ │ → parse boxes {type, bbox(0–1), angle} + reading order
59
+ └─ per region (in reading order):
60
+ crop(bbox) → vision .aimodel → image_embeds
61
+ → decoder [ …<image>×N, prompt ] (image ids → V+slot; shift_start=img_start+N,
62
+ shift_amount=N−max(gh,gw))
63
+ prompt by type: text → "Text Recognition:" · table → "Table Recognition:"
64
+ formula → "Formula Recognition:" · figure → "Image Analysis:"
65
+ → assemble content_list (type + text) in reading order → json2md → Markdown / JSON
66
+ ```
67
+
68
+ Prompt/box formats follow [`opendatalab/mineru-vl-utils`](https://github.com/opendatalab/mineru-vl-utils)
69
+ (`MinerUClient.two_step_extract` + `post_process.json2md`).
70
+
71
+ ## Use / reproduce
72
+
73
+ - **Convert**: [`conversion/export_mineru_pipelined.py`](../conversion/export_mineru_pipelined.py)
74
+ (`fp16` / `int8lin` / `int8hu`; vision stays fp16).
75
+ - **Run (Mac)**: drive the S=1 decoder bundle on the pipelined engine with three static inputs
76
+ (`image_embeds` + `rope_shift_start` + `rope_shift_amount`) and `COREAI_CHUNK_THRESHOLD=1`; feed the
77
+ prompt with the image placeholders rewritten to `V+slot`. Large decode graphs need AOT on macOS 27
78
+ (`xcrun coreai-build compile … --architecture h16c --preferred-compute gpu --expect-frequent-reshapes`);
79
+ h18p bundles are prepared for iPhone.
80
+ - **Knowledge**: [`knowledge/mineru-port.md`](../knowledge/mineru-port.md).
81
+
82
+ ## Notes
83
+
84
+ - **Whole-page structuring is in the model** (layout + per-region recognition, `json2md` reading-order
85
+ assembly) — the value over per-region OCRs. The 2-stage orchestration is host-side (Python
86
+ `mineru-vl-utils` is the source of truth; a Swift host is the app-integration step).
87
+ - **License**: base `MinerU2.5-2509` is AGPL-3.0 — this port uses **`MinerU2.5-Pro-2605` (Apache-2.0)**.
88
+ - **Appropriate input**: single-page documents; layout runs on a downsampled page, recognition on
89
+ native-res crops (pick a larger export grid for dense small text).
90
+ - int4 not shipped (weight-only int4 without QAT risks a quality cliff on a 1.2B model). iPhone (h18p)
91
+ throughput pending device verification.
92
+ - *Community port — not affiliated with Apple or OpenDataLab.*
decoder/metadata.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata_version": "0.2",
3
+ "kind": "llm",
4
+ "name": "mineru_decode_int8lin_s1",
5
+ "assets": {
6
+ "main": "mineru_decode_int8lin_s1.aimodel"
7
+ },
8
+ "language": {
9
+ "tokenizer": "tokenizer",
10
+ "vocab_size": 151936,
11
+ "max_context_length": 4096,
12
+ "embedded_tokenizer": true,
13
+ "function_map": {
14
+ "main": [
15
+ "main"
16
+ ]
17
+ }
18
+ },
19
+ "source": {
20
+ "model_definition": "torch",
21
+ "hf_model_id": "opendatalab/MinerU2.5-Pro-2605-1.2B"
22
+ },
23
+ "compression": null,
24
+ "compilation": {
25
+ "date": "2026-07-04T09:08:45.208212+00:00",
26
+ "targets": []
27
+ }
28
+ }
decoder/mineru_decode_int8lin_s1.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ .m���~ۯ��UV}�A�L@N�};�?�
decoder/mineru_decode_int8lin_s1.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e6ddaf482ca7edbafe58ee71a5556187da0418e160f4c404ee47d3bbe3f8112
3
+ size 652928624
decoder/mineru_decode_int8lin_s1.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion": "2.0"
3
+ }
decoder/tokenizer/chat_template.jinja ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
2
+ You are a helpful assistant.<|im_end|>
3
+ {% endif %}<|im_start|>{{ message['role'] }}
4
+ {% if message['content'] is string %}{{ message['content'] }}<|im_end|>
5
+ {% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
6
+ {% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
7
+ {% endif %}
decoder/tokenizer/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dceac5fc54a795ee7570d17902b47bd05412dc2afa62bdf325c3f97fcb5b87fe
3
+ size 11423550
decoder/tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,297 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "151643": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "151644": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "151645": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "151646": {
29
+ "content": "<|object_ref_start|>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "151647": {
37
+ "content": "<|object_ref_end|>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "151648": {
45
+ "content": "<|box_start|>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "151649": {
53
+ "content": "<|box_end|>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "151650": {
61
+ "content": "<|quad_start|>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "151651": {
69
+ "content": "<|quad_end|>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "151652": {
77
+ "content": "<|vision_start|>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "151653": {
85
+ "content": "<|vision_end|>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "151654": {
93
+ "content": "<|vision_pad|>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "151655": {
101
+ "content": "<|image_pad|>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "151656": {
109
+ "content": "<|video_pad|>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "151657": {
117
+ "content": "<|ref_start|>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": true
123
+ },
124
+ "151658": {
125
+ "content": "<|ref_end|>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": true
131
+ },
132
+ "151659": {
133
+ "content": "<|md_start|>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": true
139
+ },
140
+ "151660": {
141
+ "content": "<|md_end|>",
142
+ "lstrip": false,
143
+ "normalized": false,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": true
147
+ },
148
+ "151661": {
149
+ "content": "<ched>",
150
+ "lstrip": false,
151
+ "normalized": false,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": true
155
+ },
156
+ "151662": {
157
+ "content": "<ecel>",
158
+ "lstrip": false,
159
+ "normalized": false,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": true
163
+ },
164
+ "151663": {
165
+ "content": "<fcel>",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": true
171
+ },
172
+ "151664": {
173
+ "content": "<lcel>",
174
+ "lstrip": false,
175
+ "normalized": false,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": true
179
+ },
180
+ "151665": {
181
+ "content": "<ucel>",
182
+ "lstrip": false,
183
+ "normalized": false,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": true
187
+ },
188
+ "151666": {
189
+ "content": "<xcel>",
190
+ "lstrip": false,
191
+ "normalized": false,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": true
195
+ },
196
+ "151667": {
197
+ "content": "<nl>",
198
+ "lstrip": false,
199
+ "normalized": false,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": true
203
+ },
204
+ "151668": {
205
+ "content": "<|rotate_up|>",
206
+ "lstrip": false,
207
+ "normalized": false,
208
+ "rstrip": false,
209
+ "single_word": false,
210
+ "special": true
211
+ },
212
+ "151669": {
213
+ "content": "<|rotate_down|>",
214
+ "lstrip": false,
215
+ "normalized": false,
216
+ "rstrip": false,
217
+ "single_word": false,
218
+ "special": true
219
+ },
220
+ "151670": {
221
+ "content": "<|rotate_left|>",
222
+ "lstrip": false,
223
+ "normalized": false,
224
+ "rstrip": false,
225
+ "single_word": false,
226
+ "special": true
227
+ },
228
+ "151671": {
229
+ "content": "<|rotate_right|>",
230
+ "lstrip": false,
231
+ "normalized": false,
232
+ "rstrip": false,
233
+ "single_word": false,
234
+ "special": true
235
+ },
236
+ "151672": {
237
+ "content": "<|txt_contd|>",
238
+ "lstrip": false,
239
+ "normalized": false,
240
+ "rstrip": false,
241
+ "single_word": false,
242
+ "special": true
243
+ },
244
+ "151673": {
245
+ "content": "<|paratext|>",
246
+ "lstrip": false,
247
+ "normalized": false,
248
+ "rstrip": false,
249
+ "single_word": false,
250
+ "special": true
251
+ }
252
+ },
253
+ "additional_special_tokens": [
254
+ "<|im_start|>",
255
+ "<|im_end|>",
256
+ "<|object_ref_start|>",
257
+ "<|object_ref_end|>",
258
+ "<|box_start|>",
259
+ "<|box_end|>",
260
+ "<|quad_start|>",
261
+ "<|quad_end|>",
262
+ "<|vision_start|>",
263
+ "<|vision_end|>",
264
+ "<|vision_pad|>",
265
+ "<|image_pad|>",
266
+ "<|video_pad|>",
267
+ "<|ref_start|>",
268
+ "<|ref_end|>",
269
+ "<|md_start|>",
270
+ "<|md_end|>",
271
+ "<ched>",
272
+ "<ecel>",
273
+ "<fcel>",
274
+ "<lcel>",
275
+ "<ucel>",
276
+ "<xcel>",
277
+ "<nl>",
278
+ "<|rotate_up|>",
279
+ "<|rotate_down|>",
280
+ "<|rotate_left|>",
281
+ "<|rotate_right|>",
282
+ "<|txt_contd|>",
283
+ "<|paratext|>"
284
+ ],
285
+ "bos_token": null,
286
+ "clean_up_tokenization_spaces": false,
287
+ "eos_token": "<|im_end|>",
288
+ "errors": "replace",
289
+ "extra_special_tokens": {},
290
+ "model_max_length": 16384,
291
+ "pad_token": "<|endoftext|>",
292
+ "padding_side": "left",
293
+ "processor_class": "Qwen2VLProcessor",
294
+ "split_special_tokens": false,
295
+ "tokenizer_class": "Qwen2Tokenizer",
296
+ "unk_token": null
297
+ }
tokenizer/chat_template.jinja ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system
2
+ You are a helpful assistant.<|im_end|>
3
+ {% endif %}<|im_start|>{{ message['role'] }}
4
+ {% if message['content'] is string %}{{ message['content'] }}<|im_end|>
5
+ {% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>
6
+ {% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
7
+ {% endif %}
tokenizer/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dceac5fc54a795ee7570d17902b47bd05412dc2afa62bdf325c3f97fcb5b87fe
3
+ size 11423550
tokenizer/tokenizer_config.json ADDED
@@ -0,0 +1,297 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "151643": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "151644": {
13
+ "content": "<|im_start|>",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "151645": {
21
+ "content": "<|im_end|>",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "151646": {
29
+ "content": "<|object_ref_start|>",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "151647": {
37
+ "content": "<|object_ref_end|>",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "151648": {
45
+ "content": "<|box_start|>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "151649": {
53
+ "content": "<|box_end|>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ },
60
+ "151650": {
61
+ "content": "<|quad_start|>",
62
+ "lstrip": false,
63
+ "normalized": false,
64
+ "rstrip": false,
65
+ "single_word": false,
66
+ "special": true
67
+ },
68
+ "151651": {
69
+ "content": "<|quad_end|>",
70
+ "lstrip": false,
71
+ "normalized": false,
72
+ "rstrip": false,
73
+ "single_word": false,
74
+ "special": true
75
+ },
76
+ "151652": {
77
+ "content": "<|vision_start|>",
78
+ "lstrip": false,
79
+ "normalized": false,
80
+ "rstrip": false,
81
+ "single_word": false,
82
+ "special": true
83
+ },
84
+ "151653": {
85
+ "content": "<|vision_end|>",
86
+ "lstrip": false,
87
+ "normalized": false,
88
+ "rstrip": false,
89
+ "single_word": false,
90
+ "special": true
91
+ },
92
+ "151654": {
93
+ "content": "<|vision_pad|>",
94
+ "lstrip": false,
95
+ "normalized": false,
96
+ "rstrip": false,
97
+ "single_word": false,
98
+ "special": true
99
+ },
100
+ "151655": {
101
+ "content": "<|image_pad|>",
102
+ "lstrip": false,
103
+ "normalized": false,
104
+ "rstrip": false,
105
+ "single_word": false,
106
+ "special": true
107
+ },
108
+ "151656": {
109
+ "content": "<|video_pad|>",
110
+ "lstrip": false,
111
+ "normalized": false,
112
+ "rstrip": false,
113
+ "single_word": false,
114
+ "special": true
115
+ },
116
+ "151657": {
117
+ "content": "<|ref_start|>",
118
+ "lstrip": false,
119
+ "normalized": false,
120
+ "rstrip": false,
121
+ "single_word": false,
122
+ "special": true
123
+ },
124
+ "151658": {
125
+ "content": "<|ref_end|>",
126
+ "lstrip": false,
127
+ "normalized": false,
128
+ "rstrip": false,
129
+ "single_word": false,
130
+ "special": true
131
+ },
132
+ "151659": {
133
+ "content": "<|md_start|>",
134
+ "lstrip": false,
135
+ "normalized": false,
136
+ "rstrip": false,
137
+ "single_word": false,
138
+ "special": true
139
+ },
140
+ "151660": {
141
+ "content": "<|md_end|>",
142
+ "lstrip": false,
143
+ "normalized": false,
144
+ "rstrip": false,
145
+ "single_word": false,
146
+ "special": true
147
+ },
148
+ "151661": {
149
+ "content": "<ched>",
150
+ "lstrip": false,
151
+ "normalized": false,
152
+ "rstrip": false,
153
+ "single_word": false,
154
+ "special": true
155
+ },
156
+ "151662": {
157
+ "content": "<ecel>",
158
+ "lstrip": false,
159
+ "normalized": false,
160
+ "rstrip": false,
161
+ "single_word": false,
162
+ "special": true
163
+ },
164
+ "151663": {
165
+ "content": "<fcel>",
166
+ "lstrip": false,
167
+ "normalized": false,
168
+ "rstrip": false,
169
+ "single_word": false,
170
+ "special": true
171
+ },
172
+ "151664": {
173
+ "content": "<lcel>",
174
+ "lstrip": false,
175
+ "normalized": false,
176
+ "rstrip": false,
177
+ "single_word": false,
178
+ "special": true
179
+ },
180
+ "151665": {
181
+ "content": "<ucel>",
182
+ "lstrip": false,
183
+ "normalized": false,
184
+ "rstrip": false,
185
+ "single_word": false,
186
+ "special": true
187
+ },
188
+ "151666": {
189
+ "content": "<xcel>",
190
+ "lstrip": false,
191
+ "normalized": false,
192
+ "rstrip": false,
193
+ "single_word": false,
194
+ "special": true
195
+ },
196
+ "151667": {
197
+ "content": "<nl>",
198
+ "lstrip": false,
199
+ "normalized": false,
200
+ "rstrip": false,
201
+ "single_word": false,
202
+ "special": true
203
+ },
204
+ "151668": {
205
+ "content": "<|rotate_up|>",
206
+ "lstrip": false,
207
+ "normalized": false,
208
+ "rstrip": false,
209
+ "single_word": false,
210
+ "special": true
211
+ },
212
+ "151669": {
213
+ "content": "<|rotate_down|>",
214
+ "lstrip": false,
215
+ "normalized": false,
216
+ "rstrip": false,
217
+ "single_word": false,
218
+ "special": true
219
+ },
220
+ "151670": {
221
+ "content": "<|rotate_left|>",
222
+ "lstrip": false,
223
+ "normalized": false,
224
+ "rstrip": false,
225
+ "single_word": false,
226
+ "special": true
227
+ },
228
+ "151671": {
229
+ "content": "<|rotate_right|>",
230
+ "lstrip": false,
231
+ "normalized": false,
232
+ "rstrip": false,
233
+ "single_word": false,
234
+ "special": true
235
+ },
236
+ "151672": {
237
+ "content": "<|txt_contd|>",
238
+ "lstrip": false,
239
+ "normalized": false,
240
+ "rstrip": false,
241
+ "single_word": false,
242
+ "special": true
243
+ },
244
+ "151673": {
245
+ "content": "<|paratext|>",
246
+ "lstrip": false,
247
+ "normalized": false,
248
+ "rstrip": false,
249
+ "single_word": false,
250
+ "special": true
251
+ }
252
+ },
253
+ "additional_special_tokens": [
254
+ "<|im_start|>",
255
+ "<|im_end|>",
256
+ "<|object_ref_start|>",
257
+ "<|object_ref_end|>",
258
+ "<|box_start|>",
259
+ "<|box_end|>",
260
+ "<|quad_start|>",
261
+ "<|quad_end|>",
262
+ "<|vision_start|>",
263
+ "<|vision_end|>",
264
+ "<|vision_pad|>",
265
+ "<|image_pad|>",
266
+ "<|video_pad|>",
267
+ "<|ref_start|>",
268
+ "<|ref_end|>",
269
+ "<|md_start|>",
270
+ "<|md_end|>",
271
+ "<ched>",
272
+ "<ecel>",
273
+ "<fcel>",
274
+ "<lcel>",
275
+ "<ucel>",
276
+ "<xcel>",
277
+ "<nl>",
278
+ "<|rotate_up|>",
279
+ "<|rotate_down|>",
280
+ "<|rotate_left|>",
281
+ "<|rotate_right|>",
282
+ "<|txt_contd|>",
283
+ "<|paratext|>"
284
+ ],
285
+ "bos_token": null,
286
+ "clean_up_tokenization_spaces": false,
287
+ "eos_token": "<|im_end|>",
288
+ "errors": "replace",
289
+ "extra_special_tokens": {},
290
+ "model_max_length": 16384,
291
+ "pad_token": "<|endoftext|>",
292
+ "padding_side": "left",
293
+ "processor_class": "Qwen2VLProcessor",
294
+ "split_special_tokens": false,
295
+ "tokenizer_class": "Qwen2Tokenizer",
296
+ "unk_token": null
297
+ }
vision/mineru_vision.aimodel/main.hash ADDED
@@ -0,0 +1 @@
 
 
1
+ �O-<w�YeOd4���r�u6�AC�e{�'�gY
vision/mineru_vision.aimodel/main.mlirb ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd4f2d3c77be59654f6434d9c2ee72c67536c94110438f657bd127f18cac6759
3
+ size 1325089300
vision/mineru_vision.aimodel/metadata.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "assetVersion": "2.0"
3
+ }