ReadME / ModelCard 优化

#9
by HwwwH - opened
Files changed (1) hide show
  1. README.md +64 -73
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
 
17
  <div align="center">
18
 
19
- [GitHub](https://github.com/OpenBMB/MiniCPM-OCR) · [API](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpm_ocr_api_en.md) · [Demo](https://platform.modelbest.ai/console/playground/document-parse) · [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) · [Skill](https://github.com/OpenBMB/MiniCPM-OCR#skill)
20
 
21
  </div>
22
 
@@ -39,7 +39,7 @@ and balances quality against speed on real-world captures.
39
  information extraction (KIE) for downstream automation.
40
  - 🛠️ **Easy to deploy and develop**: **vLLM, SGLang, llama.cpp, and Ollama from day one**, with
41
  quantized variants and multi-chip builds. Supports further fine-tuning, and ships a pip SDK,
42
- a full API, and agent workflows. Online demos: [China](https://platform.modelbest.cn/console/playground/document-parse) ·
43
  [Global](https://platform.modelbest.ai/console/playground/document-parse).
44
 
45
  ## SOTA quality at higher throughput
@@ -92,7 +92,8 @@ Left to right: original document, detected layout, and rendered Markdown.
92
 
93
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif" alt="Chinese document animation: original, detected layout, and rendered Markdown." width="100%"></p>
94
 
95
- ### More examples
 
96
 
97
  Original documents and parsed results, side by side.
98
 
@@ -104,6 +105,8 @@ Original documents and parsed results, side by side.
104
 
105
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-ocr-chinese-cases.png" alt="Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction." width="100%"></p>
106
 
 
 
107
  ## Model architecture
108
 
109
  | Item | Description |
@@ -121,81 +124,17 @@ deployment options, and benchmark breakdowns.
121
 
122
  ## Quick start
123
 
124
- ### Document parsing (SDK)
125
-
126
- The SDK parses whole documents; the model recognises regions:
127
-
128
- 1. **Read** — accept PDF/PNG/JPG and rasterise PDF pages.
129
- 2. **Layout** — PP-DocLayoutV3 detects regions and reading order, then crops them.
130
- 3. **Recognise** — regions are sent to the MiniCPM-OCR 1.0 service concurrently.
131
- 4. **Assemble** — results are merged into Markdown in reading order; tables become HTML,
132
- formulas stay LaTeX.
133
-
134
- SDK and inference service use separate Python environments and may run on different machines.
135
- See the [SDK tutorial](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_tutorial_en.md).
136
-
137
- ### vLLM
138
 
139
- ```bash
140
- vllm serve openbmb/MiniCPM-OCR-1.0 \
141
- --served-model-name MiniCPM-OCR \
142
- --trust-remote-code \
143
- --max-model-len 40960
144
- ```
145
 
146
- Full API example (request fields, image encoding, response checks):
147
- [vLLM deploy guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md).
148
 
149
- ### Other runtimes
150
 
151
- | Runtime | Guide |
152
- | :--- | :--- |
153
- | vLLM | [API service and single-region recognition](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md) |
154
- | SGLang | [Deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/sglang_deploy.md) |
155
- | llama.cpp | [GGUF deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/llamacpp_deploy.md) |
156
- | Ollama | [SDK service connection and backend configuration](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_api_en.md) |
157
- | Transformers | [Service deployment](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md#43-transformers-serve) |
158
- | FlagOS (multi-chip) | [Deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/flagos_deploy_en.md) |
159
-
160
- ### Prompts
161
-
162
- The prompt selects the task and must match the region you cropped:
163
-
164
- | Region | Prompt | Output |
165
- | :--- | :--- | :--- |
166
- | Text block / paragraph / title / page | `Text Recognition:` | plain text |
167
- | Table | `Table Recognition:` | HTML (via SDK post-processing) |
168
- | Formula | `Formula Recognition:` | LaTeX |
169
- | Inline formula | `Text Recognition:` | LaTeX |
170
- | Seal | `Seal Recognition:` | seal text |
171
- | Chart | `Chart to Table:` | table |
172
 
173
- If `finish_reason` is `length`, the output was truncated — raise the token budget.
174
-
175
- ### Key information extraction (KIE)
176
-
177
- MiniCPM-OCR 1.0 can pull specified fields straight out of an image — for example the store name
178
- and amount on a receipt. Send the image with a field-extraction prompt and ask for a JSON object;
179
- you do not need to produce page Markdown first.
180
-
181
- Use the [image recognition API example](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md#22-image-recognition) and
182
- replace `Text Recognition:` with:
183
-
184
- ```text
185
- Extract the "store_name", "amount". Format the output as a JSON object with keys: "store_name", "amount". If an item is not in the image, use "###".
186
- ```
187
-
188
- Replace both field lists with your own field names (Chinese or English). The response is
189
- model-generated text: parse the JSON and check fields and values before using it downstream.
190
-
191
- ### Agent skill
192
-
193
- Have a coding agent follow the skill to set up the environment, download the model, and parse
194
- a first document:
195
-
196
- ```text
197
- Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md
198
- ```
199
 
200
  ### Fine-tuning
201
 
@@ -203,6 +142,58 @@ Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.
203
  [ms-swift](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/swift_minicpm_ocr_french_en.md) — data preparation, full-parameter SFT with
204
  the multimodal projector frozen, training, and evaluation (French handwritten text lines).
205
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
  ## Related checkpoints
207
 
208
  | Format | Hugging Face | ModelScope |
 
16
 
17
  <div align="center">
18
 
19
+ [GitHub](https://github.com/OpenBMB/MiniCPM-OCR) · [API](https://platform-dev.modelbest.co/console/docs/api/document-parse) · [Demo](https://platform.modelbest.ai/console/playground/document-parse) · [Usage](https://github.com/OpenBMB/MiniCPM-OCR#using-minicpm-ocr) · [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) · [Skill](https://github.com/OpenBMB/MiniCPM-OCR#skill)
20
 
21
  </div>
22
 
 
39
  information extraction (KIE) for downstream automation.
40
  - 🛠️ **Easy to deploy and develop**: **vLLM, SGLang, llama.cpp, and Ollama from day one**, with
41
  quantized variants and multi-chip builds. Supports further fine-tuning, and ships a pip SDK,
42
+ a full API, and agent workflows. Online demos: [CN](https://platform.modelbest.cn/console/playground/document-parse) ·
43
  [Global](https://platform.modelbest.ai/console/playground/document-parse).
44
 
45
  ## SOTA quality at higher throughput
 
92
 
93
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif" alt="Chinese document animation: original, detected layout, and rendered Markdown." width="100%"></p>
94
 
95
+ <details>
96
+ <summary><b>More examples</b></summary>
97
 
98
  Original documents and parsed results, side by side.
99
 
 
105
 
106
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-ocr-chinese-cases.png" alt="Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction." width="100%"></p>
107
 
108
+ </details>
109
+
110
  ## Model architecture
111
 
112
  | Item | Description |
 
124
 
125
  ## Quick start
126
 
127
+ ### Online Demo
 
 
 
 
 
 
 
 
 
 
 
 
 
128
 
129
+ To quickly try text-centric document understanding and structured information extraction, we recommend the official Web Demo ([CN](https://platform.modelbest.cn/console/playground/document-parse), [Global](https://platform.modelbest.ai/console/playground/document-parse)) / [API](https://platform-dev.modelbest.co/console/docs/api/document-parse).
 
 
 
 
 
130
 
131
+ ### Local deployment
 
132
 
133
+ For local deployment of MiniCPM-OCR, we recommend our SDK ([tutorial](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_tutorial_en.md)) with [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) enabled by default. For document parsing task, It reads a PDF or image, detects layout with PP-DocLayoutV3, recognises each region, and assembles Markdown in reading order. Our SDK supports different inference backends, including [vLLM](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md), [SGLang](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/sglang_deploy.md), [llama.cpp](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/llamacpp_deploy.md), [Ollama](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_api_en.md), [Transformers](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md#43-transformers-serve), and [FlagOS](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/flagos_deploy_en.md).
134
 
135
+ ### Agent Skill
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
 
137
+ You can also have a coding agent follow the [Agent Skill](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md) to set up the environment, download the model, and parse a first document.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
 
139
  ### Fine-tuning
140
 
 
142
  [ms-swift](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/swift_minicpm_ocr_french_en.md) — data preparation, full-parameter SFT with
143
  the multimodal projector frozen, training, and evaluation (French handwritten text lines).
144
 
145
+ ### Recognition guide
146
+
147
+ <table>
148
+ <thead>
149
+ <tr>
150
+ <th>Task</th>
151
+ <th>Region</th>
152
+ <th>Prompt</th>
153
+ <th>Output</th>
154
+ </tr>
155
+ </thead>
156
+ <tbody>
157
+ <tr>
158
+ <td rowspan="6">Text-centric document understanding</td>
159
+ <td>Text block / Paragraph / Title / Page</td>
160
+ <td><code>Text Recognition:</code></td>
161
+ <td>Plain text</td>
162
+ </tr>
163
+ <tr>
164
+ <td>Table</td>
165
+ <td><code>Table Recognition:</code></td>
166
+ <td>Table in HTML format(via SDK post-processing)</td>
167
+ </tr>
168
+ <tr>
169
+ <td>Formula</td>
170
+ <td><code>Formula Recognition:</code></td>
171
+ <td>LaTeX</td>
172
+ </tr>
173
+ <tr>
174
+ <td>Inline formula</td>
175
+ <td><code>Text Recognition:</code></td>
176
+ <td>LaTeX</td>
177
+ </tr>
178
+ <tr>
179
+ <td>Seal</td>
180
+ <td><code>Seal Recognition:</code></td>
181
+ <td>Plain text</td>
182
+ </tr>
183
+ <tr>
184
+ <td>Chart</td>
185
+ <td><code>Chart to Table:</code></td>
186
+ <td>Table in HTML format(via SDK post-processing)</td>
187
+ </tr>
188
+ <tr>
189
+ <td>Structured information extraction</td>
190
+ <td>Specified fields</td>
191
+ <td><code>Extract the "field", .... Format the output as a JSON object with keys: "field", .... If an item is not in the image, use "###".</code></td>
192
+ <td>JSON</td>
193
+ </tr>
194
+ </tbody>
195
+ </table>
196
+
197
  ## Related checkpoints
198
 
199
  | Format | Hugging Face | ModelScope |