ReadME / ModelCard 优化

#8
by HwwwH - opened
Files changed (1) hide show
  1. README.md +67 -70
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
 
17
  <div align="center">
18
 
19
- [GitHub](https://github.com/OpenBMB/MiniCPM-OCR) · [API](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpm_ocr_api_en.md) · [Demo](https://platform.modelbest.ai/console/playground/document-parse) · [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) · [Skill](https://github.com/OpenBMB/MiniCPM-OCR#skill)
20
 
21
  </div>
22
 
@@ -39,8 +39,10 @@ and balances quality against speed on real-world captures.
39
  information extraction (KIE) for downstream automation.
40
  - 🛠️ **Easy to deploy and develop**: **vLLM, SGLang, llama.cpp, and Ollama from day one**, with
41
  quantized variants and multi-chip builds. Supports further fine-tuning, and ships a pip SDK,
42
- a full API, and agent workflows. Online demos: [China](https://platform.modelbest.cn/console/playground/document-parse) ·
43
- [Global](https://platform.modelbest.ai/console/playground/document-parse).
 
 
44
 
45
  ## SOTA quality at higher throughput
46
 
@@ -92,7 +94,8 @@ Left to right: original document, detected layout, and rendered Markdown.
92
 
93
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif" alt="Chinese document animation: original, detected layout, and rendered Markdown." width="100%"></p>
94
 
95
- ### More examples
 
96
 
97
  Original documents and parsed results, side by side.
98
 
@@ -104,6 +107,8 @@ Original documents and parsed results, side by side.
104
 
105
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-ocr-chinese-cases.png" alt="Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction." width="100%"></p>
106
 
 
 
107
  ## Model architecture
108
 
109
  | Item | Description |
@@ -121,77 +126,17 @@ deployment options, and benchmark breakdowns.
121
 
122
  ## Quick start
123
 
124
- ### Document parsing (SDK)
125
-
126
- The SDK parses whole documents; the model recognises regions:
127
-
128
- 1. **Read** — accept PDF/PNG/JPG and rasterise PDF pages.
129
- 2. **Layout** — PP-DocLayoutV3 detects regions and reading order, then crops them.
130
- 3. **Recognise** — regions are sent to the MiniCPM-OCR 1.0 service concurrently.
131
- 4. **Assemble** — results are merged into Markdown in reading order; tables become HTML,
132
- formulas stay LaTeX.
133
 
134
- SDK and inference service use separate Python environments and may run on different machines.
135
- See the [SDK tutorial](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_tutorial_en.md).
136
-
137
- ### vLLM
138
-
139
- ```bash
140
- vllm serve openbmb/MiniCPM-OCR-1.0 \
141
- --served-model-name MiniCPM-OCR \
142
- --trust-remote-code \
143
- --max-model-len 40960
144
- ```
145
 
146
- Full API example (request fields, image encoding, response checks):
147
- [vLLM deploy guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md).
148
 
149
- ### Other runtimes
150
 
151
- | Runtime | Guide |
152
- | :--- | :--- |
153
- | vLLM | [API service and single-region recognition](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md) |
154
- | SGLang | [Deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/sglang_deploy.md) |
155
- | llama.cpp | [GGUF deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/llamacpp_deploy.md) |
156
- | Ollama | [SDK service connection and backend configuration](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_api_en.md) |
157
- | Transformers | [Service deployment](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md#43-transformers-serve) |
158
- | FlagOS (multi-chip) | [Deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/flagos_deploy_en.md) |
159
-
160
- ### Prompts
161
-
162
- The prompt selects the task and must match the region you cropped:
163
-
164
- | Region | Prompt | Output |
165
- | :--- | :--- | :--- |
166
- | Text block / paragraph / title / page | `Text Recognition:` | plain text |
167
- | Table | `Table Recognition:` | HTML (via SDK post-processing) |
168
- | Formula | `Formula Recognition:` | LaTeX |
169
- | Inline formula | `Text Recognition:` | LaTeX |
170
- | Seal | `Seal Recognition:` | seal text |
171
- | Chart | `Chart to Table:` | table |
172
 
173
- If `finish_reason` is `length`, the output was truncated — raise the token budget.
174
-
175
- ### Key information extraction (KIE)
176
-
177
- MiniCPM-OCR 1.0 can pull specified fields straight out of an image — for example the store name
178
- and amount on a receipt. Send the image with a field-extraction prompt and ask for a JSON object;
179
- you do not need to produce page Markdown first.
180
-
181
- Use the [image recognition API example](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md#22-image-recognition) and
182
- replace `Text Recognition:` with:
183
-
184
- ```text
185
- Extract the "store_name", "amount". Format the output as a JSON object with keys: "store_name", "amount". If an item is not in the image, use "###".
186
- ```
187
-
188
- Replace both field lists with your own field names (Chinese or English). The response is
189
- model-generated text: parse the JSON and check fields and values before using it downstream.
190
-
191
- ### Agent skill
192
-
193
- Have a coding agent follow the skill to set up the environment, download the model, and parse
194
- a first document:
195
 
196
  ```text
197
  Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md
@@ -203,6 +148,58 @@ Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.
203
  [ms-swift](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/swift_minicpm_ocr_french_en.md) — data preparation, full-parameter SFT with
204
  the multimodal projector frozen, training, and evaluation (French handwritten text lines).
205
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
  ## Related checkpoints
207
 
208
  | Format | Hugging Face | ModelScope |
 
16
 
17
  <div align="center">
18
 
19
+ [GitHub](https://github.com/OpenBMB/MiniCPM-OCR) · [API](https://platform.modelbest.ai/console/docs/api/document-parse) · [Official Demo](https://platform.modelbest.ai/console/playground/document-parse) · [HF Spaces](https://huggingface.co/spaces/openbmb/MiniCPM-OCR) · [Usage](https://github.com/OpenBMB/MiniCPM-OCR#using-minicpm-ocr) · [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) · [Skill](https://github.com/OpenBMB/MiniCPM-OCR#skill)
20
 
21
  </div>
22
 
 
39
  information extraction (KIE) for downstream automation.
40
  - 🛠️ **Easy to deploy and develop**: **vLLM, SGLang, llama.cpp, and Ollama from day one**, with
41
  quantized variants and multi-chip builds. Supports further fine-tuning, and ships a pip SDK,
42
+ a full API, and agent workflows. Online demos: [HF Spaces](https://huggingface.co/spaces/openbmb/MiniCPM-OCR) ·
43
+ [CN](http://82.157.64.212:10111/playground/document-parse) ·
44
+ [Official CN](https://platform.modelbest.cn/console/playground/document-parse) ·
45
+ [Official Global](https://platform.modelbest.ai/console/playground/document-parse).
46
 
47
  ## SOTA quality at higher throughput
48
 
 
94
 
95
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif" alt="Chinese document animation: original, detected layout, and rendered Markdown." width="100%"></p>
96
 
97
+ <details>
98
+ <summary><b>More examples</b></summary>
99
 
100
  Original documents and parsed results, side by side.
101
 
 
107
 
108
  <p align="center"><img src="https://raw.githubusercontent.com/OpenBMB/MiniCPM-OCR/main/assets/minicpm-ocr/minicpm-ocr-chinese-cases.png" alt="Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction." width="100%"></p>
109
 
110
+ </details>
111
+
112
  ## Model architecture
113
 
114
  | Item | Description |
 
126
 
127
  ## Quick start
128
 
129
+ ### Online Demo
 
 
 
 
 
 
 
 
130
 
131
+ To quickly try text-centric document understanding and structured information extraction, we recommend [HF Spaces](https://huggingface.co/spaces/openbmb/MiniCPM-OCR), [CN](http://82.157.64.212:10111/playground/document-parse), [Official CN](https://platform.modelbest.cn/console/playground/document-parse), [Official Global](https://platform.modelbest.ai/console/playground/document-parse) / [API](https://platform.modelbest.ai/console/docs/api/document-parse).
 
 
 
 
 
 
 
 
 
 
132
 
133
+ ### Local deployment
 
134
 
135
+ For local deployment of MiniCPM-OCR, we recommend our SDK ([tutorial](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_tutorial_en.md)) with [DSpark](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark) enabled by default. For document parsing task, It reads a PDF or image, detects layout with PP-DocLayoutV3, recognises each region, and assembles Markdown in reading order. Our SDK supports different inference backends, including [vLLM](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md), [SGLang](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/sglang_deploy.md), [llama.cpp](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/llamacpp_deploy.md), [Ollama](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpmvocr_api_en.md), [Transformers](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md#43-transformers-serve), and [FlagOS](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/flagos_deploy_en.md).
136
 
137
+ ### Agent Skill
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
 
139
+ You can also have a coding agent follow the [Agent Skill](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md) to set up the environment, download the model, and parse a first document.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
140
 
141
  ```text
142
  Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md
 
148
  [ms-swift](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/swift_minicpm_ocr_french_en.md) — data preparation, full-parameter SFT with
149
  the multimodal projector frozen, training, and evaluation (French handwritten text lines).
150
 
151
+ ### Recognition guide
152
+
153
+ <table>
154
+ <thead>
155
+ <tr>
156
+ <th>Task</th>
157
+ <th>Region</th>
158
+ <th>Prompt</th>
159
+ <th>Output</th>
160
+ </tr>
161
+ </thead>
162
+ <tbody>
163
+ <tr>
164
+ <td rowspan="6">Text-centric document understanding</td>
165
+ <td>Text block / Paragraph / Title / Page</td>
166
+ <td><code>Text Recognition:</code></td>
167
+ <td>Plain text</td>
168
+ </tr>
169
+ <tr>
170
+ <td>Table</td>
171
+ <td><code>Table Recognition:</code></td>
172
+ <td>Table in HTML format(via SDK post-processing)</td>
173
+ </tr>
174
+ <tr>
175
+ <td>Formula</td>
176
+ <td><code>Formula Recognition:</code></td>
177
+ <td>LaTeX</td>
178
+ </tr>
179
+ <tr>
180
+ <td>Inline formula</td>
181
+ <td><code>Text Recognition:</code></td>
182
+ <td>LaTeX</td>
183
+ </tr>
184
+ <tr>
185
+ <td>Seal</td>
186
+ <td><code>Seal Recognition:</code></td>
187
+ <td>Plain text</td>
188
+ </tr>
189
+ <tr>
190
+ <td>Chart</td>
191
+ <td><code>Chart to Table:</code></td>
192
+ <td>Table in HTML format(via SDK post-processing)</td>
193
+ </tr>
194
+ <tr>
195
+ <td>Structured information extraction</td>
196
+ <td>Specified fields</td>
197
+ <td><code>Extract the "field", .... Format the output as a JSON object with keys: "field", .... If an item is not in the image, use "###".</code></td>
198
+ <td>JSON</td>
199
+ </tr>
200
+ </tbody>
201
+ </table>
202
+
203
  ## Related checkpoints
204
 
205
  | Format | Hugging Face | ModelScope |