Instructions to use AtomicChat/MiniCPM-OCR-1.0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtomicChat/MiniCPM-OCR-1.0-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtomicChat/MiniCPM-OCR-1.0-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Ollama
How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with Ollama:
```
ollama run hf.co/AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
```
Unsloth Desktop

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with Docker Model Runner:
```
docker model run hf.co/AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M
```

Lemonade

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiniCPM-OCR-1.0-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AtomicChat/MiniCPM-OCR-1.0-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AtomicChat/MiniCPM-OCR-1.0-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiniCPM-OCR 1.0 GGUF

You can now run MiniCPM-OCR 1.0 in Atomic Chat with image input: drop in a page and ask Text Recognition: (llama.cpp engine b11946 or newer, offered as an engine update).

GGUF builds of openbmb/MiniCPM-OCR-1.0, OpenBMB's 1.3B model that reads document images into text, tables (HTML), formulas (LaTeX), charts and seals, in Chinese and English. By AtomicChat. Every file was run on 40 real document pages and compared with the original.

From 1.15 GB on disk for the whole set: AD-Q4_K_M (0.53 GB) with the Q8_0 vision projector (0.62 GB). OpenBMB's smallest recommended pair is 1.43 GB (Q4_K_M + F16 projector). On our 40 pages, every file here reads within 1.5 points of the original's character error (table below).
The text model drifts half as much as stock quants of the same size: KLD to BF16 0.087 against 0.165 at 0.53 GB (AD-Q4_K_M vs Q4_K_M), 0.027 against 0.051 at 0.58-0.59 GB (AD-Q5_K_M vs Q5_K_M).
Thinking stays off in the embedded chat template, as in OpenBMB's own GGUF. With the original template, llama-server returns the transcription in reasoning_content and an empty content, and llama-mtmd-cli reads worse (+6.6 points of character error on our pages, 95% interval +1.6 to +12.4).
1.7x faster generation with OpenBMB's DSpark draft (below): same text on 11 of 12 pages.

Files

Every text file needs a vision projector (--mmproj). The Q8_0 projector is the one to download; the F16 one is the reference.

File	Size on disk	Use	Pages within 1% of the original	Reading error vs annotation, change against the original	Text model: KLD / top-1
`MiniCPM-OCR-1.0-Q8_0.gguf`	0.81 GB	closest to the original	30/40	+0.2 [-0.8, +1.1]	0.0039 / 96.1%
`MiniCPM-OCR-1.0-AD-Q6_K.gguf`	0.73 GB	near Q8_0, 0.08 GB smaller	32/40	+0.3 [-0.2, +0.8]	0.0064 / 95.0%
`MiniCPM-OCR-1.0-AD-Q5_K_M.gguf`	0.59 GB	recommended	27/40	+1.5 [+0.1, +3.2]; with the Q8_0 projector -0.3 [-1.5, +0.7]	0.0266 / 89.9%
`MiniCPM-OCR-1.0-AD-Q4_K_M.gguf`	0.53 GB	smallest	19/40	-0.4 [-4.2, +2.6]	0.0871 / 83.4%
`mmproj-MiniCPM-OCR-1.0-Q8_0.gguf`	0.62 GB	vision projector	33/40 (with the BF16 text model)	-0.7 [-1.9, +0.3]
`mmproj-MiniCPM-OCR-1.0-F16.gguf`	0.90 GB	vision projector, reference

Pages: 40 from OmniDocBench, 4 from each of its 10 sources (slides, papers, books, textbooks, exams, historical documents, magazines, newspapers, notes, reports; seed 0), English and Chinese. One request per page, Text Recognition:, greedy decoding, llama.cpp b11953.
The original is the BF16 GGUF with the F16 projector, on the same stand. It reads these pages with a mean character error of 21.6% against OmniDocBench's annotation (median 13.1%). Much of that is format rather than misread characters (reading order and markup conventions differ), so the number only compares files with each other.
Change against the original: the file's character error minus the original's, in points, averaged over the pages, with a 95% bootstrap interval over pages. An interval that holds 0 means no difference this set can show.
Pages within 1%: the file's text differs from the original's by under 1% of characters. Greedy decoding turns one near-tie into a different rest of the page, so this count moves by a few pages between files of the same quality (our Q8_0 and OpenBMB's Q8_0, with the same KLD, land at 30 and 35).
KLD / top-1: the text model alone against BF16 logits on 87 x 4096 tokens of general text in 30 languages. It speaks to free-form prompts (questions about a page, extraction to JSON) more than to transcription.
Numbers for all 16 sizes we built (down to 0.30 GB), page by page, are in the metrics repo. Below 0.53 GB some pages start to repeat until the token limit (2 of 40 at 0.45 GB), so those files are not here. The pages' text is not republished: OmniDocBench is for research use only.

Other GGUF files on the same pages

Same pages, prompt, reference and llama.cpp build. Each file runs with its own publisher's projector (F16 or BF16, 0.90 GB).

File	Size on disk, with projector	Pages within 1% of the original	Reading error vs annotation, change against the original	Text model: KLD / top-1
openbmb F16	1.52 + 0.90 GB	38/40	+0.05 [-0.02, +0.17]	0.0002 / 99.0%
openbmb Q8_0	0.81 + 0.90 GB	35/40	+0.3 [0.0, +0.8]	0.0039 / 96.1%
openbmb Q6_K	0.63 + 0.90 GB	26/40	+4.5 [-0.4, +13.7]	0.0200 / 91.2%
openbmb Q5_K_M	0.58 + 0.90 GB	25/40	-0.6 [-1.7, +0.4]	0.0513 / 86.5%
openbmb Q4_K_M	0.53 + 0.90 GB	18/40	+0.3 [-3.9, +3.6]	0.1650 / 77.2%
openbmb Q4_0	0.50 + 0.90 GB	14/40	-0.1 [-1.9, +1.7]	0.2641 / 72.2%
prithivMLmods Q4_K_M	0.53 + 0.90 GB	5/40	+14.5 [0.0, +38.6]	0.1650 / 77.3%
prithivMLmods Q3_K_M	0.47 + 0.90 GB	2/40	+141 [+6.9, +375]	0.5642 / 62.2%

On reading error, OpenBMB's files and ours are within the noise of 40 pages at every size. The difference is the projector (0.28 GB) and the text model's drift.
prithivMLmods' Q4_K_M has the same weights as OpenBMB's (same KLD) but the original chat template, which opens a thinking block in llama.cpp; that, not the quantization, is most of its gap.

Run

llama.cpp b11826 or newer (MiniCPM-OCR support: #30979). These files were built and measured with b11953 (commit a7af149).

hf download AtomicChat/MiniCPM-OCR-1.0-GGUF MiniCPM-OCR-1.0-AD-Q5_K_M.gguf mmproj-MiniCPM-OCR-1.0-Q8_0.gguf --local-dir .

# one image, one region type (prompts below)
llama-mtmd-cli -m MiniCPM-OCR-1.0-AD-Q5_K_M.gguf --mmproj mmproj-MiniCPM-OCR-1.0-Q8_0.gguf \
  --jinja --temp 0 -c 16384 -n 8192 --image page.png -p "Text Recognition:"

# OpenAI-compatible server: send the image and the prompt as one user message, temperature 0
llama-server -m MiniCPM-OCR-1.0-AD-Q5_K_M.gguf --mmproj mmproj-MiniCPM-OCR-1.0-Q8_0.gguf \
  --jinja -c 16384 -ngl 99

What is in the image	Prompt	You get
a page, paragraph, title or text block	`Text Recognition:`	plain text (inline formulas as LaTeX)
a table	`Table Recognition:`	HTML
a display formula	`Formula Recognition:`	LaTeX
a seal	`Seal Recognition:`	the seal's text
a chart	`Chart to Table:`	a table

For whole PDFs with layout (reading order, tables and formulas cropped and recognized separately), OpenBMB's minicpmvocr SDK runs on top of llama-server (--backend llamacpp).

Faster with DSpark

OpenBMB's DSpark draft speeds up generation with these files too (llama-server or llama-cli; llama-mtmd-cli takes no draft model):

hf download openbmb/MiniCPM-OCR-1.0-DSpark-gguf dspark-MiniCPM-OCR-1_0-Q8_0.gguf --local-dir .
llama-server -m MiniCPM-OCR-1.0-AD-Q5_K_M.gguf --mmproj mmproj-MiniCPM-OCR-1.0-Q8_0.gguf \
  -md dspark-MiniCPM-OCR-1_0-Q8_0.gguf -ngld 99 --spec-type draft-dspark --jinja -c 16384 -ngl 99

Text model	Generation, no draft	With DSpark	Same text
`Q8_0`	315 tokens/s	527 tokens/s (1.67x)	11/12 pages
`AD-Q4_K_M`	337 tokens/s	565 tokens/s (1.68x)	11/12 pages

12 of the pages above, one at a time, one A100 40 GB; 86% of drafted tokens accepted. With greedy decoding a draft should not change the text; on one page it did, most likely through batched-verification numerics (not checked further).

How the files are built

The text model (Qwen3.5 0.8B, 24 blocks) is quantized with an importance matrix from about 5M tokens of text in the model's chat markup (many languages, code, tables, math, long documents), shared with nothing in the page set above. The vision encoder is not calibrated: images enter only through the projector.
AD layouts: the embedding table, which is also the output head here, and the attention output stay at Q6_K or higher; the FFN of the first two and last five blocks, and of blocks 2-4, sits one step above the rest.
mmproj-...-Q8_0.gguf: llama.cpp's converter at Q8_0. The SigLIP ffn_down weights (rows 4304 wide, not a multiple of 32) stay F16 in it, so it is 0.62 GB rather than half of the F16 file.
The embedded chat template is the original with one change: the assistant turn always starts with an empty <think></think> block. If you convert the weights yourself, run llama-server with --reasoning off instead.

Limits

Measured on whole pages with Text Recognition:. Crops from a layout model (tables, formulas, seals), the way OpenBMB's SDK works, are not measured separately here.
Chinese and English pages only, as the model is trained.
40 pages: differences under about 2 points of character error are within the noise.
Speeds are from one A100 with llama.cpp b11953; other hardware differs.

License

Apache-2.0, as openbmb/MiniCPM-OCR-1.0 (SigLIP vision encoder and Qwen3.5 text model, trained by OpenBMB from MiniCPM-V 4.6). Built by AtomicChat. Measurements and logs: AtomicChat/MiniCPM-OCR-1.0-GGUF-metrics.

Downloads last month: -

GGUF

Model size

0.8B params

Architecture

qwen35

Hardware compatibility

4-bit

5-bit

6-bit

8-bit

Model tree for AtomicChat/MiniCPM-OCR-1.0-GGUF

Base model

openbmb/MiniCPM-OCR-1.0

Quantized

(6)

this model

Collection including AtomicChat/MiniCPM-OCR-1.0-GGUF

MiniCPM-OCR 1.0

Collection

OpenBMB's 1.3B OCR model as GGUF: every file measured on real document pages. By AtomicChat. • 2 items • Updated about 7 hours ago