README.md

---
library_name: onnx
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
  - onnx
  - ocr
  - translation
  - pdf
base_model:
  - deepseek-ai/DeepSeek-OCR-2
  - Helsinki-NLP/opus-mt-es-en
  - storia/font-classify-onnx
---

# ocrtool models

ONNX models used by ocrtool, a C++ command-line tool that converts PDFs to Markdown, translates
PDFs in place and identifies the fonts in an image. They run on ONNX Runtime with no
Python involved.

Nothing here was trained or fine-tuned by us. Each folder is a format conversion of the original
weights, or an unchanged copy, and stays under the original author's licence.

| Folder | Size | Use | Original weights | Licence |
| --- | --- | --- | --- | --- |
| `deepseek-ocr2-int4/` | 2.9 GB | PDF to Markdown, CPU or GPU (recommended download) | [deepseek-ai/DeepSeek-OCR-2](https://huggingface.co/deepseek-ai/DeepSeek-OCR-2) | Apache-2.0 |
| `deepseek-ocr2-fp16/` | 6.6 GB | PDF to Markdown, GPU, fastest | [deepseek-ai/DeepSeek-OCR-2](https://huggingface.co/deepseek-ai/DeepSeek-OCR-2) | Apache-2.0 |
| `deepseek-ocr2-fp32/` | 13 GB | Full precision; the source for quantizing | [deepseek-ai/DeepSeek-OCR-2](https://huggingface.co/deepseek-ai/DeepSeek-OCR-2) | Apache-2.0 |
| `opus-mt-es-en/` | 0.6 GB | Spanish to English translation | [Helsinki-NLP/opus-mt-es-en](https://huggingface.co/Helsinki-NLP/opus-mt-es-en) | Apache-2.0 |
| `font-classify/` | 64 MB | Font identification | [storia/font-classify-onnx](https://huggingface.co/storia/font-classify-onnx) | MIT |

Each folder is self-contained. Download the whole folder and keep the file names: an `.onnx` file
loads its weights from the `.onnx.data` file beside it.

```sh
hf download baker76/ocrtool-models --include "deepseek-ocr2-int4/*" --local-dir models
ocrtool convert in.pdf -o out.md --model models/deepseek-ocr2-int4
ocrtool translate in.pdf -o out.pdf --model models/opus-mt-es-en
ocrtool fonts scan.png -o fonts.json --model models/font-classify
```

## DeepSeek-OCR-2

- `vision_sam_1024.onnx`, `vision_sam_768.onnx`: the SAM stage for the full-page view and the tiles.
- `vision_encoder.onnx`: the Qwen2-based encoder and projector.
- `decoder.onnx`: token embedding, image-feature merge, the 12-layer decoder with a key/value cache,
  and the output head. The mixture-of-experts routing is evaluated densely (all experts, masked to
  the top 6), which gives the same result as the original sparse routing.
- `tokenizer.json`, `config.json`: tokenizer and the prompt token ids.

On one test page the fp32 export reproduced the PyTorch model's output token for token; the fp16
export produced the same text with two bounding-box coordinates differing by 1.

`deepseek-ocr2-int4` is the fp32 export with the decoder's weights stored in 4 bits and the vision
encoder's in 8 (ONNX Runtime `MatMulNBits`); the two SAM graphs are unchanged. On four test pages its
text matched the fp16 model's apart from a few formatting differences. It decodes more slowly than
fp16 on a GPU (about 50 against 80 tokens per second on an RTX 4090).

## OPUS-MT

`encoder.onnx` and `decoder.onnx` (no cache; the decoder is re-run each step), with the original
`source.spm`, `target.spm` and `vocab.json`. Logits match PyTorch to within 1e-5.

## Font classifier

An unmodified copy of [storia/font-classify-onnx](https://huggingface.co/storia/font-classify-onnx)
at revision `86a4e77`: an EfficientNet-B3 fine-tuned by Storia on synthetic images of about 3,470
Google Fonts. `model.onnx` takes one 320 x 320 image; `model_config.yaml` lists the class names and
`fonts_mapping.yaml` the font file behind each class. `google_fonts_mapping.tsv`, from Storia's
[training repository](https://github.com/Storia-AI/font-classify), maps each class to the family
name and variant used by Google Fonts.

The scripts that produced the exported files are in the ocrtool repository under `tools/export/`.