cstr/paddleocr-vl-1.6-GGUF

PaddleOCR-VL-1.6 — CrispEmbed GGUF

CrispEmbed-native GGUF quantizations of PaddlePaddle/PaddleOCR-VL-1.6.

Latest PaddleOCR-VL model with improved accuracy on OmniDocBench (96.3% SOTA). End-to-end VLM-based OCR: text recognition, table extraction, formula recognition, chart understanding. 109+ languages.

Files

File	Size	Description
paddleocr-vl-1.6-q4_k.gguf	1.3 GB	4-bit K-quant — smallest, lossy (see note)
paddleocr-vl-1.6-q8_0.gguf	1.4 GB	8-bit quantization — recommended
paddleocr-vl-1.6-f16.gguf	2.3 GB	fp16 reference

Quantization quality. This is a small 0.9B model, so 4-bit quantization is noticeably lossy. In testing, q8_0 transcribes fully and matches the fp16 reference, while q4_k tends to terminate early (e.g. it emitted only "The quick brown fox jumps" and stopped, dropping the rest of the line) — a flattened output distribution from 4-bit weights, not a bug. Behavior is identical on CPU and Metal. Use q8_0 for accurate OCR; reach for q4_k only when memory is tight and truncated/approximate output is acceptable.

Model

• Architecture: NaViT-style ViT (27L, 1152d) + ERNIE-4.5-0.3B LLM (18L, 1024d, 16/2 GQA, MRoPE, SwiGLU)
• Parameters: ~0.9B (same architecture as PaddleOCR-VL-0.9B, improved training)
• OmniDocBench: 96.3% (SOTA)
• Languages: 109+ (multilingual)
• License: Apache 2.0

Usage

./crispembed -m paddleocr-vl-1.6-q8_0.gguf --ocr document.png

License

Apache 2.0