README.md

---
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- ERNIE4.5
- PaddleOCR
- PaddlePaddle
- image-to-text
- ocr
- document-parse
- layout
- table
- formula
- chart
- seal
- spotting
- mlx
language:
- en
- zh
- multilingual
library_name: mlx
base_model: PaddlePaddle/PaddleOCR-VL-1.6
---

# PaddleOCR-VL-1.6-MLX-8bit

MLX 8-bit quantization of [PaddlePaddle/PaddleOCR-VL-1.6](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6), a 0.9B-parameter vision-language model for document parsing. Converted for Apple Silicon, running locally with [mlx-vlm](https://github.com/Blaizzy/mlx-vlm).

## Model details

- **Base model:** [PaddlePaddle/PaddleOCR-VL-1.6](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6) (0.9B, ERNIE-4.5-based)
- **Quantization:** 8-bit (MLX `mlx_vlm.convert`), ~1.09 GB on disk
- **Architecture:** `PaddleOCRVLForConditionalGeneration` (custom `paddleocr_vl` model type; trust_remote_code required)
- **Context:** up to 131,072 tokens
- **Vision encoder:** 14px patches, up to ~1M pixels per image

## Capabilities

- Full-page OCR in Markdown and JSON formats
- Layout analysis, table recognition (structure), formula recognition (LaTeX)
- Chart and seal recognition, text spotting (detection + recognition)

## Intended uses

- On-device document parsing on Apple Silicon (M-series)
- Local OCR pipelines where no data may leave the machine
- Served via `mlx_vlm.server` or the MLX Python API with `trust_remote_code=True`

## Usage

```bash
pip install mlx-vlm
```

```python
from mlx_vlm import load, generate

model, processor = load("OpenGryd/PaddleOCR-VL-1.6-MLX-8bit")
output = generate(model, processor, "media/document.png", max_tokens=4096)
print(output)
```

To serve OpenAI-compatible endpoints:

```bash
python -m mlx_vlm.server --model OpenGryd/PaddleOCR-VL-1.6-MLX-8bit --port 8090
```

## Limitations

- 8-bit quantization introduces a small accuracy delta vs. the bf16/fp16 base model
- Requires `trust_remote_code=True` (custom modeling code ships with the repo)
- No text-only inputs: image required

## Evaluation

The base model achieves 96.3% on OmniDocBench v1.6, leading on text, formula, and table recognition. See the [base model card](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6) for full benchmarks.

## License

Apache-2.0 — inherited from the base model.

## Citation

```bibtex
@misc{paddleocrvl2025,
  title={PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training},
  author={PaddlePaddle Team},
  year={2025},
  howpublished={\url{https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6}},
}
```