---
license: apache-2.0
library_name: mlx
pipeline_tag: object-detection
tags:
  - document-layout
  - layout-detection
  - rt-detr
  - deformable-detr
  - paddleocr
  - pp-doclayout
  - mlx
  - apple-silicon
---

# PP-DocLayoutV3 — MLX

Apple **MLX** port of **PP-DocLayoutV3**, the RT-DETR-style document-layout
detector used by [model-pp-doclayoutv3](https://github.com/agentable/model-pp-doclayoutv3)
(image → 25-class layout regions with boxes, instance masks, and reading order).
A single `PP-DocLayoutV3.safetensors` (~142 MB fp32), **self-converted from the
ONNX graph** ([Bei0001/PP-DocLayoutV3-ONNX](https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX)
— the only export keeping all 4 heads) — no retraining, only a layout conversion
gated on numerical parity. The file also carries the AIFI positional embedding
and the baked decoder constants (order masks, coordinate grids) the MLX forward
needs.

## Architecture

RT-DETR family: **HGNetV2** CNN backbone → **hybrid encoder** (AIFI transformer
layer + CCFM FPN/PAN fusion + mask-feature head) → **6-layer transformer decoder**
with **multi-scale deformable attention** (300 queries) → 4 heads:

- `logits` (1,300,25) — class scores
- `pred_boxes` (1,300,4) — DETR cxcywh
- `out_masks` (1,300,200,200) — instance masks
- `order_logits` (1,300,300) — reading order

The MLX backend replays the whole ONNX graph faithfully op-by-op (convs via an
NHWC transpose-dance; the deformable GridSample, TopK query selection, GatherND,
and bilinear Resize all in MLX), so no part of the intricate MSDeformAttn / box
decode is re-derived by hand.

## Parity

The full forward matches the fp32 ONNX outputs (relative error vs each output's
own scale — the order/mask heads carry ±10000 mask-fill magnitudes):

| output | relative |
|---|---|
| logits | 1.6e-5 |
| pred_boxes | 2.2e-5 |
| out_masks | 1.3e-5 |
| order_logits | 3.2e-6 |

End-to-end, the full `Detect` pipeline (preprocess → forward → DB-style postproc)
produces **byte-identical layout elements** (class / box / reading order) to the
ONNX backend on a real document.

## Usage

Backs the MLX backend of `github.com/agentable/model-pp-doclayoutv3`. Build with
`-tags mlx` on Apple Silicon and place `PP-DocLayoutV3.safetensors` next to the
ONNX graph in the model directory. The default build stays pure-ONNX (no
MLX/Metal links).

## License

Apache-2.0, following upstream
[PaddleOCR / PaddleX](https://github.com/PaddlePaddle/PaddleOCR).