File size: 2,483 Bytes
01ff83c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
license: apache-2.0
library_name: mlx
pipeline_tag: object-detection
tags:
- document-layout
- layout-detection
- rt-detr
- deformable-detr
- paddleocr
- pp-doclayout
- mlx
- apple-silicon
---
# PP-DocLayoutV3 — MLX
Apple **MLX** port of **PP-DocLayoutV3**, the RT-DETR-style document-layout
detector used by [model-pp-doclayoutv3](https://github.com/agentable/model-pp-doclayoutv3)
(image → 25-class layout regions with boxes, instance masks, and reading order).
A single `PP-DocLayoutV3.safetensors` (~142 MB fp32), **self-converted from the
ONNX graph** ([Bei0001/PP-DocLayoutV3-ONNX](https://huggingface.co/Bei0001/PP-DocLayoutV3-ONNX)
— the only export keeping all 4 heads) — no retraining, only a layout conversion
gated on numerical parity. The file also carries the AIFI positional embedding
and the baked decoder constants (order masks, coordinate grids) the MLX forward
needs.
## Architecture
RT-DETR family: **HGNetV2** CNN backbone → **hybrid encoder** (AIFI transformer
layer + CCFM FPN/PAN fusion + mask-feature head) → **6-layer transformer decoder**
with **multi-scale deformable attention** (300 queries) → 4 heads:
- `logits` (1,300,25) — class scores
- `pred_boxes` (1,300,4) — DETR cxcywh
- `out_masks` (1,300,200,200) — instance masks
- `order_logits` (1,300,300) — reading order
The MLX backend replays the whole ONNX graph faithfully op-by-op (convs via an
NHWC transpose-dance; the deformable GridSample, TopK query selection, GatherND,
and bilinear Resize all in MLX), so no part of the intricate MSDeformAttn / box
decode is re-derived by hand.
## Parity
The full forward matches the fp32 ONNX outputs (relative error vs each output's
own scale — the order/mask heads carry ±10000 mask-fill magnitudes):
| output | relative |
|---|---|
| logits | 1.6e-5 |
| pred_boxes | 2.2e-5 |
| out_masks | 1.3e-5 |
| order_logits | 3.2e-6 |
End-to-end, the full `Detect` pipeline (preprocess → forward → DB-style postproc)
produces **byte-identical layout elements** (class / box / reading order) to the
ONNX backend on a real document.
## Usage
Backs the MLX backend of `github.com/agentable/model-pp-doclayoutv3`. Build with
`-tags mlx` on Apple Silicon and place `PP-DocLayoutV3.safetensors` next to the
ONNX graph in the model directory. The default build stays pure-ONNX (no
MLX/Metal links).
## License
Apache-2.0, following upstream
[PaddleOCR / PaddleX](https://github.com/PaddlePaddle/PaddleOCR).
|