agentable/pp-doclayoutv3-mlx

PP-DocLayoutV3 — MLX

Apple MLX port of PP-DocLayoutV3, the RT-DETR-style document-layout detector used by model-pp-doclayoutv3 (image → 25-class layout regions with boxes, instance masks, and reading order). A single PP-DocLayoutV3.safetensors (~142 MB fp32), self-converted from the ONNX graph (Bei0001/PP-DocLayoutV3-ONNX — the only export keeping all 4 heads) — no retraining, only a layout conversion gated on numerical parity. The file also carries the AIFI positional embedding and the baked decoder constants (order masks, coordinate grids) the MLX forward needs.

Architecture

RT-DETR family: HGNetV2 CNN backbone → hybrid encoder (AIFI transformer layer + CCFM FPN/PAN fusion + mask-feature head) → 6-layer transformer decoder with multi-scale deformable attention (300 queries) → 4 heads:

• logits (1,300,25) — class scores
• pred_boxes (1,300,4) — DETR cxcywh
• out_masks (1,300,200,200) — instance masks
• order_logits (1,300,300) — reading order

The MLX backend replays the whole ONNX graph faithfully op-by-op (convs via an NHWC transpose-dance; the deformable GridSample, TopK query selection, GatherND, and bilinear Resize all in MLX), so no part of the intricate MSDeformAttn / box decode is re-derived by hand.

Parity

The full forward matches the fp32 ONNX outputs (relative error vs each output's own scale — the order/mask heads carry ±10000 mask-fill magnitudes):

output	relative
logits	1.6e-5
pred_boxes	2.2e-5
out_masks	1.3e-5
order_logits	3.2e-6

End-to-end, the full Detect pipeline (preprocess → forward → DB-style postproc) produces byte-identical layout elements (class / box / reading order) to the ONNX backend on a real document.

Usage

Backs the MLX backend of github.com/agentable/model-pp-doclayoutv3. Build with -tags mlx on Apple Silicon and place PP-DocLayoutV3.safetensors next to the ONNX graph in the model directory. The default build stays pure-ONNX (no MLX/Metal links).

License

Apache-2.0, following upstream PaddleOCR / PaddleX.