| --- |
| license: agpl-3.0 |
| library_name: ax-doclayout |
| pipeline_tag: object-detection |
| tags: |
| - document-layout |
| - object-detection |
| - onnx |
| - safetensors |
| - cuda |
| - tf32 |
| --- |
| |
| # AXDocLayout-M v1.3 CUDA TF32 |
|
|
| AX DocLayout detects layout regions in a decoded RGB page image and returns |
| class labels, confidence scores and boxes in the original raster coordinates. |
| This model updates the weights while retaining the v1.2 CUDA TF32 inference |
| standard. The predecessor remains available at |
| [AXDocLayout-M v1.2 CUDA TF32](https://huggingface.co/AutomatosX/AXDocLayout-M-v1.2-CUDA-TF32). |
|
|
| ## Model and runtime |
|
|
| Architecture: RT-DETRv2 `rtdetrv2_r50vd_m`. Stored weights and ONNX inputs are |
| FP32. CUDA TF32 is the execution policy, not a BF16 weight format. The bundle |
| contains FP32 safetensors, a standard-operator ONNX graph (opset 16) and the |
| complete inference source. Integer quantization is not used. |
|
|
| The ONNX backend requires Linux x86-64, Python 3.12 or 3.13 and an NVIDIA |
| TF32-capable Ampere or newer device with a compatible CUDA 12.8 driver. Install |
| the pinned ONNX Runtime GPU 1.23.2, CUDA 12.8 and cuDNN 9 dependencies through |
| the `onnx` extra. CUDAExecutionProvider is required, TF32 is explicitly enabled, |
| and startup checks verify recognized compute placement. Shape/control nodes |
| may use the CPU. An unavailable CUDA backend raises an error. |
|
|
| A separate explicit PyTorch CPU FP32 reference is available with the `torch` |
| extra and `backend="pytorch"`. Vectorized postprocessing follows v1.2. |
|
|
| ## Input and output |
|
|
| Input is one HWC uint8 RGB raster. Preserve aspect ratio, resize the long edge |
| to 1024, center-pad to 1024 x 1024 with white, and convert to NCHW float32 in |
| [0, 1]. Contract: `rgb-letterbox-1024/1`. |
|
|
| Direct ONNX inputs are `images` float32 [N, 3, 1024, 1024] and |
| `orig_target_sizes` int64 [N, 2], always [1024, 1024] per image. Outputs are `labels`, |
| `boxes`, `scores`. Raw boxes are letterbox-space xyxy; the Python API undoes |
| padding/scaling and clips boxes to the page. Default score floor is 0.30; |
| invalid or reserved labels are removed. |
|
|
| The JSON schema is `ax-doclayout.detection/1`; taxonomy is `ax-doclayout-11/1`. |
| Boxes use half-open pixel edges, origin top-left, x rightward, y downward. |
|
|
| | ID | Label | |
| |---|---| |
| | 1 | `caption` | |
| | 2 | `footnote` | |
| | 3 | `formula` | |
| | 4 | `list_item` | |
| | 5 | `page_footer` | |
| | 6 | `page_header` | |
| | 7 | `picture` | |
| | 8 | `section_header` | |
| | 9 | `table` | |
| | 10 | `text` | |
| | 11 | `title` | |
|
|
| ## Qualification and limitations |
|
|
| The following measurements use internal development cohorts on one NVIDIA |
| RTX PRO 6000 Blackwell Server Edition. They are not independent acceptance |
| or a universal benchmark superiority claim. The predecessor and this release |
| were measured on the same host with the same CUDA TF32 runtime. AP uses |
| score floor 0.001, COCO IoU 0.50:0.95, area all and maxDets 100. |
|
|
| | Development cohort | Model | AP | AP50 | AP70 | AP90 | |
| |---|---|---:|---:|---:|---:| |
| |General, 6489 pages, 11 classes|v1.2|76.2397|90.9894|86.8779|55.9050| |
| |General, 6489 pages, 11 classes|v1.3|76.1623|90.9487|86.7953|55.8282| |
| |Financial layout, 100 pages, table|v1.2|66.7463|78.4563|71.8082|54.7579| |
| |Financial layout, 100 pages, table|v1.3|69.4385|80.4662|74.8691|58.1819| |
| |Financial tables, 256 pages|v1.2|90.8539|95.5602|94.5320|89.6775| |
| |Financial tables, 256 pages|v1.3|90.9530|95.3348|94.2652|89.3608| |
|
|
| These cohorts show mixed tradeoffs: financial-layout table AP improves, while |
| some general and high-IoU table metrics decrease. Validate on representative |
| inputs before deployment. Dataset definitions differ from other comparisons; |
| their numbers should not be mixed. |
|
|
| Public API latency uses the same 100 decoded RGB pages, three counterbalanced |
| rounds per model, 10 untimed initial calls per session and the default score floor 0.30. |
| The timed region ends at PageStructure and excludes raster decoding, JSON |
| serialization and worker transport. Only the production session is retained. |
|
|
| | Same-host CUDA TF32 API | Median ms | p95 ms | Mean ms | |
| |---|---:|---:|---:| |
| |v1.2|17.959|18.268|17.998| |
| |v1.3|17.945|18.563|18.013| |
|
|
| The ONNX operator structure and numerical inference code follow v1.2; model |
| weights and immutable artifact bindings are updated. Conversion checks passed |
| against native FP32 reference weights. The CUDA-only inference environment |
| contains no PyTorch. A 128-page check verified preprocessing and API equivalence |
| using captured same-invocation outputs. GPU execution can vary slightly across |
| invocations; no bitwise repeatability guarantee is made. |
|
|
| The detector accepts RGB page rasters. It does not open PDFs, perform OCR, |
| extract text, predict reading order or hierarchy, or reconstruct table cells. |
|
|
| ## License |
|
|
| The model weights and modified inference source are **AGPL-3.0-only**. |
| Commercial use is permitted subject to the license terms, including applicable |
| source-disclosure obligations. See LICENSE and NOTICE. Preserved Apache-2.0 |
| component notices and rights remain in NOTICE. The complete inference source |
| is provided in `inference/`. |
|
|
| See the root README for immutable source installation and model revisions. |
|
|
