File size: 7,799 Bytes

d2446c1

---
library_name: transformers
license: apache-2.0
pipeline_tag: object-detection
base_model: docling-project/docling-layout-heron
base_model_relation: finetune
tags:
- document-layout-analysis
- finance
- rtdetr-v2
- cuda
- tf32
- ax-pdf
---

# AX-DocLayout-M v1.4 - H5 CUDA / cuDNN TF32

This is the owner-selected **H5 final EMA** model, fine-tuned from Docling Heron.
It detects page layout from decoded RGB rasters and maps17native classes to
**AX PDF SDK IDs1-17**, taxonomy `ax-document-unified-25/1`.
This repository is a separate Heron-derived model identity. It is not D6,
not the old11-class v1.0 model, and not the ax-doclayout Python package v1.4.0.

The owner selected H5 after qualitative comparison with D5 and D6 using a150-page
HKEX-oriented review packet. The number of pages actually inspected and structured
ratings were not recorded. This selection does not mean H5 wins every benchmark.

## Exact artifact and runtime

- Parent: `docling-project/docling-layout-heron` at
  `8f39ad3c0b4c58e9c2d2c84a38465abf757272d8`.
- H1:2048updates; H5:1024updates from finalH1EMA.42,761,907parameters.
- Weight SHA256: `a00ac115f20a9b00ababec445c526771e0d6dcf8ad41cad30049bcaa40fc0ec8`.
- FP32 safetensors and activations; no quantization, autocast or FP16 conversion.
- Native **640x640 direct resize**, original pinned image processor. Do not apply
  D5's1024letterbox preprocessing to this model.
- Qualified CUDA settings: **cuDNN TF32 allowed, CUDA matmul TF32 disabled**.
  The repository suffix does not mean all matrix multiplications use TF32.
  TF32 is an allowed arithmetic mode, not a different weight file format.
- Existing GPU evidence: PyTorch2.12.1+cu129, CUDA12.9, cuDNN92000,
  Transformers4.53.3, RTX PRO6000Blackwell. See runtime-profile.json.
- The standalone example is newly packaged and CPU-tested; it was not freshly
  rerun on CUDA for this publication. The underlying exact H5 checkpoint was
  evaluated on CUDA and its exported tensors were verified exactly after reload.

The independent controls for matmul and cuDNN are documented by
[PyTorch2.12](https://docs.pytorch.org/docs/2.12/notes/cuda.html#tensorfloat-32-tf32-on-ampere-and-later-devices).
An all-matmul-TF32, BF16, FP16, ONNX or TensorRT variant is not qualified here.

## Classes

Transformers returns native IDs0-16. Map them using
`sdk-taxonomy.json` -> `native_index_to_sdk_id`, not an11-class consumer enum.

| SDK ID | Name |
|---:|---|
|1|caption|
|2|footnote|
|3|formula|
|4|list_item|
|5|page_footer|
|6|page_header|
|7|picture|
|8|section_header|
|9|table|
|10|text|
|11|title|
|12|document_index|
|13|code|
|14|checkbox_selected|
|15|checkbox_unselected|
|16|form|
|17|key_value_region|

SDK0isreserved. SDK18-25 are unsupported by this head, although the SDK catalog
contains their definitions. `document_index` detects a TOC/index region; it does
not imply entry parsing, reading order or document hierarchy.

Fine-tuning targets cover SDK1-11. Loss masking protected unknown classes from
negative supervision, but **accuracy retention of classes12-17 is unmeasured**.

## Evaluation

All values below are percentages. AP averages IoU0.50:0.95; AP50/AP70/AP90 use the
named IoU threshold. AR100 is COCOmaximumrecall, not recall at confidence0.30.
These are previously exposed development panels, not independent final tests.

| Panel | Pages | H5 AP | D5 AP | H5 AP50 | H5 AP70 | H5 AP90 | H5 AR100 |
|---|---:|---:|---:|---:|---:|---:|---:|
|DocLayNet gold validation|6489|70.6572|76.1917|88.4057|78.1385|50.4501|82.8326|
|Financial validation subset|1745|61.5113|59.6151|74.8695|68.4569|43.3226|83.2427|
|FinTabNet validation|256|91.5856|91.2829|96.4231|95.6897|89.0473|97.1835|
|Exposed DocLayNet test|4999|73.3731|74.5767|87.7283|81.9561|54.2318|84.6841|
|Exposed synthetic test|100|32.1854|35.6326|41.6041|34.7108|23.4697|52.6072|
|Exposed FinTabNet test|255|87.8283|88.1351|92.4317|91.9150|85.4955|97.4497|

The financial validation panel is the `financial_reports` category within gold
validation, not the independently annotated HKEX corpus. No HKEX paired final-bot
label accuracy is claimed. The table-only panels score SDK9; synthetic scores
SDK5,7,8,9,10; the general panels score the shared11classes (absent GTclasses are
excluded). H5 uses640resize versus D5's1024letterbox and a different encoder
expansion; this is a practical comparison, not a pure pretraining ablation.

**Known limitation: document Title.** Gold-validation Title AP is35.9637 forH5
versus80.7431 forD5. H5 passes its improvement gate versusH0 but fails the prior
numeric D5 replacement gate. The owner nevertheless selected H5 for this release
on qualitative visual suitability. Historical gate results remain unchanged.
Title improvements will be separate candidates, not silent replacement of these
weights. See evaluation.json for complete metrics and per-classAP.

## Quick start

Use an immutable40-character Hub commit for `AX_DOCLAYOUT_REVISION`; never load
`main` or `latest` in a production consumer. The final publication receipt gives
the exact revision. A private repository requires your own Hugging Face access.
Do not embed tokens in source or command-line arguments.

Use Python3.12 or3.13. Install the matching CUDA12.9 PyTorch runtime using the
[official PyTorch distribution](https://download.pytorch.org/whl/cu129), then the
pinned requirements.txt from this snapshot. Example:

```bash
python -m pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu129
python -m pip install transformers==4.53.3 Pillow==12.3.0 numpy==2.5.2 safetensors==0.8.0 huggingface-hub==0.36.2 tokenizers==0.21.4
```

Download and verify the bundle at an exact revision:

```python
import hashlib, json, os, re
from pathlib import Path
from huggingface_hub import snapshot_download

revision = os.environ["AX_DOCLAYOUT_REVISION"]
assert re.fullmatch(r"[0-9a-f]{40}", revision)
folder = Path(snapshot_download(
    "AutomatosX/AXDocLayout-M-v1.4-CUDA-TF32",
    revision=revision,
))
manifest = json.loads((folder / "bundle-manifest.json").read_text())
for name, expected in manifest["files"].items():
    with (folder / name).open("rb") as stream:
        assert hashlib.file_digest(stream, "sha256").hexdigest() == expected["sha256"]
print(folder)
```

Run the included example using that verified local folder:

```bash
python /path/to/verified/snapshot/predict_image.py \
  --model-dir /path/to/verified/snapshot \
  --image page.png --output detections.json --device cuda --threshold 0.30
```

The example accepts raster images, not PDF files. It verifies the four core
artifact hashes and exact SDK mapping, refuses CUDA-to-CPU fallback, and returns
original-raster xyxy boxes. Native top300 postprocessing is retained. Threshold0.30
is a display convention, not calibrated probability. There is no addedNMS,
ensemble, PDF geometry replacement or bot curation. Explicit `--device cpu` is
available for inspection; it is not a CUDA performance claim.

This model is not loadable by the old11-class `AXDocLayout.from_pretrained` pin.
Consumers must explicitly adopt the17-class native model and SDK mapping.

## Training and attribution

H1/H5 continue the pinned Heron model with human DocLayNet gold replay and finance
pools (FinTabNet.c, IIIT-AR-13K inH1, and financial-layout-analysis synthetic data).
Training inputs and source documents are not redistributed. No new HKEX pseudo-label
fine-tuning was used in this H5 run. Complete and partial-label supervision masks
were preserved. The training corpus is not licensed by this weights repository.

Apache-2.0. See LICENSE and NOTICE. Upstream model:
[Docling Heron](https://huggingface.co/docling-project/docling-layout-heron).
Technical report: [Livathinos et al.,2025](https://arxiv.org/abs/2509.11720).