| --- |
| license: apache-2.0 |
| base_model: jbarrow/FFDetr |
| datasets: |
| - jbarrow/CommonForms |
| pipeline_tag: object-detection |
| library_name: onnx |
| tags: |
| - form-field-detection |
| - rf-detr |
| - onnx |
| --- |
| |
| # FFDetr, ONNX at 1024x1024 |
|
|
| [jbarrow/FFDetr](https://huggingface.co/jbarrow/FFDetr) converted to ONNX. FFDetr is Joe Barrow's form field |
| detector, an RF-DETR Medium model trained on the [CommonForms](https://huggingface.co/datasets/jbarrow/CommonForms) |
| dataset. Given an image of a form page, it finds the text boxes, checkboxes and signature fields. |
|
|
| DocSpring uses this file to detect the fields on uploaded PDFs. We publish it so our builds can download one pinned |
| file instead of exporting it from PyTorch each time, and so anyone else who wants to run FFDetr without PyTorch can |
| use it too. |
|
|
| ## What the conversion changed |
|
|
| The weights are FFDetr's, unchanged. The export differs from rfdetr's defaults in two ways: |
|
|
| - **Input size 1024x1024.** The checkpoint was trained at 1024x1024 (its saved training arguments say so), but |
| rfdetr's `RFDETRMedium` runs at 576 by default. At 1024 it finds noticeably more fields (see below). |
| - **Position embeddings stored as a constant.** At any size but 576, the DINOv2 backbone resizes its position |
| embeddings with antialiased bicubic interpolation, which ONNX can't express. The result depends only on the input |
| size, so it was computed once in PyTorch and stored in the graph. |
|
|
| ONNX Runtime gives the same boxes as the PyTorch model. |
|
|
| | File | SHA-256 | |
| | ------------------ | ------------------------------------------------------------------ | |
| | `ffdetr-1024.onnx` | `4b98b3e25341f64a289034a8f7a42d945c7e528aae23fa912eaa2ba5f610c6e2` | |
|
|
| Exported from `FFDetr.pth` at revision `56f4e4235e28dcb2953513dc020bb191a2f54cfe` of jbarrow/FFDetr, with rfdetr |
| 1.6.5.post0 and torch 2.11.0, ONNX opset 17. |
|
|
| ## Input and output |
|
|
| - `input`: float32 `[1, 3, 1024, 1024]`. The page as RGB scaled to 0-1, resized to 1024x1024 with an antialiased |
| bilinear filter (as torchvision and Pillow do it), then normalised with the ImageNet mean `[0.485, 0.456, 0.406]` |
| and standard deviation `[0.229, 0.224, 0.225]`. |
| - `dets`: float32 `[1, 300, 4]`. One box per query as centre x, centre y, width and height, from 0 to 1 relative to |
| the page. |
| - `labels`: float32 `[1, 300, 91]`. Class logits. FFDetr uses the first three classes: 0 text box, 1 choice button |
| (a checkbox or radio button), 2 signature. |
|
|
| To get boxes the way rfdetr does: apply a sigmoid to the logits, take the 300 best query and class scores, keep the |
| ones above a confidence threshold, scale the boxes to the page and remove overlaps with class-agnostic NMS. DocSpring |
| uses a confidence of 0.3 and an NMS IoU of 0.1. |
|
|
| ## Accuracy |
|
|
| On the 164 pages of the CommonForms test split (5,294 fields), counting a field as found when a box overlaps it at |
| IoU 0.3 or more: |
|
|
| | Input size | Fields found | |
| | ---------------------- | -----------: | |
| | 576 (rfdetr's default) | 83.5% | |
| | 1024 (this file) | 91.5% | |
|
|
| ## Licence and credit |
|
|
| Apache-2.0, the same as FFDetr. The model and the CommonForms dataset are by Joe Barrow: |
| [CommonForms: A Large, Diverse Dataset for Form Field Detection](https://arxiv.org/abs/2509.16506). FFDetr was |
| trained with Roboflow's [RF-DETR](https://github.com/roboflow/rf-detr) (Apache-2.0). DocSpring converted the file to |
| ONNX as described above. |
|
|
