---
license: apache-2.0
base_model: jbarrow/FFDetr
pipeline_tag: object-detection
tags:
  - rf-detr
  - form-fields
  - document-ai
  - commonforms
---

# FFDetr — filled-form fine-tune

Fine-tune of [jbarrow/FFDetr](https://huggingface.co/jbarrow/FFDetr) (RF-DETR Medium) that detects
form fields **even when they already contain values** — typed text, handwriting, checkbox marks,
signatures — on both born-digital and scanned pages. The base model only detects blank fields
reliably; on filled fields its recall collapses to near zero.

## Classes

Same head layout and index order as the base model:

| id | class |
|----|-------|
| 0 | Text (TextBox) |
| 1 | CheckBox (ChoiceButton) |
| 2 | Signature |

## Results

Evaluated on a held-out 300-image synthetic validation set (including 60 field-less negative
pages) built from the CommonForms `valid` split — document-disjoint from training, with disjoint
fill-content generator profiles. IoU 0.5, confidence 0.5:

| Slice | Base P / R / F1 | This model P / R / F1 |
|---|---|---|
| filled_flat | 0.34 / 0.05 / 0.09 | **0.77 / 0.71 / 0.74** |
| filled_scan | 0.62 / 0.02 / 0.04 | **0.96 / 0.86 / 0.91** |
| hybrid_flat | 0.70 / 0.42 / 0.53 | **0.80 / 0.75 / 0.78** |
| blank_flat | 0.90 / 0.78 / 0.83 | 0.89 / 0.76 / 0.82 |
| blank_scan | 0.87 / 0.82 / 0.84 | 0.92 / 0.71 / 0.80 |
| **overall** | 0.84 / 0.53 / 0.65 | **0.87 / 0.77 / 0.82** |

- Overall AP@0.5: 0.635 (base) → **0.815**
- False positives on field-less pages: 1.47 → **1.32** per negative page
- On pages that do contain fields, absolute FP rises 438 → 502 (+15%) because the model now
  makes far more detections there — true positives rise 2669 → 3833 (+44%) — so the share of
  false boxes among all detections **falls from 16.5% to 13.2%**, and precision improves on
  every slice except blank_flat (flat: 0.90 → 0.89). The base model's low absolute FP on
  filled pages is an artifact of it barely detecting anything there.
- Inference latency unchanged (~57 ms/img on A100 at resolution 1024; same architecture,
  weights-only checkpoint)

On real filled documents (scanned bank forms with handwriting, typewriter-filled government
forms), the fine-tune matches the base model on strongly-outlined blank fields and additionally
detects value-bearing and graphics-obstructed fields the base model misses, without producing
detections on field-less pages.

## Training data

8,000 images derived from 4,000 pages of [jbarrow/CommonForms](https://huggingface.co/datasets/jbarrow/CommonForms)
(3,200 pages with fields + 800 negatives, revision-pinned). Every page is paired with a synthetic
augmented counterpart: values rendered into field boxes (multilingual text incl. CJK/Arabic/Cyrillic,
choice marks, signature strokes; partial-fill variants) and/or scan degradation
(rotation, blur, contrast shift, noise, JPEG artifacts).

## Recipe

Initialized from the base FFDetr checkpoint; detection head reinitialized for 3 classes.
8 epochs, lr 1e-5, batch 4 with grad-accum 4 (effective 16), resolution 1024, on 1× A100 40GB.

## Usage

Pin a revision in production so a re-upload can never silently change your model:

```python
from huggingface_hub import hf_hub_download
from commonforms.inference import FFDetrDetector

path = hf_hub_download(
    "pdf-net/ffdetr-filled",
    "FFDetr.pth",
    revision="e4fc1a8654915fe2f9bae63327062bc15711ecd6",
)
detector = FFDetrDetector(path)
```

The checkpoint is a drop-in replacement for the base model: same class indices, same input
resolution, same inference code path.

## Known limitations

- Detected boxes follow the CommonForms field-rect convention (the fillable area). Ink written
  outside that area — on or below an underline, spilling past the box — may not be fully covered
  by the box. A future revision trains with overflowing fills and ink-union target boxes.
- `blank_scan` recall is ~0.11 below the base model at conf 0.5 (precision is higher); expected
  to recover with the larger final training run.

## Status

Interim checkpoint from an 8k-image pilot. A final fine-tune on a larger dataset is planned and
will be published here as a new revision — pin a specific commit revision in production.

License: Apache-2.0, same as the base model (jbarrow/FFDetr), the CommonForms dataset, and the
rfdetr training library.