README.md

---
language:
- vi
license: apache-2.0
library_name: d-fine
pipeline_tag: object-detection
tags:
- d-fine
- object-detection
- document-layout
- fine-tuned
- hgnetv2

---

<div align="center">

# 🔍 D-FINE X — Object Detection for Document Layout

**D-FINE HGNetv2-X fine-tuned for document layout analysis**

[![license](https://img.shields.io/badge/license-Apache%202.0-green.svg)](https://github.com/Peterande/D-FINE/blob/master/LICENSE)
[![mAP50:95](https://img.shields.io/badge/mAP@50%3A95-83.75%25-blue)]()
[![mAP50](https://img.shields.io/badge/mAP@50-91.82%25-blue)]()
[![Python](https://img.shields.io/badge/python-3.8%2B-blue)]()

</div>

---

## Overview

Mô hình **D-FINE HGNetv2-X** được fine-tune cho bài toán nhận diện bố cục tài liệu (Document Layout Analysis). Mô hình sử dụng kiến trúc D-FINE tiên tiến với cơ chế Fine-grained Distribution Refinement (FDR) để tinh chỉnh bounding box, kết hợp backbone HGNetv2-B5 mạnh mẽ.

| Property | Value |
|---|---|
| Base model | D-FINE (HGNetv2-X) |
| Task | Object Detection |
| Domain | Document Layout |
| Input resolution | 640 × 640 |
| Training | 80 epochs (72 Stage 1 + 8 Stage 2) |

---

## Classes

| ID | Label | Description |
|:---:|---|---|
| 0 | `Object-detection` | Generic detectable region |
| 1 | `Figure` | Diagrams, illustrations, charts |
| 2 | `Icon` | Small symbolic graphics |
| 3 | `Table` | Tabular data structures |

---

## Benchmark

### ✅ Best validation metrics (Stage 2)

| Metric | Value |
|---|---:|
| mAP @ \[0.50 : 0.95\] | **0.8375** |
| mAP @ 0.50 | **0.9182** |
| mAP @ 0.75 | **0.8966** |
| AP (Medium) | **0.8703** |
| AP (Large) | **0.8384** |
| AR @ 100 | **0.9348** |

---

## Usage

### Option 1 — HuggingFace Transformers (recommended)

```bash
pip install transformers torch torchvision
# Clone D-FINE source code (bắt buộc)
git clone https://github.com/Peterande/D-FINE.git
cd D-FINE
pip install -r requirements.txt
```

```python
from transformers import AutoModel, AutoImageProcessor
from PIL import Image

processor = AutoImageProcessor.from_pretrained(
    "ducnhan0804/dfine-x-obj-detection",
    trust_remote_code=True
)
model = AutoModel.from_pretrained(
    "ducnhan0804/dfine-x-obj-detection",
    trust_remote_code=True
)

image = Image.open("path/to/your/document.png")
inputs = processor.preprocess(image)
outputs = model(inputs["images"], threshold=0.4)
results = processor.post_process_object_detection(outputs, threshold=0.4)

# results[0]["boxes"]  → xyxy coordinates (tensor)
# results[0]["scores"] → confidence scores (tensor)
# results[0]["labels"] → class ids (tensor)
print(results)
```

> **Note:** `trust_remote_code=True` is required because this model uses custom `modeling_dfine.py`. The checkpoint (~1GB) is downloaded automatically to `~/.cache/huggingface/` on first run.

> **Important:** You must run this code from the D-FINE root directory so that the source code (`src/`) can be imported.

### Option 2 — D-FINE directly

```bash
git clone https://github.com/Peterande/D-FINE.git
cd D-FINE
pip install -r requirements.txt
```

```python
# Download checkpoint from HuggingFace
from huggingface_hub import hf_hub_download

weights = hf_hub_download(
    repo_id="ducnhan0804/dfine-x-obj-detection",
    filename="best_stg2.pth"
)

# Run inference using D-FINE's built-in script
# python tools/inference/torch_inf.py -c configs/dfine/custom/dfine_hgnetv2_x_custom.yml -r <weights_path> -i <image_path> -d cuda:0
```

---

## Training Configuration

```yaml
model:
  architecture: D-FINE (HGNetv2-X)
  backbone: HGNetv2-B5

training:
  total_epochs: 35
  stage1_epochs: 30 (heavy augmentation)
  stage2_epochs: 5 (fine-tuning, no heavy augmentation)
  resolution: 640
  device: cuda

optimizer:
  type: AdamW
  lr: 0.00025
  backbone_lr: 0.0000025
```

### Dataset split (COCO format)

```
datasets/
├── train/
│   ├── _annotations.coco.json
│   └── *.jpg
└── valid/
    ├── _annotations.coco.json
    └── *.jpg
```

---

## Repository Structure

```
ducnhan0804/dfine-x-obj-detection
├── README.md                        # Model card
├── config.json                      # HuggingFace model config
├── preprocessor_config.json         # Image processor config
├── modeling_dfine.py                # Custom modeling code (trust_remote_code)
├── best_stg2.pth                    # Best checkpoint (Stage 2)
├── dfine_hgnetv2_x_custom.yml      # D-FINE training config
└── custom_detection.yml             # Dataset config
```

---

## Limitations

- Trained on a specific document layout dataset; generalization to other document types is not guaranteed.
- Requires D-FINE source code to be cloned and available locally for inference.
- The checkpoint file is ~1GB in size.

---

## License

This model is released under the **Apache License 2.0**, inherited from the D-FINE base model.

---

## Citation

**Base model (D-FINE):**

```bibtex
@article{peng2024d,
  title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author={Peng, Yansong and Songtao, Liu and others},
  journal={arXiv preprint arXiv:2407.06537},
  year={2024}
}
```