Instructions to use ArgoSA/D-FINE-seg with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use ArgoSA/D-FINE-seg with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
D-FINE-seg
Real-time object detection, keypoint estimation, instance and semantic segmentation - one codebase, one config flag (task: detect | segment | sem_seg | kpts), five model sizes (N -> X), and fused end-to-end deployment graphs for ONNX, TensorRT, OpenVINO, CoreML and LiteRT.
This repo hosts the released weights; the code, full benchmark tables and docs live in the GitHub repo. See every task on real footage at dfine-seg.argoai.space. π Paper (arXiv:2602.23043) Β· π¬ Video tutorial Β· π§ͺ Colab
One frame, three tasks, one config flag.
Cityscapes semantic segmentation (released)
Five semantic-segmentation checkpoints trained on Cityscapes at native 1024x2048, benchmarked against 23 official checkpoints of four other families - each through its own framework's shipped code, one validator, 500 val images, TensorRT fp16, batch 1, RTX 5070 Ti. Full 28-model table, per-class IoU and protocol: cityscapes-benchmark.
| model | params (M) | mIoU | pixel acc | engine ms | e2e ms |
|---|---|---|---|---|---|
| D-FINE-seg-N | 3.36 | 77.99 | 96.03 | 3.02 | 4.21 |
| D-FINE-seg-S | 7.99 | 81.12 | 96.47 | 5.38 | 6.37 |
| D-FINE-seg-M | 15.95 | 81.41 | 96.55 | 7.42 | 8.26 |
| D-FINE-seg-L | 24.97 | 83.80 | 96.93 | 8.98 | 9.77 |
| D-FINE-seg-X | 56.3 | 84.84 | 97.10 | 14.18 | 14.64 |
| YOLO26x-sem | 40.13 | 83.58 | 96.79 | 10.37 | 13.58 |
| EfficientViT-L2 | 52.61 | 83.20 | 96.87 | 9.56 | 19.87 |
| Mask2Former-Swin-L | 215.46 | 83.33 | 96.91 | 64.51 | 94.07 |
| SegFormer-B5 | 84.61 | 81.41 | 96.62 | 38.81 | 42.12 |
D-FINE-seg-X: 84.84 mIoU at 14.6 ms end-to-end, the highest mIoU of the 28 models - +1.26 mIoU over YOLO26-x, and +1.5 mIoU over Mask2Former Swin-L with 6.4x lower latency and a quarter of the parameters. D-FINE-seg-L matches or beats YOLO26-x on every axis.
Load with one line (weights auto-download):
model = load_model("x", task="sem_seg") # names and 1024x2048 input size come from the checkpoint
They are also a strong init for your own driving/outdoor data: point train.pretrained_model_path at one with a different label_to_name and the class heads re-initialize while the rest keeps the Cityscapes training (N initialized from COCO, S/M/L/X from obj2coco). A model fine-tuned from them is still derived from Cityscapes and inherits its non-commercial terms (see License) - for commercial use, start from the COCO / obj2coco weights instead.
COCO person keypoints (released)
dfine_kpts_s_coco.pt: the S model trained on COCO person keypoints (17 points). COCO val2017 at 640, official COCO AP; end-to-end latency with each framework's own pipeline, TensorRT fp16, batch 1, RTX 5070 Ti.
| model | keypoint AP | AP50 / AP75 | person box AP | e2e ms |
|---|---|---|---|---|
| D-FINE-seg S (40 epochs) | 65.4 | 88.0 / 71.1 | 64.2 | 2.10 |
| YOLO26s-pose (released, 300 epochs, Objects365 init) | 62.7 | 85.8 / 68.8 | 54.3 | 2.16 |
model = load_model("s", task="kpts") # out["kpts"]: [N, 17, 4] = x, y, presence, visible
The keypoint head is not tied to human pose: every detection predicts K points with a presence and a visibility score, so it learns whatever points your labels define (YOLO-pose or COCO keypoint labels). Use these weights as the init for your own person keypoint data, or start from the detection weights for other objects. Larger sizes are coming.
Highlights
- Four tasks, one model family - DETR-style detection (queries -> boxes), plus a lightweight Mask-DINO-style mask head for instance segmentation, a query-free head for semantic segmentation, and a keypoint head that predicts K points per detection (refined by a keypoint decoder, one token per point).
- Five sizes (N/S/M/L/X) - pretrained weights for every task: detection on COCO (plain and Objects365-pretrained), COCO instance segmentation, Cityscapes semantic segmentation, COCO person keypoints (S for now).
- Multi-backend deployment - one export pipeline produces ONNX / TensorRT / OpenVINO / CoreML / LiteRT graphs with the postprocessor fused; INT8 quantization (OpenVINO, CoreML, LiteRT); ByteTrack tracking; SAM3 auto-labeling; Gradio demo.
- Fine-tuning - on 640 Cityscapes, beats Ultralytics YOLO26 and RF-DETR on detection & instance-segmentation F1; on TACO and VisDrone, beats YOLO26 across all sizes (TensorRT FP16, end-to-end protocol).
- Not a fork - the detection core follows the D-FINE paper; segmentation heads, training, export and inference were reimplemented from scratch, with the mask-head design following the Mask DINO paradigm.
Checkpoints
All files are PyTorch .pt; weights auto-download on first use (no manual download needed).
| Task | Files | Params (M) |
|---|---|---|
| Detection (COCO-pretrained) | dfine_{n,s,m,l,x}_coco.pt |
3.8 / 10.4 / 19.7 / 31.4 / 62.9 |
| Detection (Objects365 -> COCO) | dfine_{s,m,l,x}_obj2coco.pt |
10.4 / 19.7 / 31.4 / 62.9 |
| Instance segmentation (COCO-pretrained) | dfine_seg_{n,s,m,l,x}_coco.pt |
4.2 / 12.0 / 21.3 / 33.0 / 64.6 |
| Semantic segmentation (Cityscapes) | dfine_sem_{n,s,m,l,x}_cityscapes.pt |
3.36 / 7.99 / 15.95 / 24.97 / 56.3 |
| Keypoints (COCO person, 17 points) | dfine_kpts_s_coco.pt |
13.5 |
Usage
pip install dfine-seg # inference + training
pip install 'dfine-seg[all]' # + every export backend, SAM3, Gradio demo
from dfine_seg import load_model, read_image
model = load_model("s") # COCO detection, weights auto-downloaded
model = load_model("s", task="segment") # COCO instance segmentation
model = load_model("x", task="sem_seg") # Cityscapes semantic segmentation
model = load_model("s", task="kpts") # COCO person keypoints (17 points)
model = load_model("path/to/model.pt") # your checkpoint - size/task/classes auto-detected
model = load_model("path/to/model.engine") # any exported artifact, picked by extension
out = model(read_image("path/to/image.jpg"))[0]
print(out["boxes"], out["scores"], [model.names[int(i)] for i in out["labels"]])
Train, export and bench from a config file:
dfine init # writes ./config.yaml (task: detect | segment | sem_seg | kpts)
dfine split && dfine train && dfine export && dfine bench
Everything else - data formats, config keys, export knobs, inference flags - is in the GitHub README.
Example outputs
Detection on VisDrone (aerial) Β· instance segmentation on TACO (waste).
Architecture
Notes
- Weights are an init, not a final model - the COCO checkpoints cover detection, instance segmentation and person keypoints, the Cityscapes ones semantic segmentation; fine-tune on your own data for the task and classes you need.
load_modeldetects task, classes and input size from the weights. - TensorRT engines are GPU-specific - re-export on the target GPU. INT8 quantization shows a noticeable F1 drop on segmentation; FP16 is the recommended trade-off.
- Mosaic augmentation (
mosaic_augs.mosaic_prob) is not recommended for the segmentation task - lower it toward 0 if masks look wrong. - Older fine-tuning benchmarks (640 Cityscapes detection/instance segmentation, VisDrone, TACO, COCO-style APs, cross-format latency): GitHub README -> Benchmarks.
License
- The code and the COCO-pretrained checkpoints (detection, instance segmentation, person keypoints) are Apache-2.0.
- The Cityscapes semantic-segmentation checkpoints (
dfine_sem_*_cityscapes.pt) are derivative works of the Cityscapes dataset and inherit its terms of use: non-commercial purposes only (academic research, teaching, scientific publications, personal experimentation), no commercial gain. Train on your own data and the resulting model is yours to ship - the code stays Apache-2.0 (no AGPL).
Citation
@article{saakyan2026dfineseg,
title = {D-FINE-seg: Object Detection and Instance Segmentation Framework with Multi-Backend Deployment},
author = {Saakyan, Argo and Solntsev, Dmitry},
journal = {arXiv preprint arXiv:2602.23043},
year = {2026},
eprint = {2602.23043}
}
@misc{peng2024dfine,
title = {D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
author = {Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
year = {2024},
eprint = {2410.13842},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
- Downloads last month
- 1,087
Space using ArgoSA/D-FINE-seg 1
Papers for ArgoSA/D-FINE-seg
D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
Evaluation results
- mIoU (val) on Cityscapesvalidation set self-reported84.840
- Pixel acc (val) on Cityscapesvalidation set self-reported97.100
- Keypoint AP (val2017) on COCOself-reported65.400
- Person box AP (val2017) on COCOself-reported64.200