Improve model card and add metadata
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,10 +1,82 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
tags:
|
| 3 |
- model_hub_mixin
|
| 4 |
- pytorch_model_hub_mixin
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: object-detection
|
| 5 |
tags:
|
| 6 |
- model_hub_mixin
|
| 7 |
- pytorch_model_hub_mixin
|
| 8 |
+
- object-detection
|
| 9 |
+
- detrs
|
| 10 |
+
- dinov3
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# DEIMv2: Real-Time Object Detection Meets DINOv3
|
| 14 |
+
|
| 15 |
+
DEIMv2 is an evolution of the DEIM framework that leverages features from DINOv3. It spans eight model sizes (from Atto to X), covering GPU, edge, and mobile deployment scenarios. DEIMv2 achieves state-of-the-art results by combining DINOv3-pretrained backbones with a Spatial Tuning Adapter (STA) for larger models, and using pruned HGNetv2 for ultra-lightweight variants.
|
| 16 |
+
|
| 17 |
+
- **Paper:** [Real-Time Object Detection Meets DINOv3](https://huggingface.co/papers/2509.20787)
|
| 18 |
+
- **Repository:** [GitHub - DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2)
|
| 19 |
+
- **Project Page:** [DEIMv2 Project](https://intellindust-ai-lab.github.io/projects/DEIMv2/)
|
| 20 |
+
|
| 21 |
+
## Model Zoo (COCO)
|
| 22 |
+
|
| 23 |
+
| Model | AP | #Params | GFLOPs |
|
| 24 |
+
| :---: | :---: | :---: | :---: |
|
| 25 |
+
| **Atto** | 23.8 | 0.5M | 0.8 |
|
| 26 |
+
| **Femto** | 31.0 | 1.0M | 1.7 |
|
| 27 |
+
| **Pico** | 38.5 | 1.5M | 5.2 |
|
| 28 |
+
| **N** | 43.0 | 3.6M | 6.8 |
|
| 29 |
+
| **S** | 50.9 | 9.7M | 25.6 |
|
| 30 |
+
| **M** | 53.0 | 18.1M | 52.2 |
|
| 31 |
+
| **L** | 56.0 | 32.2M | 96.7 |
|
| 32 |
+
| **X** | 57.8 | 50.3M | 151.6 |
|
| 33 |
+
|
| 34 |
+
## Usage
|
| 35 |
+
|
| 36 |
+
This model can be loaded using the `PyTorchModelHubMixin` integration. Please ensure you have the necessary components from the official [DEIMv2 repository](https://github.com/Intellindust-AI-Lab/DEIMv2) in your Python path.
|
| 37 |
+
|
| 38 |
+
```python
|
| 39 |
+
import torch.nn as nn
|
| 40 |
+
from huggingface_hub import PyTorchModelHubMixin
|
| 41 |
+
|
| 42 |
+
from engine.backbone import HGNetv2, DINOv3STAs
|
| 43 |
+
from engine.deim import HybridEncoder, LiteEncoder
|
| 44 |
+
from engine.deim import DFINETransformer, DEIMTransformer
|
| 45 |
+
from engine.deim.postprocessor import PostProcessor
|
| 46 |
+
|
| 47 |
+
class DEIMv2(nn.Module, PyTorchModelHubMixin):
|
| 48 |
+
def __init__(self, config):
|
| 49 |
+
super().__init__()
|
| 50 |
+
# Select backbone based on the configuration
|
| 51 |
+
if "HGNetv2" in config:
|
| 52 |
+
self.backbone = HGNetv2(**config["HGNetv2"])
|
| 53 |
+
else:
|
| 54 |
+
self.backbone = DINOv3STAs(**config["DINOv3STAs"])
|
| 55 |
+
|
| 56 |
+
self.encoder = HybridEncoder(**config["HybridEncoder"])
|
| 57 |
+
self.decoder = DEIMTransformer(**config["DEIMTransformer"])
|
| 58 |
+
self.postprocessor = PostProcessor(**config["PostProcessor"])
|
| 59 |
+
|
| 60 |
+
def forward(self, x, orig_target_sizes):
|
| 61 |
+
x = self.backbone(x)
|
| 62 |
+
x = self.encoder(x)
|
| 63 |
+
x = self.decoder(x)
|
| 64 |
+
x = self.postprocessor(x, orig_target_sizes)
|
| 65 |
+
|
| 66 |
+
return x
|
| 67 |
+
|
| 68 |
+
# Load the model from the Hub
|
| 69 |
+
# Replace the model ID with the specific variant you wish to use
|
| 70 |
+
model = DEIMv2.from_pretrained("Intellindust/DEIMv2_DINOv3_S_COCO")
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
## Citation
|
| 74 |
+
|
| 75 |
+
```bibtex
|
| 76 |
+
@article{huang2025deimv2,
|
| 77 |
+
title={Real-Time Object Detection Meets DINOv3},
|
| 78 |
+
author={Huang, Shihua and Hou, Yongjie and Liu, Longfei and Yu, Xuanlong and Shen, Xi},
|
| 79 |
+
journal={arXiv preprint arXiv:2509.20787},
|
| 80 |
+
year={2025}
|
| 81 |
+
}
|
| 82 |
+
```
|