---
license: apache-2.0
tags:
- vision
- image-detection
- image-segmentation
datasets:
- COCO
---
# DETR for TI EdgeAI
### Set-Prediction Object Detection and Panoptic Segmentation via Transformers
[](https://opensource.org/licenses/Apache-2.0)
[](https://onnx.ai/)
[](https://github.com/TexasInstruments/edgeai)
[](https://cocodataset.org)
---
## Overview
**DETR** (DEtection TRansformer) is a transformer-based object detection architecture from Facebook Research that eliminates the need for hand-crafted components like anchor generation and NMS post-processing. It reformulates object detection as a direct set-prediction problem, using bipartite matching together with a transformer encoder-decoder to predict a fixed set of 100 object queries per image.
DETR matches Faster R-CNN with a ResNet-50 backbone in AP while using half the FLOPs, and extends naturally to panoptic segmentation by adding a mask head on top of the detection queries. The original [facebookresearch/detr](https://github.com/facebookresearch/detr) repository is archived (Apache 2.0), and pretrained COCO weights are pulled automatically via `torch.hub`.
This export covers the four ResNet-backbone **detection** variants (`detr_resnet50`, `detr_resnet50_dc5`, `detr_resnet101`, `detr_resnet101_dc5`). The panoptic segmentation variants (`detr_resnet50_panoptic`, `detr_resnet50_dc5_panoptic`, `detr_resnet101_panoptic`) are supported by the upstream repository and by `prepare_model.py`, but are not included as pre-exported artifacts in this folder.
---
## Model Variants
| Model | Backbone | Reference mAP[.5:.95]% | Reference mAP[.50]% | Validated Devices | Config |
|-------|----------|-------------|-----------|--------------------|--------|
| `detr_resnet50` | ResNet-50 | 42.0 | 62.4 | TDA4VH | [detr_resnet50_config.yaml](detr_resnet50_config.yaml) |
| `detr_resnet50_dc5` | ResNet-50 DC5 | 43.3 | 63.1 | TDA4VH | [detr_resnet50_dc5_config.yaml](detr_resnet50_dc5_config.yaml) |
| `detr_resnet101` | ResNet-101 | 43.5 | 63.8 | TDA4VH | [detr_resnet101_config.yaml](detr_resnet101_config.yaml) |
| `detr_resnet101_dc5` | ResNet-101 DC5 | 44.9 | 64.7 | TDA4VH | [detr_resnet101_dc5_config.yaml](detr_resnet101_dc5_config.yaml) |
> mAP values are on COCO val2017. DC5 = dilated convolutions in the last ResNet block (stride 16→32 kept at stride 8→16), giving higher-resolution features at the cost of higher compute.
**Recommended for edge deployment:** `detr_resnet50` (best accuracy/compute trade-off)
---
## Quick Start
### Prerequisites
```bash
pip install torch>=1.12.0 torchvision>=0.13.0 onnx>=1.14.0 scipy
pip install onnxruntime>=1.15.0
```
`scipy` is required because DETR imports it at module load time (`models/matcher.py`). All of the above are auto-installed by `prepare_model.py` if missing.
### Export the Model
```bash
# Export the default model (detr_resnet50)
python prepare_model.py
# Export a specific model variant
python prepare_model.py --model detr_resnet101
# Export multiple variants at once
python prepare_model.py --model detr_resnet50 detr_resnet101
# List all available variants with accuracy info
python prepare_model.py --list-models
# Export from a locally trained checkpoint
python prepare_model.py --model detr_resnet50 --weights /path/to/checkpoint.pth
```
The script automatically:
- Installs missing dependencies (`torch`, `torchvision`, `onnx`, `scipy`) if not present
- Loads the pretrained model via `torch.hub` (`facebookresearch/detr:main`), cloning the DETR source and downloading pretrained COCO weights from `dl.fbaipublicfiles.com` on first use
- Wraps the model to accept a plain `(N, 3, H, W)` tensor instead of a `NestedTensor`
- Exports to ONNX (opset 17 by default) with constant folding enabled, and validates the exported graph
- Saves the result as `.onnx` in the output directory
> **Note:** DC5 (`_dc5`) variants are currently skipped by `prepare_model.py` with a warning, since TIDL does not yet support the dilated-conv backbone for compilation. The pre-exported `.onnx`/config artifacts for these variants remain in this folder for reference.
### Compile and Infer uing edgeai-tidlrunner
> **Note:** Run the commands below from inside the `tidlrunner` directory (the cloned [edgeai-tidlrunner](https://github.com/TexasInstruments/edgeai-tidlrunner) repository), with `--config_path` pointing to this model's config file.
**Compile using edgeai-tidlrunner - on PC**
```bash
cd /path/to/edgeai-tidlrunner
tidlrunner-cli compile --target_device J784S4 \
--config_path /path/to/detr_resnet50_config.yaml
```
**Run Inference Benchmark - on device**
```bash
cd /path/to/edgeai-tidlrunner
tidlrunner-cli infer --target_device J784S4 \
--config_path /path/to/detr_resnet50_config.yaml
```
Swap `detr_resnet50_config.yaml` for `detr_resnet50_dc5_config.yaml`, `detr_resnet101_config.yaml`, or `detr_resnet101_dc5_config.yaml` to compile/infer the other variants.
### Compile and Infer using edgeai-tidl-tools (Advanced):
Follow the instructions at https://github.com/TexasInstruments/edgeai-tidl-tools
### Deploy using edgeai-tidl-tools:
Deplyment can be done using **[edgeai-tidl-tools](https://github.com/TexasInstruments/edgeai-tidl-tools)**. For ONNX models, onnxruntime-tidl with TIDL acceleration can be used. Consult the documentation of edgeai-tidl-tools for more details.
---
## Citation
If you use these models, please cite:
```bibtex
@inproceedings{carion2020end,
title = {End-to-End Object Detection with Transformers},
author = {Carion, Nicolas and Massa, Francisco and Synnaeve, Gabriel and
Usunier, Nicolas and Kirillov, Alexander and Zagoruyko, Sergey},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}
```
---
## 🔗 Resources
| Resource | Link |
|----------|------|
| **Paper** | [arXiv:2005.12872](https://arxiv.org/abs/2005.12872) |
| **Source Code** | [facebookresearch/detr](https://github.com/facebookresearch/detr) |
| **Blog Post** | [End-to-End Object Detection with Transformers](https://ai.facebook.com/blog/end-to-end-object-detection-with-transformers) |
| **COCO Dataset** | [cocodataset.org](https://cocodataset.org) |
| **edgeai-tidl-tools** | [GitHub](https://github.com/TexasInstruments/edgeai-tidl-tools) |
| **edgeai-tidlrunner** | [GitHub](https://github.com/TexasInstruments/edgeai-tidlrunner) |
| **EdgeAI SDK** | [Documentation](https://github.com/TexasInstruments/edgeai/blob/main/edgeai-mpu/readme_sdk.md) |
| **TI EdgeAI Ecosystem** | [GitHub](https://github.com/TexasInstruments/edgeai) |
---
## Related Models
|
**Deformable-DETR**
Deformable attention
Faster convergence
|
**RT-DETRv2**
Real-time transformer
NMS-free detection
|
**RF-DETR**
Receptive-field DETR
Lightweight edge variant
|
**DEIMv2**
Improved DETR training
Higher accuracy/epoch
|
---
**Maintained by:** Texas Instruments EdgeAI Team
**Last Updated:** August 2026