zeromodels/detr-resnet-50

See our collection for all versions of DETR.

Run DETR with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/detr-resnet-50

Paper: End-to-End Object Detection with Transformers (arXiv:2005.12872) · HF Papers

DETR (DEtection TRansformer) treats object detection as direct set prediction. A ResNet backbone produces a feature map, a transformer encoder-decoder attends over it with a fixed set of learned object queries, and each query emits one class and one box. Training uses a bipartite (Hungarian) matching loss, so every ground-truth object is assigned exactly one query. That framing removes anchors and NMS. Panoptic checkpoints add a mask head for things and stuff.

For more details on the model, please go to Facebook's original model card.

Pure-Keras 3 conversion of facebook/detr-resnet-50 for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an object detection checkpoint (DETRDetect): each query predicts a class and box on COCO.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.detr import DETRDetect, DETRImageProcessor

model = DETRDetect.from_weights("kerasformers/detr-resnet-50")
processor = DETRImageProcessor.from_weights("kerasformers/detr-resnet-50")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
    output, threshold=0.9, target_sizes=[(image.height, image.width)]
)[0]
for score, name, box in zip(
    results["scores"], results["label_names"], results["boxes"]
):
    print(f"{name}: {float(score):.3f} {box}")

Load any DETR variant the same way with from_weights("kerasformers/<variant>") (use DETRDetect for this repo):

Variant	Hub	Task
detr-resnet-50	kerasformers/detr-resnet-50	object detection
detr-resnet-101	kerasformers/detr-resnet-101	object detection
detr-resnet-50-panoptic	kerasformers/detr-resnet-50-panoptic	panoptic segmentation
detr-resnet-101-panoptic	kerasformers/detr-resnet-101-panoptic	panoptic segmentation

Tips

• Set KERAS_BACKEND before importing Keras / kerasformers.
• Detection: DETRDetect + post_process_object_detection (try threshold=0.9 on clean COCO scenes).
• Panoptic: DETRPanopticSegment returns pred_masks per query; threshold at zero and resize to the image size yourself.
• See DETR docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. DETRDetect.from_weights("hf:facebook/detr-resnet-50").

Special Thanks

A huge thank you to the Facebook AI Research DETR authors for creating and releasing these models.

License: Apache 2.0.