See our collection for all versions of RF-DETR.

Run RF-DETR with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/rfdetr-base

Paper: RF-DETR: Neural Architecture Search for Real-Time Detection Transformers (arXiv:2511.09554) · HF Papers

RF-DETR is Roboflow's real-time DETR, built on a windowed DINOv2 backbone with a lightweight deformable decoder. Configurations came out of a neural architecture search, so variants differ in resolution, patch size, window count, and decoder depth. Instance-segmentation checkpoints add a mask head.

For more details on the model, please go to Roboflow's original model card.

Pure-Keras 3 conversion of Roboflow/rf-detr-base for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is an object detection checkpoint (RFDETRDetect): each query predicts a class and box.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.rf_detr import RFDETRDetect, RFDETRImageProcessor

model = RFDETRDetect.from_weights("zeromodels/rfdetr-base")
processor = RFDETRImageProcessor.from_weights("zeromodels/rfdetr-base")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
    output, threshold=0.5, target_sizes=[(image.height, image.width)]
)[0]
for score, name, box in zip(
    results["scores"], results["label_names"], results["boxes"]
):
    print(f"{name}: {float(score):.3f} {box}")

Load any RF-DETR variant the same way with from_weights("zeromodels/<variant>") (use RFDETRDetect for this repo):

Variant	Hub	Task
`rfdetr-nano`	`zeromodels/rfdetr-nano`	object detection
`rfdetr-small`	`zeromodels/rfdetr-small`	object detection
`rfdetr-medium`	`zeromodels/rfdetr-medium`	object detection
`rfdetr-base`	`zeromodels/rfdetr-base`	object detection
`rfdetr-large`	`zeromodels/rfdetr-large`	object detection
`rfdetr-seg-preview`	`zeromodels/rfdetr-seg-preview`	instance segmentation
`rfdetr-seg-nano`	`zeromodels/rfdetr-seg-nano`	instance segmentation
`rfdetr-seg-small`	`zeromodels/rfdetr-seg-small`	instance segmentation
`rfdetr-seg-medium`	`zeromodels/rfdetr-seg-medium`	instance segmentation
`rfdetr-seg-large`	`zeromodels/rfdetr-seg-large`	instance segmentation
`rfdetr-seg-xlarge`	`zeromodels/rfdetr-seg-xlarge`	instance segmentation
`rfdetr-seg-xxlarge`	`zeromodels/rfdetr-seg-xxlarge`	instance segmentation

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
Prefer RFDETRImageProcessor.from_weights(...) so the processor resolution matches the variant (bare constructor defaults to base's 560).
Detection: RFDETRDetect + post_process_object_detection.
Segmentation: RFDETRInstanceSegment + post_process_instance_segmentation.
See RF-DETR docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. RFDETRDetect.from_weights("hf:Roboflow/rf-detr-base").