Remove Edge Inference Benchmarks link

6933a8c 9 days ago

11.8 kB

	---
	license: other
	license_name: embedl-models-community-licence-1.0
	license_link: https://github.com/embedl/embedl-models/blob/main/LICENSE
	base_model:
	- Roboflow/rf-detr-base
	quantized_from:
	- Roboflow/rf-detr-base
	tags:
	- object-detection
	- quantization
	- onnx
	- tensorrt
	- edge
	- embedl
	gated: true
	extra_gated_heading: "Access Embedl RF-DETR Base"
	extra_gated_description: "To access this model, please review and accept the terms below. Your contact information is collected solely to manage access and, with your explicit consent, to notify you about updated or new optimized models from Embedl."
	extra_gated_button_content: "Agree and request access"
	extra_gated_prompt: "By requesting access you agree to the Embedl Models Community Licence and the upstream RF-DETR License"
	extra_gated_fields:
	Company: text
	I agree to the Embedl Models Community Licence and upstream RF-DETR License: checkbox
	I consent to being contacted by Embedl about products and services (optional): checkbox
	---
	<!-- embedl-banner:start -->
	<style>
	.embedl-btn-primary { transition: background 160ms ease, box-shadow 160ms ease; }
	.embedl-btn-primary:hover { background: #4FDCE4 !important; box-shadow: 0 8px 22px rgba(45,212,221,0.45) !important; }
	.embedl-btn-secondary { transition: background 160ms ease; }
	.embedl-btn-secondary:hover { background: rgba(45,212,221,0.15) !important; }
	.embedl-headline { font-size: clamp(11px, 2.15vw, 15px) !important; }
	.embedl-btn-primary, .embedl-btn-secondary {
	font-size: clamp(11px, 1.65vw, 13px) !important;
	padding: clamp(6px, 1.1vw, 9px) clamp(10px, 1.6vw, 14px) !important;
	}
	</style>
	<div style="background:radial-gradient(600px 220px at 0% 50%,rgba(45,212,221,0.22) 0%,rgba(45,212,221,0) 60%),radial-gradient(400px 180px at 100% 100%,rgba(45,212,221,0.10) 0%,rgba(45,212,221,0) 55%),linear-gradient(135deg,#0B1626 0%,#142338 100%);border:1px solid rgba(45,212,221,0.28);border-radius:12px;padding:22px 24px;margin:0 0 24px 0;color:#F2F6FA;box-shadow:0 4px 16px rgba(11,22,38,0.18);overflow:hidden;box-sizing:border-box;max-width:100%;">
	<table style="width:100%;border-collapse:collapse;border:0;background:transparent;">
	<tr style="background:transparent;">
	<td style="vertical-align:middle;border:0;padding:0;background:transparent;">
	<div style="display:inline-block;font-size:10px;letter-spacing:0.08em;text-transform:uppercase;font-weight:700;color:#2DD4DD;background:rgba(45,212,221,0.15);border:1px solid rgba(45,212,221,0.35);padding:4px 10px;border-radius:999px;margin-bottom:10px;white-space:nowrap;">Optimized by Embedl</div>
	<div class="embedl-headline" style="font-size:15px;font-weight:700;line-height:1.35;color:#F2F6FA;margin-bottom:4px;">Need to <span style="color:#2DD4DD;white-space:nowrap;">fine-tune</span>, hit <span style="color:#2DD4DD;white-space:nowrap;">performance targets</span>, or deploy on <span style="color:#2DD4DD;white-space:nowrap;">specific hardware</span>?</div>
	<div style="font-size:13px;color:#9BA7B5;">We've got you covered.</div>
	</td>
	<td width="1%" style="vertical-align:middle;border:0;padding:0 0 0 18px;white-space:nowrap;text-align:right;background:transparent;">
	<a href="https://www.embedl.com/models" class="embedl-btn-secondary" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #2DD4DD;color:#2DD4DD;text-decoration:none;margin-right:8px;">Learn more</a>
	<a href="https://www.embedl.com/contact" class="embedl-btn-primary" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #2DD4DD;background:#2DD4DD;color:#0B1626;text-decoration:none;box-shadow:0 6px 18px rgba(45,212,221,0.28);">Get in touch →</a>
	</td>
	</tr>
	</table>
	</div>
	<!-- embedl-banner:end -->

	# Embedl RF-DETR Base (Quantized for TensorRT)

	Deployable INT8-quantized version of [`Roboflow/rf-detr-base`](https://huggingface.co/Roboflow/rf-detr-base),
	optimized with [embedl-deploy](https://github.com/embedl/embedl-deploy)
	for low-latency NVIDIA TensorRT inference on edge GPUs.

	## Upstream Model

	<a href="https://hfviewer.com/Roboflow/rf-detr-base?utm_source=huggingface&utm_medium=embedded_model_card&utm_campaign=Roboflow__rf-detr-base_card" target="_blank" rel="noopener">
	<img
	src="https://hfviewer.com/api/card.svg?source=Roboflow%2Frf-detr-base&v=20260501clipcard"
	alt="Open Roboflow/rf-detr-base in hfviewer"
	width="100%"
	/>
	</a>

	## Highlights

	- Mixed-precision INT8/FP16 quantization with hardware-aware
	optimizations from [embedl-deploy](https://github.com/embedl/embedl-deploy).
	- Drop-in replacement for `Roboflow/rf-detr-base` in TensorRT pipelines:
	same image input (560×560), same output semantics.
	- Validated accuracy within 1.19 pp of the FP32 baseline on COCO
	(see Accuracy table below).
	- Faster than `trtexec --best` on supported NVIDIA hardware (see Performance table below).
	- Includes both ONNX (for TensorRT) and PT2
	(`torch.export`-loadable) artifacts plus runnable inference scripts.

	## Quick Start

	```bash
	pip install huggingface_hub onnxruntime-gpu pillow numpy
	python -c "from huggingface_hub import snapshot_download; snapshot_download('embedl/rf-detr-base-quantized-tensorrt', local_dir='.')"
	python infer_trt.py --image path/to/image.jpg # TensorRT
	# or
	python infer_pt2.py --image path/to/image.jpg # pure PyTorch via torch.export
	```

	## Files

	\| File \| Purpose \|
	\|---\|---\|
	\| `embedl_rf-detr-base_int8.onnx` \| INT8-quantized ONNX with Q/DQ nodes, ready for TensorRT. \|
	\| `embedl_rf-detr-base_int8.pt2` \| INT8-quantized `torch.export` ExportedProgram. \|
	\| `infer_trt.py` \| Build a TRT engine from the ONNX and run sample inference. \|
	\| `infer_pt2.py` \| Load the `.pt2` with `torch.export.load` and run sample inference. \|

	## Performance

	Latency measured with TensorRT + `trtexec`, GPU compute time only
	(`--noDataTransfers`), CUDA Graph + Spin Wait enabled, clocks locked
	(max-performance `nvpmodel` mode + `jetson_clocks` on Jetson).

	<a href="https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks" target="_blank" rel="noopener"><img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/rf-detr-base__orin-nano-super-palo-alto__latency.svg" alt="RF-DETR Base latency on NVIDIA Jetson Orin Nano Super"></a>

	<a href="https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks" target="_blank" rel="noopener"><img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/rf-detr-base__orin-nano-super-palo-alto__memory.svg" alt="RF-DETR Base peak memory on NVIDIA Jetson Orin Nano Super"></a>

	### NVIDIA Jetson Orin Nano Super

	\| Configuration \| Mean Latency \| Speedup vs FP16 \|
	\|---\|---\|---\|
	\| TensorRT FP16 \| 14.83 ms \| 1.00x \|
	\| TensorRT --best (unconstrained) \| 14.87 ms \| 1.00x \|
	\| Embedl Deploy INT8 \| 11.77 ms \| 1.26x \|

	## Example Detections

	Both panels in every figure below are decoded from the **actual
	TensorRT engines running on the Jetson Orin Nano Super**: inputs fed
	with `trtexec --loadInputs`, raw `dets`/`labels` outputs exported and
	drawn offline. Small per-detection score differences between the two
	precisions are expected for a two-stage DETR (proposal selection
	reorders under precision changes); the detection sets match.

	<img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/detections_street.png" alt="RF-DETR Base FP16 vs Embedl INT8 detections - street scene">

	<img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/detections_livingroom.png" alt="RF-DETR Base FP16 vs Embedl INT8 detections - living room">

	<img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/detections_cats.png" alt="RF-DETR Base FP16 vs Embedl INT8 detections - COCO cats">

	Confidence-threshold sweep from 0.80 down to 0.30, both engines in
	sync; detections appear at nearly identical thresholds:

	<img src="https://huggingface.co/datasets/embedl/documentation-images/resolve/main/rf-detr-base-quantized-tensorrt/threshold_sweep_street.gif" alt="Synced confidence-threshold sweep, TensorRT FP16 vs Embedl INT8">

	## Accuracy

	Evaluated on the COCO validation split. The quantized model
	retains nearly all of the FP32 accuracy with a small tolerance.

	\| Model \| mAP@50-95 \| mAP@50 \|
	\|---\|---\|---\|
	\| `Roboflow/rf-detr-base` FP32 (ours) \| 53.32% \| 72.16% \|
	\| Embedl RF-DETR Base INT8 \| 52.14% \| 71.26% \|

	## Creating Your Own Optimized Models

	This artifact was produced with
	[embedl-deploy](https://github.com/embedl/embedl-deploy),
	Embedl's open-source PyTorch-to-TensorRT deployment library. You can
	apply the same workflow to your own models - see
	[the documentation](https://github.com/embedl/embedl-deploy#readme)
	for installation and usage.

	## License

	\| Component \| License \|
	\|---\|---\|
	\| Optimized model artifacts (this repo) \| [Embedl Models Community Licence v1.0](https://github.com/embedl/embedl-models/blob/main/LICENSE) (no redistribution as a hosted service) \|
	\| Upstream architecture and weights \| [RF-DETR License](https://github.com/roboflow/rf-detr/blob/main/LICENSE) \|

	## Contact

	We offer engineering support for on-prem/edge deployments and partner
	co-marketing opportunities. Reach out at
	[contact@embedl.com](mailto:contact@embedl.com), or open an issue on
	[GitHub](https://github.com/embedl/embedl-deploy).

	<!-- embedl-discord-banner:start -->
	<style>
	.embedl-discord-btn { transition: background 160ms ease, box-shadow 160ms ease; }
	.embedl-discord-btn:hover { background: #6C77F5 !important; box-shadow: 0 8px 22px rgba(88,101,242,0.55) !important; }
	</style>
	<div style="background:radial-gradient(600px 220px at 0% 50%,rgba(88,101,242,0.22) 0%,rgba(88,101,242,0) 60%),radial-gradient(400px 180px at 100% 100%,rgba(88,101,242,0.10) 0%,rgba(88,101,242,0) 55%),linear-gradient(135deg,#0B1626 0%,#142338 100%);border:1px solid rgba(88,101,242,0.35);border-radius:12px;padding:22px 24px;margin:24px 0 0 0;color:#F2F6FA;box-shadow:0 4px 16px rgba(11,22,38,0.18);overflow:hidden;box-sizing:border-box;max-width:100%;">
	<table style="width:100%;border-collapse:collapse;border:0;background:transparent;">
	<tr style="background:transparent;">
	<td style="vertical-align:middle;border:0;padding:0;background:transparent;">
	<div style="display:inline-block;font-size:10px;letter-spacing:0.08em;text-transform:uppercase;font-weight:700;color:#A5B4FC;background:rgba(88,101,242,0.18);border:1px solid rgba(88,101,242,0.45);padding:4px 10px;border-radius:999px;margin-bottom:10px;white-space:nowrap;">Community & support</div>
	<div style="font-size:15px;font-weight:700;line-height:1.35;color:#F2F6FA;margin-bottom:4px;">Need help with this model? Chat with the Embedl team and other engineers on <span style="color:#A5B4FC;white-space:nowrap;">Discord</span>.</div>
	<div style="font-size:13px;color:#9BA7B5;">Quantization gotchas, hardware questions, fine-tuning tips — bring them all.</div>
	</td>
	<td width="1%" style="vertical-align:middle;border:0;padding:0 0 0 18px;white-space:nowrap;text-align:right;background:transparent;">
	<a href="https://discord.gg/MTbMWdKqE" class="embedl-discord-btn" style="display:inline-block;font-size:13px;font-weight:600;padding:9px 14px;border-radius:6px;border:1px solid #5865F2;background:#5865F2;color:#FFFFFF;text-decoration:none;box-shadow:0 6px 18px rgba(88,101,242,0.35);">Join our Discord →</a>
	</td>
	</tr>
	</table>
	</div>
	<!-- embedl-discord-banner:end -->