vagheshpatel commited on
Commit
baa49ed
·
verified ·
1 Parent(s): 6d20ffd

Update native RC1 samples and supported device documentation

Browse files
Files changed (4) hide show
  1. README.md +50 -13
  2. config.json +11 -3
  3. dlstreamer_sample.py +99 -0
  4. export_and_quantize.sh +119 -11
README.md CHANGED
@@ -8,6 +8,7 @@ tags:
8
  - intel
9
  - yolo
10
  - yolo26
 
11
  - loitering-detection
12
  - zone-analytics
13
  - tracking
@@ -25,7 +26,8 @@ language:
25
  | Property | Value |
26
  |---|---|
27
  | **Category** | Object Detection + Tracking + Zone Analytics (GstAnalytics) |
28
- | **Source Framework** | PyTorch (Ultralytics) |
 
29
  | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
30
  | **Inference Engine** | OpenVINO |
31
  | **Hardware** | CPU, GPU, NPU |
@@ -47,15 +49,34 @@ Typical Metro deployments include:
47
  - **ATM and Ticketing Security** -- identify suspicious dwell at unattended kiosks.
48
  - **Crowd-Free Zone Enforcement** -- monitor emergency exits and corridors that must remain clear.
49
 
50
- Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`.
51
  Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment.
52
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  ---
54
 
55
  ## Prerequisites
56
 
 
 
 
 
57
  - Python 3.11+
58
- - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html)
59
 
60
  Create and activate a Python virtual environment before running the scripts:
61
 
@@ -83,25 +104,27 @@ chmod +x export_and_quantize.sh
83
 
84
  This exports the default **yolo26n** model in **FP16** precision.
85
 
86
- #### Optional: Select a Different Variant or Precision
87
 
88
  ```bash
89
- ./export_and_quantize.sh yolo26n FP32 # full-precision
90
- ./export_and_quantize.sh yolo26n INT8 # quantized
91
- ./export_and_quantize.sh yolo26s # larger variant, default FP16
 
 
92
  ```
93
 
94
- Replace `yolo26n` with any variant (`yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`).
95
  The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**.
96
 
97
  The script performs the following steps:
98
 
99
- 1. Installs dependencies (`openvino`, `ultralytics`; adds `nncf` for INT8).
100
  2. Downloads the sample surveillance video (`VIRAT_S_000101.mp4`) from the Intel Metro AI Suite project into the current directory.
101
  3. Downloads the PyTorch weights and exports to OpenVINO IR.
102
  4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.
103
 
104
- Output files:
105
 
106
  - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
107
  - `yolo26n_loitering_int8.xml` / `yolo26n_loitering_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*.
@@ -170,7 +193,16 @@ export PYTHONPATH=/opt/intel/dlstreamer/python:/opt/intel/dlstreamer/gstreamer/l
170
 
171
  Run loitering detection:
172
 
 
 
 
173
  ```python
 
 
 
 
 
 
174
  from collections import defaultdict
175
  import json
176
  import sys
@@ -188,7 +220,9 @@ _ov = sys.modules["gi.overrides.GstAnalytics"]
188
  _ov.__mtd_types__[DLStreamerMeta.ZoneMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_zone_mtd
189
  _ov.__mtd_types__[DLStreamerMeta.TripwireMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_tripwire_mtd
190
 
191
- MODEL = "yolo26n_openvino_model/yolo26n.xml"
 
 
192
  VIDEO = "VIRAT_S_000101.mp4"
193
  ZONE_JSON = json.dumps([{
194
  "id": "loiter_zone",
@@ -200,7 +234,7 @@ LOITERING_SECONDS = 5.0
200
 
201
  pipeline = Gst.parse_launch(
202
  f"filesrc location={VIDEO} ! decodebin3 ! videoconvert ! "
203
- f"gvadetect model={MODEL} device=GPU threshold=0.5 ! queue ! "
204
  f"gvatrack tracking-type=short-term-imageless ! queue ! "
205
  f"gvaanalytics name=analytics draw-zones=true ! "
206
  f"gvafpscounter ! identity name=probe ! gvawatermark name=watermark ! "
@@ -263,6 +297,7 @@ pipeline.set_state(Gst.State.PLAYING)
263
  pipeline.get_bus().timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
264
  pipeline.set_state(Gst.State.NULL)
265
  ```
 
266
 
267
  Expected output:
268
 
@@ -294,8 +329,10 @@ Licensed under the MIT License. See [LICENSE](LICENSE) for details.
294
  ## References
295
 
296
  - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
 
 
297
  - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb)
298
- - [Intel DLStreamer Object Tracking](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/elements/gvatrack.html)
299
  - [Intel DLStreamer gvaanalytics](https://github.com/dlstreamer/dlstreamer/blob/main/src/monolithic/gst/elements/gvaanalytics/README.md)
300
  - [OpenVINO Documentation](https://docs.openvino.ai/)
301
  - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html)
 
8
  - intel
9
  - yolo
10
  - yolo26
11
+ - detr
12
  - loitering-detection
13
  - zone-analytics
14
  - tracking
 
26
  | Property | Value |
27
  |---|---|
28
  | **Category** | Object Detection + Tracking + Zone Analytics (GstAnalytics) |
29
+ | **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics, default) or [DETR](https://huggingface.co/facebook/detr-resnet-50) (Facebook, alternative) |
30
+ | **Source Framework** | PyTorch (Ultralytics, Hugging Face Transformers) |
31
  | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
32
  | **Inference Engine** | OpenVINO |
33
  | **Hardware** | CPU, GPU, NPU |
 
49
  - **ATM and Ticketing Security** -- identify suspicious dwell at unattended kiosks.
50
  - **Crowd-Free Zone Enforcement** -- monitor emergency exits and corridors that must remain clear.
51
 
52
+ Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`.
53
  Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment.
54
 
55
+ ### Model Selection
56
+
57
+ [DETR](https://huggingface.co/facebook/detr-resnet-50) (DEtection TRansformer) is available as an alternative to YOLO26.
58
+ DETR is a transformer-based, NMS-free detector trained on COCO and released under the Apache-2.0 license.
59
+ The export script embeds DETR normalization, box decoding, and the class remap into the OpenVINO IR, so the DETR model produces the same output layout and class ids as YOLO26.
60
+ The same DLStreamer sample run either model; select one with the `MODEL` constant at the top of the sample.
61
+
62
+ | Model | Variants | Input Size | Output | License |
63
+ |---|---|---|---|---|
64
+ | YOLO26 (default) | `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l` | 640 x 640 | `[1, 300, 6]` | AGPL-3.0 |
65
+ | DETR | `detr-resnet-50` | 800 x 800 | `[1, 100, 6]` | Apache-2.0 |
66
+
67
+ Choose YOLO26 for maximum throughput on edge devices.
68
+ Choose DETR when a permissively licensed transformer detector is preferred; it is heavier and runs at a lower frame rate.
69
+
70
  ---
71
 
72
  ## Prerequisites
73
 
74
+ Target stack: Intel DLStreamer **2026.3.0 RC1** with the matching OpenVINO GenAI bundle **2026.5.0.0.dev20261030** (OpenVINO Runtime **2026.5.0.dev20261030**).
75
+ Use the [native RC1 setup](../README.md#native-rc1-runtime) before running these samples; this is a development stack, not a stable release.
76
+ See the [device validation matrix](../DEVICE_SUPPORT_MATRIX.md) for measured CPU, GPU, and NPU results.
77
+
78
  - Python 3.11+
79
+ - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html)
80
 
81
  Create and activate a Python virtual environment before running the scripts:
82
 
 
104
 
105
  This exports the default **yolo26n** model in **FP16** precision.
106
 
107
+ #### Optional: Select a Different Model, Variant, or Precision
108
 
109
  ```bash
110
+ ./export_and_quantize.sh yolo26n FP32 # full-precision
111
+ ./export_and_quantize.sh yolo26n INT8 # quantized
112
+ ./export_and_quantize.sh yolo26s # larger variant, default FP16
113
+ ./export_and_quantize.sh detr-resnet-50 # DETR alternative, default FP16
114
+ ./export_and_quantize.sh detr-resnet-50 INT8 # DETR quantized
115
  ```
116
 
117
+ The first argument selects the model: any YOLO26 variant (`yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`) or DETR (`detr-resnet-50`).
118
  The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**.
119
 
120
  The script performs the following steps:
121
 
122
+ 1. Installs dependencies (`openvino`, `ultralytics` for YOLO26 or `transformers`, `torch` for DETR; adds `nncf` for INT8).
123
  2. Downloads the sample surveillance video (`VIRAT_S_000101.mp4`) from the Intel Metro AI Suite project into the current directory.
124
  3. Downloads the PyTorch weights and exports to OpenVINO IR.
125
  4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.
126
 
127
+ Output files (replace `yolo26n` with the selected model, for example `detr-resnet-50`):
128
 
129
  - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
130
  - `yolo26n_loitering_int8.xml` / `yolo26n_loitering_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*.
 
193
 
194
  Run loitering detection:
195
 
196
+ The full sample is in [dlstreamer_sample.py](dlstreamer_sample.py); run it with `python3 dlstreamer_sample.py` or copy the code below.
197
+
198
+ <!-- sample: dlstreamer_sample.py -->
199
  ```python
200
+ # SPDX-License-Identifier: MIT
201
+ # Copyright (C) Intel Corporation
202
+ #
203
+ # Loitering Detection: DLStreamer pipeline sample.
204
+ # See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py
205
+
206
  from collections import defaultdict
207
  import json
208
  import sys
 
220
  _ov.__mtd_types__[DLStreamerMeta.ZoneMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_zone_mtd
221
  _ov.__mtd_types__[DLStreamerMeta.TripwireMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_tripwire_mtd
222
 
223
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
224
+ MODEL = "yolo26n"
225
+ MODEL_PATH = f"{MODEL}_openvino_model/{MODEL}.xml"
226
  VIDEO = "VIRAT_S_000101.mp4"
227
  ZONE_JSON = json.dumps([{
228
  "id": "loiter_zone",
 
234
 
235
  pipeline = Gst.parse_launch(
236
  f"filesrc location={VIDEO} ! decodebin3 ! videoconvert ! "
237
+ f"gvadetect model={MODEL_PATH} device=GPU threshold=0.5 ! queue ! "
238
  f"gvatrack tracking-type=short-term-imageless ! queue ! "
239
  f"gvaanalytics name=analytics draw-zones=true ! "
240
  f"gvafpscounter ! identity name=probe ! gvawatermark name=watermark ! "
 
297
  pipeline.get_bus().timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
298
  pipeline.set_state(Gst.State.NULL)
299
  ```
300
+ <!-- /sample -->
301
 
302
  Expected output:
303
 
 
329
  ## References
330
 
331
  - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
332
+ - [DETR Model Card (facebook/detr-resnet-50)](https://huggingface.co/facebook/detr-resnet-50)
333
+ - [End-to-End Object Detection with Transformers (DETR paper)](https://arxiv.org/abs/2005.12872)
334
  - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb)
335
+ - [Intel DLStreamer Object Tracking](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/elements/gvatrack.html)
336
  - [Intel DLStreamer gvaanalytics](https://github.com/dlstreamer/dlstreamer/blob/main/src/monolithic/gst/elements/gvaanalytics/README.md)
337
  - [OpenVINO Documentation](https://docs.openvino.ai/)
338
  - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html)
config.json CHANGED
@@ -3,8 +3,8 @@
3
  "library_name": "openvino",
4
  "pipeline_tag": "object-detection",
5
  "category": "Object Detection + Tracking + Zone Analytics (GstAnalytics)",
6
- "base_model": null,
7
- "source_framework": "PyTorch (Ultralytics)",
8
  "supported_precisions": [
9
  "FP32",
10
  "FP16",
@@ -18,5 +18,13 @@
18
  ],
19
  "detected_classes": "`person` (COCO class 0)",
20
  "hf_repo_id": "Intel/loitering-detection",
21
- "description": "Loitering detection use case using YOLO26 person detection with zone-based dwell-time tracking"
 
 
 
 
 
 
 
 
22
  }
 
3
  "library_name": "openvino",
4
  "pipeline_tag": "object-detection",
5
  "category": "Object Detection + Tracking + Zone Analytics (GstAnalytics)",
6
+ "base_model": "YOLO26 (Ultralytics, default) or DETR facebook/detr-resnet-50 (alternative)",
7
+ "source_framework": "PyTorch (Ultralytics, Hugging Face Transformers)",
8
  "supported_precisions": [
9
  "FP32",
10
  "FP16",
 
18
  ],
19
  "detected_classes": "`person` (COCO class 0)",
20
  "hf_repo_id": "Intel/loitering-detection",
21
+ "description": "Loitering detection use case using YOLO26 (default) or DETR person detection with zone-based dwell-time tracking",
22
+ "selectable_models": [
23
+ "yolo26n",
24
+ "yolo26s",
25
+ "yolo26m",
26
+ "yolo26l",
27
+ "detr-resnet-50"
28
+ ],
29
+ "default_model": "yolo26n"
30
  }
dlstreamer_sample.py ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-License-Identifier: MIT
2
+ # Copyright (C) Intel Corporation
3
+ #
4
+ # Loitering Detection: DLStreamer pipeline sample.
5
+ # See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py
6
+
7
+ from collections import defaultdict
8
+ import json
9
+ import sys
10
+ import gi
11
+ gi.require_version("Gst", "1.0")
12
+ gi.require_version("GstAnalytics", "1.0")
13
+ gi.require_version("DLStreamerMeta", "1.0")
14
+ gi.require_version("DLStreamerWatermarkMeta", "1.0")
15
+ from gi.repository import Gst, GLib, GstAnalytics, DLStreamerMeta, DLStreamerWatermarkMeta
16
+
17
+ Gst.init([])
18
+
19
+ # Register DLStreamerMeta types so GstAnalytics iteration can handle them
20
+ _ov = sys.modules["gi.overrides.GstAnalytics"]
21
+ _ov.__mtd_types__[DLStreamerMeta.ZoneMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_zone_mtd
22
+ _ov.__mtd_types__[DLStreamerMeta.TripwireMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_tripwire_mtd
23
+
24
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
25
+ MODEL = "yolo26n"
26
+ MODEL_PATH = f"{MODEL}_openvino_model/{MODEL}.xml"
27
+ VIDEO = "VIRAT_S_000101.mp4"
28
+ ZONE_JSON = json.dumps([{
29
+ "id": "loiter_zone",
30
+ "type": "polygon",
31
+ "points": [{"x": 0, "y": 200}, {"x": 300, "y": 200},
32
+ {"x": 300, "y": 400}, {"x": 0, "y": 400}]
33
+ }])
34
+ LOITERING_SECONDS = 5.0
35
+
36
+ pipeline = Gst.parse_launch(
37
+ f"filesrc location={VIDEO} ! decodebin3 ! videoconvert ! "
38
+ f"gvadetect model={MODEL_PATH} device=GPU threshold=0.5 ! queue ! "
39
+ f"gvatrack tracking-type=short-term-imageless ! queue ! "
40
+ f"gvaanalytics name=analytics draw-zones=true ! "
41
+ f"gvafpscounter ! identity name=probe ! gvawatermark name=watermark ! "
42
+ f"videoconvert ! video/x-raw,format=I420 ! "
43
+ f"openh264enc ! h264parse ! mp4mux ! filesink location=output_dlstreamer.mp4"
44
+ )
45
+
46
+ pipeline.get_by_name("analytics").set_property("zones", ZONE_JSON)
47
+ pipeline.get_by_name("watermark").set_property("displ-cfg", "hide-roi=person")
48
+
49
+ dwell = defaultdict(float)
50
+ last_seen = {}
51
+ flagged = set()
52
+
53
+ def on_buffer(pad, info):
54
+ buf = info.get_buffer()
55
+ now = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else 0.0
56
+ rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
57
+ if not rmeta:
58
+ return Gst.PadProbeReturn.OK
59
+
60
+ # Iterate only over object-detection entries
61
+ for od in rmeta.iter_on_type(GstAnalytics.ODMtd):
62
+ label = GLib.quark_to_string(od.get_obj_type())
63
+ if label != "person":
64
+ continue
65
+
66
+ # Find tracking ID via direct relation
67
+ track_id = None
68
+ for trk in od.iter_direct_related(GstAnalytics.RelTypes.RELATE_TO, GstAnalytics.TrackingMtd):
69
+ success, tracking_id, *_ = trk.get_info()
70
+ if success:
71
+ track_id = tracking_id
72
+ break
73
+ if track_id is None:
74
+ continue
75
+
76
+ # Check if gvaanalytics placed this detection inside the zone
77
+ in_zone = False
78
+ for zone in od.iter_direct_related(GstAnalytics.RelTypes.RELATE_TO, DLStreamerMeta.ZoneMtd):
79
+ in_zone = True
80
+ break
81
+
82
+ if not in_zone:
83
+ continue
84
+
85
+ # Accumulate dwell time for persons inside the zone
86
+ dwell[track_id] += now - last_seen.get(track_id, now)
87
+ last_seen[track_id] = now
88
+
89
+ if dwell[track_id] >= LOITERING_SECONDS and track_id not in flagged:
90
+ flagged.add(track_id)
91
+ _, x, y, w, h, _ = od.get_location()
92
+ print(f"LOITERING id={track_id} dwell={dwell[track_id]:.1f}s pos=({int(x + w/2)},{int(y + h)})")
93
+
94
+ return Gst.PadProbeReturn.OK
95
+
96
+ pipeline.get_by_name("probe").get_static_pad("src").add_probe(Gst.PadProbeType.BUFFER, on_buffer)
97
+ pipeline.set_state(Gst.State.PLAYING)
98
+ pipeline.get_bus().timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
99
+ pipeline.set_state(Gst.State.NULL)
export_and_quantize.sh CHANGED
@@ -2,9 +2,19 @@
2
  # SPDX-License-Identifier: MIT
3
  # Copyright (C) Intel Corporation
4
  #
5
- # Export a YOLO26 person detector for loitering detection to OpenVINO IR.
6
  # Usage: ./export_and_quantize.sh [MODEL_VARIANT] [PRECISION]
7
  # Example: ./export_and_quantize.sh yolo26n FP16
 
 
 
 
 
 
 
 
 
 
8
  #
9
  # Supported precisions:
10
  # FP32 -- Full-precision floating-point weights
@@ -29,12 +39,20 @@ if [[ "${PRECISION}" != "FP32" && "${PRECISION}" != "FP16" && "${PRECISION}" !=
29
  exit 1
30
  fi
31
 
32
- echo "--- Installing dependencies ---"
33
- if [[ "${PRECISION}" == "INT8" ]]; then
34
- pip install -qU openvino nncf ultralytics
35
- else
36
- pip install -qU openvino ultralytics
37
- fi
 
 
 
 
 
 
 
 
38
 
39
  # Ask for approval before downloading models and sample files
40
  echo ""
@@ -59,6 +77,17 @@ else
59
  echo "WARNING: HuggingFace tracking ping failed (offline?); continuing." >&2
60
  fi
61
  echo ""
 
 
 
 
 
 
 
 
 
 
 
62
  echo "--- Downloading sample test video ---"
63
  if [[ ! -f VIRAT_S_000101.mp4 ]]; then
64
  wget -O VIRAT_S_000101.mp4 \
@@ -76,14 +105,92 @@ else
76
  EXPORT_LABEL="FP16"
77
  fi
78
 
79
- echo "--- Exporting ${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
80
- python3 -c "
 
81
  from ultralytics import YOLO
82
 
83
  model = YOLO('${MODEL_NAME}.pt')
84
- model.export(format='openvino', half=${HALF_FLAG}, dynamic=False, imgsz=640)
85
  print('Export complete: ${MODEL_NAME}_openvino_model/')
86
  "
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
  if [[ "${PRECISION}" == "INT8" ]]; then
89
  echo "--- Quantizing to INT8 with NNCF ---"
@@ -104,7 +211,7 @@ while len(frames) < 300:
104
  if not ret:
105
  cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
106
  continue
107
- img = cv2.resize(frame, (640, 640))
108
  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
109
  img = img.transpose(2, 0, 1)[np.newaxis, ...]
110
  frames.append(img)
@@ -120,6 +227,7 @@ quantized = nncf.quantize(
120
  calibration_dataset,
121
  preset=nncf.QuantizationPreset.MIXED,
122
  subset_size=300,
 
123
  )
124
 
125
  ov.save_model(quantized, '${MODEL_NAME}_loitering_int8.xml')
 
2
  # SPDX-License-Identifier: MIT
3
  # Copyright (C) Intel Corporation
4
  #
5
+ # Export a YOLO26 or DETR person detector for loitering detection to OpenVINO IR.
6
  # Usage: ./export_and_quantize.sh [MODEL_VARIANT] [PRECISION]
7
  # Example: ./export_and_quantize.sh yolo26n FP16
8
+ # Example: ./export_and_quantize.sh detr-resnet-50 FP16
9
+ #
10
+ # Supported model variants:
11
+ # yolo26n, yolo26s, yolo26m, yolo26l -- Ultralytics YOLO26 (default)
12
+ # detr-resnet-50 -- Facebook DETR (Hugging Face)
13
+ #
14
+ # DETR is exported end-to-end: ImageNet normalization, softmax,
15
+ # box decoding, and the COCO-91 to COCO-80 class remap are embedded in the
16
+ # graph, so the IR produces the same [1, N, 6] output layout and class ids as
17
+ # YOLO26 and runs with the same OpenVINO and DLStreamer samples.
18
  #
19
  # Supported precisions:
20
  # FP32 -- Full-precision floating-point weights
 
39
  exit 1
40
  fi
41
 
42
+ case "${MODEL_NAME}" in
43
+ yolo26n|yolo26s|yolo26m|yolo26l)
44
+ MODEL_FAMILY="yolo"
45
+ INPUT_SIZE=640
46
+ ;;
47
+ detr-resnet-50)
48
+ MODEL_FAMILY="detr"
49
+ INPUT_SIZE=800
50
+ ;;
51
+ *)
52
+ echo "ERROR: unsupported model '${MODEL_NAME}'. Choose yolo26[n|s|m|l] or detr-resnet-50." >&2
53
+ exit 1
54
+ ;;
55
+ esac
56
 
57
  # Ask for approval before downloading models and sample files
58
  echo ""
 
77
  echo "WARNING: HuggingFace tracking ping failed (offline?); continuing." >&2
78
  fi
79
  echo ""
80
+ echo "--- Installing dependencies ---"
81
+ if [[ "${MODEL_FAMILY}" == "detr" ]]; then
82
+ DEPS=(openvino transformers torch pyyaml opencv-python)
83
+ else
84
+ DEPS=(openvino ultralytics)
85
+ fi
86
+ if [[ "${PRECISION}" == "INT8" ]]; then
87
+ DEPS+=(nncf)
88
+ fi
89
+ pip install -qU "${DEPS[@]}"
90
+
91
  echo "--- Downloading sample test video ---"
92
  if [[ ! -f VIRAT_S_000101.mp4 ]]; then
93
  wget -O VIRAT_S_000101.mp4 \
 
105
  EXPORT_LABEL="FP16"
106
  fi
107
 
108
+ if [[ "${MODEL_FAMILY}" == "yolo" ]]; then
109
+ echo "--- Exporting ${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
110
+ python3 -c "
111
  from ultralytics import YOLO
112
 
113
  model = YOLO('${MODEL_NAME}.pt')
114
+ model.export(format='openvino', half=${HALF_FLAG}, dynamic=False, imgsz=${INPUT_SIZE}, nms=False)
115
  print('Export complete: ${MODEL_NAME}_openvino_model/')
116
  "
117
+ else
118
+ echo "--- Exporting facebook/${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
119
+ python3 - "${MODEL_NAME}" "${HALF_FLAG}" "${INPUT_SIZE}" <<'PYEXPORT'
120
+ import sys
121
+ from pathlib import Path
122
+
123
+ import openvino as ov
124
+ import torch
125
+ import yaml
126
+ from transformers import DetrForObjectDetection
127
+
128
+ model_name, half_flag, size = sys.argv[1], sys.argv[2] == "True", int(sys.argv[3])
129
+ model_id = f"facebook/{model_name}"
130
+
131
+ # The "no_timm" revision uses the transformers ResNet backbone (no timm dependency).
132
+ detr = DetrForObjectDetection.from_pretrained(model_id, revision="no_timm").eval()
133
+ id2label = detr.config.id2label
134
+ # DETR predicts 91 COCO category ids (with "N/A" gaps) plus a no-object class.
135
+ # Keep the 80 real categories so class ids match YOLO26 (person = 0).
136
+ coco80 = [k for k in sorted(id2label) if id2label[k] != "N/A"]
137
+ assert len(coco80) == 80, f"unexpected label map with {len(coco80)} classes"
138
+
139
+
140
+ class DetrEndToEnd(torch.nn.Module):
141
+ """Input: RGB [0, 1] NCHW. Output: [1, 100, 6] = [x1, y1, x2, y2, conf, class_id] in input pixels."""
142
+
143
+ def __init__(self, model: torch.nn.Module) -> None:
144
+ super().__init__()
145
+ self.model = model
146
+ self.register_buffer("mean", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
147
+ self.register_buffer("std", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
148
+ self.register_buffer("coco80", torch.tensor(coco80))
149
+
150
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
151
+ out = self.model(pixel_values=(x - self.mean) / self.std, return_dict=True)
152
+ probs = out.logits.softmax(-1).index_select(-1, self.coco80)
153
+ conf, cls = probs.max(-1)
154
+ cx, cy, w, h = (out.pred_boxes * size).unbind(-1)
155
+ return torch.stack([cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2, conf, cls.float()], -1)
156
+
157
+
158
+ ov_model = ov.convert_model(
159
+ DetrEndToEnd(detr), example_input=torch.zeros(1, 3, size, size), input=[1, 3, size, size]
160
+ )
161
+ ov_model.outputs[0].get_tensor().set_names({"output0"})
162
+
163
+ names = {i: id2label[k] for i, k in enumerate(coco80)}
164
+ # DLStreamer reads preprocessing and the post-processor type from model_info.
165
+ model_info = {
166
+ "model_type": "YOLO",
167
+ "labels": " ".join(n.replace(" ", "_") for n in names.values()),
168
+ "resize_type": "standard",
169
+ "reverse_input_channels": "YES",
170
+ "scale_values": "255",
171
+ "iou_threshold": "0.7",
172
+ }
173
+ for key, value in model_info.items():
174
+ ov_model.set_rt_info(value, ["model_info", key])
175
+
176
+ out_dir = Path(f"{model_name}_openvino_model")
177
+ out_dir.mkdir(exist_ok=True)
178
+ ov.save_model(ov_model, out_dir / f"{model_name}.xml", compress_to_fp16=half_flag)
179
+ metadata = {
180
+ "description": f"{model_id} exported end-to-end with YOLO26-compatible output",
181
+ "license": "Apache-2.0",
182
+ "task": "detect",
183
+ "batch": 1,
184
+ "imgsz": [size, size],
185
+ "names": names,
186
+ "channels": 3,
187
+ "end2end": True,
188
+ }
189
+ with open(out_dir / "metadata.yaml", "w", encoding="utf-8") as f:
190
+ yaml.safe_dump(metadata, f, sort_keys=False)
191
+ print(f"Export complete: {out_dir}/")
192
+ PYEXPORT
193
+ fi
194
 
195
  if [[ "${PRECISION}" == "INT8" ]]; then
196
  echo "--- Quantizing to INT8 with NNCF ---"
 
211
  if not ret:
212
  cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
213
  continue
214
+ img = cv2.resize(frame, (${INPUT_SIZE}, ${INPUT_SIZE}))
215
  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
216
  img = img.transpose(2, 0, 1)[np.newaxis, ...]
217
  frames.append(img)
 
227
  calibration_dataset,
228
  preset=nncf.QuantizationPreset.MIXED,
229
  subset_size=300,
230
+ model_type=nncf.ModelType.TRANSFORMER if '${MODEL_FAMILY}' == 'detr' else None,
231
  )
232
 
233
  ov.save_model(quantized, '${MODEL_NAME}_loitering_int8.xml')