vagheshpatel commited on
Commit
953285e
·
verified ·
1 Parent(s): a77c745

Sync running-detection from metro-analytics-catalog

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ expected_output_dlstreamer.gif filter=lfs diff=lfs merge=lfs -text
37
+ expected_output_openvino.gif filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) Intel Corporation.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE
README.md ADDED
@@ -0,0 +1,545 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ license_link: LICENSE
4
+ library_name: openvino
5
+ pipeline_tag: object-detection
6
+ tags:
7
+ - openvino
8
+ - intel
9
+ - yolo
10
+ - yolo26
11
+ - detr
12
+ - running-detection
13
+ - speed-estimation
14
+ - tracking
15
+ - edge-ai
16
+ - metro
17
+ - dlstreamer
18
+ language:
19
+ - en
20
+ ---
21
+
22
+ # Running Detection
23
+
24
+ | Property | Value |
25
+ |---|---|
26
+ | **Category** | Object Detection + Tracking + Speed Estimation |
27
+ | **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics, default) or [DETR](https://huggingface.co/facebook/detr-resnet-50) (Facebook, alternative) + DLStreamer `gvatrack` (Kalman filter tracker) |
28
+ | **Source Framework** | PyTorch (Ultralytics, Hugging Face Transformers) |
29
+ | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
30
+ | **Inference Engine** | OpenVINO |
31
+ | **Hardware** | CPU, GPU, NPU |
32
+ | **Detected Class** | `person` (COCO class 0) |
33
+
34
+ ---
35
+
36
+ ## Overview
37
+
38
+ Running Detection is a Metro Analytics use case that flags people who are running or moving faster than a configurable speed threshold.
39
+ It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a state-of-the-art real-time object detector trained on the COCO dataset, quantized to INT8 and filtered at runtime to the `person` class.
40
+ Each detected person is assigned a persistent track ID across frames, and per-track speed is estimated from the frame-to-frame displacement of the bounding-box center.
41
+ A person is flagged as running when the estimated speed stays above the threshold for a short, sustained window, which suppresses single-frame jitter.
42
+
43
+ Typical Metro deployments include:
44
+
45
+ - **Platform Safety** -- flag people sprinting across platforms or toward closing train doors.
46
+ - **Incident Detection** -- surface sudden running that may indicate a chase, altercation, or emergency.
47
+ - **Crowd Flow Monitoring** -- distinguish normal walking pace from abnormal fast movement in concourses.
48
+ - **Restricted-Speed Zones** -- enforce walk-only areas such as escalators, ramps, and stairwells.
49
+
50
+ Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`.
51
+ Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment; larger variants improve recall in dense scenes.
52
+
53
+ ### Model Selection
54
+
55
+ [DETR](https://huggingface.co/facebook/detr-resnet-50) (DEtection TRansformer) is available as an alternative to YOLO26.
56
+ DETR is a transformer-based, NMS-free detector trained on COCO and released under the Apache-2.0 license.
57
+ The export script embeds DETR normalization, box decoding, and the class remap into the OpenVINO IR, so the DETR model produces the same output layout and class ids as YOLO26.
58
+ The same OpenVINO and DLStreamer samples run either model; select one with the `MODEL` constant at the top of each sample.
59
+
60
+ | Model | Variants | Input Size | Output | License |
61
+ |---|---|---|---|---|
62
+ | YOLO26 (default) | `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l` | 640 x 640 | `[1, 300, 6]` | AGPL-3.0 |
63
+ | DETR | `detr-resnet-50` | 800 x 800 | `[1, 100, 6]` | Apache-2.0 |
64
+
65
+ Choose YOLO26 for maximum throughput on edge devices.
66
+ Choose DETR when a permissively licensed transformer detector is preferred; it is heavier and runs at a lower frame rate.
67
+
68
+ ---
69
+
70
+ ## Prerequisites
71
+
72
+ Target stack: Intel DLStreamer **2026.3.0 RC1** with the matching OpenVINO GenAI bundle **2026.5.0.0.dev20261030** (OpenVINO Runtime **2026.5.0.dev20261030**).
73
+ Use the [native RC1 setup](../README.md#native-rc1-runtime) before running these samples; this is a development stack, not a stable release.
74
+ See the [device validation matrix](../DEVICE_SUPPORT_MATRIX.md) for measured CPU, GPU, and NPU results.
75
+
76
+ - Python 3.11+
77
+ - `ffmpeg` (`sudo apt install ffmpeg`) -- used by the samples to encode output video
78
+ - [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
79
+ - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)
80
+
81
+ Create and activate a Python virtual environment before running the scripts:
82
+
83
+ ```bash
84
+ python3 -m venv .venv --system-site-packages
85
+ source .venv/bin/activate
86
+ ```
87
+
88
+ > **Note:** The `--system-site-packages` flag is required so the virtual
89
+ > environment can access the system-installed OpenVINO and DLStreamer Python
90
+ > packages.
91
+
92
+ ---
93
+
94
+ ## Getting Started
95
+
96
+ ### Download and Quantize Model
97
+
98
+ Run the provided script to download, export to OpenVINO IR, and optionally quantize:
99
+
100
+ ```bash
101
+ wget https://huggingface.co/open-edge-platform/running-detection/resolve/main/export_and_quantize.sh
102
+ chmod +x export_and_quantize.sh
103
+ ./export_and_quantize.sh
104
+ ```
105
+
106
+ This exports the default **yolo26n** model in **FP16** precision.
107
+
108
+ #### Optional: Select a Different Model, Variant, or Precision
109
+
110
+ ```bash
111
+ ./export_and_quantize.sh yolo26n FP32 # full-precision
112
+ ./export_and_quantize.sh yolo26n INT8 # quantized
113
+ ./export_and_quantize.sh yolo26s # larger variant, default FP16
114
+ ./export_and_quantize.sh detr-resnet-50 # DETR alternative, default FP16
115
+ ./export_and_quantize.sh detr-resnet-50 INT8 # DETR quantized
116
+ ```
117
+
118
+ The first argument selects the model: any YOLO26 variant (`yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`) or DETR (`detr-resnet-50`).
119
+ The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**.
120
+
121
+ The script performs the following steps:
122
+
123
+ 1. Installs dependencies (`openvino`, `ultralytics` for YOLO26 or `transformers`, `torch` for DETR, `opencv-python`; adds `nncf` for INT8).
124
+ 2. Downloads the sample running video (`running.mp4`) and extracts a calibration frame (`test.jpg`).
125
+ 3. Downloads the PyTorch weights and exports to OpenVINO IR.
126
+ 4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.
127
+
128
+ Output files (replace `yolo26n` with the selected model, for example `detr-resnet-50`):
129
+
130
+ - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
131
+ - `yolo26n_running_int8.xml` / `yolo26n_running_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*.
132
+
133
+ #### Precision / Device Compatibility
134
+
135
+ | Precision | CPU | GPU | NPU |
136
+ |---|---|---|---|
137
+ | FP32 | Yes | Yes | No |
138
+ | FP16 | Yes | Yes | Yes |
139
+ | INT8 | Yes | Yes | Yes |
140
+
141
+ > **Note:** The INT8 calibration uses the extracted sample frame.
142
+ > For production accuracy, replace it with a representative set of frames from
143
+ > the target deployment site.
144
+
145
+ ### Speed Threshold
146
+
147
+ Running is defined by a per-track speed threshold expressed in pixels per second:
148
+
149
+ ```text
150
+ RUNNING_SPEED = 250.0 # pixels/second (demo value for the sample clip)
151
+ MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging
152
+ ```
153
+
154
+ > **Note:** Pixel speed depends on camera resolution, framing, and distance to
155
+ > the subject, so `RUNNING_SPEED` must be tuned per site. For a calibrated
156
+ > metric speed (meters/second), convert pixel displacement using the known
157
+ > ground-sampling distance of the scene.
158
+
159
+ ### OpenVINO Sample
160
+
161
+ The sample below runs YOLO26 inference on the sample video, filters to the `person` class, assigns track IDs with a lightweight nearest-center tracker, estimates per-track pixel speed, and flags people who run faster than `RUNNING_SPEED` for at least `MIN_RUN_FRAMES` frames.
162
+ YOLO26 is end-to-end (NMS-free), so no manual non-maximum suppression is needed.
163
+ The annotated result is written to `output_openvino.mp4`, with a latched
164
+ `RUNNING DETECTED` / `NO RUNNING DETECTED` status banner across the top.
165
+ Change the `DEVICE` string to run on CPU, GPU, or NPU.
166
+
167
+ The full sample is in [openvino_sample.py](openvino_sample.py); run it with `python3 openvino_sample.py` or copy the code below.
168
+
169
+ <!-- sample: openvino_sample.py -->
170
+ ```python
171
+ # SPDX-License-Identifier: MIT
172
+ # Copyright (C) Intel Corporation
173
+ #
174
+ # Running Detection: OpenVINO Runtime sample.
175
+ # See README.md for prerequisites and inputs. Run: python3 openvino_sample.py
176
+
177
+ import subprocess
178
+
179
+ import cv2
180
+ import numpy as np
181
+ import openvino as ov
182
+
183
+ PERSON_CLASS_ID = 0
184
+ CONF_THRESHOLD = 0.4
185
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
186
+ MODEL = "yolo26n"
187
+ INPUT_SIZE = 800 if MODEL.startswith("detr") else 640
188
+ RUNNING_SPEED = 250.0 # pixels/second
189
+ MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging
190
+ MAX_MATCH_DIST = 120 # max center distance (px) to link a track across frames
191
+ ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering
192
+
193
+ # Change DEVICE to "GPU" or "NPU" to run on integrated GPU or NPU.
194
+ DEVICE = "CPU"
195
+ INPUT_VIDEO = "running.mp4"
196
+
197
+ core = ov.Core()
198
+ model = core.read_model(f"{MODEL}_openvino_model/{MODEL}.xml")
199
+ compiled = core.compile_model(model, DEVICE)
200
+ output_port = compiled.output(0)
201
+
202
+ cap = cv2.VideoCapture(INPUT_VIDEO)
203
+ fps = cap.get(cv2.CAP_PROP_FPS) or 25.0
204
+ width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
205
+ height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
206
+ ALERT_HOLD_FRAMES = max(1, int(ALERT_HOLD_SECONDS * fps))
207
+
208
+ proc = subprocess.Popen(
209
+ ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24",
210
+ "-s", f"{width}x{height}", "-r", str(fps),
211
+ "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p",
212
+ "-movflags", "+faststart", "output_openvino.mp4"],
213
+ stdin=subprocess.PIPE, stderr=subprocess.DEVNULL,
214
+ )
215
+
216
+ tracks: dict[int, dict] = {} # id -> {cx, cy, run_frames}
217
+ next_id = 0
218
+ flagged: set[int] = set()
219
+ alert_hold = 0
220
+ frame_idx = 0
221
+
222
+ while True:
223
+ ok, frame = cap.read()
224
+ if not ok:
225
+ break
226
+ frame_idx += 1
227
+ dt = 1.0 / fps
228
+
229
+ blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
230
+ blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
231
+ blob = blob.transpose(2, 0, 1)[np.newaxis, ...] # NCHW
232
+
233
+ output = compiled([blob])[output_port][0]
234
+ mask = (output[:, 4] >= CONF_THRESHOLD) & (output[:, 5].astype(int) == PERSON_CLASS_ID)
235
+ dets = output[mask]
236
+
237
+ sx, sy = width / INPUT_SIZE, height / INPUT_SIZE
238
+ detections = []
239
+ for det in dets:
240
+ x1, y1 = int(det[0] * sx), int(det[1] * sy)
241
+ x2, y2 = int(det[2] * sx), int(det[3] * sy)
242
+ detections.append((x1, y1, x2, y2, (x1 + x2) // 2, (y1 + y2) // 2))
243
+
244
+ # Greedy nearest-center association to the previous frame's tracks.
245
+ used = set()
246
+ assignments = {}
247
+ for i, (_, _, _, _, cx, cy) in enumerate(detections):
248
+ best_id, best_dist = None, MAX_MATCH_DIST
249
+ for tid, tr in tracks.items():
250
+ if tid in used:
251
+ continue
252
+ d = np.hypot(cx - tr["cx"], cy - tr["cy"])
253
+ if d < best_dist:
254
+ best_id, best_dist = tid, d
255
+ if best_id is None:
256
+ best_id = next_id
257
+ next_id += 1
258
+ tracks[best_id] = {"cx": cx, "cy": cy, "run_frames": 0}
259
+ used.add(best_id)
260
+ assignments[i] = best_id
261
+
262
+ new_tracks = {}
263
+ frame_running = False
264
+ for i, (x1, y1, x2, y2, cx, cy) in enumerate(detections):
265
+ tid = assignments[i]
266
+ prev = tracks.get(tid, {"cx": cx, "cy": cy, "run_frames": 0})
267
+ speed = np.hypot(cx - prev["cx"], cy - prev["cy"]) / dt
268
+ run_frames = prev["run_frames"] + 1 if speed >= RUNNING_SPEED else 0
269
+ new_tracks[tid] = {"cx": cx, "cy": cy, "run_frames": run_frames}
270
+
271
+ is_running = run_frames >= MIN_RUN_FRAMES
272
+ frame_running = frame_running or is_running
273
+ color = (0, 0, 255) if is_running else (0, 255, 0)
274
+ cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
275
+ label = f"RUNNING {int(speed)}px/s" if is_running else f"{int(speed)}px/s"
276
+ cv2.putText(frame, label, (x1, max(y1 - 8, 12)),
277
+ cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2)
278
+ if is_running and tid not in flagged:
279
+ flagged.add(tid)
280
+ print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True)
281
+
282
+ tracks = new_tracks
283
+
284
+ # Latch the alert so the banner reflects a sustained state, not a single
285
+ # transient frame: once running is seen it stays on for ALERT_HOLD_FRAMES.
286
+ alert_hold = ALERT_HOLD_FRAMES if frame_running else max(0, alert_hold - 1)
287
+ alert_on = alert_hold > 0
288
+ banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED"
289
+ banner_color = (0, 0, 255) if alert_on else (0, 180, 0)
290
+ cv2.rectangle(frame, (0, 0), (width, 40), (0, 0, 0), -1)
291
+ cv2.putText(frame, banner, (10, 28),
292
+ cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2)
293
+
294
+ proc.stdin.write(frame.tobytes())
295
+
296
+ cap.release()
297
+ proc.stdin.close()
298
+ proc.wait()
299
+ print("Wrote output_openvino.mp4", flush=True)
300
+ ```
301
+ <!-- /sample -->
302
+
303
+ **Device targets:**
304
+
305
+ - `"CPU"` -- default, works on all Intel platforms.
306
+ - `"GPU"` -- Intel integrated or discrete GPU.
307
+ - `"NPU"` -- Intel NPU (different throughput profile; validate with `benchmark_app -d NPU`).
308
+
309
+ Expected console output:
310
+
311
+ ```text
312
+ RUNNING id=0 speed=312px/s frame=14
313
+ ...
314
+ Wrote output_openvino.mp4
315
+ ```
316
+
317
+ `output_openvino.mp4` shows a green box around each person, turning red with a
318
+ `RUNNING` label when the person's speed exceeds the threshold.
319
+
320
+ #### Expected Output
321
+
322
+ ![OpenVINO expected output](expected_output_openvino.gif)
323
+
324
+ ### DLStreamer Sample
325
+
326
+ The pipeline below runs the FP16 YOLO26 detector via `gvadetect` on the sample
327
+ video, attaches persistent track IDs with `gvatrack`
328
+ (`short-term-imageless` tracker), and overlays bounding boxes with
329
+ `gvawatermark`. Frames are pulled from an `appsink`; per-track pixel speed is
330
+ computed from the frame-to-frame displacement of each track center, and a
331
+ `RUNNING` event is raised when the speed stays above `RUNNING_SPEED` for at
332
+ least `MIN_RUN_FRAMES` frames. A latched `RUNNING DETECTED` /
333
+ `NO RUNNING DETECTED` status banner is drawn across the top of every frame.
334
+ `gvawatermark` renders boxes for the `person` class only. The annotated result
335
+ is muxed to `output_dlstreamer.mp4`.
336
+
337
+ > **Notes on running this sample:**
338
+ >
339
+ > - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml` or `detr-resnet-50_openvino_model/detr-resnet-50.xml`). Class names are
340
+ > read automatically from the model's embedded `metadata.yaml` by
341
+ > DLStreamer 2026.0+ -- no external `labels-file` is required.
342
+ > - Export `PYTHONPATH` so the DLStreamer Python module is importable:
343
+ >
344
+ > ```bash
345
+ > source /opt/intel/openvino_2026/setupvars.sh
346
+ > source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
347
+ > export PYTHONPATH=/opt/intel/dlstreamer/python:\
348
+ > /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
349
+ > ```
350
+
351
+ The full sample is in [dlstreamer_sample.py](dlstreamer_sample.py); run it with `python3 dlstreamer_sample.py` or copy the code below.
352
+
353
+ <!-- sample: dlstreamer_sample.py -->
354
+ ```python
355
+ # SPDX-License-Identifier: MIT
356
+ # Copyright (C) Intel Corporation
357
+ #
358
+ # Running Detection: DLStreamer pipeline sample.
359
+ # See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py
360
+
361
+ import subprocess
362
+ from collections import defaultdict
363
+
364
+ import numpy as np
365
+ import gi
366
+
367
+ gi.require_version("Gst", "1.0")
368
+ gi.require_version("GstAnalytics", "1.0")
369
+ gi.require_version("GstVideo", "1.0")
370
+ from gi.repository import Gst, GLib, GstAnalytics, GstVideo
371
+
372
+ Gst.init([])
373
+
374
+ # Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
375
+ import cv2
376
+
377
+ INPUT_VIDEO = "running.mp4"
378
+ RUNNING_SPEED = 250.0 # pixels/second
379
+ MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging
380
+ ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering
381
+
382
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
383
+ MODEL = "yolo26n"
384
+
385
+ # For CPU: change device=GPU to device=CPU.
386
+ # For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
387
+ # gvawatermark draws only person ROIs (displ-cfg=show-roi=person) so boxes for
388
+ # other COCO classes are not rendered.
389
+ pipeline_str = (
390
+ f"filesrc location={INPUT_VIDEO} ! decodebin3 ! "
391
+ "videoconvert ! "
392
+ f"gvadetect model={MODEL}_openvino_model/{MODEL}.xml "
393
+ "device=GPU "
394
+ "threshold=0.4 ! queue ! "
395
+ "gvatrack tracking-type=short-term-imageless ! queue ! "
396
+ "gvawatermark displ-cfg=show-roi=person ! "
397
+ "videoconvert ! video/x-raw,format=BGR ! "
398
+ "appsink name=sink emit-signals=false sync=false"
399
+ )
400
+ pipeline = Gst.parse_launch(pipeline_str)
401
+ appsink = pipeline.get_by_name("sink")
402
+
403
+ pipeline.set_state(Gst.State.PLAYING)
404
+
405
+ proc = None
406
+ prev_center: dict[int, tuple[int, int]] = {}
407
+ run_frames: dict[int, int] = defaultdict(int)
408
+ flagged: set[int] = set()
409
+ prev_pts = None
410
+ alert_hold = 0
411
+ alert_hold_frames = 40 # updated from the real framerate on the first frame
412
+ frame_idx = 0
413
+
414
+ while True:
415
+ sample = appsink.emit("pull-sample")
416
+ if sample is None:
417
+ break
418
+
419
+ buf = sample.get_buffer()
420
+ caps = sample.get_caps()
421
+ struct = caps.get_structure(0)
422
+ width = struct.get_value("width")
423
+ height = struct.get_value("height")
424
+ frame_idx += 1
425
+
426
+ # Start ffmpeg encoder on the first frame.
427
+ if proc is None:
428
+ ok, fps_num, fps_den = struct.get_fraction("framerate")
429
+ fps = fps_num / fps_den if ok and fps_den > 0 else 25.0
430
+ alert_hold_frames = max(1, int(ALERT_HOLD_SECONDS * fps))
431
+ proc = subprocess.Popen(
432
+ ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24",
433
+ "-s", f"{width}x{height}", "-r", str(fps),
434
+ "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p",
435
+ "-movflags", "+faststart", "output_dlstreamer.mp4"],
436
+ stdin=subprocess.PIPE, stderr=subprocess.DEVNULL,
437
+ )
438
+
439
+ # Elapsed time since the previous frame from buffer timestamps.
440
+ pts = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else frame_idx / fps
441
+ dt = (pts - prev_pts) if (prev_pts is not None and pts > prev_pts) else 1.0 / fps
442
+ prev_pts = pts
443
+
444
+ # Read detection / tracking metadata via GstAnalytics.
445
+ rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
446
+ regions = []
447
+ if rmeta is not None:
448
+ od_entries = []
449
+ trk_map = {} # metadata_id -> tracking_id
450
+ idx = 1
451
+ while True:
452
+ ok_od, od = rmeta.get_od_mtd(idx)
453
+ ok_trk, trk = rmeta.get_tracking_mtd(idx)
454
+ if not ok_od and not ok_trk:
455
+ break
456
+ if ok_od:
457
+ label = GLib.quark_to_string(od.get_obj_type())
458
+ _, x, y, w, h, _ = od.get_location()
459
+ od_entries.append((idx, label, int(x + w / 2), int(y + h / 2)))
460
+ if ok_trk:
461
+ ok2, tid, _, _, _ = trk.get_info()
462
+ if ok2:
463
+ trk_map[idx] = tid
464
+ idx += 1
465
+ for od_id, label, cx, cy in od_entries:
466
+ if label != "person":
467
+ continue
468
+ tid = 0
469
+ for trk_meta_id, tracking_id in trk_map.items():
470
+ if rmeta.get_relation(od_id, trk_meta_id) != GstAnalytics.RelTypes.NONE:
471
+ tid = tracking_id
472
+ break
473
+ regions.append((tid, cx, cy))
474
+
475
+ # Map buffer read-only and copy pixels to a writable numpy array.
476
+ success, map_info = buf.map(Gst.MapFlags.READ)
477
+ if not success:
478
+ continue
479
+ video_info = GstVideo.VideoInfo.new_from_caps(caps)
480
+ arr = np.ndarray((height, width, 3), dtype=np.uint8,
481
+ buffer=map_info.data, strides=(video_info.stride[0], 3, 1)).copy()
482
+ buf.unmap(map_info)
483
+
484
+ frame_running = False
485
+ for tid, cx, cy in regions:
486
+ px, py = prev_center.get(tid, (cx, cy))
487
+ speed = np.hypot(cx - px, cy - py) / dt if dt > 0 else 0.0
488
+ prev_center[tid] = (cx, cy)
489
+ run_frames[tid] = run_frames[tid] + 1 if speed >= RUNNING_SPEED else 0
490
+
491
+ if run_frames[tid] >= MIN_RUN_FRAMES:
492
+ frame_running = True
493
+ cv2.putText(arr, f"RUNNING {int(speed)}px/s", (cx - 40, max(cy - 20, 12)),
494
+ cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
495
+ if tid not in flagged:
496
+ flagged.add(tid)
497
+ print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True)
498
+
499
+ # Latch the alert so the banner reflects a sustained state, not a single
500
+ # transient frame: once running is seen it stays on for alert_hold_frames.
501
+ alert_hold = alert_hold_frames if frame_running else max(0, alert_hold - 1)
502
+ alert_on = alert_hold > 0
503
+ banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED"
504
+ banner_color = (0, 0, 255) if alert_on else (0, 180, 0)
505
+ cv2.rectangle(arr, (0, 0), (width, 40), (0, 0, 0), -1)
506
+ cv2.putText(arr, banner, (10, 28),
507
+ cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2)
508
+
509
+ proc.stdin.write(arr.tobytes())
510
+
511
+ pipeline.set_state(Gst.State.NULL)
512
+ if proc:
513
+ proc.stdin.close()
514
+ proc.wait()
515
+ print("Wrote output_dlstreamer.mp4", flush=True)
516
+ ```
517
+ <!-- /sample -->
518
+
519
+ #### Expected Output
520
+
521
+ ![DLStreamer expected output](expected_output_dlstreamer.gif)
522
+
523
+ **Device targets:**
524
+
525
+ - `device=GPU` -- default in the sample code.
526
+ - `device=CPU` -- change `device=GPU` to `device=CPU`.
527
+ - `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization.
528
+
529
+ ---
530
+
531
+ ## License
532
+
533
+ Licensed under the MIT License. See [LICENSE](LICENSE) for details.
534
+
535
+ ## References
536
+
537
+ - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
538
+ - [DETR Model Card (facebook/detr-resnet-50)](https://huggingface.co/facebook/detr-resnet-50)
539
+ - [End-to-End Object Detection with Transformers (DETR paper)](https://arxiv.org/abs/2005.12872)
540
+ - [Ultralytics Multi-Object Tracking](https://docs.ultralytics.com/modes/track/)
541
+ - [Intel DLStreamer gvatrack](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/elements/gvatrack.html)
542
+ - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb)
543
+ - [OpenVINO Documentation](https://docs.openvino.ai/)
544
+ - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html)
545
+ - [Intel DLStreamer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/index.html)
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "running-detection",
3
+ "library_name": "openvino",
4
+ "pipeline_tag": "object-detection",
5
+ "category": "Object Detection + Tracking + Speed Estimation",
6
+ "base_model": "YOLO26 (Ultralytics, default) or DETR facebook/detr-resnet-50 (alternative) + DLStreamer `gvatrack` (Kalman filter tracker)",
7
+ "source_framework": "PyTorch (Ultralytics, Hugging Face Transformers)",
8
+ "supported_precisions": [
9
+ "FP32",
10
+ "FP16",
11
+ "INT8"
12
+ ],
13
+ "inference_engine": "OpenVINO",
14
+ "hardware": [
15
+ "CPU",
16
+ "GPU",
17
+ "NPU"
18
+ ],
19
+ "detected_classes": "`person` (COCO class 0)",
20
+ "hf_repo_id": "open-edge-platform/running-detection",
21
+ "description": "Running detection use case using YOLO26 (default) or DETR person detection with gvatrack tracking and per-track pixel-speed estimation to flag people running faster than a configurable threshold",
22
+ "selectable_models": [
23
+ "yolo26n",
24
+ "yolo26s",
25
+ "yolo26m",
26
+ "yolo26l",
27
+ "detr-resnet-50"
28
+ ],
29
+ "default_model": "yolo26n"
30
+ }
dlstreamer_sample.py ADDED
@@ -0,0 +1,161 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-License-Identifier: MIT
2
+ # Copyright (C) Intel Corporation
3
+ #
4
+ # Running Detection: DLStreamer pipeline sample.
5
+ # See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py
6
+
7
+ import subprocess
8
+ from collections import defaultdict
9
+
10
+ import numpy as np
11
+ import gi
12
+
13
+ gi.require_version("Gst", "1.0")
14
+ gi.require_version("GstAnalytics", "1.0")
15
+ gi.require_version("GstVideo", "1.0")
16
+ from gi.repository import Gst, GLib, GstAnalytics, GstVideo
17
+
18
+ Gst.init([])
19
+
20
+ # Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
21
+ import cv2
22
+
23
+ INPUT_VIDEO = "running.mp4"
24
+ RUNNING_SPEED = 250.0 # pixels/second
25
+ MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging
26
+ ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering
27
+
28
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
29
+ MODEL = "yolo26n"
30
+
31
+ # For CPU: change device=GPU to device=CPU.
32
+ # For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
33
+ # gvawatermark draws only person ROIs (displ-cfg=show-roi=person) so boxes for
34
+ # other COCO classes are not rendered.
35
+ pipeline_str = (
36
+ f"filesrc location={INPUT_VIDEO} ! decodebin3 ! "
37
+ "videoconvert ! "
38
+ f"gvadetect model={MODEL}_openvino_model/{MODEL}.xml "
39
+ "device=GPU "
40
+ "threshold=0.4 ! queue ! "
41
+ "gvatrack tracking-type=short-term-imageless ! queue ! "
42
+ "gvawatermark displ-cfg=show-roi=person ! "
43
+ "videoconvert ! video/x-raw,format=BGR ! "
44
+ "appsink name=sink emit-signals=false sync=false"
45
+ )
46
+ pipeline = Gst.parse_launch(pipeline_str)
47
+ appsink = pipeline.get_by_name("sink")
48
+
49
+ pipeline.set_state(Gst.State.PLAYING)
50
+
51
+ proc = None
52
+ prev_center: dict[int, tuple[int, int]] = {}
53
+ run_frames: dict[int, int] = defaultdict(int)
54
+ flagged: set[int] = set()
55
+ prev_pts = None
56
+ alert_hold = 0
57
+ alert_hold_frames = 40 # updated from the real framerate on the first frame
58
+ frame_idx = 0
59
+
60
+ while True:
61
+ sample = appsink.emit("pull-sample")
62
+ if sample is None:
63
+ break
64
+
65
+ buf = sample.get_buffer()
66
+ caps = sample.get_caps()
67
+ struct = caps.get_structure(0)
68
+ width = struct.get_value("width")
69
+ height = struct.get_value("height")
70
+ frame_idx += 1
71
+
72
+ # Start ffmpeg encoder on the first frame.
73
+ if proc is None:
74
+ ok, fps_num, fps_den = struct.get_fraction("framerate")
75
+ fps = fps_num / fps_den if ok and fps_den > 0 else 25.0
76
+ alert_hold_frames = max(1, int(ALERT_HOLD_SECONDS * fps))
77
+ proc = subprocess.Popen(
78
+ ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24",
79
+ "-s", f"{width}x{height}", "-r", str(fps),
80
+ "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p",
81
+ "-movflags", "+faststart", "output_dlstreamer.mp4"],
82
+ stdin=subprocess.PIPE, stderr=subprocess.DEVNULL,
83
+ )
84
+
85
+ # Elapsed time since the previous frame from buffer timestamps.
86
+ pts = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else frame_idx / fps
87
+ dt = (pts - prev_pts) if (prev_pts is not None and pts > prev_pts) else 1.0 / fps
88
+ prev_pts = pts
89
+
90
+ # Read detection / tracking metadata via GstAnalytics.
91
+ rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
92
+ regions = []
93
+ if rmeta is not None:
94
+ od_entries = []
95
+ trk_map = {} # metadata_id -> tracking_id
96
+ idx = 1
97
+ while True:
98
+ ok_od, od = rmeta.get_od_mtd(idx)
99
+ ok_trk, trk = rmeta.get_tracking_mtd(idx)
100
+ if not ok_od and not ok_trk:
101
+ break
102
+ if ok_od:
103
+ label = GLib.quark_to_string(od.get_obj_type())
104
+ _, x, y, w, h, _ = od.get_location()
105
+ od_entries.append((idx, label, int(x + w / 2), int(y + h / 2)))
106
+ if ok_trk:
107
+ ok2, tid, _, _, _ = trk.get_info()
108
+ if ok2:
109
+ trk_map[idx] = tid
110
+ idx += 1
111
+ for od_id, label, cx, cy in od_entries:
112
+ if label != "person":
113
+ continue
114
+ tid = 0
115
+ for trk_meta_id, tracking_id in trk_map.items():
116
+ if rmeta.get_relation(od_id, trk_meta_id) != GstAnalytics.RelTypes.NONE:
117
+ tid = tracking_id
118
+ break
119
+ regions.append((tid, cx, cy))
120
+
121
+ # Map buffer read-only and copy pixels to a writable numpy array.
122
+ success, map_info = buf.map(Gst.MapFlags.READ)
123
+ if not success:
124
+ continue
125
+ video_info = GstVideo.VideoInfo.new_from_caps(caps)
126
+ arr = np.ndarray((height, width, 3), dtype=np.uint8,
127
+ buffer=map_info.data, strides=(video_info.stride[0], 3, 1)).copy()
128
+ buf.unmap(map_info)
129
+
130
+ frame_running = False
131
+ for tid, cx, cy in regions:
132
+ px, py = prev_center.get(tid, (cx, cy))
133
+ speed = np.hypot(cx - px, cy - py) / dt if dt > 0 else 0.0
134
+ prev_center[tid] = (cx, cy)
135
+ run_frames[tid] = run_frames[tid] + 1 if speed >= RUNNING_SPEED else 0
136
+
137
+ if run_frames[tid] >= MIN_RUN_FRAMES:
138
+ frame_running = True
139
+ cv2.putText(arr, f"RUNNING {int(speed)}px/s", (cx - 40, max(cy - 20, 12)),
140
+ cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
141
+ if tid not in flagged:
142
+ flagged.add(tid)
143
+ print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True)
144
+
145
+ # Latch the alert so the banner reflects a sustained state, not a single
146
+ # transient frame: once running is seen it stays on for alert_hold_frames.
147
+ alert_hold = alert_hold_frames if frame_running else max(0, alert_hold - 1)
148
+ alert_on = alert_hold > 0
149
+ banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED"
150
+ banner_color = (0, 0, 255) if alert_on else (0, 180, 0)
151
+ cv2.rectangle(arr, (0, 0), (width, 40), (0, 0, 0), -1)
152
+ cv2.putText(arr, banner, (10, 28),
153
+ cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2)
154
+
155
+ proc.stdin.write(arr.tobytes())
156
+
157
+ pipeline.set_state(Gst.State.NULL)
158
+ if proc:
159
+ proc.stdin.close()
160
+ proc.wait()
161
+ print("Wrote output_dlstreamer.mp4", flush=True)
expected_output_dlstreamer.gif ADDED

Git LFS Details

  • SHA256: fee2bcb5e41d9d7fec7c4df99d96f358cc62d59b519c009e0fc15bcf27f1d471
  • Pointer size: 132 Bytes
  • Size of remote file: 7.73 MB
expected_output_openvino.gif ADDED

Git LFS Details

  • SHA256: f2b82067634f39529bcce2ea65a309118b06b6defbd70ca00f8f81604210e8a8
  • Pointer size: 133 Bytes
  • Size of remote file: 26.3 MB
export_and_quantize.sh ADDED
@@ -0,0 +1,257 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ # SPDX-License-Identifier: MIT
3
+ # Copyright (C) Intel Corporation
4
+ #
5
+ # Export a YOLO26 or DETR person detector to OpenVINO IR for running detection.
6
+ # Usage: ./export_and_quantize.sh [MODEL_VARIANT] [PRECISION]
7
+ # Example: ./export_and_quantize.sh yolo26n FP16
8
+ # Example: ./export_and_quantize.sh detr-resnet-50 FP16
9
+ #
10
+ # Supported model variants:
11
+ # yolo26n, yolo26s, yolo26m, yolo26l -- Ultralytics YOLO26 (default)
12
+ # detr-resnet-50 -- Facebook DETR (Hugging Face)
13
+ #
14
+ # DETR is exported end-to-end: ImageNet normalization, softmax,
15
+ # box decoding, and the COCO-91 to COCO-80 class remap are embedded in the
16
+ # graph, so the IR produces the same [1, N, 6] output layout and class ids as
17
+ # YOLO26 and runs with the same OpenVINO and DLStreamer samples.
18
+ #
19
+ # Supported precisions:
20
+ # FP32 -- Full-precision floating-point weights
21
+ # FP16 -- Half-precision floating-point weights (default)
22
+ # INT8 -- Quantized 8-bit integer weights (requires NNCF)
23
+ #
24
+ # Precision / device compatibility:
25
+ # | Precision | CPU | GPU | NPU |
26
+ # |-----------|-----|-----|-----|
27
+ # | FP32 | Yes | Yes | No |
28
+ # | FP16 | Yes | Yes | Yes |
29
+ # | INT8 | Yes | Yes | Yes |
30
+
31
+ set -euo pipefail
32
+
33
+ MODEL_NAME="${1:-yolo26n}"
34
+ PRECISION="${2:-FP16}"
35
+ PRECISION="$(echo "${PRECISION}" | tr '[:lower:]' '[:upper:]')"
36
+
37
+ if [[ "${PRECISION}" != "FP32" && "${PRECISION}" != "FP16" && "${PRECISION}" != "INT8" ]]; then
38
+ echo "ERROR: unsupported precision '${PRECISION}'. Choose FP32, FP16, or INT8." >&2
39
+ exit 1
40
+ fi
41
+
42
+ case "${MODEL_NAME}" in
43
+ yolo26n|yolo26s|yolo26m|yolo26l)
44
+ MODEL_FAMILY="yolo"
45
+ INPUT_SIZE=640
46
+ ;;
47
+ detr-resnet-50)
48
+ MODEL_FAMILY="detr"
49
+ INPUT_SIZE=800
50
+ ;;
51
+ *)
52
+ echo "ERROR: unsupported model '${MODEL_NAME}'. Choose yolo26[n|s|m|l] or detr-resnet-50." >&2
53
+ exit 1
54
+ ;;
55
+ esac
56
+
57
+ # Pre-downscaled sample clip of a man running on an outdoor track (720x1280, 25 fps).
58
+ VIDEO_URL="https://www.pexels.com/download/video/37709462/?fps=25.0&h=1280&w=720"
59
+
60
+ # Ask for approval before downloading models and sample files
61
+ echo ""
62
+ echo "This script will download:"
63
+ echo " - ${MODEL_NAME} model weights (if not cached locally)"
64
+ echo " - Sample running video and a calibration frame"
65
+ echo ""
66
+ read -p "Continue with downloads? (yes/no): " APPROVAL
67
+ if [[ "${APPROVAL}" != "yes" ]]; then
68
+ echo "Download cancelled by user."
69
+ exit 0
70
+ fi
71
+
72
+ # Ping the HuggingFace repo to register a tracked download of config.json.
73
+ # This is best-effort: a failure (offline, or before the repo is published)
74
+ # must not stop the export.
75
+ echo "--- Registering HuggingFace download (tracking ping) ---"
76
+ HF_REPO_ID="open-edge-platform/running-detection"
77
+ HF_CONFIG_URL="https://huggingface.co/${HF_REPO_ID}/resolve/main/config.json"
78
+ if curl -fsSL -o /dev/null "${HF_CONFIG_URL}"; then
79
+ echo "Registered HuggingFace download for ${HF_REPO_ID}"
80
+ else
81
+ echo "WARNING: HuggingFace tracking ping failed (offline?); continuing." >&2
82
+ fi
83
+ echo ""
84
+ echo "--- Installing dependencies ---"
85
+ if [[ "${MODEL_FAMILY}" == "detr" ]]; then
86
+ DEPS=(openvino transformers torch pyyaml opencv-python)
87
+ else
88
+ DEPS=(openvino ultralytics opencv-python)
89
+ fi
90
+ if [[ "${PRECISION}" == "INT8" ]]; then
91
+ DEPS+=(nncf)
92
+ fi
93
+ pip install -qU "${DEPS[@]}"
94
+
95
+ echo "--- Downloading sample running video ---"
96
+ if [[ ! -f running.mp4 ]]; then
97
+ wget -q -O running.mp4 "${VIDEO_URL}"
98
+ echo "Downloaded: running.mp4"
99
+ else
100
+ echo "Already present: running.mp4"
101
+ fi
102
+
103
+ echo "--- Extracting a calibration frame (test.jpg) ---"
104
+ if [[ ! -f test.jpg ]]; then
105
+ python3 -c "
106
+ import cv2
107
+ cap = cv2.VideoCapture('running.mp4')
108
+ cap.set(cv2.CAP_PROP_POS_FRAMES, 30)
109
+ ok, frame = cap.read()
110
+ if not ok:
111
+ cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
112
+ ok, frame = cap.read()
113
+ cap.release()
114
+ if not ok:
115
+ raise SystemExit('Could not read a frame from running.mp4')
116
+ cv2.imwrite('test.jpg', frame)
117
+ print('Extracted: test.jpg')
118
+ "
119
+ else
120
+ echo "Already present: test.jpg"
121
+ fi
122
+
123
+ if [[ "${PRECISION}" == "FP32" ]]; then
124
+ HALF_FLAG="False"
125
+ EXPORT_LABEL="FP32"
126
+ else
127
+ HALF_FLAG="True"
128
+ EXPORT_LABEL="FP16"
129
+ fi
130
+
131
+ if [[ "${MODEL_FAMILY}" == "yolo" ]]; then
132
+ echo "--- Exporting ${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
133
+ python3 -c "
134
+ from ultralytics import YOLO
135
+
136
+ model = YOLO('${MODEL_NAME}.pt')
137
+ model.export(format='openvino', half=${HALF_FLAG}, dynamic=False, imgsz=${INPUT_SIZE}, nms=False)
138
+ print('Export complete: ${MODEL_NAME}_openvino_model/')
139
+ "
140
+ else
141
+ echo "--- Exporting facebook/${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
142
+ python3 - "${MODEL_NAME}" "${HALF_FLAG}" "${INPUT_SIZE}" <<'PYEXPORT'
143
+ import sys
144
+ from pathlib import Path
145
+
146
+ import openvino as ov
147
+ import torch
148
+ import yaml
149
+ from transformers import DetrConfig, DetrForObjectDetection
150
+
151
+ model_name, half_flag, size = sys.argv[1], sys.argv[2] == "True", int(sys.argv[3])
152
+ model_id = f"facebook/{model_name}"
153
+
154
+ # The "no_timm" revision uses the transformers ResNet backbone (no timm dependency).
155
+ config_dict, _ = DetrConfig.get_config_dict(model_id, revision="no_timm")
156
+ # Older no_timm configs serialize the standard non-dilated backbone as null.
157
+ if config_dict.get("dilation") is None:
158
+ config_dict["dilation"] = False
159
+ config = DetrConfig.from_dict(config_dict)
160
+ detr = DetrForObjectDetection.from_pretrained(model_id, revision="no_timm", config=config).eval()
161
+ id2label = detr.config.id2label
162
+ # DETR predicts 91 COCO category ids (with "N/A" gaps) plus a no-object class.
163
+ # Keep the 80 real categories so class ids match YOLO26 (person = 0).
164
+ coco80 = [k for k in sorted(id2label) if id2label[k] != "N/A"]
165
+ assert len(coco80) == 80, f"unexpected label map with {len(coco80)} classes"
166
+
167
+
168
+ class DetrEndToEnd(torch.nn.Module):
169
+ """Input: RGB [0, 1] NCHW. Output: [1, 100, 6] = [x1, y1, x2, y2, conf, class_id] in input pixels."""
170
+
171
+ def __init__(self, model: torch.nn.Module) -> None:
172
+ super().__init__()
173
+ self.model = model
174
+ self.register_buffer("mean", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
175
+ self.register_buffer("std", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
176
+ self.register_buffer("coco80", torch.tensor(coco80))
177
+
178
+ def forward(self, x: torch.Tensor) -> torch.Tensor:
179
+ out = self.model(pixel_values=(x - self.mean) / self.std, return_dict=True)
180
+ probs = out.logits.softmax(-1).index_select(-1, self.coco80)
181
+ conf, cls = probs.max(-1)
182
+ cx, cy, w, h = (out.pred_boxes * size).unbind(-1)
183
+ return torch.stack([cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2, conf, cls.float()], -1)
184
+
185
+
186
+ ov_model = ov.convert_model(
187
+ DetrEndToEnd(detr), example_input=torch.zeros(1, 3, size, size), input=[1, 3, size, size]
188
+ )
189
+ ov_model.outputs[0].get_tensor().set_names({"output0"})
190
+
191
+ names = {i: id2label[k] for i, k in enumerate(coco80)}
192
+ # DLStreamer reads preprocessing and the post-processor type from model_info.
193
+ model_info = {
194
+ "model_type": "YOLO",
195
+ "labels": " ".join(n.replace(" ", "_") for n in names.values()),
196
+ "resize_type": "standard",
197
+ "reverse_input_channels": "YES",
198
+ "scale_values": "255",
199
+ "iou_threshold": "0.7",
200
+ }
201
+ for key, value in model_info.items():
202
+ ov_model.set_rt_info(value, ["model_info", key])
203
+
204
+ out_dir = Path(f"{model_name}_openvino_model")
205
+ out_dir.mkdir(exist_ok=True)
206
+ ov.save_model(ov_model, out_dir / f"{model_name}.xml", compress_to_fp16=half_flag)
207
+ metadata = {
208
+ "description": f"{model_id} exported end-to-end with YOLO26-compatible output",
209
+ "license": "Apache-2.0",
210
+ "task": "detect",
211
+ "batch": 1,
212
+ "imgsz": [size, size],
213
+ "names": names,
214
+ "channels": 3,
215
+ "end2end": True,
216
+ }
217
+ with open(out_dir / "metadata.yaml", "w", encoding="utf-8") as f:
218
+ yaml.safe_dump(metadata, f, sort_keys=False)
219
+ print(f"Export complete: {out_dir}/")
220
+ PYEXPORT
221
+ fi
222
+
223
+ if [[ "${PRECISION}" == "INT8" ]]; then
224
+ echo "--- Quantizing to INT8 with NNCF ---"
225
+ python3 -c "
226
+ import nncf
227
+ import openvino as ov
228
+ import numpy as np
229
+ import cv2
230
+
231
+ core = ov.Core()
232
+ model = core.read_model('${MODEL_NAME}_openvino_model/${MODEL_NAME}.xml')
233
+
234
+ # Use the extracted calibration frame instead of random noise.
235
+ img = cv2.imread('test.jpg')
236
+ img = cv2.resize(img, (${INPUT_SIZE}, ${INPUT_SIZE}))
237
+ img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
238
+ img = img.transpose(2, 0, 1)[np.newaxis, ...] # NCHW
239
+
240
+ def transform_fn(data_item):
241
+ return img
242
+
243
+ calibration_dataset = nncf.Dataset(list(range(300)), transform_fn)
244
+
245
+ quantized = nncf.quantize(
246
+ model,
247
+ calibration_dataset,
248
+ preset=nncf.QuantizationPreset.MIXED,
249
+ subset_size=300,
250
+ model_type=nncf.ModelType.TRANSFORMER if '${MODEL_FAMILY}' == 'detr' else None,
251
+ )
252
+
253
+ ov.save_model(quantized, '${MODEL_NAME}_running_int8.xml')
254
+ print('Quantization complete: ${MODEL_NAME}_running_int8.xml')
255
+ "
256
+ fi
257
+ echo "--- Done ---"
openvino_sample.py ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-License-Identifier: MIT
2
+ # Copyright (C) Intel Corporation
3
+ #
4
+ # Running Detection: OpenVINO Runtime sample.
5
+ # See README.md for prerequisites and inputs. Run: python3 openvino_sample.py
6
+
7
+ import subprocess
8
+
9
+ import cv2
10
+ import numpy as np
11
+ import openvino as ov
12
+
13
+ PERSON_CLASS_ID = 0
14
+ CONF_THRESHOLD = 0.4
15
+ # Select the model: "yolo26n" (default) or "detr-resnet-50".
16
+ MODEL = "yolo26n"
17
+ INPUT_SIZE = 800 if MODEL.startswith("detr") else 640
18
+ RUNNING_SPEED = 250.0 # pixels/second
19
+ MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging
20
+ MAX_MATCH_DIST = 120 # max center distance (px) to link a track across frames
21
+ ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering
22
+
23
+ # Change DEVICE to "GPU" or "NPU" to run on integrated GPU or NPU.
24
+ DEVICE = "CPU"
25
+ INPUT_VIDEO = "running.mp4"
26
+
27
+ core = ov.Core()
28
+ model = core.read_model(f"{MODEL}_openvino_model/{MODEL}.xml")
29
+ compiled = core.compile_model(model, DEVICE)
30
+ output_port = compiled.output(0)
31
+
32
+ cap = cv2.VideoCapture(INPUT_VIDEO)
33
+ fps = cap.get(cv2.CAP_PROP_FPS) or 25.0
34
+ width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
35
+ height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
36
+ ALERT_HOLD_FRAMES = max(1, int(ALERT_HOLD_SECONDS * fps))
37
+
38
+ proc = subprocess.Popen(
39
+ ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24",
40
+ "-s", f"{width}x{height}", "-r", str(fps),
41
+ "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p",
42
+ "-movflags", "+faststart", "output_openvino.mp4"],
43
+ stdin=subprocess.PIPE, stderr=subprocess.DEVNULL,
44
+ )
45
+
46
+ tracks: dict[int, dict] = {} # id -> {cx, cy, run_frames}
47
+ next_id = 0
48
+ flagged: set[int] = set()
49
+ alert_hold = 0
50
+ frame_idx = 0
51
+
52
+ while True:
53
+ ok, frame = cap.read()
54
+ if not ok:
55
+ break
56
+ frame_idx += 1
57
+ dt = 1.0 / fps
58
+
59
+ blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
60
+ blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
61
+ blob = blob.transpose(2, 0, 1)[np.newaxis, ...] # NCHW
62
+
63
+ output = compiled([blob])[output_port][0]
64
+ mask = (output[:, 4] >= CONF_THRESHOLD) & (output[:, 5].astype(int) == PERSON_CLASS_ID)
65
+ dets = output[mask]
66
+
67
+ sx, sy = width / INPUT_SIZE, height / INPUT_SIZE
68
+ detections = []
69
+ for det in dets:
70
+ x1, y1 = int(det[0] * sx), int(det[1] * sy)
71
+ x2, y2 = int(det[2] * sx), int(det[3] * sy)
72
+ detections.append((x1, y1, x2, y2, (x1 + x2) // 2, (y1 + y2) // 2))
73
+
74
+ # Greedy nearest-center association to the previous frame's tracks.
75
+ used = set()
76
+ assignments = {}
77
+ for i, (_, _, _, _, cx, cy) in enumerate(detections):
78
+ best_id, best_dist = None, MAX_MATCH_DIST
79
+ for tid, tr in tracks.items():
80
+ if tid in used:
81
+ continue
82
+ d = np.hypot(cx - tr["cx"], cy - tr["cy"])
83
+ if d < best_dist:
84
+ best_id, best_dist = tid, d
85
+ if best_id is None:
86
+ best_id = next_id
87
+ next_id += 1
88
+ tracks[best_id] = {"cx": cx, "cy": cy, "run_frames": 0}
89
+ used.add(best_id)
90
+ assignments[i] = best_id
91
+
92
+ new_tracks = {}
93
+ frame_running = False
94
+ for i, (x1, y1, x2, y2, cx, cy) in enumerate(detections):
95
+ tid = assignments[i]
96
+ prev = tracks.get(tid, {"cx": cx, "cy": cy, "run_frames": 0})
97
+ speed = np.hypot(cx - prev["cx"], cy - prev["cy"]) / dt
98
+ run_frames = prev["run_frames"] + 1 if speed >= RUNNING_SPEED else 0
99
+ new_tracks[tid] = {"cx": cx, "cy": cy, "run_frames": run_frames}
100
+
101
+ is_running = run_frames >= MIN_RUN_FRAMES
102
+ frame_running = frame_running or is_running
103
+ color = (0, 0, 255) if is_running else (0, 255, 0)
104
+ cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
105
+ label = f"RUNNING {int(speed)}px/s" if is_running else f"{int(speed)}px/s"
106
+ cv2.putText(frame, label, (x1, max(y1 - 8, 12)),
107
+ cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2)
108
+ if is_running and tid not in flagged:
109
+ flagged.add(tid)
110
+ print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True)
111
+
112
+ tracks = new_tracks
113
+
114
+ # Latch the alert so the banner reflects a sustained state, not a single
115
+ # transient frame: once running is seen it stays on for ALERT_HOLD_FRAMES.
116
+ alert_hold = ALERT_HOLD_FRAMES if frame_running else max(0, alert_hold - 1)
117
+ alert_on = alert_hold > 0
118
+ banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED"
119
+ banner_color = (0, 0, 255) if alert_on else (0, 180, 0)
120
+ cv2.rectangle(frame, (0, 0), (width, 40), (0, 0, 0), -1)
121
+ cv2.putText(frame, banner, (10, 28),
122
+ cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2)
123
+
124
+ proc.stdin.write(frame.tobytes())
125
+
126
+ cap.release()
127
+ proc.stdin.close()
128
+ proc.wait()
129
+ print("Wrote output_openvino.mp4", flush=True)