--- license: mit license_link: LICENSE library_name: openvino pipeline_tag: object-detection tags: - openvino - yolo26 - detr - running-detection - speed-estimation - tracking - dlstreamer language: - en --- # Running Detection | Property | Value | |---|---| | **Category** | Object Detection + Tracking + Speed Estimation | | **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics, default) or [DETR](https://huggingface.co/facebook/detr-resnet-50) (Facebook, alternative) + DLStreamer `gvatrack` (Kalman filter tracker) | | **Source Framework** | PyTorch (Ultralytics, Hugging Face Transformers) | | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) | | **Inference Engine** | OpenVINO | | **Hardware** | CPU, GPU, NPU | | **Detected Class** | `person` (COCO class 0) | --- ## Overview Running Detection is a Metro Analytics use case that flags people who are running or moving faster than a configurable speed threshold. It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a state-of-the-art real-time object detector trained on the COCO dataset, quantized to INT8 and filtered at runtime to the `person` class. Each detected person is assigned a persistent track ID across frames, and per-track speed is estimated from the frame-to-frame displacement of the bounding-box center. A person is flagged as running when the estimated speed stays above the threshold for a short, sustained window, which suppresses single-frame jitter. Typical Metro deployments include: - **Platform Safety** -- flag people sprinting across platforms or toward closing train doors. - **Incident Detection** -- surface sudden running that may indicate a chase, altercation, or emergency. - **Crowd Flow Monitoring** -- distinguish normal walking pace from abnormal fast movement in concourses. - **Restricted-Speed Zones** -- enforce walk-only areas such as escalators, ramps, and stairwells. Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`. Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment; larger variants improve recall in dense scenes. ### Model Selection [DETR](https://huggingface.co/facebook/detr-resnet-50) (DEtection TRansformer) is available as an alternative to YOLO26. DETR is a transformer-based, NMS-free detector trained on COCO and released under the Apache-2.0 license. The export script embeds DETR normalization, box decoding, and the class remap into the OpenVINO IR, so the DETR model produces the same output layout and class ids as YOLO26. The same OpenVINO and DLStreamer samples run either model; select one with the `MODEL` constant at the top of each sample. | Model | Variants | Input Size | Output | License | |---|---|---|---|---| | YOLO26 (default) | `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l` | 640 x 640 | `[1, 300, 6]` | AGPL-3.0 | | DETR | `detr-resnet-50` | 800 x 800 | `[1, 100, 6]` | Apache-2.0 | Choose YOLO26 for maximum throughput on edge devices. Choose DETR when a permissively licensed transformer detector is preferred; it is heavier and runs at a lower frame rate. --- ## Prerequisites See the [released runtime setup](../README.md#released-runtimes) and [device support matrix](../DEVICE_SUPPORT_MATRIX.md) before choosing a runtime. The previous validation used the RC1 development stack. - Python 3.11+ - `ffmpeg` (`sudo apt install ffmpeg`) -- used by the samples to encode output video Create and activate a Python virtual environment before running the scripts: ```bash python3 -m venv .venv --system-site-packages source .venv/bin/activate ``` > **Note:** The `--system-site-packages` flag is required so the virtual > environment can access the system-installed OpenVINO and DLStreamer Python > packages. --- ## Getting Started ### Download and Quantize Model Run the provided script to download, export to OpenVINO IR, and optionally quantize: ```bash wget https://huggingface.co/open-edge-platform/running-detection/resolve/main/export_and_quantize.sh chmod +x export_and_quantize.sh ./export_and_quantize.sh ``` This exports the default **yolo26n** model in **FP16** precision. #### Optional: Select a Different Model, Variant, or Precision ```bash ./export_and_quantize.sh yolo26n FP32 # full-precision ./export_and_quantize.sh yolo26n INT8 # quantized ./export_and_quantize.sh yolo26s # larger variant, default FP16 ./export_and_quantize.sh detr-resnet-50 # DETR alternative, default FP16 ./export_and_quantize.sh detr-resnet-50 INT8 # DETR quantized ``` The first argument selects the model: any YOLO26 variant (`yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`) or DETR (`detr-resnet-50`). The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**. The script performs the following steps: 1. Installs dependencies (`openvino`, `ultralytics` for YOLO26 or `transformers`, `torch` for DETR, `opencv-python`; adds `nncf` for INT8). 2. Downloads the sample running video (`running.mp4`) and extracts a calibration frame (`test.jpg`). 3. Downloads the PyTorch weights and exports to OpenVINO IR. 4. *(INT8 only)* Quantizes the model using NNCF post-training quantization. Output files (replace `yolo26n` with the selected model, for example `detr-resnet-50`): - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory. - `yolo26n_running_int8.xml` / `yolo26n_running_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*. #### Precision / Device Compatibility | Precision | CPU | GPU | NPU | |---|---|---|---| | FP32 | Yes | Yes | No | | FP16 | Yes | Yes | Yes | | INT8 | Yes | Yes | Yes | > **Note:** The INT8 calibration uses the extracted sample frame. > For production accuracy, replace it with a representative set of frames from > the target deployment site. ### Speed Threshold Running is defined by a per-track speed threshold expressed in pixels per second: ```text RUNNING_SPEED = 250.0 # pixels/second (demo value for the sample clip) MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging ``` > **Note:** Pixel speed depends on camera resolution, framing, and distance to > the subject, so `RUNNING_SPEED` must be tuned per site. For a calibrated > metric speed (meters/second), convert pixel displacement using the known > ground-sampling distance of the scene. ### OpenVINO Sample The sample below runs YOLO26 inference on the sample video, filters to the `person` class, assigns track IDs with a lightweight nearest-center tracker, estimates per-track pixel speed, and flags people who run faster than `RUNNING_SPEED` for at least `MIN_RUN_FRAMES` frames. YOLO26 is end-to-end (NMS-free), so no manual non-maximum suppression is needed. The annotated result is written to `output_openvino.mp4`, with a latched `RUNNING DETECTED` / `NO RUNNING DETECTED` status banner across the top. Change the `DEVICE` string to run on CPU, GPU, or NPU. The full sample is in [openvino_sample.py](openvino_sample.py); run it with `python3 openvino_sample.py` or copy the code below. ```python # SPDX-License-Identifier: MIT # Copyright (C) Intel Corporation # # Running Detection: OpenVINO Runtime sample. # See README.md for prerequisites and inputs. Run: python3 openvino_sample.py import subprocess import cv2 import numpy as np import openvino as ov PERSON_CLASS_ID = 0 CONF_THRESHOLD = 0.4 # Select the model: "yolo26n" (default) or "detr-resnet-50". MODEL = "yolo26n" INPUT_SIZE = 800 if MODEL.startswith("detr") else 640 RUNNING_SPEED = 250.0 # pixels/second MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging MAX_MATCH_DIST = 120 # max center distance (px) to link a track across frames ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering # Change DEVICE to "GPU" or "NPU" to run on integrated GPU or NPU. DEVICE = "CPU" INPUT_VIDEO = "running.mp4" core = ov.Core() model = core.read_model(f"{MODEL}_openvino_model/{MODEL}.xml") compiled = core.compile_model(model, DEVICE) output_port = compiled.output(0) cap = cv2.VideoCapture(INPUT_VIDEO) fps = cap.get(cv2.CAP_PROP_FPS) or 25.0 width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) ALERT_HOLD_FRAMES = max(1, int(ALERT_HOLD_SECONDS * fps)) proc = subprocess.Popen( ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24", "-s", f"{width}x{height}", "-r", str(fps), "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-movflags", "+faststart", "output_openvino.mp4"], stdin=subprocess.PIPE, stderr=subprocess.DEVNULL, ) tracks: dict[int, dict] = {} # id -> {cx, cy, run_frames} next_id = 0 flagged: set[int] = set() alert_hold = 0 frame_idx = 0 while True: ok, frame = cap.read() if not ok: break frame_idx += 1 dt = 1.0 / fps blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE)) blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0 blob = blob.transpose(2, 0, 1)[np.newaxis, ...] # NCHW output = compiled([blob])[output_port][0] mask = (output[:, 4] >= CONF_THRESHOLD) & (output[:, 5].astype(int) == PERSON_CLASS_ID) dets = output[mask] sx, sy = width / INPUT_SIZE, height / INPUT_SIZE detections = [] for det in dets: x1, y1 = int(det[0] * sx), int(det[1] * sy) x2, y2 = int(det[2] * sx), int(det[3] * sy) detections.append((x1, y1, x2, y2, (x1 + x2) // 2, (y1 + y2) // 2)) # Greedy nearest-center association to the previous frame's tracks. used = set() assignments = {} for i, (_, _, _, _, cx, cy) in enumerate(detections): best_id, best_dist = None, MAX_MATCH_DIST for tid, tr in tracks.items(): if tid in used: continue d = np.hypot(cx - tr["cx"], cy - tr["cy"]) if d < best_dist: best_id, best_dist = tid, d if best_id is None: best_id = next_id next_id += 1 tracks[best_id] = {"cx": cx, "cy": cy, "run_frames": 0} used.add(best_id) assignments[i] = best_id new_tracks = {} frame_running = False for i, (x1, y1, x2, y2, cx, cy) in enumerate(detections): tid = assignments[i] prev = tracks.get(tid, {"cx": cx, "cy": cy, "run_frames": 0}) speed = np.hypot(cx - prev["cx"], cy - prev["cy"]) / dt run_frames = prev["run_frames"] + 1 if speed >= RUNNING_SPEED else 0 new_tracks[tid] = {"cx": cx, "cy": cy, "run_frames": run_frames} is_running = run_frames >= MIN_RUN_FRAMES frame_running = frame_running or is_running color = (0, 0, 255) if is_running else (0, 255, 0) cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2) label = f"RUNNING {int(speed)}px/s" if is_running else f"{int(speed)}px/s" cv2.putText(frame, label, (x1, max(y1 - 8, 12)), cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2) if is_running and tid not in flagged: flagged.add(tid) print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True) tracks = new_tracks # Latch the alert so the banner reflects a sustained state, not a single # transient frame: once running is seen it stays on for ALERT_HOLD_FRAMES. alert_hold = ALERT_HOLD_FRAMES if frame_running else max(0, alert_hold - 1) alert_on = alert_hold > 0 banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED" banner_color = (0, 0, 255) if alert_on else (0, 180, 0) cv2.rectangle(frame, (0, 0), (width, 40), (0, 0, 0), -1) cv2.putText(frame, banner, (10, 28), cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2) proc.stdin.write(frame.tobytes()) cap.release() proc.stdin.close() proc.wait() print("Wrote output_openvino.mp4", flush=True) ``` **Device targets:** - `"CPU"` -- default, works on all Intel platforms. - `"GPU"` -- Intel integrated or discrete GPU. - `"NPU"` -- Intel NPU (different throughput profile; validate with `benchmark_app -d NPU`). Expected console output: ```text RUNNING id=0 speed=312px/s frame=14 ... Wrote output_openvino.mp4 ``` `output_openvino.mp4` shows a green box around each person, turning red with a `RUNNING` label when the person's speed exceeds the threshold. #### Expected Output ![OpenVINO expected output](expected_output_openvino.gif) ### DLStreamer Sample The pipeline below runs the FP16 YOLO26 detector via `gvadetect` on the sample video, attaches persistent track IDs with `gvatrack` (`short-term-imageless` tracker), and overlays bounding boxes with `gvawatermark`. Frames are pulled from an `appsink`; per-track pixel speed is computed from the frame-to-frame displacement of each track center, and a `RUNNING` event is raised when the speed stays above `RUNNING_SPEED` for at least `MIN_RUN_FRAMES` frames. A latched `RUNNING DETECTED` / `NO RUNNING DETECTED` status banner is drawn across the top of every frame. `gvawatermark` renders boxes for the `person` class only. The annotated result is muxed to `output_dlstreamer.mp4`. > **Notes on running this sample:** > > - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml` or `detr-resnet-50_openvino_model/detr-resnet-50.xml`). Class names are > read automatically from the model's embedded `metadata.yaml` by > DLStreamer 2026.0+ -- no external `labels-file` is required. > - Export `PYTHONPATH` so the DLStreamer Python module is importable: > > ```bash > source /opt/intel/openvino_2026/setupvars.sh > source /opt/intel/dlstreamer/scripts/setup_dls_env.sh > export PYTHONPATH=/opt/intel/dlstreamer/python:\ > /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-} > ``` The full sample is in [dlstreamer_sample.py](dlstreamer_sample.py); run it with `python3 dlstreamer_sample.py` or copy the code below. ```python # SPDX-License-Identifier: MIT # Copyright (C) Intel Corporation # # Running Detection: DLStreamer pipeline sample. # See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py import subprocess from collections import defaultdict import numpy as np import gi gi.require_version("Gst", "1.0") gi.require_version("GstAnalytics", "1.0") gi.require_version("GstVideo", "1.0") from gi.repository import Gst, GLib, GstAnalytics, GstVideo Gst.init([]) # Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts. import cv2 INPUT_VIDEO = "running.mp4" RUNNING_SPEED = 250.0 # pixels/second MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering # Select the model: "yolo26n" (default) or "detr-resnet-50". MODEL = "yolo26n" # For CPU: change device=GPU to device=CPU. # For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended). # gvawatermark draws only person ROIs (displ-cfg=show-roi=person) so boxes for # other COCO classes are not rendered. pipeline_str = ( f"filesrc location={INPUT_VIDEO} ! decodebin3 ! " "videoconvert ! " f"gvadetect model={MODEL}_openvino_model/{MODEL}.xml " "device=GPU " "threshold=0.4 ! queue ! " "gvatrack tracking-type=short-term-imageless ! queue ! " "gvawatermark displ-cfg=show-roi=person ! " "videoconvert ! video/x-raw,format=BGR ! " "appsink name=sink emit-signals=false sync=false" ) pipeline = Gst.parse_launch(pipeline_str) appsink = pipeline.get_by_name("sink") pipeline.set_state(Gst.State.PLAYING) proc = None prev_center: dict[int, tuple[int, int]] = {} run_frames: dict[int, int] = defaultdict(int) flagged: set[int] = set() prev_pts = None alert_hold = 0 alert_hold_frames = 40 # updated from the real framerate on the first frame frame_idx = 0 while True: sample = appsink.emit("pull-sample") if sample is None: break buf = sample.get_buffer() caps = sample.get_caps() struct = caps.get_structure(0) width = struct.get_value("width") height = struct.get_value("height") frame_idx += 1 # Start ffmpeg encoder on the first frame. if proc is None: ok, fps_num, fps_den = struct.get_fraction("framerate") fps = fps_num / fps_den if ok and fps_den > 0 else 25.0 alert_hold_frames = max(1, int(ALERT_HOLD_SECONDS * fps)) proc = subprocess.Popen( ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24", "-s", f"{width}x{height}", "-r", str(fps), "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-movflags", "+faststart", "output_dlstreamer.mp4"], stdin=subprocess.PIPE, stderr=subprocess.DEVNULL, ) # Elapsed time since the previous frame from buffer timestamps. pts = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else frame_idx / fps dt = (pts - prev_pts) if (prev_pts is not None and pts > prev_pts) else 1.0 / fps prev_pts = pts # Read detection / tracking metadata via GstAnalytics. rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf) regions = [] if rmeta is not None: od_entries = [] trk_map = {} # metadata_id -> tracking_id idx = 1 while True: ok_od, od = rmeta.get_od_mtd(idx) ok_trk, trk = rmeta.get_tracking_mtd(idx) if not ok_od and not ok_trk: break if ok_od: label = GLib.quark_to_string(od.get_obj_type()) _, x, y, w, h, _ = od.get_location() od_entries.append((idx, label, int(x + w / 2), int(y + h / 2))) if ok_trk: ok2, tid, _, _, _ = trk.get_info() if ok2: trk_map[idx] = tid idx += 1 for od_id, label, cx, cy in od_entries: if label != "person": continue tid = 0 for trk_meta_id, tracking_id in trk_map.items(): if rmeta.get_relation(od_id, trk_meta_id) != GstAnalytics.RelTypes.NONE: tid = tracking_id break regions.append((tid, cx, cy)) # Map buffer read-only and copy pixels to a writable numpy array. success, map_info = buf.map(Gst.MapFlags.READ) if not success: continue video_info = GstVideo.VideoInfo.new_from_caps(caps) arr = np.ndarray((height, width, 3), dtype=np.uint8, buffer=map_info.data, strides=(video_info.stride[0], 3, 1)).copy() buf.unmap(map_info) frame_running = False for tid, cx, cy in regions: px, py = prev_center.get(tid, (cx, cy)) speed = np.hypot(cx - px, cy - py) / dt if dt > 0 else 0.0 prev_center[tid] = (cx, cy) run_frames[tid] = run_frames[tid] + 1 if speed >= RUNNING_SPEED else 0 if run_frames[tid] >= MIN_RUN_FRAMES: frame_running = True cv2.putText(arr, f"RUNNING {int(speed)}px/s", (cx - 40, max(cy - 20, 12)), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2) if tid not in flagged: flagged.add(tid) print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True) # Latch the alert so the banner reflects a sustained state, not a single # transient frame: once running is seen it stays on for alert_hold_frames. alert_hold = alert_hold_frames if frame_running else max(0, alert_hold - 1) alert_on = alert_hold > 0 banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED" banner_color = (0, 0, 255) if alert_on else (0, 180, 0) cv2.rectangle(arr, (0, 0), (width, 40), (0, 0, 0), -1) cv2.putText(arr, banner, (10, 28), cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2) proc.stdin.write(arr.tobytes()) pipeline.set_state(Gst.State.NULL) if proc: proc.stdin.close() proc.wait() print("Wrote output_dlstreamer.mp4", flush=True) ``` #### Expected Output ![DLStreamer expected output](expected_output_dlstreamer.gif) **Device targets:** - `device=GPU` -- default in the sample code. - `device=CPU` -- change `device=GPU` to `device=CPU`. - `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization. --- ## License Licensed under the MIT License. See [LICENSE](LICENSE) for details. ## References - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/) - [DETR Model Card (facebook/detr-resnet-50)](https://huggingface.co/facebook/detr-resnet-50) - [End-to-End Object Detection with Transformers (DETR paper)](https://arxiv.org/abs/2005.12872) - [Ultralytics Multi-Object Tracking](https://docs.ultralytics.com/modes/track/) - [Intel DLStreamer gvatrack](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/elements/gvatrack.html) - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb) - [OpenVINO Documentation](https://docs.openvino.ai/) - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html) - [Intel DLStreamer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/index.html)