| --- |
| license: mit |
| license_link: LICENSE |
| library_name: openvino |
| pipeline_tag: object-detection |
| tags: |
| - openvino |
| - intel |
| - yolo |
| - yolo26 |
| - running-detection |
| - speed-estimation |
| - tracking |
| - edge-ai |
| - metro |
| - dlstreamer |
| language: |
| - en |
| --- |
| |
| # Running Detection |
|
|
| | Property | Value | |
| |---|---| |
| | **Category** | Object Detection + Tracking + Speed Estimation | |
| | **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics) + DLStreamer `gvatrack` (Kalman filter tracker) | |
| | **Source Framework** | PyTorch (Ultralytics) | |
| | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) | |
| | **Inference Engine** | OpenVINO | |
| | **Hardware** | CPU, GPU, NPU | |
| | **Detected Class** | `person` (COCO class 0) | |
|
|
| --- |
|
|
| ## Overview |
|
|
| Running Detection is a Metro Analytics use case that flags people who are running or moving faster than a configurable speed threshold. |
| It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a state-of-the-art real-time object detector trained on the COCO dataset, quantized to INT8 and filtered at runtime to the `person` class. |
| Each detected person is assigned a persistent track ID across frames, and per-track speed is estimated from the frame-to-frame displacement of the bounding-box center. |
| A person is flagged as running when the estimated speed stays above the threshold for a short, sustained window, which suppresses single-frame jitter. |
|
|
| Typical Metro deployments include: |
|
|
| - **Platform Safety** -- flag people sprinting across platforms or toward closing train doors. |
| - **Incident Detection** -- surface sudden running that may indicate a chase, altercation, or emergency. |
| - **Crowd Flow Monitoring** -- distinguish normal walking pace from abnormal fast movement in concourses. |
| - **Restricted-Speed Zones** -- enforce walk-only areas such as escalators, ramps, and stairwells. |
|
|
| Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`. |
| Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment; larger variants improve recall in dense scenes. |
|
|
| --- |
|
|
| ## Prerequisites |
|
|
| - Python 3.11+ |
| - `ffmpeg` (`sudo apt install ffmpeg`) -- used by the samples to encode output video |
| - [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version) |
| - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version) |
|
|
| Create and activate a Python virtual environment before running the scripts: |
|
|
| ```bash |
| python3 -m venv .venv --system-site-packages |
| source .venv/bin/activate |
| ``` |
|
|
| > **Note:** The `--system-site-packages` flag is required so the virtual |
| > environment can access the system-installed OpenVINO and DLStreamer Python |
| > packages. |
|
|
| --- |
|
|
| ## Getting Started |
|
|
| ### Download and Quantize Model |
|
|
| Run the provided script to download, export to OpenVINO IR, and optionally quantize: |
|
|
| ```bash |
| chmod +x export_and_quantize.sh |
| ./export_and_quantize.sh |
| ``` |
|
|
| This exports the default **yolo26n** model in **FP16** precision. |
|
|
| #### Optional: Select a Different Variant or Precision |
|
|
| ```bash |
| ./export_and_quantize.sh yolo26n FP32 # full-precision |
| ./export_and_quantize.sh yolo26n INT8 # quantized |
| ./export_and_quantize.sh yolo26s # larger variant, default FP16 |
| ``` |
|
|
| Replace `yolo26n` with any variant (`yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`). |
| The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**. |
|
|
| The script performs the following steps: |
|
|
| 1. Installs dependencies (`openvino`, `ultralytics`, `opencv-python`; adds `nncf` for INT8). |
| 2. Downloads the sample running video (`running.mp4`) and extracts a calibration frame (`test.jpg`). |
| 3. Downloads the PyTorch weights and exports to OpenVINO IR. |
| 4. *(INT8 only)* Quantizes the model using NNCF post-training quantization. |
|
|
| Output files: |
|
|
| - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory. |
| - `yolo26n_running_int8.xml` / `yolo26n_running_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*. |
|
|
| #### Precision / Device Compatibility |
|
|
| | Precision | CPU | GPU | NPU | |
| |---|---|---|---| |
| | FP32 | Yes | Yes | No | |
| | FP16 | Yes | Yes | Yes | |
| | INT8 | Yes | Yes | Yes | |
|
|
| > **Note:** The INT8 calibration uses the extracted sample frame. |
| > For production accuracy, replace it with a representative set of frames from |
| > the target deployment site. |
|
|
| ### Speed Threshold |
|
|
| Running is defined by a per-track speed threshold expressed in pixels per second: |
|
|
| ```text |
| RUNNING_SPEED = 250.0 # pixels/second (demo value for the sample clip) |
| MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging |
| ``` |
|
|
| > **Note:** Pixel speed depends on camera resolution, framing, and distance to |
| > the subject, so `RUNNING_SPEED` must be tuned per site. For a calibrated |
| > metric speed (meters/second), convert pixel displacement using the known |
| > ground-sampling distance of the scene. |
| |
| ### OpenVINO Sample |
| |
| The sample below runs YOLO26 inference on the sample video, filters to the `person` class, assigns track IDs with a lightweight nearest-center tracker, estimates per-track pixel speed, and flags people who run faster than `RUNNING_SPEED` for at least `MIN_RUN_FRAMES` frames. |
| YOLO26 is end-to-end (NMS-free), so no manual non-maximum suppression is needed. |
| The annotated result is written to `output_openvino.mp4`, with a latched |
| `RUNNING DETECTED` / `NO RUNNING DETECTED` status banner across the top. |
| Change the `DEVICE` string to run on CPU, GPU, or NPU. |
|
|
| ```python |
| import subprocess |
| |
| import cv2 |
| import numpy as np |
| import openvino as ov |
| |
| PERSON_CLASS_ID = 0 |
| CONF_THRESHOLD = 0.4 |
| INPUT_SIZE = 640 |
| RUNNING_SPEED = 250.0 # pixels/second |
| MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging |
| MAX_MATCH_DIST = 120 # max center distance (px) to link a track across frames |
| ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering |
| |
| # Change DEVICE to "GPU" or "NPU" to run on integrated GPU or NPU. |
| DEVICE = "CPU" |
| INPUT_VIDEO = "running.mp4" |
| |
| core = ov.Core() |
| model = core.read_model("yolo26n_openvino_model/yolo26n.xml") |
| compiled = core.compile_model(model, DEVICE) |
| output_port = compiled.output(0) |
| |
| cap = cv2.VideoCapture(INPUT_VIDEO) |
| fps = cap.get(cv2.CAP_PROP_FPS) or 25.0 |
| width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) |
| height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) |
| ALERT_HOLD_FRAMES = max(1, int(ALERT_HOLD_SECONDS * fps)) |
| |
| proc = subprocess.Popen( |
| ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24", |
| "-s", f"{width}x{height}", "-r", str(fps), |
| "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p", |
| "-movflags", "+faststart", "output_openvino.mp4"], |
| stdin=subprocess.PIPE, stderr=subprocess.DEVNULL, |
| ) |
| |
| tracks: dict[int, dict] = {} # id -> {cx, cy, run_frames} |
| next_id = 0 |
| flagged: set[int] = set() |
| alert_hold = 0 |
| frame_idx = 0 |
| |
| while True: |
| ok, frame = cap.read() |
| if not ok: |
| break |
| frame_idx += 1 |
| dt = 1.0 / fps |
| |
| blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE)) |
| blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0 |
| blob = blob.transpose(2, 0, 1)[np.newaxis, ...] # NCHW |
| |
| output = compiled([blob])[output_port][0] |
| mask = (output[:, 4] >= CONF_THRESHOLD) & (output[:, 5].astype(int) == PERSON_CLASS_ID) |
| dets = output[mask] |
| |
| sx, sy = width / INPUT_SIZE, height / INPUT_SIZE |
| detections = [] |
| for det in dets: |
| x1, y1 = int(det[0] * sx), int(det[1] * sy) |
| x2, y2 = int(det[2] * sx), int(det[3] * sy) |
| detections.append((x1, y1, x2, y2, (x1 + x2) // 2, (y1 + y2) // 2)) |
| |
| # Greedy nearest-center association to the previous frame's tracks. |
| used = set() |
| assignments = {} |
| for i, (_, _, _, _, cx, cy) in enumerate(detections): |
| best_id, best_dist = None, MAX_MATCH_DIST |
| for tid, tr in tracks.items(): |
| if tid in used: |
| continue |
| d = np.hypot(cx - tr["cx"], cy - tr["cy"]) |
| if d < best_dist: |
| best_id, best_dist = tid, d |
| if best_id is None: |
| best_id = next_id |
| next_id += 1 |
| tracks[best_id] = {"cx": cx, "cy": cy, "run_frames": 0} |
| used.add(best_id) |
| assignments[i] = best_id |
| |
| new_tracks = {} |
| frame_running = False |
| for i, (x1, y1, x2, y2, cx, cy) in enumerate(detections): |
| tid = assignments[i] |
| prev = tracks.get(tid, {"cx": cx, "cy": cy, "run_frames": 0}) |
| speed = np.hypot(cx - prev["cx"], cy - prev["cy"]) / dt |
| run_frames = prev["run_frames"] + 1 if speed >= RUNNING_SPEED else 0 |
| new_tracks[tid] = {"cx": cx, "cy": cy, "run_frames": run_frames} |
| |
| is_running = run_frames >= MIN_RUN_FRAMES |
| frame_running = frame_running or is_running |
| color = (0, 0, 255) if is_running else (0, 255, 0) |
| cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2) |
| label = f"RUNNING {int(speed)}px/s" if is_running else f"{int(speed)}px/s" |
| cv2.putText(frame, label, (x1, max(y1 - 8, 12)), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2) |
| if is_running and tid not in flagged: |
| flagged.add(tid) |
| print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True) |
| |
| tracks = new_tracks |
| |
| # Latch the alert so the banner reflects a sustained state, not a single |
| # transient frame: once running is seen it stays on for ALERT_HOLD_FRAMES. |
| alert_hold = ALERT_HOLD_FRAMES if frame_running else max(0, alert_hold - 1) |
| alert_on = alert_hold > 0 |
| banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED" |
| banner_color = (0, 0, 255) if alert_on else (0, 180, 0) |
| cv2.rectangle(frame, (0, 0), (width, 40), (0, 0, 0), -1) |
| cv2.putText(frame, banner, (10, 28), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2) |
| |
| proc.stdin.write(frame.tobytes()) |
| |
| cap.release() |
| proc.stdin.close() |
| proc.wait() |
| print("Wrote output_openvino.mp4", flush=True) |
| ``` |
|
|
| **Device targets:** |
|
|
| - `"CPU"` -- default, works on all Intel platforms. |
| - `"GPU"` -- Intel integrated or discrete GPU. |
| - `"NPU"` -- Intel NPU (different throughput profile; validate with `benchmark_app -d NPU`). |
|
|
| Expected console output: |
|
|
| ```text |
| RUNNING id=0 speed=312px/s frame=14 |
| ... |
| Wrote output_openvino.mp4 |
| ``` |
|
|
| `output_openvino.mp4` shows a green box around each person, turning red with a |
| `RUNNING` label when the person's speed exceeds the threshold. |
|
|
| #### Expected Output |
|
|
|  |
|
|
| ### DLStreamer Sample |
|
|
| The pipeline below runs the FP16 YOLO26 detector via `gvadetect` on the sample |
| video, attaches persistent track IDs with `gvatrack` |
| (`short-term-imageless` tracker), and overlays bounding boxes with |
| `gvawatermark`. Frames are pulled from an `appsink`; per-track pixel speed is |
| computed from the frame-to-frame displacement of each track center, and a |
| `RUNNING` event is raised when the speed stays above `RUNNING_SPEED` for at |
| least `MIN_RUN_FRAMES` frames. A latched `RUNNING DETECTED` / |
| `NO RUNNING DETECTED` status banner is drawn across the top of every frame. |
| `gvawatermark` renders boxes for the `person` class only. The annotated result |
| is muxed to `output_dlstreamer.mp4`. |
|
|
| > **Notes on running this sample:** |
| > |
| > - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml`). Class names are |
| > read automatically from the model's embedded `metadata.yaml` by |
| > DLStreamer 2026.0+ -- no external `labels-file` is required. |
| > - Export `PYTHONPATH` so the DLStreamer Python module is importable: |
| > |
| > ```bash |
| > source /opt/intel/openvino_2026/setupvars.sh |
| > source /opt/intel/dlstreamer/scripts/setup_dls_env.sh |
| > export PYTHONPATH=/opt/intel/dlstreamer/python:\ |
| > /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-} |
| > ``` |
| |
| ```python |
| import subprocess |
| from collections import defaultdict |
| |
| import numpy as np |
| import gi |
| |
| gi.require_version("Gst", "1.0") |
| gi.require_version("GstAnalytics", "1.0") |
| from gi.repository import Gst, GLib, GstAnalytics |
| |
| Gst.init([]) |
| |
| # Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts. |
| import cv2 |
| |
| INPUT_VIDEO = "running.mp4" |
| RUNNING_SPEED = 250.0 # pixels/second |
| MIN_RUN_FRAMES = 3 # sustained frames above the threshold before flagging |
| ALERT_HOLD_SECONDS = 2.0 # latch the alert banner to keep it from flickering |
| |
| # For CPU: change device=GPU to device=CPU. |
| # For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended). |
| # gvawatermark draws only person ROIs (displ-cfg=show-roi=person) so boxes for |
| # other COCO classes are not rendered. |
| pipeline_str = ( |
| f"filesrc location={INPUT_VIDEO} ! decodebin3 ! " |
| "videoconvert ! " |
| "gvadetect model=yolo26n_openvino_model/yolo26n.xml " |
| "device=GPU " |
| "threshold=0.4 ! queue ! " |
| "gvatrack tracking-type=short-term-imageless ! queue ! " |
| "gvawatermark displ-cfg=show-roi=person ! appsink name=sink emit-signals=false sync=false" |
| ) |
| pipeline = Gst.parse_launch(pipeline_str) |
| appsink = pipeline.get_by_name("sink") |
| |
| pipeline.set_state(Gst.State.PLAYING) |
| |
| proc = None |
| prev_center: dict[int, tuple[int, int]] = {} |
| run_frames: dict[int, int] = defaultdict(int) |
| flagged: set[int] = set() |
| prev_pts = None |
| alert_hold = 0 |
| alert_hold_frames = 40 # updated from the real framerate on the first frame |
| frame_idx = 0 |
|
|
| while True: |
| sample = appsink.emit("pull-sample") |
| if sample is None: |
| break |
| |
| buf = sample.get_buffer() |
| caps = sample.get_caps() |
| struct = caps.get_structure(0) |
| width = struct.get_value("width") |
| height = struct.get_value("height") |
| frame_idx += 1 |
| |
| # Start ffmpeg encoder on the first frame. |
| if proc is None: |
| ok, fps_num, fps_den = struct.get_fraction("framerate") |
| fps = fps_num / fps_den if ok and fps_den > 0 else 25.0 |
| alert_hold_frames = max(1, int(ALERT_HOLD_SECONDS * fps)) |
| proc = subprocess.Popen( |
| ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24", |
| "-s", f"{width}x{height}", "-r", str(fps), |
| "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p", |
| "-movflags", "+faststart", "output_dlstreamer.mp4"], |
| stdin=subprocess.PIPE, stderr=subprocess.DEVNULL, |
| ) |
| |
| # Elapsed time since the previous frame from buffer timestamps. |
| pts = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else frame_idx / fps |
| dt = (pts - prev_pts) if (prev_pts is not None and pts > prev_pts) else 1.0 / fps |
| prev_pts = pts |
| |
| # Read detection / tracking metadata via GstAnalytics. |
| rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf) |
| regions = [] |
| if rmeta is not None: |
| od_entries = [] |
| trk_map = {} # metadata_id -> tracking_id |
| idx = 1 |
| while True: |
| ok_od, od = rmeta.get_od_mtd(idx) |
| ok_trk, trk = rmeta.get_tracking_mtd(idx) |
| if not ok_od and not ok_trk: |
| break |
| if ok_od: |
| label = GLib.quark_to_string(od.get_obj_type()) |
| _, x, y, w, h, _ = od.get_location() |
| od_entries.append((idx, label, int(x + w / 2), int(y + h / 2))) |
| if ok_trk: |
| ok2, tid, _, _, _ = trk.get_info() |
| if ok2: |
| trk_map[idx] = tid |
| idx += 1 |
| for od_id, label, cx, cy in od_entries: |
| if label != "person": |
| continue |
| tid = 0 |
| for trk_meta_id, tracking_id in trk_map.items(): |
| if rmeta.get_relation(od_id, trk_meta_id) != GstAnalytics.RelTypes.NONE: |
| tid = tracking_id |
| break |
| regions.append((tid, cx, cy)) |
| |
| # Map buffer read-only and copy pixels to a writable numpy array. |
| success, map_info = buf.map(Gst.MapFlags.READ) |
| if not success: |
| continue |
| arr = np.ndarray((height, width, 3), dtype=np.uint8, |
| buffer=map_info.data).copy() |
| buf.unmap(map_info) |
| |
| frame_running = False |
| for tid, cx, cy in regions: |
| px, py = prev_center.get(tid, (cx, cy)) |
| speed = np.hypot(cx - px, cy - py) / dt if dt > 0 else 0.0 |
| prev_center[tid] = (cx, cy) |
| run_frames[tid] = run_frames[tid] + 1 if speed >= RUNNING_SPEED else 0 |
| |
| if run_frames[tid] >= MIN_RUN_FRAMES: |
| frame_running = True |
| cv2.putText(arr, f"RUNNING {int(speed)}px/s", (cx - 40, max(cy - 20, 12)), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2) |
| if tid not in flagged: |
| flagged.add(tid) |
| print(f"RUNNING id={tid} speed={int(speed)}px/s frame={frame_idx}", flush=True) |
| |
| # Latch the alert so the banner reflects a sustained state, not a single |
| # transient frame: once running is seen it stays on for alert_hold_frames. |
| alert_hold = alert_hold_frames if frame_running else max(0, alert_hold - 1) |
| alert_on = alert_hold > 0 |
| banner = "RUNNING DETECTED" if alert_on else "NO RUNNING DETECTED" |
| banner_color = (0, 0, 255) if alert_on else (0, 180, 0) |
| cv2.rectangle(arr, (0, 0), (width, 40), (0, 0, 0), -1) |
| cv2.putText(arr, banner, (10, 28), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.9, banner_color, 2) |
| |
| proc.stdin.write(arr.tobytes()) |
| |
| pipeline.set_state(Gst.State.NULL) |
| if proc: |
| proc.stdin.close() |
| proc.wait() |
| print("Wrote output_dlstreamer.mp4", flush=True) |
| ``` |
| |
| #### Expected Output |
| |
|  |
| |
| **Device targets:** |
| |
| - `device=GPU` -- default in the sample code. |
| - `device=CPU` -- change `device=GPU` to `device=CPU`. |
| - `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization. |
| |
| --- |
| |
| ## License |
| |
| Licensed under the MIT License. See [LICENSE](LICENSE) for details. |
| |
| ## References |
| |
| - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/) |
| - [Ultralytics Multi-Object Tracking](https://docs.ultralytics.com/modes/track/) |
| - [Intel DLStreamer gvatrack](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/elements/gvatrack.html) |
| - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb) |
| - [OpenVINO Documentation](https://docs.openvino.ai/) |
| - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html) |
| - [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html) |
| |
