File size: 13,229 Bytes
de2b807
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
d3a091b
 
de2b807
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
---
license: mit
license_link: LICENSE
library_name: openvino
pipeline_tag: object-detection
tags:
  - openvino
  - yolo26
  - detr
  - motion-tracking
  - multi-object-tracking
  - dlstreamer
language:
  - en
---

# Motion Tracking

| Property | Value |
|---|---|
| **Category** | Object Detection + Multi-Object Tracking |
| **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics, default) or [DETR](https://huggingface.co/facebook/detr-resnet-50) (Facebook, alternative) + DLStreamer `gvatrack` (Kalman filter tracker) |
| **Source Framework** | PyTorch (Ultralytics, Hugging Face Transformers) |
| **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
| **Inference Engine** | OpenVINO |
| **Hardware** | CPU, GPU, NPU |
| **Detected Class(es)** | Configurable (default: all 80 COCO classes) |

---

## Overview

Motion Tracking is a Metro Analytics use case that detects objects and assigns persistent track IDs across frames, enabling trajectory analysis and temporal event detection.
It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a state-of-the-art real-time object detector quantized to INT8, paired with a multi-object tracker:

- **DLStreamer pipeline:** YOLO26 FP16 detection via `gvadetect` + `gvatrack` element with `tracking-type=short-term-imageless`.

Each detected object receives a unique `track_id` that persists across frames as long as the object remains visible.
Outputs include per-object trajectories suitable for path analysis, dwell-time computation, and zone-based event triggers.

Typical Metro deployments include:

- **Pedestrian Trajectory Analysis** -- map walking paths through stations for flow optimization.
- **Dwell-Time Measurement** -- measure how long individuals stay in specific zones.
- **Zone-Based Event Detection** -- trigger alerts when tracked objects enter or exit defined areas.
- **Traffic Flow Analytics** -- track vehicles through intersections for signal timing optimization.
- **Incident Replay** -- reconstruct object paths for post-event forensic review.

Available YOLO26 variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`.
Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment.
The default tracker is `short-term-imageless` (Kalman filter-based, no image data required).
DLStreamer also supports `tracking-type=deep-sort` for more robust re-identification using a feature extraction model (e.g., mars-small128).

### Model Selection

[DETR](https://huggingface.co/facebook/detr-resnet-50) (DEtection TRansformer) is available as an alternative to YOLO26.
DETR is a transformer-based, NMS-free detector trained on COCO and released under the Apache-2.0 license.
The export script embeds DETR normalization, box decoding, and the class remap into the OpenVINO IR, so the DETR model produces the same output layout and class ids as YOLO26.
The same DLStreamer sample run either model; select one with the `MODEL` constant at the top of the sample.

| Model | Variants | Input Size | Output | License |
|---|---|---|---|---|
| YOLO26 (default) | `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l` | 640 x 640 | `[1, 300, 6]` | AGPL-3.0 |
| DETR | `detr-resnet-50` | 800 x 800 | `[1, 100, 6]` | Apache-2.0 |

Choose YOLO26 for maximum throughput on edge devices.
Choose DETR when a permissively licensed transformer detector is preferred; it is heavier and runs at a lower frame rate.

---

## Prerequisites

See the [released runtime setup](../README.md#released-runtimes) and [device support matrix](../DEVICE_SUPPORT_MATRIX.md) before choosing a runtime.
The previous validation used the RC1 development stack.

- Python 3.11+
- `ffmpeg` (`sudo apt install ffmpeg`) -- used by both samples to encode output video

Create and activate a Python virtual environment before running the scripts:

```bash
python3 -m venv .venv --system-site-packages
source .venv/bin/activate
```

> **Note:** The `--system-site-packages` flag is required so the virtual
> environment can access the system-installed OpenVINO and DLStreamer Python
> packages.

---

## Getting Started

### Download and Quantize Model

Run the provided script to download, export to OpenVINO IR, and optionally quantize:

```bash
wget https://huggingface.co/open-edge-platform/motion-tracking/resolve/main/export_and_quantize.sh
chmod +x export_and_quantize.sh
./export_and_quantize.sh
```

This exports the default **yolo26n** model in **FP16** precision.

#### Optional: Select a Different Model, Variant, or Precision

```bash
./export_and_quantize.sh yolo26n FP32          # full-precision
./export_and_quantize.sh yolo26n INT8          # quantized
./export_and_quantize.sh yolo26s               # larger variant, default FP16
./export_and_quantize.sh detr-resnet-50        # DETR alternative, default FP16
./export_and_quantize.sh detr-resnet-50 INT8   # DETR quantized
```

The first argument selects the model: any YOLO26 variant (`yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`) or DETR (`detr-resnet-50`).
The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**.

The script performs the following steps:

1. Installs dependencies (`openvino`, `ultralytics` for YOLO26 or `transformers`, `torch` for DETR; adds `nncf` for INT8).
2. Downloads the sample test video (`test_video.mp4`) and a sample test image (`test.jpg`).
3. Downloads the PyTorch weights and exports to OpenVINO IR.
4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.

Output files (replace `yolo26n` with the selected model, for example `detr-resnet-50`):

- `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
- `yolo26n_tracking_int8.xml` / `yolo26n_tracking_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*.

#### Precision / Device Compatibility

| Precision | CPU | GPU | NPU |
|---|---|---|---|
| FP32 | Yes | Yes | No |
| FP16 | Yes | Yes | Yes |
| INT8 | Yes | Yes | Yes |

> **Note:** The INT8 calibration uses frames from the bundled sample video.
> For production accuracy, replace it with a representative set of frames from
> the target deployment site.

### DLStreamer Sample

The pipeline below runs the YOLO26 FP16 detector via `gvadetect` on
`test_video.mp4`, attaches persistent track IDs with `gvatrack`
(`short-term-imageless` tracker), and overlays bounding boxes with
`gvawatermark`.  Frames are pulled from an `appsink`, per-track trajectory
polylines are drawn with OpenCV, and the result is muxed to `output_dlstreamer.mp4`
(H.264 via ffmpeg).

> **Notes on running this sample:**
>
> - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml` or `detr-resnet-50_openvino_model/detr-resnet-50.xml`). Class names are
>   read automatically from the model's embedded `metadata.yaml` by
>   DLStreamer 2026.0+ -- no external `labels-file` is required.
> - Export `PYTHONPATH` so the DLStreamer Python module is importable:
>
>   ```bash
>   source /opt/intel/openvino_2026/setupvars.sh
>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
>   ```

The full sample is in [dlstreamer_sample.py](dlstreamer_sample.py); run it with `python3 dlstreamer_sample.py` or copy the code below.

<!-- sample: dlstreamer_sample.py -->
```python
# SPDX-License-Identifier: MIT
# Copyright (C) Intel Corporation
#
# Motion Tracking: DLStreamer pipeline sample.
# See README.md for prerequisites and inputs. Run: python3 dlstreamer_sample.py

import subprocess
from collections import defaultdict

import numpy as np
import gi

gi.require_version("Gst", "1.0")
gi.require_version("GstAnalytics", "1.0")
gi.require_version("GstVideo", "1.0")
from gi.repository import Gst, GLib, GstAnalytics, GstVideo

Gst.init([])

# Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
import cv2

# Select the model: "yolo26n" (default) or "detr-resnet-50".
MODEL = "yolo26n"

# For CPU: change device=GPU to device=CPU.
# For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
pipeline_str = (
    "filesrc location=test_video.mp4 ! decodebin3 ! "
    "videoconvert ! "
    f"gvadetect model={MODEL}_openvino_model/{MODEL}.xml "
    "device=GPU "
    "threshold=0.4 ! queue ! "
    "gvatrack tracking-type=short-term-imageless ! queue ! "
    "gvawatermark ! videoconvert ! video/x-raw,format=BGR ! "
    "appsink name=sink emit-signals=false sync=false"
)
pipeline = Gst.parse_launch(pipeline_str)
appsink = pipeline.get_by_name("sink")

# Distinct colors for trajectory lines (one per track ID).
COLORS = [
    (255, 0, 0), (0, 255, 0), (0, 0, 255), (255, 255, 0),
    (255, 0, 255), (0, 255, 255), (128, 0, 255), (255, 128, 0),
]
track_history: dict[int, list[tuple[int, int]]] = defaultdict(list)

pipeline.set_state(Gst.State.PLAYING)

proc = None

while True:
    sample = appsink.emit("pull-sample")
    if sample is None:
        break

    buf = sample.get_buffer()
    caps = sample.get_caps()
    struct = caps.get_structure(0)
    width = struct.get_value("width")
    height = struct.get_value("height")

    # Start ffmpeg encoder on the first frame.
    if proc is None:
        ok, fps_num, fps_den = struct.get_fraction("framerate")
        fps = fps_num / fps_den if ok and fps_den > 0 else 30.0
        proc = subprocess.Popen(
            ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "bgr24",
             "-s", f"{width}x{height}", "-r", str(fps),
             "-i", "pipe:0", "-c:v", "libx264", "-pix_fmt", "yuv420p",
             "-movflags", "+faststart", "output_dlstreamer.mp4"],
            stdin=subprocess.PIPE, stderr=subprocess.DEVNULL,
        )

    # Read detection / tracking metadata via GstAnalytics.
    rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
    regions_data = []
    if rmeta is not None:
        od_entries = []
        trk_map = {}  # metadata_id -> tracking_id
        idx = 1
        while True:
            ok_od, od = rmeta.get_od_mtd(idx)
            ok_trk, trk = rmeta.get_tracking_mtd(idx)
            if not ok_od and not ok_trk:
                break
            if ok_od:
                label = GLib.quark_to_string(od.get_obj_type())
                _, x, y, w, h, conf = od.get_location()
                od_entries.append((idx, label, int(x + w / 2), int(y + h / 2)))
            if ok_trk:
                ok2, tid, _, _, _ = trk.get_info()
                if ok2:
                    trk_map[idx] = tid
            idx += 1
        for od_id, label, cx, cy in od_entries:
            tid = 0
            for trk_meta_id, tracking_id in trk_map.items():
                if rmeta.get_relation(od_id, trk_meta_id) != GstAnalytics.RelTypes.NONE:
                    tid = tracking_id
                    break
            regions_data.append((tid, label, cx, cy))

    # Map buffer read-only and copy pixels to a writable numpy array.
    success, map_info = buf.map(Gst.MapFlags.READ)
    if not success:
        continue
    video_info = GstVideo.VideoInfo.new_from_caps(caps)
    stride = video_info.stride[0]
    arr = np.ndarray((height, width, 3), dtype=np.uint8,
                     buffer=map_info.data, strides=(stride, 3, 1)).copy()
    buf.unmap(map_info)

    # Draw per-track trajectory polylines on the frame copy.
    for tid, label, cx, cy in regions_data:
        track = track_history[tid]
        track.append((cx, cy))
        if len(track) > 30:
            track.pop(0)
        color = COLORS[tid % len(COLORS)]
        pts = np.array(track, dtype=np.int32).reshape((-1, 1, 2))
        cv2.polylines(arr, [pts], False, color, 2)
        print(f"  Track {tid}: {label} center=({cx},{cy})", flush=True)

    proc.stdin.write(arr.tobytes())

pipeline.set_state(Gst.State.NULL)
if proc:
    proc.stdin.close()
    proc.wait()
print("Wrote output_dlstreamer.mp4", flush=True)
```
<!-- /sample -->

#### Expected Output

![DLStreamer expected output](expected_output_dlstreamer.gif)

**Device targets:**

- `device=GPU` -- default in the sample code.
- `device=CPU` -- change `device=GPU` to `device=CPU`.
- `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization.

---

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for details.

## References

- [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
- [DETR Model Card (facebook/detr-resnet-50)](https://huggingface.co/facebook/detr-resnet-50)
- [End-to-End Object Detection with Transformers (DETR paper)](https://arxiv.org/abs/2005.12872)
- [Ultralytics Multi-Object Tracking](https://docs.ultralytics.com/modes/track/)
- [Intel DLStreamer gvatrack](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/elements/gvatrack.html)
- [DLStreamer Object Tracking Guide](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/dev_guide/object_tracking.html)
- [OpenVINO Documentation](https://docs.openvino.ai/)
- [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html)
- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dlstreamer/index.html)