README.md

---
license: mit
license_link: LICENSE
library_name: openvino
pipeline_tag: object-detection
tags:
  - openvino
  - intel
  - yolo
  - yolo26
  - intrusion-detection
  - zone-analytics
  - tracking
  - gstanalytics
  - gvaanalytics
  - edge-ai
  - metro
  - dlstreamer
language:
  - en
---

# Intrusion Detection

| Property | Value |
|---|---|
| **Category** | Object Detection + Tracking + Zone Analytics (GstAnalytics) |
| **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) |
| **Source Framework** | PyTorch (Ultralytics) |
| **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
| **Inference Engine** | OpenVINO |
| **Hardware** | CPU, GPU, NPU |
| **Detected Class** | `person` (COCO class 0) |

---

## Overview

Intrusion Detection is a Metro Analytics use case that flags unauthorized entry into a restricted region of interest.
It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/) for person detection, paired with a multi-object tracker that assigns persistent IDs across frames.
DLStreamer's `gvaanalytics` element defines the protected zone and automatically attaches `GstAnalyticsZoneMtd` metadata to every tracked person whose center falls inside the polygon.
A Python probe reads this GstAnalytics metadata and raises an intrusion event the moment a tracked person first crosses into the restricted zone.
The model is a quantized (INT8) state-of-the-art detector; smaller variants run at high FPS on edge hardware.

Typical Metro deployments include:

- **Restricted-Area Monitoring** -- raise alerts when a person enters track beds, equipment rooms, or after-hours zones.
- **Utility-Site Protection** -- detect entry into substations, pump houses, and fenced infrastructure.
- **Secured-Perimeter Enforcement** -- trigger on anyone crossing a fence line or standoff boundary.
- **Off-Limits Zone Compliance** -- monitor emergency exits, tunnels, and maintenance corridors that must stay clear.

Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`.
Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment.

---

## Prerequisites

- Python 3.11+
- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html)

Create and activate a Python virtual environment before running the scripts:

```bash
python3 -m venv .venv --system-site-packages
source .venv/bin/activate
```

> **Note:** The `--system-site-packages` flag is required so the virtual
> environment can access the system-installed OpenVINO and DLStreamer Python
> packages.

---

## Getting Started

### Download and Quantize Model

Run the provided script to download, export to OpenVINO IR, and optionally quantize:

```bash
chmod +x export_and_quantize.sh
./export_and_quantize.sh
```

This exports the default **yolo26n** model in **FP16** precision.

#### Optional: Select a Different Variant or Precision

```bash
./export_and_quantize.sh yolo26n FP32   # full-precision
./export_and_quantize.sh yolo26n INT8   # quantized
./export_and_quantize.sh yolo26s        # larger variant, default FP16
```

Replace `yolo26n` with any variant (`yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`).
The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**.

The script performs the following steps:

1. Installs dependencies (`openvino`, `ultralytics`; adds `nncf` for INT8).
2. Downloads the sample surveillance video (`VIRAT_S_000101.mp4`) from the Intel Metro AI Suite project into the current directory.
3. Downloads the PyTorch weights and exports to OpenVINO IR.
4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.

Output files:

- `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
- `yolo26n_intrusion_int8.xml` / `yolo26n_intrusion_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*.

#### Precision / Device Compatibility

| Precision | CPU | GPU | NPU |
|---|---|---|---|
| FP32 | Yes | Yes | No |
| FP16 | Yes | Yes | Yes |
| INT8 | Yes | Yes | Yes |

> **Note:** The INT8 calibration uses frames from the bundled sample video.
> For production accuracy, replace it with a representative set of frames from
> the target deployment site.

### Defining the Restricted Zone

The zone is a polygon defined in JSON and passed to DLStreamer's
`gvaanalytics` element, which automatically detects when tracked objects
are inside the zone using GstAnalytics metadata -- no Python polygon math
required.
A typical restricted-zone configuration on a 1280x720 source might be:

```json
[
  {
    "id": "restricted_zone",
    "type": "polygon",
    "points": [
      {"x": 0, "y": 200},
      {"x": 300, "y": 200},
      {"x": 300, "y": 400},
      {"x": 0, "y": 400}
    ]
  }
]
```

The `gvaanalytics` element attaches `GstAnalyticsZoneMtd` to each detection
whose center falls inside the polygon.
The Python probe checks for this metadata and raises an intrusion event the
first time each tracked person enters the zone.

> **Note:** The zone polygon supports arbitrary shapes (not just rectangles).
> Use `draw-zones=true` (the default) so that `gvawatermark` renders the zone
> boundary on the output video.

### DLStreamer Sample

Set up the environment:

```bash
source /opt/intel/openvino_2026/setupvars.sh
source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
export PYTHONPATH=/opt/intel/dlstreamer/python:/opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
```

Run intrusion detection:

```python
import json
import sys
import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstAnalytics", "1.0")
gi.require_version("DLStreamerMeta", "1.0")
gi.require_version("DLStreamerWatermarkMeta", "1.0")
from gi.repository import Gst, GLib, GstAnalytics, DLStreamerMeta, DLStreamerWatermarkMeta

Gst.init([])

# Register DLStreamerMeta types so GstAnalytics iteration can handle them
_ov = sys.modules["gi.overrides.GstAnalytics"]
_ov.__mtd_types__[DLStreamerMeta.ZoneMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_zone_mtd
_ov.__mtd_types__[DLStreamerMeta.TripwireMtd.get_mtd_type()] = DLStreamerMeta.relation_meta_get_tripwire_mtd

MODEL = "yolo26n_openvino_model/yolo26n.xml"
VIDEO = "VIRAT_S_000101.mp4"
ZONE_JSON = json.dumps([{
    "id": "restricted_zone",
    "type": "polygon",
    "points": [{"x": 0, "y": 200}, {"x": 300, "y": 200},
               {"x": 300, "y": 400}, {"x": 0, "y": 400}]
}])

pipeline = Gst.parse_launch(
    f"filesrc location={VIDEO} ! decodebin3 ! videoconvert ! "
    f"gvadetect model={MODEL} device=GPU threshold=0.5 ! queue ! "
    f"gvatrack tracking-type=short-term-imageless ! queue ! "
    f"gvaanalytics name=analytics draw-zones=true ! "
    f"gvafpscounter ! identity name=probe ! gvawatermark name=watermark ! "
    f"videoconvert ! video/x-raw,format=I420 ! "
    f"openh264enc ! h264parse ! mp4mux ! filesink location=output_dlstreamer.mp4"
)

pipeline.get_by_name("analytics").set_property("zones", ZONE_JSON)
pipeline.get_by_name("watermark").set_property("displ-cfg", "hide-roi=person")

# Track IDs that have already triggered an intrusion event, so each intruder
# is reported only once.
flagged = set()

def on_buffer(pad, info):
    buf = info.get_buffer()
    now = buf.pts / Gst.SECOND if buf.pts != Gst.CLOCK_TIME_NONE else 0.0
    rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
    if not rmeta:
        return Gst.PadProbeReturn.OK

    # Iterate only over object-detection entries
    for od in rmeta.iter_on_type(GstAnalytics.ODMtd):
        label = GLib.quark_to_string(od.get_obj_type())
        if label != "person":
            continue

        # Find tracking ID via direct relation
        track_id = None
        for trk in od.iter_direct_related(GstAnalytics.RelTypes.RELATE_TO, GstAnalytics.TrackingMtd):
            success, tracking_id, *_ = trk.get_info()
            if success:
                track_id = tracking_id
            break
        if track_id is None:
            continue

        # Check if gvaanalytics placed this detection inside the restricted zone
        in_zone = False
        for zone in od.iter_direct_related(GstAnalytics.RelTypes.RELATE_TO, DLStreamerMeta.ZoneMtd):
            in_zone = True
            break

        if not in_zone:
            continue

        # Raise an intrusion event the first time each person enters the zone
        if track_id not in flagged:
            flagged.add(track_id)
            _, x, y, w, h, _ = od.get_location()
            print(f"INTRUSION id={track_id} t={now:.1f}s entered restricted zone at ({int(x + w/2)},{int(y + h)})")

    return Gst.PadProbeReturn.OK

pipeline.get_by_name("probe").get_static_pad("src").add_probe(Gst.PadProbeType.BUFFER, on_buffer)
pipeline.set_state(Gst.State.PLAYING)
pipeline.get_bus().timed_pop_filtered(Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
pipeline.set_state(Gst.State.NULL)
```

Expected output:

```text
INTRUSION id=26 t=3.2s entered restricted zone at (147,341)
INTRUSION id=27 t=4.6s entered restricted zone at (122,337)
...
```

The annotated video is saved to `output_dlstreamer.mp4`.
The `gvaanalytics` element also draws the zone polygon on each frame via `gvawatermark`.

#### Expected Output

![DLStreamer expected output](expected_output_dlstreamer.gif)

**Device targets:**

- `device=GPU` -- default in the sample code.
- `device=CPU` -- change `device=GPU` to `device=CPU`.
- `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization.

---

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for details.

## References

- [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
- [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb)
- [Intel DLStreamer Object Tracking](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/elements/gvatrack.html)
- [Intel DLStreamer gvaanalytics](https://github.com/dlstreamer/dlstreamer/blob/main/src/monolithic/gst/elements/gvaanalytics/README.md)
- [OpenVINO Documentation](https://docs.openvino.ai/)
- [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html)
- [COCO Dataset](https://cocodataset.org/)