--- license: mit license_link: LICENSE library_name: openvino pipeline_tag: object-detection tags: - openvino - intel - yolo - yolo26 - crowd-analysis - crowd-density - movement-patterns - person-counting - edge-ai - metro - dlstreamer language: - en --- # Crowd Analysis | Property | Value | |---|---| | **Category** | Object Detection (Crowd Density + Movement) | | **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics) | | **Source Framework** | PyTorch (Ultralytics) | | **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) | | **Inference Engine** | OpenVINO | | **Hardware** | CPU, GPU, NPU | | **Detected Class** | `person` (COCO class 0) | --- ## Overview Crowd Analysis is a Metro Analytics use case that estimates **crowd density** and **movement patterns** in video streams. It detects people frame by frame, reports a per-frame count with a simple density level (`LOW` / `MEDIUM` / `HIGH`), and tracks each person across frames to estimate the dominant flow direction of the crowd. It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a state-of-the-art real-time object detector trained on the COCO dataset, exported to OpenVINO IR and filtered at runtime to the `person` class. Typical Metro deployments include: - **Platform & Concourse Density** -- gauge how crowded station platforms and concourses are and flag build-up before it becomes unsafe. - **Pedestrian Flow Analysis** -- estimate the dominant direction people move through corridors, gates, and crossings. - **Public-Venue Occupancy** -- monitor crowd density at stadiums, transit hubs, and event entrances. - **Situational Awareness** -- combine density level and flow to support operator decisions in public venues and transportation hubs. Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`. Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge deployment; larger variants improve recall in dense crowds. > **Density levels** are defined by two count thresholds (defaults: `LOW` for > fewer than 10 people, `MEDIUM` for 10-25, `HIGH` for more than 25). Tune these > to the field of view and expected occupancy of your deployment site. --- ## Prerequisites - Python 3.11+ - [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version) - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) Create and activate a Python virtual environment before running the scripts: ```bash python3 -m venv .venv --system-site-packages source .venv/bin/activate ``` > **Note:** The `--system-site-packages` flag is required so the virtual > environment can access the system-installed OpenVINO and DLStreamer Python > packages. --- ## Getting Started ### Download and Quantize Model Run the provided script to download, export to OpenVINO IR, and optionally quantize: ```bash chmod +x export_and_quantize.sh ./export_and_quantize.sh ``` This exports the default **yolo26n** model in **FP16** precision. #### Optional: Select a Different Variant or Precision ```bash ./export_and_quantize.sh yolo26n FP32 # full-precision ./export_and_quantize.sh yolo26n INT8 # quantized ./export_and_quantize.sh yolo26s # larger variant, default FP16 ``` Replace `yolo26n` with any variant (`yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`). The second argument selects the precision (`FP32`, `FP16`, `INT8`); the default is **FP16**. The script performs the following steps: 1. Installs dependencies (`openvino`, `ultralytics`; adds `nncf` for INT8). 2. Downloads a sample test image (`test.jpg`) and a sample test video (`test_video.mp4`). 3. Downloads the PyTorch weights and exports to OpenVINO IR. 4. *(INT8 only)* Quantizes the model using NNCF post-training quantization. The sample video is a free-to-use [pedestrians-crossing-the-street clip from Pexels](https://www.pexels.com/video/pedestrians-crossing-the-street-27700659/). Output files: - `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory. - `yolo26n_crowdanalysis_int8.xml` / `yolo26n_crowdanalysis_int8.bin` -- INT8 quantized model *(only when `INT8` is selected)*. #### Precision / Device Compatibility | Precision | CPU | GPU | NPU | |---|---|---|---| | FP32 | Yes | Yes | No | | FP16 | Yes | Yes | Yes | | INT8 | Yes | Yes | Yes | > **Note:** The INT8 calibration uses the bundled sample image. > For production accuracy, replace it with a representative set of frames from > the target deployment site. ### OpenVINO Sample The sample below runs YOLO26 inference on the sample video, filters to the `person` class, reports the crowd count and density level per frame, tracks each person with a lightweight IoU tracker to estimate the dominant flow direction, and writes the annotated result to `output_openvino.mp4`. Change the `device` string to run on CPU, GPU, or NPU. ```python import cv2 import numpy as np import openvino as ov PERSON_CLASS_ID = 0 CONF_THRESHOLD = 0.4 INPUT_SIZE = 640 # Crowd-density thresholds (person count per frame). DENSITY_LOW_MAX = 10 # fewer than 10 -> LOW DENSITY_MEDIUM_MAX = 25 # 10-25 -> MEDIUM, more than 25 -> HIGH # Movement tracking. IOU_MATCH_THRESHOLD = 0.3 MAX_MISSED_FRAMES = 15 def density_level(count): if count < DENSITY_LOW_MAX: return "LOW", (0, 200, 0) if count <= DENSITY_MEDIUM_MAX: return "MEDIUM", (0, 200, 255) return "HIGH", (0, 0, 255) def iou(box_a, box_b): ax1, ay1, ax2, ay2 = box_a bx1, by1, bx2, by2 = box_b ix1, iy1 = max(ax1, bx1), max(ay1, by1) ix2, iy2 = min(ax2, bx2), min(ay2, by2) inter = max(0, ix2 - ix1) * max(0, iy2 - iy1) if inter == 0: return 0.0 area_a = max(0, ax2 - ax1) * max(0, ay2 - ay1) area_b = max(0, bx2 - bx1) * max(0, by2 - by1) return inter / float(area_a + area_b - inter) class CentroidTracker: """Minimal IoU tracker that records each track's last centroid so we can estimate per-frame movement (flow) vectors.""" def __init__(self): self._next_id = 1 self._tracks = {} # id -> {"box", "centroid", "missed"} def update(self, boxes): unmatched = set(self._tracks) assignments, moves = [], [] for box in boxes: cx = (box[0] + box[2]) / 2.0 cy = (box[1] + box[3]) / 2.0 best_id, best_iou = None, IOU_MATCH_THRESHOLD for tid in unmatched: score = iou(box, self._tracks[tid]["box"]) if score > best_iou: best_id, best_iou = tid, score if best_id is not None: tid = best_id unmatched.discard(tid) pcx, pcy = self._tracks[tid]["centroid"] moves.append((cx - pcx, cy - pcy)) else: tid = self._next_id self._next_id += 1 self._tracks[tid] = {"box": box, "centroid": (cx, cy), "missed": 0} assignments.append((box, tid)) for tid in unmatched: self._tracks[tid]["missed"] += 1 if self._tracks[tid]["missed"] > MAX_MISSED_FRAMES: del self._tracks[tid] return assignments, moves core = ov.Core() model = core.read_model("yolo26n_openvino_model/yolo26n.xml") compiled = core.compile_model(model, "CPU") # or "GPU", "NPU" cap = cv2.VideoCapture("test_video.mp4") fps = cap.get(cv2.CAP_PROP_FPS) or 30.0 width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) writer = cv2.VideoWriter( "output_openvino.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (width, height)) tracker = CentroidTracker() while True: ok, frame = cap.read() if not ok: break h0, w0 = frame.shape[:2] sx, sy = w0 / INPUT_SIZE, h0 / INPUT_SIZE blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE)) blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0 blob = blob.transpose(2, 0, 1)[np.newaxis, ...] # NCHW # YOLO26 end-to-end output: [1, 300, 6] = [x1, y1, x2, y2, confidence, class_id] output = compiled([blob])[compiled.output(0)][0] mask = (output[:, 4] >= CONF_THRESHOLD) & (output[:, 5].astype(int) == PERSON_CLASS_ID) dets = output[mask] boxes = [(d[0] * sx, d[1] * sy, d[2] * sx, d[3] * sy) for d in dets] assignments, moves = tracker.update(boxes) for box, _tid in assignments: x1, y1, x2, y2 = (int(v) for v in box) cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2) count = len(boxes) level, color = density_level(count) cv2.putText(frame, f"Crowd: {count} ({level})", (10, 40), cv2.FONT_HERSHEY_SIMPLEX, 1.0, color, 2) # Movement: mean of all per-track displacements -> dominant flow arrow. if moves: mdx = float(np.mean([m[0] for m in moves])) mdy = float(np.mean([m[1] for m in moves])) ox, oy = width // 2, height - 40 cv2.arrowedLine(frame, (ox, oy), (int(ox + mdx * 10), int(oy + mdy * 10)), (255, 0, 0), 3, tipLength=0.3) cv2.putText(frame, f"Flow dx={mdx:+.1f} dy={mdy:+.1f}", (10, 75), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 0, 0), 2) writer.write(frame) cap.release() writer.release() print("Saved: output_openvino.mp4") ``` ### Try It on the Sample Video The `export_and_quantize.sh` script downloads `test_video.mp4` automatically. Run the OpenVINO sample above. It reads `test_video.mp4`, prints the crowd count and density level per frame, and writes the annotated video to `output_openvino.mp4` with a green box around each detected person, the `Crowd: N (LEVEL)` overlay, and a blue arrow showing the dominant crowd flow. > **Tip:** For production testing, replace the bundled `test_video.mp4` with > footage from your target deployment site and re-tune the density thresholds. #### Expected Output ![OpenVINO expected output](expected_output_openvino.gif) ### DLStreamer Sample The pipeline below runs the FP16 YOLO26 detector on the sample video via `gvadetect`, assigns a stable track ID to each person with `gvatrack`, filters detections to the `person` class in a buffer probe using the GStreamer Analytics metadata API (`GstAnalytics`), overlays bounding boxes, and saves the annotated result to `output_dlstreamer.mp4`. The probe prints the crowd count, density level, and dominant flow direction per frame. > **Notes on running this sample:** > > - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml`). > On DLStreamer 2026.0.0, `gvadetect` cannot auto-derive a YOLO post-processor > from the INT8 model produced by the bundled script. > To use the INT8 model, supply a matching `model-proc` JSON. > - Class names are read automatically from the model's embedded > `metadata.yaml` by DLStreamer 2026.0+ -- no external `labels-file` is > required. > - Filtering with `object-class=person` directly on `gvadetect` is rejected > when `inference-region` is `full-frame` (the default), so the sample > filters by detection label in the buffer probe instead. > - Export `PYTHONPATH` so the DLStreamer Python module is importable: > > ```bash > source /opt/intel/openvino_2026/setupvars.sh > source /opt/intel/dlstreamer/scripts/setup_dls_env.sh > export PYTHONPATH=/opt/intel/dlstreamer/python:\ > /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-} > ``` ```python import gi gi.require_version("Gst", "1.0") gi.require_version("GstAnalytics", "1.0") from gi.repository import Gst, GLib, GstAnalytics Gst.init([]) INPUT_VIDEO = "test_video.mp4" # Crowd-density thresholds (person count per frame). DENSITY_LOW_MAX = 10 # fewer than 10 -> LOW DENSITY_MEDIUM_MAX = 25 # 10-25 -> MEDIUM, more than 25 -> HIGH def density_level(count): if count < DENSITY_LOW_MAX: return "LOW" if count <= DENSITY_MEDIUM_MAX: return "MEDIUM" return "HIGH" # For CPU: change device=GPU to device=CPU. # For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended). pipeline_str = ( f"filesrc location={INPUT_VIDEO} ! decodebin3 ! " "videoconvert ! " "gvadetect model=yolo26n_openvino_model/yolo26n.xml " "device=GPU " "threshold=0.4 ! queue ! " "gvatrack tracking-type=zero-term-imageless ! queue ! " "gvawatermark displ-cfg=show-roi=person ! " "videoconvert ! video/x-raw,format=I420 ! " "openh264enc ! h264parse ! " "mp4mux ! filesink name=sink location=output_dlstreamer.mp4" ) pipeline = Gst.parse_launch(pipeline_str) sink = pipeline.get_by_name("sink") sink_pad = sink.get_static_pad("sink") prev_centroid = {} # track_id -> (cx, cy) def on_buffer(pad, info): buf = info.get_buffer() rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf) if rmeta is None: return Gst.PadProbeReturn.OK # OD and tracking metadata share one id space and can be interleaved # (id=1 -> ODMtd, id=2 -> TrackingMtd, ...), so scan every id and stop only # after several consecutive misses. ods, tracks = [], [] idx, misses = 1, 0 while misses < 20: ok_od, od = rmeta.get_od_mtd(idx) ok_trk, trk = rmeta.get_tracking_mtd(idx) if ok_od: ods.append(od) misses = 0 elif ok_trk: tracks.append(trk) misses = 0 else: misses += 1 idx += 1 count, moves = 0, [] for od in ods: if GLib.quark_to_string(od.get_obj_type()) != "person": continue count += 1 _, x, y, w, h, _ = od.get_location() cx, cy = x + w / 2.0, y + h / 2.0 for trk in tracks: if rmeta.get_relation(od.id, trk.id) == GstAnalytics.RelTypes.NONE: continue ok_trk, track_id, _, _, _ = trk.get_info() if not ok_trk: continue if track_id in prev_centroid: pcx, pcy = prev_centroid[track_id] moves.append((cx - pcx, cy - pcy)) prev_centroid[track_id] = (cx, cy) break if count: level = density_level(count) if moves: mdx = sum(m[0] for m in moves) / len(moves) mdy = sum(m[1] for m in moves) / len(moves) print(f"Crowd: {count} ({level}) flow dx={mdx:+.1f} dy={mdy:+.1f}", flush=True) else: print(f"Crowd: {count} ({level})", flush=True) return Gst.PadProbeReturn.OK sink_pad.add_probe(Gst.PadProbeType.BUFFER, on_buffer) pipeline.set_state(Gst.State.PLAYING) bus = pipeline.get_bus() bus.timed_pop_filtered( Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR, ) pipeline.set_state(Gst.State.NULL) ``` #### Expected Output ![DLStreamer expected output](expected_output_dlstreamer.gif) **Device targets:** - `device=GPU` -- default in the sample code. - `device=CPU` -- change `device=GPU` to `device=CPU`. - `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization. --- ## License Licensed under the MIT License. See [LICENSE](LICENSE) for details. ## References - [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/) - [OpenVINO YOLO26 Notebook](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov26-optimization/yolov26-object-detection.ipynb) - [COCO Dataset](https://cocodataset.org/) - [OpenVINO Documentation](https://docs.openvino.ai/) - [NNCF Post-Training Quantization](https://docs.openvino.ai/latest/nncf_ptq_introduction.html) - [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html) - [Sample video: Pedestrians crossing the street (Pexels)](https://www.pexels.com/video/pedestrians-crossing-the-street-27700659/)