| --- |
| license: mit |
| license_link: LICENSE |
| library_name: openvino |
| pipeline_tag: image-classification |
| tags: |
| - openvino |
| - intel |
| - person-detection |
| - person-reidentification |
| - re-identification |
| - edge-ai |
| - metro |
| - dlstreamer |
| language: |
| - en |
| --- |
| |
| # Person Re-Identification |
|
|
| | Property | Value | |
| |---|---| |
| | **Category** | Person Detection + Cross-Camera Re-Identification | |
| | **Base Model** | [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html) + [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html) (Open Model Zoo) | |
| | **Source Framework** | Caffe / PyTorch (Open Model Zoo) | |
| | **Supported Precisions** | FP32, FP16 | |
| | **Inference Engine** | OpenVINO | |
| | **Hardware** | CPU, GPU, NPU | |
| | **Detected Class(es)** | Persons (detection) + 256-d appearance embeddings (re-identification) | |
|
|
| --- |
|
|
| ## Overview |
|
|
| Person Re-Identification is a Metro Analytics use case that tracks the same |
| individual across multiple camera views. |
| Given a reference person seen on one camera, it locates that same person on |
| another camera even though the pose, scale, and viewing angle differ. |
| Each detected person is compared to the reference by cosine similarity of its |
| appearance embedding vector. |
|
|
| It uses a two-stage pipeline: |
|
|
| - **person-detection-retail-0013** -- detects every person in the scene. |
| - **person-reidentification-retail-0287** -- computes a 256-d appearance |
| embedding per person that is robust to viewpoint and lighting changes. |
|
|
| Unlike face-based matching, re-identification relies on whole-body appearance |
| (clothing, build, gait cues), so it works at surveillance distances where faces |
| are not clearly visible. |
|
|
| To demonstrate cross-camera behaviour from a single downloadable clip, the wide |
| surveillance video is treated as two virtual cameras by time window: an earlier |
| enrollment window is **Camera A** (where the reference identity is first seen) |
| and a later query window is **Camera B** (where the person is re-identified as |
| they continue to move through the scene). This emulates a person first seen on |
| one camera and later re-identified on another using the same embedding-matching |
| logic that links identities across a real multi-camera network. |
|
|
| Typical Metro deployments include: |
|
|
| - **Multi-Camera Tracking** -- follow a person across cameras in campuses, |
| airports, and transit hubs. |
| - **Lost-and-Found / Person of Interest** -- locate where a flagged individual |
| appears across a camera network. |
| - **Journey Analytics** -- reconstruct a person's path through a facility. |
| - **Tailgating and Zone Analytics** -- confirm the same person across entry and |
| interior cameras. |
|
|
| > **Privacy Note:** Person re-identification processes biometric-adjacent |
| > appearance data. |
| > Ensure your deployment complies with applicable privacy regulations |
| > (GDPR, BIPA, etc.) and has proper consent and retention policies in place. |
|
|
| --- |
|
|
| ## Prerequisites |
|
|
| - Python 3.11+ |
| - [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version) |
| - [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version) |
|
|
| Create and activate a Python virtual environment before running the scripts: |
|
|
| ```bash |
| python3 -m venv .venv --system-site-packages |
| source .venv/bin/activate |
| ``` |
|
|
| > **Note:** The `--system-site-packages` flag is required so the virtual |
| > environment can access the system-installed OpenVINO and DLStreamer Python |
| > packages. |
|
|
| --- |
|
|
| ## Getting Started |
|
|
| ### Download Models |
|
|
| Run the provided script to download the person detection and re-identification |
| models from the Open Model Zoo: |
|
|
| ```bash |
| chmod +x export_and_quantize.sh |
| ./export_and_quantize.sh |
| ``` |
|
|
| The script downloads `person-detection-retail-0013` and |
| `person-reidentification-retail-0287` in FP16, downloads the sample surveillance |
| video (`test_video.mp4`), and captures a reference person crop (`person_a.jpg`) |
| of the most prominent person seen in the Camera A enrollment window. |
|
|
| ### OpenVINO Sample |
|
|
| The sample below re-identifies the reference person across camera views. |
| It loads the captured reference image (`person_a.jpg`, enrolled from Camera A), |
| computes its embedding, then scans frames of the Camera B query window. |
| In each Camera B frame it detects every person, embeds each one, and keeps the |
| person whose similarity to the reference is highest. |
| It writes the Camera B frame with the strongest match, drawing a green box only |
| on the re-identified person. |
| Change the `device` string to run on CPU, GPU, or NPU. |
|
|
| ```python |
| import cv2 |
| import numpy as np |
| import openvino as ov |
| |
| DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml" |
| REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml" |
| REFERENCE_IMAGE = "person_a.jpg" # reference identity enrolled from Camera A |
| SCENE_VIDEO = "test_video.mp4" # wide feed; a later window acts as Camera B |
| CAMERA_B_START_FRAME = 450 # query window begins ~15s into the clip |
| CONF_THRESHOLD = 0.6 |
| MATCH_THRESHOLD = 0.6 |
| |
| core = ov.Core() |
| |
| # Change device to "GPU" or "NPU" to run on integrated GPU or NPU. |
| det_compiled = core.compile_model(core.read_model(DETECTION_MODEL), "CPU") |
| reid_compiled = core.compile_model(core.read_model(REID_MODEL), "CPU") |
| |
| det_input = det_compiled.input(0) |
| det_h, det_w = det_input.shape[2], det_input.shape[3] |
| reid_input = reid_compiled.input(0) |
| reid_h, reid_w = reid_input.shape[2], reid_input.shape[3] |
| |
| |
| def detect_persons(img): |
| h0, w0 = img.shape[:2] |
| blob = cv2.resize(img, (det_w, det_h)) |
| blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32) |
| dets = det_compiled([blob])[det_compiled.output(0)][0][0] |
| persons = [] |
| for d in dets: |
| if float(d[2]) < CONF_THRESHOLD: |
| continue |
| x1 = max(0, int(d[3] * w0)) |
| y1 = max(0, int(d[4] * h0)) |
| x2 = min(w0, int(d[5] * w0)) |
| y2 = min(h0, int(d[6] * h0)) |
| if x2 > x1 and y2 > y1: |
| persons.append((x1, y1, x2, y2)) |
| return persons |
| |
| |
| def get_embedding(img, bbox): |
| x1, y1, x2, y2 = bbox |
| crop = img[y1:y2, x1:x2] |
| blob = cv2.resize(crop, (reid_w, reid_h)) |
| blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32) |
| emb = reid_compiled([blob])[reid_compiled.output(0)].flatten() |
| return emb / (np.linalg.norm(emb) + 1e-9) |
| |
| |
| # 1. Embed the reference person enrolled from Camera A. |
| # person_a.jpg is already a cropped person, so embed the whole image directly |
| # (the re-identification model expects a person crop as its input). |
| reference = cv2.imread(REFERENCE_IMAGE) |
| if reference is None: |
| raise SystemExit("Could not read the reference image") |
| ref_emb = get_embedding(reference, (0, 0, reference.shape[1], reference.shape[0])) |
| |
| # 2. Scan the Camera B window and keep the frame with the strongest re-id match. |
| cap = cv2.VideoCapture(SCENE_VIDEO) |
| total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 900 |
| best = {"sim": 0.0, "frame": None, "bbox": None} |
| for frame_idx in range(CAMERA_B_START_FRAME, total, 15): |
| cap.set(cv2.CAP_PROP_POS_FRAMES, frame_idx) |
| ok, frame = cap.read() |
| if not ok: |
| break |
| for bbox in detect_persons(frame): |
| sim = float(np.dot(get_embedding(frame, bbox), ref_emb)) |
| if sim > best["sim"]: |
| best = {"sim": sim, "frame": frame.copy(), "bbox": bbox} |
| cap.release() |
| |
| # 3. Annotate and save the best Camera B match. |
| if best["frame"] is None: |
| raise SystemExit("No person detected in the Camera B window") |
| frame = best["frame"] |
| if best["sim"] >= MATCH_THRESHOLD: |
| x1, y1, x2, y2 = best["bbox"] |
| cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2) |
| cv2.putText(frame, f"RE-ID {best['sim']:.2f}", (x1, max(15, y1 - 8)), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2) |
| print(f"Re-identified reference person in Camera B, similarity {best['sim']:.4f}") |
| else: |
| print(f"No matching person found (best similarity {best['sim']:.4f})") |
| |
| cv2.imwrite("output_openvino.jpg", frame) |
| print("Saved: output_openvino.jpg") |
| ``` |
|
|
| **Device targets:** |
|
|
| - `"CPU"` -- default, works on all Intel platforms. |
| - `"GPU"` -- Intel integrated or discrete GPU. |
| - `"NPU"` -- Intel NPU; both FP16 models are NPU-compatible. |
|
|
| #### Expected Output |
|
|
|  |
|
|
| ### DLStreamer Sample |
|
|
| The pipeline below runs the person detector via `gvadetect` and the |
| re-identification model via `gvaclassify` on the video. |
| Frames are pulled through an `appsink`, where each detected person's embedding |
| is compared to the reference embedding computed from `person_a.jpg`. |
| Only persons that match the reference identity are boxed, so the annotated |
| `output_dlstreamer.mp4` highlights the same person as they move across the scene |
| even when other people are present. |
|
|
| > **Notes on running this sample:** |
| > |
| > - Export `PYTHONPATH` so the DLStreamer Python modules (`gi`, `gstgva`) are |
| > importable: |
| > |
| > ```bash |
| > source /opt/intel/openvino_2026/setupvars.sh |
| > source /opt/intel/dlstreamer/scripts/setup_dls_env.sh |
| > export PYTHONPATH=/opt/intel/dlstreamer/python:\ |
| > /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-} |
| > ``` |
| > |
| > - The re-identification embedding is attached as a tensor on each person's |
| > region-of-interest metadata. Convert the stream to `BGR` **before** |
| > `gvadetect`/`gvaclassify` so a downstream format conversion does not strip |
| > those tensors before the `appsink` reads them. |
| |
| ```python |
| import gi |
| |
| gi.require_version("Gst", "1.0") |
| from gi.repository import Gst |
|
|
| Gst.init([]) |
|
|
| import numpy as np |
| import cv2 |
| from gstgva import VideoFrame |
|
|
| INPUT_VIDEO = "test_video.mp4" |
| REFERENCE_IMAGE = "person_a.jpg" |
| OUTPUT_VIDEO = "output_dlstreamer.mp4" |
| DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml" |
| REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml" |
| # For CPU: change "GPU" to "CPU". For NPU: change "GPU" to "NPU". |
| DEVICE = "GPU" |
| DET_THRESHOLD = 0.6 |
| MATCH_THRESHOLD = 0.6 |
|
|
|
|
| def person_embeddings(video_frame): |
| """Yield ((x, y, w, h), normalized_embedding) for each classified person.""" |
| for region in video_frame.regions(): |
| rect = region.rect() |
| emb = None |
| for tensor in region.tensors(): |
| if tensor.is_detection(): |
| continue |
| data = np.array(tensor.data(), dtype=np.float32) |
| if data.size >= 256: |
| emb = data[:256] |
| if emb is None: |
| continue |
| emb = emb / (np.linalg.norm(emb) + 1e-9) |
| yield (int(rect.x), int(rect.y), int(rect.w), int(rect.h)), emb |
| |
|
|
| def run_pipeline(source_desc, on_frame): |
| # Convert to BGR before inference so gvaclassify's embedding tensors survive |
| # to the appsink (a later format-changing videoconvert would strip them). |
| pipeline = Gst.parse_launch( |
| f"{source_desc} ! videoconvert ! video/x-raw,format=BGR ! " |
| f"gvadetect model={DETECTION_MODEL} device={DEVICE} " |
| f"threshold={DET_THRESHOLD} ! queue ! " |
| f"gvaclassify model={REID_MODEL} device={DEVICE} ! queue ! " |
| "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false" |
| ) |
| sink = pipeline.get_by_name("sink") |
| sink.connect("new-sample", on_frame) |
| pipeline.set_state(Gst.State.PLAYING) |
| pipeline.get_bus().timed_pop_filtered( |
| Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR) |
| pipeline.set_state(Gst.State.NULL) |
| |
|
|
| # 1. Compute the reference embedding from the enrolled person image. |
| # person_a.jpg is already a person crop, so run the re-identification model on |
| # the whole frame (inference-region=full-frame) instead of detecting first. |
| ref = {"emb": None} |
| |
| |
| def on_reference(sink): |
| sample = sink.emit("pull-sample") |
| if sample is None: |
| return Gst.FlowReturn.OK |
| vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps()) |
| for tensor in vf.tensors(): |
| data = np.array(tensor.data(), dtype=np.float32) |
| if data.size >= 256: |
| emb = data[:256] |
| ref["emb"] = emb / (np.linalg.norm(emb) + 1e-9) |
| return Gst.FlowReturn.OK |
| |
|
|
| reference_pipeline = Gst.parse_launch( |
| f"filesrc location={REFERENCE_IMAGE} ! jpegdec ! videoconvert ! " |
| f"video/x-raw,format=BGR ! " |
| f"gvainference model={REID_MODEL} device={DEVICE} " |
| f"inference-region=full-frame ! queue ! " |
| "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false" |
| ) |
| ref_sink = reference_pipeline.get_by_name("sink") |
| ref_sink.connect("new-sample", on_reference) |
| reference_pipeline.set_state(Gst.State.PLAYING) |
| reference_pipeline.get_bus().timed_pop_filtered( |
| Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR) |
| reference_pipeline.set_state(Gst.State.NULL) |
| if ref["emb"] is None: |
| raise SystemExit("Could not compute the reference embedding") |
| ref_emb = ref["emb"] |
| |
| # 2. Process the video, boxing only persons that match the reference identity. |
| writer = {"w": None} |
| match_frames = 0 |
| |
| |
| def on_video(sink): |
| global match_frames |
| sample = sink.emit("pull-sample") |
| if sample is None: |
| return Gst.FlowReturn.OK |
| vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps()) |
| matches = [] |
| for (x, y, w, h), emb in person_embeddings(vf): |
| similarity = float(np.dot(emb, ref_emb)) |
| if similarity >= MATCH_THRESHOLD: |
| matches.append((x, y, w, h, similarity)) |
| |
| with vf.data() as mat: |
| frame = mat.copy() |
| |
| if writer["w"] is None: |
| frame_h, frame_w = frame.shape[:2] |
| structure = sample.get_caps().get_structure(0) |
| ok_fr, fps_n, fps_d = structure.get_fraction("framerate") |
| fps = fps_n / fps_d if ok_fr and fps_d else 12 |
| writer["w"] = cv2.VideoWriter( |
| OUTPUT_VIDEO, cv2.VideoWriter_fourcc(*"mp4v"), fps, (frame_w, frame_h)) |
| |
| for x, y, w, h, similarity in matches: |
| cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2) |
| cv2.putText(frame, f"RE-ID {similarity:.2f}", (x, max(15, y - 8)), |
| cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2) |
| if matches: |
| match_frames += 1 |
| writer["w"].write(frame) |
| return Gst.FlowReturn.OK |
| |
|
|
| run_pipeline(f"filesrc location={INPUT_VIDEO} ! decodebin3", on_video) |
| if writer["w"] is not None: |
| writer["w"].release() |
| print(f"Frames with a re-identified person: {match_frames}", flush=True) |
| print(f"Saved: {OUTPUT_VIDEO}", flush=True) |
| ``` |
| |
| **Device targets:** |
| |
| - `DEVICE = "GPU"` -- default in the sample code. |
| - `DEVICE = "CPU"` -- change `"GPU"` to `"CPU"`. |
| - `DEVICE = "NPU"` -- change `"GPU"` to `"NPU"`; use `batch-size=1` and `nireq=4` for best NPU utilization. |
| |
| #### Expected Output |
| |
|  |
| |
| --- |
| |
| ## License |
| |
| Licensed under the MIT License. See [LICENSE](LICENSE) for details. |
| |
| ## References |
| |
| - [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html) |
| - [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html) |
| - [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo) |
| - [OpenVINO Documentation](https://docs.openvino.ai/) |
| - [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html) |
| |
