README.md

---
license: mit
license_link: LICENSE
library_name: openvino
pipeline_tag: image-classification
tags:
  - openvino
  - intel
  - person-detection
  - person-reidentification
  - re-identification
  - edge-ai
  - metro
  - dlstreamer
language:
  - en
---

# Person Re-Identification

| Property | Value |
|---|---|
| **Category** | Person Detection + Cross-Camera Re-Identification |
| **Base Model** | [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html) + [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html) (Open Model Zoo) |
| **Source Framework** | Caffe / PyTorch (Open Model Zoo) |
| **Supported Precisions** | FP32, FP16 |
| **Inference Engine** | OpenVINO |
| **Hardware** | CPU, GPU, NPU |
| **Detected Class(es)** | Persons (detection) + 256-d appearance embeddings (re-identification) |

---

## Overview

Person Re-Identification is a Metro Analytics use case that tracks the same
individual across multiple camera views.
Given a reference person seen on one camera, it locates that same person on
another camera even though the pose, scale, and viewing angle differ.
Each detected person is compared to the reference by cosine similarity of its
appearance embedding vector.

It uses a two-stage pipeline:

- **person-detection-retail-0013** -- detects every person in the scene.
- **person-reidentification-retail-0287** -- computes a 256-d appearance
  embedding per person that is robust to viewpoint and lighting changes.

Unlike face-based matching, re-identification relies on whole-body appearance
(clothing, build, gait cues), so it works at surveillance distances where faces
are not clearly visible.

To demonstrate cross-camera behaviour from a single downloadable clip, the wide
surveillance video is treated as two virtual cameras by time window: an earlier
enrollment window is **Camera A** (where the reference identity is first seen)
and a later query window is **Camera B** (where the person is re-identified as
they continue to move through the scene). This emulates a person first seen on
one camera and later re-identified on another using the same embedding-matching
logic that links identities across a real multi-camera network.

Typical Metro deployments include:

- **Multi-Camera Tracking** -- follow a person across cameras in campuses,
  airports, and transit hubs.
- **Lost-and-Found / Person of Interest** -- locate where a flagged individual
  appears across a camera network.
- **Journey Analytics** -- reconstruct a person's path through a facility.
- **Tailgating and Zone Analytics** -- confirm the same person across entry and
  interior cameras.

> **Privacy Note:** Person re-identification processes biometric-adjacent
> appearance data.
> Ensure your deployment complies with applicable privacy regulations
> (GDPR, BIPA, etc.) and has proper consent and retention policies in place.

---

## Prerequisites

- Python 3.11+
- [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)

Create and activate a Python virtual environment before running the scripts:

```bash
python3 -m venv .venv --system-site-packages
source .venv/bin/activate
```

> **Note:** The `--system-site-packages` flag is required so the virtual
> environment can access the system-installed OpenVINO and DLStreamer Python
> packages.

---

## Getting Started

### Download Models

Run the provided script to download the person detection and re-identification
models from the Open Model Zoo:

```bash
chmod +x export_and_quantize.sh
./export_and_quantize.sh
```

The script downloads `person-detection-retail-0013` and
`person-reidentification-retail-0287` in FP16, downloads the sample surveillance
video (`test_video.mp4`), and captures a reference person crop (`person_a.jpg`)
of the most prominent person seen in the Camera A enrollment window.

### OpenVINO Sample

The sample below re-identifies the reference person across camera views.
It loads the captured reference image (`person_a.jpg`, enrolled from Camera A),
computes its embedding, then scans frames of the Camera B query window.
In each Camera B frame it detects every person, embeds each one, and keeps the
person whose similarity to the reference is highest.
It writes the Camera B frame with the strongest match, drawing a green box only
on the re-identified person.
Change the `device` string to run on CPU, GPU, or NPU.

```python
import cv2
import numpy as np
import openvino as ov

DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml"
REFERENCE_IMAGE = "person_a.jpg"     # reference identity enrolled from Camera A
SCENE_VIDEO = "test_video.mp4"       # wide feed; a later window acts as Camera B
CAMERA_B_START_FRAME = 450           # query window begins ~15s into the clip
CONF_THRESHOLD = 0.6
MATCH_THRESHOLD = 0.6

core = ov.Core()

# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
det_compiled = core.compile_model(core.read_model(DETECTION_MODEL), "CPU")
reid_compiled = core.compile_model(core.read_model(REID_MODEL), "CPU")

det_input = det_compiled.input(0)
det_h, det_w = det_input.shape[2], det_input.shape[3]
reid_input = reid_compiled.input(0)
reid_h, reid_w = reid_input.shape[2], reid_input.shape[3]


def detect_persons(img):
    h0, w0 = img.shape[:2]
    blob = cv2.resize(img, (det_w, det_h))
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
    dets = det_compiled([blob])[det_compiled.output(0)][0][0]
    persons = []
    for d in dets:
        if float(d[2]) < CONF_THRESHOLD:
            continue
        x1 = max(0, int(d[3] * w0))
        y1 = max(0, int(d[4] * h0))
        x2 = min(w0, int(d[5] * w0))
        y2 = min(h0, int(d[6] * h0))
        if x2 > x1 and y2 > y1:
            persons.append((x1, y1, x2, y2))
    return persons


def get_embedding(img, bbox):
    x1, y1, x2, y2 = bbox
    crop = img[y1:y2, x1:x2]
    blob = cv2.resize(crop, (reid_w, reid_h))
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
    emb = reid_compiled([blob])[reid_compiled.output(0)].flatten()
    return emb / (np.linalg.norm(emb) + 1e-9)


# 1. Embed the reference person enrolled from Camera A.
# person_a.jpg is already a cropped person, so embed the whole image directly
# (the re-identification model expects a person crop as its input).
reference = cv2.imread(REFERENCE_IMAGE)
if reference is None:
    raise SystemExit("Could not read the reference image")
ref_emb = get_embedding(reference, (0, 0, reference.shape[1], reference.shape[0]))

# 2. Scan the Camera B window and keep the frame with the strongest re-id match.
cap = cv2.VideoCapture(SCENE_VIDEO)
total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) or 900
best = {"sim": 0.0, "frame": None, "bbox": None}
for frame_idx in range(CAMERA_B_START_FRAME, total, 15):
    cap.set(cv2.CAP_PROP_POS_FRAMES, frame_idx)
    ok, frame = cap.read()
    if not ok:
        break
    for bbox in detect_persons(frame):
        sim = float(np.dot(get_embedding(frame, bbox), ref_emb))
        if sim > best["sim"]:
            best = {"sim": sim, "frame": frame.copy(), "bbox": bbox}
cap.release()

# 3. Annotate and save the best Camera B match.
if best["frame"] is None:
    raise SystemExit("No person detected in the Camera B window")
frame = best["frame"]
if best["sim"] >= MATCH_THRESHOLD:
    x1, y1, x2, y2 = best["bbox"]
    cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
    cv2.putText(frame, f"RE-ID {best['sim']:.2f}", (x1, max(15, y1 - 8)),
                cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
    print(f"Re-identified reference person in Camera B, similarity {best['sim']:.4f}")
else:
    print(f"No matching person found (best similarity {best['sim']:.4f})")

cv2.imwrite("output_openvino.jpg", frame)
print("Saved: output_openvino.jpg")
```

**Device targets:**

- `"CPU"` -- default, works on all Intel platforms.
- `"GPU"` -- Intel integrated or discrete GPU.
- `"NPU"` -- Intel NPU; both FP16 models are NPU-compatible.

#### Expected Output

![OpenVINO expected output](expected_output_openvino.jpg)

### DLStreamer Sample

The pipeline below runs the person detector via `gvadetect` and the
re-identification model via `gvaclassify` on the video.
Frames are pulled through an `appsink`, where each detected person's embedding
is compared to the reference embedding computed from `person_a.jpg`.
Only persons that match the reference identity are boxed, so the annotated
`output_dlstreamer.mp4` highlights the same person as they move across the scene
even when other people are present.

> **Notes on running this sample:**
>
> - Export `PYTHONPATH` so the DLStreamer Python modules (`gi`, `gstgva`) are
>   importable:
>
>   ```bash
>   source /opt/intel/openvino_2026/setupvars.sh
>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
>   ```
>
> - The re-identification embedding is attached as a tensor on each person's
>   region-of-interest metadata. Convert the stream to `BGR` **before**
>   `gvadetect`/`gvaclassify` so a downstream format conversion does not strip
>   those tensors before the `appsink` reads them.

```python
import gi

gi.require_version("Gst", "1.0")
from gi.repository import Gst

Gst.init([])

import numpy as np
import cv2
from gstgva import VideoFrame

INPUT_VIDEO = "test_video.mp4"
REFERENCE_IMAGE = "person_a.jpg"
OUTPUT_VIDEO = "output_dlstreamer.mp4"
DETECTION_MODEL = "intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml"
REID_MODEL = "intel/person-reidentification-retail-0287/FP16/person-reidentification-retail-0287.xml"
# For CPU: change "GPU" to "CPU". For NPU: change "GPU" to "NPU".
DEVICE = "GPU"
DET_THRESHOLD = 0.6
MATCH_THRESHOLD = 0.6


def person_embeddings(video_frame):
    """Yield ((x, y, w, h), normalized_embedding) for each classified person."""
    for region in video_frame.regions():
        rect = region.rect()
        emb = None
        for tensor in region.tensors():
            if tensor.is_detection():
                continue
            data = np.array(tensor.data(), dtype=np.float32)
            if data.size >= 256:
                emb = data[:256]
        if emb is None:
            continue
        emb = emb / (np.linalg.norm(emb) + 1e-9)
        yield (int(rect.x), int(rect.y), int(rect.w), int(rect.h)), emb


def run_pipeline(source_desc, on_frame):
    # Convert to BGR before inference so gvaclassify's embedding tensors survive
    # to the appsink (a later format-changing videoconvert would strip them).
    pipeline = Gst.parse_launch(
        f"{source_desc} ! videoconvert ! video/x-raw,format=BGR ! "
        f"gvadetect model={DETECTION_MODEL} device={DEVICE} "
        f"threshold={DET_THRESHOLD} ! queue ! "
        f"gvaclassify model={REID_MODEL} device={DEVICE} ! queue ! "
        "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false"
    )
    sink = pipeline.get_by_name("sink")
    sink.connect("new-sample", on_frame)
    pipeline.set_state(Gst.State.PLAYING)
    pipeline.get_bus().timed_pop_filtered(
        Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
    pipeline.set_state(Gst.State.NULL)


# 1. Compute the reference embedding from the enrolled person image.
# person_a.jpg is already a person crop, so run the re-identification model on
# the whole frame (inference-region=full-frame) instead of detecting first.
ref = {"emb": None}


def on_reference(sink):
    sample = sink.emit("pull-sample")
    if sample is None:
        return Gst.FlowReturn.OK
    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
    for tensor in vf.tensors():
        data = np.array(tensor.data(), dtype=np.float32)
        if data.size >= 256:
            emb = data[:256]
            ref["emb"] = emb / (np.linalg.norm(emb) + 1e-9)
    return Gst.FlowReturn.OK


reference_pipeline = Gst.parse_launch(
    f"filesrc location={REFERENCE_IMAGE} ! jpegdec ! videoconvert ! "
    f"video/x-raw,format=BGR ! "
    f"gvainference model={REID_MODEL} device={DEVICE} "
    f"inference-region=full-frame ! queue ! "
    "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false"
)
ref_sink = reference_pipeline.get_by_name("sink")
ref_sink.connect("new-sample", on_reference)
reference_pipeline.set_state(Gst.State.PLAYING)
reference_pipeline.get_bus().timed_pop_filtered(
    Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
reference_pipeline.set_state(Gst.State.NULL)
if ref["emb"] is None:
    raise SystemExit("Could not compute the reference embedding")
ref_emb = ref["emb"]

# 2. Process the video, boxing only persons that match the reference identity.
writer = {"w": None}
match_frames = 0


def on_video(sink):
    global match_frames
    sample = sink.emit("pull-sample")
    if sample is None:
        return Gst.FlowReturn.OK
    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
    matches = []
    for (x, y, w, h), emb in person_embeddings(vf):
        similarity = float(np.dot(emb, ref_emb))
        if similarity >= MATCH_THRESHOLD:
            matches.append((x, y, w, h, similarity))

    with vf.data() as mat:
        frame = mat.copy()

    if writer["w"] is None:
        frame_h, frame_w = frame.shape[:2]
        structure = sample.get_caps().get_structure(0)
        ok_fr, fps_n, fps_d = structure.get_fraction("framerate")
        fps = fps_n / fps_d if ok_fr and fps_d else 12
        writer["w"] = cv2.VideoWriter(
            OUTPUT_VIDEO, cv2.VideoWriter_fourcc(*"mp4v"), fps, (frame_w, frame_h))

    for x, y, w, h, similarity in matches:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
        cv2.putText(frame, f"RE-ID {similarity:.2f}", (x, max(15, y - 8)),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
    if matches:
        match_frames += 1
    writer["w"].write(frame)
    return Gst.FlowReturn.OK


run_pipeline(f"filesrc location={INPUT_VIDEO} ! decodebin3", on_video)
if writer["w"] is not None:
    writer["w"].release()
print(f"Frames with a re-identified person: {match_frames}", flush=True)
print(f"Saved: {OUTPUT_VIDEO}", flush=True)
```

**Device targets:**

- `DEVICE = "GPU"` -- default in the sample code.
- `DEVICE = "CPU"` -- change `"GPU"` to `"CPU"`.
- `DEVICE = "NPU"` -- change `"GPU"` to `"NPU"`; use `batch-size=1` and `nireq=4` for best NPU utilization.

#### Expected Output

![DLStreamer expected output](expected_output_dlstreamer.gif)

---

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for details.

## References

- [person-detection-retail-0013](https://docs.openvino.ai/2024/omz_models_model_person_detection_retail_0013.html)
- [person-reidentification-retail-0287](https://docs.openvino.ai/2024/omz_models_model_person_reidentification_retail_0287.html)
- [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo)
- [OpenVINO Documentation](https://docs.openvino.ai/)
- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html)