README.md

---
license: mit
license_link: LICENSE
library_name: openvino
pipeline_tag: image-classification
tags:
  - openvino
  - intel
  - face-detection
  - face-matching
  - face-reidentification
  - edge-ai
  - metro
  - dlstreamer
language:
  - en
---

# Face Matching

| Property | Value |
|---|---|
| **Category** | Face Detection + One-to-One Verification |
| **Base Model** | [face-detection-adas-0001](https://docs.openvino.ai/2024/omz_models_model_face_detection_adas_0001.html) + [face-reidentification-retail-0095](https://docs.openvino.ai/2024/omz_models_model_face_reidentification_retail_0095.html) (Open Model Zoo) |
| **Source Framework** | Caffe / PyTorch (Open Model Zoo) |
| **Supported Precisions** | FP32, FP16 |
| **Inference Engine** | OpenVINO |
| **Hardware** | CPU, GPU, NPU |
| **Detected Class(es)** | Human faces (detection) + 256-d face embeddings (matching) |

---

## Overview

Face Matching is a Metro Analytics use case that verifies a specific identity:
given a reference face image, it locates that person inside a scene that may
contain several people and highlights only the matching face. Each detected
face is compared to the reference by cosine similarity of its embedding vector.

It uses the same two-stage pipeline as [facial-recognition](../facial-recognition/):

- **face-detection-adas-0001** -- detects every face in the scene.
- **face-reidentification-retail-0095** -- computes a 256-d embedding per face.

The difference from full facial recognition is scope: face matching verifies a
single reference identity and highlights only that person, without maintaining a
gallery database.

Typical Metro deployments include:

- **Badge Verification** -- compare a live face to a badge photo at entry gates.
- **Document Verification** -- match a passport or ID photo to the holder.
- **Duplicate Detection** -- check if two records belong to the same person.
- **Re-identification Confirmation** -- confirm a person flagged by the search system.

> **Privacy Note:** Face matching involves biometric data.
> Ensure your deployment complies with applicable privacy regulations
> (GDPR, BIPA, etc.) and has proper consent mechanisms in place.

---

## Prerequisites

- Python 3.11+
- [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)

Create and activate a Python virtual environment before running the scripts:

```bash
python3 -m venv .venv --system-site-packages
source .venv/bin/activate
```

> **Note:** The `--system-site-packages` flag is required so the virtual
> environment can access the system-installed OpenVINO and DLStreamer Python
> packages.

---

## Getting Started

### Download Models

Run the provided script to download the face detection and re-identification
models from the Open Model Zoo:

```bash
chmod +x export_and_quantize.sh
./export_and_quantize.sh
```

The script downloads `face-detection-adas-0001` and
`face-reidentification-retail-0095` in FP16, downloads the sample video, and
captures a reference face image (`face_a.jpg`) of the left subject from it.

### OpenVINO Sample

The sample below verifies identity inside a scene: it loads the captured
reference image of the subject (`face_a.jpg`, the woman who pauses on the left of
the sample video), computes its embedding, then reads a frame from the video that
contains several people. It detects every face in the frame, embeds each one, and
draws a green box only on the face whose similarity to the reference is highest
and above the match threshold. Change the `device` string to run on CPU, GPU, or NPU.

```python
import cv2
import numpy as np
import openvino as ov

DETECTION_MODEL = "intel/face-detection-adas-0001/FP16/face-detection-adas-0001.xml"
REID_MODEL = "intel/face-reidentification-retail-0095/FP16/face-reidentification-retail-0095.xml"
REFERENCE_IMAGE = "face_a.jpg"  # reference: left subject captured from the video
SCENE_VIDEO = "test_video.mp4"  # scene containing the reference subject plus others
SCENE_FRAME = 480               # frame index of the two-person scene (~40s)
CONF_THRESHOLD = 0.5
MATCH_THRESHOLD = 0.5

core = ov.Core()

# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
det_compiled = core.compile_model(core.read_model(DETECTION_MODEL), "CPU")
reid_compiled = core.compile_model(core.read_model(REID_MODEL), "CPU")

det_input = det_compiled.input(0)
det_h, det_w = det_input.shape[2], det_input.shape[3]
reid_input = reid_compiled.input(0)
reid_h, reid_w = reid_input.shape[2], reid_input.shape[3]


def detect_faces(img):
    h0, w0 = img.shape[:2]
    blob = cv2.resize(img, (det_w, det_h))
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
    dets = det_compiled([blob])[det_compiled.output(0)][0][0]
    faces = []
    for d in dets:
        if float(d[2]) < CONF_THRESHOLD:
            continue
        x1 = max(0, int(d[3] * w0))
        y1 = max(0, int(d[4] * h0))
        x2 = min(w0, int(d[5] * w0))
        y2 = min(h0, int(d[6] * h0))
        if x2 > x1 and y2 > y1:
            faces.append((x1, y1, x2, y2))
    return faces


def get_embedding(img, bbox):
    x1, y1, x2, y2 = bbox
    crop = img[y1:y2, x1:x2]
    blob = cv2.resize(crop, (reid_w, reid_h))
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)
    emb = reid_compiled([blob])[reid_compiled.output(0)].flatten()
    return emb / np.linalg.norm(emb)


# 1. Embed the reference face.
reference = cv2.imread(REFERENCE_IMAGE)
ref_faces = detect_faces(reference)
if not ref_faces:
    raise SystemExit("No face detected in the reference image")
ref_bbox = max(ref_faces, key=lambda b: (b[2] - b[0]) * (b[3] - b[1]))
ref_emb = get_embedding(reference, ref_bbox)

# 2. Read a scene frame that contains several people.
cap = cv2.VideoCapture(SCENE_VIDEO)
cap.set(cv2.CAP_PROP_POS_FRAMES, SCENE_FRAME)
ok, frame = cap.read()
cap.release()
if not ok:
    raise SystemExit(f"Could not read frame {SCENE_FRAME} from {SCENE_VIDEO}")

# 3. Compare every face in the scene to the reference; keep the best match.
best_bbox = None
best_sim = 0.0
for bbox in detect_faces(frame):
    sim = float(np.dot(get_embedding(frame, bbox), ref_emb))
    if sim > best_sim:
        best_sim = sim
        best_bbox = bbox

if best_bbox is not None and best_sim >= MATCH_THRESHOLD:
    x1, y1, x2, y2 = best_bbox
    cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
    cv2.putText(frame, f"MATCH {best_sim:.2f}", (x1, max(15, y1 - 8)),
                cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
    print(f"Matched reference identity, similarity {best_sim:.4f}")
else:
    print(f"No matching face found (best similarity {best_sim:.4f})")

cv2.imwrite("output_openvino.jpg", frame)
print("Saved: output_openvino.jpg")
```

**Device targets:**

- `"CPU"` -- default, works on all Intel platforms.
- `"GPU"` -- Intel integrated or discrete GPU.
- `"NPU"` -- Intel NPU; face-detection-adas-0001 FP16 is NPU-compatible.

#### Expected Output

![OpenVINO expected output](expected_output_openvino.jpg)

### DLStreamer Sample

The pipeline below runs the face detector via `gvadetect` and the
re-identification model via `gvaclassify` on the video. Frames are pulled through
an `appsink`, where each detected face's embedding is compared to the reference
embedding computed from `face_a.jpg`. Only faces that match the reference are
boxed, so the annotated `output_dlstreamer.mp4` highlights just the reference
subject even when other people are present.

> **Notes on running this sample:**
>
> - Export `PYTHONPATH` so the DLStreamer Python modules (`gi`, `gstgva`) are
>   importable:
>
>   ```bash
>   source /opt/intel/openvino_2026/setupvars.sh
>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
>   ```
>
> - The re-identification embedding is attached as a tensor on each face's
>   region-of-interest metadata. Convert the stream to `BGR` **before**
>   `gvadetect`/`gvaclassify` so a downstream format conversion does not strip
>   those tensors before the `appsink` reads them.

```python
import gi

gi.require_version("Gst", "1.0")
from gi.repository import Gst

Gst.init([])

import numpy as np
import cv2
from gstgva import VideoFrame

INPUT_VIDEO = "test_video.mp4"
REFERENCE_IMAGE = "face_a.jpg"
OUTPUT_VIDEO = "output_dlstreamer.mp4"
DETECTION_MODEL = "intel/face-detection-adas-0001/FP16/face-detection-adas-0001.xml"
REID_MODEL = "intel/face-reidentification-retail-0095/FP16/face-reidentification-retail-0095.xml"
# For CPU: change "GPU" to "CPU". For NPU: change "GPU" to "NPU".
DEVICE = "GPU"
DET_THRESHOLD = 0.6
MATCH_THRESHOLD = 0.4


def face_embeddings(video_frame):
    """Yield ((x, y, w, h), normalized_embedding) for each classified face."""
    for region in video_frame.regions():
        rect = region.rect()
        emb = None
        for tensor in region.tensors():
            if tensor.is_detection():
                continue
            data = np.array(tensor.data(), dtype=np.float32)
            if data.size >= 256:
                emb = data[:256]
        if emb is None:
            continue
        emb = emb / (np.linalg.norm(emb) + 1e-9)
        yield (int(rect.x), int(rect.y), int(rect.w), int(rect.h)), emb


def run_pipeline(source_desc, on_frame):
    # Convert to BGR before inference so gvaclassify's embedding tensors survive
    # to the appsink (a later format-changing videoconvert would strip them).
    pipeline = Gst.parse_launch(
        f"{source_desc} ! videoconvert ! video/x-raw,format=BGR ! "
        f"gvadetect model={DETECTION_MODEL} device={DEVICE} "
        f"threshold={DET_THRESHOLD} ! queue ! "
        f"gvaclassify model={REID_MODEL} device={DEVICE} ! queue ! "
        "appsink name=sink emit-signals=true sync=false max-buffers=4 drop=false"
    )
    sink = pipeline.get_by_name("sink")
    sink.connect("new-sample", on_frame)
    pipeline.set_state(Gst.State.PLAYING)
    pipeline.get_bus().timed_pop_filtered(
        Gst.CLOCK_TIME_NONE, Gst.MessageType.EOS | Gst.MessageType.ERROR)
    pipeline.set_state(Gst.State.NULL)


# 1. Compute the reference embedding from the reference image.
ref = {"emb": None, "area": 0}


def on_reference(sink):
    sample = sink.emit("pull-sample")
    if sample is None:
        return Gst.FlowReturn.OK
    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
    for (x, y, w, h), emb in face_embeddings(vf):
        if w * h > ref["area"]:
            ref["area"] = w * h
            ref["emb"] = emb
    return Gst.FlowReturn.OK


run_pipeline(f"filesrc location={REFERENCE_IMAGE} ! jpegdec", on_reference)
if ref["emb"] is None:
    raise SystemExit("No face detected in the reference image")
ref_emb = ref["emb"]

# 2. Process the video, boxing only faces that match the reference identity.
writer = {"w": None}
match_frames = 0


def on_video(sink):
    global match_frames
    sample = sink.emit("pull-sample")
    if sample is None:
        return Gst.FlowReturn.OK
    vf = VideoFrame(sample.get_buffer(), caps=sample.get_caps())
    matches = []
    for (x, y, w, h), emb in face_embeddings(vf):
        similarity = float(np.dot(emb, ref_emb))
        if similarity >= MATCH_THRESHOLD:
            matches.append((x, y, w, h, similarity))

    with vf.data() as mat:
        frame = mat.copy()

    if writer["w"] is None:
        frame_h, frame_w = frame.shape[:2]
        structure = sample.get_caps().get_structure(0)
        ok_fr, fps_n, fps_d = structure.get_fraction("framerate")
        fps = fps_n / fps_d if ok_fr and fps_d else 12
        writer["w"] = cv2.VideoWriter(
            OUTPUT_VIDEO, cv2.VideoWriter_fourcc(*"mp4v"), fps, (frame_w, frame_h))

    for x, y, w, h, similarity in matches:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
        cv2.putText(frame, f"MATCH {similarity:.2f}", (x, max(15, y - 8)),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
    if matches:
        match_frames += 1
    writer["w"].write(frame)
    return Gst.FlowReturn.OK


run_pipeline(f"filesrc location={INPUT_VIDEO} ! decodebin3", on_video)
if writer["w"] is not None:
    writer["w"].release()
print(f"Frames with a matched face: {match_frames}", flush=True)
print(f"Saved: {OUTPUT_VIDEO}", flush=True)
```

**Device targets:**

- `DEVICE = "GPU"` -- default in the sample code.
- `DEVICE = "CPU"` -- change `"GPU"` to `"CPU"`.
- `DEVICE = "NPU"` -- change `"GPU"` to `"NPU"`; use `batch-size=1` and `nireq=4` for best NPU utilization.

#### Expected Output

![DLStreamer expected output](expected_output_dlstreamer.gif)

---

## License

Licensed under the MIT License. See [LICENSE](LICENSE) for details.

## References

- [face-detection-adas-0001](https://docs.openvino.ai/2024/omz_models_model_face_detection_adas_0001.html)
- [face-reidentification-retail-0095](https://docs.openvino.ai/2024/omz_models_model_face_reidentification_retail_0095.html)
- [Open Model Zoo](https://github.com/openvinotoolkit/open_model_zoo)
- [OpenVINO Documentation](https://docs.openvino.ai/)
- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html)