DeepSORT: Optimized for AMD ROCm

DeepSORT is a multi-object tracking pipeline that assigns a stable identity to every person across a video sequence. This repository packages evaluation/inference for multi-object tracking using ONNX Runtime, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.

This is based on the implementation of DeepSORT found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the DeepSORT AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Multi-object tracking (appearance Re-ID embedding + DeepSORT association)

Dataset: MOT16 train split (7 sequences, pedestrian tracking with public ground truth)

Output metrics: MOTA, MOTP, IDF1, IDP, IDR, Recall, Precision, IDSW, FP, FN, MT, ML, FM

VitisAI EP note: On the current Ryzen AI 1.7.0 / VitisAI stack, the NPU path does not execute on the AIE array — it silently falls back to CPU. The mars-small128 encoder compiles through the VitisAI frontend, but the VAIML backend fails L2 Placement Verification on the convolution layers, so ONNX Runtime falls back to CPUExecutionProvider. NPU accuracy numbers therefore equal the CPU numbers (verified bit-identical, max Δ = 0.0).

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:

• Validated backends: CPU (ONNX Runtime), GPU (MIGraphX execution provider), and NPU (VitisAI execution provider — currently runs via CPU fallback).
• The Re-ID encoder is exported once to a device-agnostic FP32 ONNX artifact; only detection-feature generation runs on the target execution provider, while the DeepSORT tracker, accuracy scoring, and overlay rendering are device-independent CPU post-processing.

Runtime	Precision	Backend	Hardware	Notes
ONNX Runtime	FP32 / FP16 / BF16 / INT8	CPU (ONNX Runtime autoquant)	AMD CPU	—
ONNX Runtime	FP32 / FP16 / BF16 / INT8	MIGraphX	AMD Instinct™ / Radeon™ GPU	—
ONNX Runtime	FP32	VitisAI EP	AMD NPU	Currently CPU fallback — AIE placement fails to compile

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the DeepSORT on GitHub.

---

Model Details

Model Type: Appearance Re-ID embedding encoder (CNN) + DeepSORT tracker

Base Model: mars-small128 (nwojke/deep_sort Re-ID encoder)

Model Stats:

• Input: cropped person patch, shape (1, 128, 64, 3), float32
• Output: 128-dimensional appearance embedding, shape (1, 128), float32
• Precision tested: FP32, FP16, BF16, INT8 (CPU/GPU); FP32 only (NPU, CPU fallback)

---

Accuracy Pipeline

MOTA and IDF1 are the two headline scores. Higher MOTA means fewer total tracking errors (false positives + misses + ID switches) relative to the number of ground-truth objects; higher IDF1 means each target keeps the same ID more consistently over its lifetime. MOTA can be negative when errors exceed the number of ground-truth boxes. On MOT16 with public detections, MOTA in the 20–40% range and IDF1 in the 40–55% range are typical for a DeepSORT-class tracker.

Metrics Explained

Metric	Description
MOTA	Multi-Object Tracking Accuracy — 1 − (FP + FN + IDSW) / GT_boxes. The single most holistic score, combining all three error types; higher is better.
IDF1	Identity F1 — the harmonic mean of identity precision and recall over the whole trajectory. Rewards keeping one consistent ID per target across the full sequence.
MOTP	Multi-Object Tracking Precision — the mean IoU-distance (1 − IoU) of matched detection/ground-truth pairs. Lower is better here (0 = perfect overlap).
IDP / IDR	Identity Precision / Recall — the precision and recall halves of IDF1. IDP falling faster than IDR points to fragmented/duplicated IDs; IDR falling faster points to identities that are dropped and never recovered.
Recall / Precision	Detection-level recall and precision (before identity is considered). Low Recall means missed objects (drives FN); low Precision means spurious boxes (drives FP).
IDSW	Number of identity switches — how often a tracked target is assigned a new ID mid-trajectory. Lower is better.
FP / FN	Total false positives and false negatives / misses, summed over all frames. Lower is better.
MT / ML	Mostly-Tracked and Mostly-Lost trajectories — targets covered for ≥ 80% (MT, higher is better) and ≤ 20% (ML, lower is better) of their lifespan.
FM	Fragmentations — how many times a trajectory is interrupted (track lost then resumed). Lower is better.

Accuracy Results

Full Dataset Evaluation (MOT16 train split, all 7 sequences, mars-small128 encoder) — filled from runs/mot_metrics_onnx_<device>.json; run make metrics to refresh:

Device	MOTA	IDF1	IDP	IDR	Rcll	Prcn	MOTP
CPU	28.90%	37.90%	75.33%	25.32%	31.36%	93.32%	0.215
GPU	28.90%	37.90%	75.33%	25.32%	31.36%	93.32%	0.215
NPU (CPU-fallback)	28.90%	37.90%	75.33%	25.32%	31.36%	93.32%	0.215

Note: the NPU numbers are CPU-fallback results (the AIE partition fails to compile), which is why they equal the CPU row exactly. All three devices produce identical tracking metrics because the encoder's appearance embeddings come out bit-identical across them (max Δ = 0.0).

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for the supported runners
• Additional model variants and datasets
• Benchmarking and reproduction instructions