VehicleDINO

Unified multi-task vehicle recognition model β€” detection, type classification, make/model identification, re-identification, and license plate OCR in a single forward pass.

Architecture

  • Backbone: DINOv2 ViT-B/14 (frozen, with LoRA adapters)
  • Neck: SimpleFPN (768 -> 256) + HybridEncoder (AIFI + CCFM)
  • Decoder: RT-DETR-style with 300 detection queries + 1 global attribute query
  • Heads: 6 task-specific heads (det, type, make, model, Re-ID, OCR)

Model Variants

File Format Size Notes
vehicledino_dinov2.onnx FP32 450 MB Full precision
vehicledino_dinov2_int8.onnx INT8 139 MB Quantized, 3.2x smaller

Input / Output

Input: images β€” float32 tensor (1, 3, 560, 560), ImageNet-normalized RGB

Outputs:

Tensor Shape Description
det_boxes (1, 300, 4) Detection boxes (cx, cy, w, h normalized)
det_classes (1, 300, 5) Detection class logits (car, suv, truck, bus, van)
vehicle_types (1, 1, 8) Vehicle type logits
makes (1, 1, 164) Make classification logits; original CompCars IDs
models (1, 1, 2005) Model classification logits; original CompCars IDs
reid_embeds (1, 1, 256) L2-normalized Re-ID embedding
ocr_logits (1, 1, 8, 37) License plate OCR logits (8 positions, 37 chars)

Class-index mappings

The mappings below are verified for vehicledino_dinov2_int8.onnx, SHA-256 c8ad401d0fe1f319ab59a12ad0033ac932741823dc732ab461c73d614467870c.

The training loader preserves the original CompCars numeric IDs without compacting or subtracting one. Use labels[predicted_index] directly. Do not sort the arrays, remove Unknown entries, or shift the indices.

The make list has 163 names plus an Unknown placeholder at index 0. The model list has 1,716 names across IDs 1–2004, with 288 unnamed positions plus index 0 represented as Unknown. These placeholders are not a calibrated unknown-vehicle class. Label availability does not establish per-class training coverage or accuracy.

For example, make indices 39, 78, and 100 map to Toyota, Audi, and Honda. Dataset spellings are preserved, including BWM at index 81. Names were checked against CompCars misc/make_model_name.mat, with surrounding whitespace removed.

The earlier 42-make/323-model output sizes were incorrect for this export. Those counts must not be used to construct or reorder its label maps.

import json

with open("vehicledino_makes.json", encoding="utf-8") as f:
    make_labels = json.load(f)
with open("vehicledino_models.json", encoding="utf-8") as f:
    model_labels = json.load(f)

# session is an ONNX Runtime session; tensor is the preprocessed float32 input.
make_logits, model_logits = session.run(
    ["makes", "models"], {"images": tensor}
)
assert make_logits.shape[-1] == len(make_labels) == 164
assert model_logits.shape[-1] == len(model_labels) == 2005
make_index = int(make_logits[0, 0].argmax())
model_index = int(model_logits[0, 0].argmax())
print(make_index, make_labels[make_index])
print(model_index, model_labels[model_index])

CompCars reference: Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang, A Large-Scale Car Dataset for Fine-Grained Categorization and Verification, CVPR 2015. Dataset page.

Performance (Test Set)

Task Metric Score
Type Classification Top-1 Accuracy 95.6%
Make Classification Top-1 Accuracy 98.4%
Model Classification Top-1 Accuracy 87.7%
Re-ID (VeRi-776) mAP 61.1%
Re-ID (VeRi-776) Rank-1 86.1%

Training Data

  • Detection + Type + Re-ID: VeRi-776 (776 vehicles, 49,360 images)
  • Make/Model: CompCars, using the original numeric make and model IDs. Output dimensions are 164 and 2,005; these dimensions do not establish training coverage for every ID.
  • OCR: CCPD-Green (Chinese license plates)

Usage with ONNX Runtime (Python)

import onnxruntime as ort
import numpy as np
from PIL import Image

session = ort.InferenceSession("vehicledino_dinov2_int8.onnx")

# Preprocess: resize to 560x560, ImageNet normalize
img = Image.open("car.jpg").resize((560, 560))
arr = np.array(img).astype(np.float32) / 255.0
mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]
arr = (arr - mean) / std
tensor = arr.transpose(2, 0, 1)[np.newaxis]  # (1, 3, 560, 560)

outputs = session.run(None, {"images": tensor.astype(np.float32)})

Usage in Browser (ONNX Runtime Web)

The INT8 model runs in the browser via ONNX Runtime Web with WebGPU or WASM backend.

Live demo: https://yolov11-plate-recognition.swmengappdev.workers.dev

Citation

@article{vehicledino2026,
  title={VehicleDINO: Unified Multi-Task Vehicle Recognition via DINOv2 Features},
  author={Soh, Wei Meng},
  year={2026}
}

License

Apache 2.0

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support