kiselyovd/vehicle-keypoints

Vehicle Keypoints

A 14-point vehicle-keypoint detector trained on real CarFusion images. Main weights (weights.pt) are YOLO26-pose; the baseline/ directory contains a ViTPose-S reference checkpoint.

Code & evaluation · Documentation · Synthetic-data study

This is an output of the companion geometric PnP baseline, not the output of weights.pt alone. The 3D evaluation protocol uses ApolloCar3D and has separate coverage and domain-shift limits.

Use the model

from urllib.request import urlopen
from PIL import Image
from huggingface_hub import hf_hub_download
from ultralytics import YOLO

checkpoint = hf_hub_download("kiselyovd/vehicle-keypoints", "weights.pt")
model = YOLO(checkpoint)
image = Image.open(urlopen("https://raw.githubusercontent.com/kiselyovd/vehicle-keypoints/f057d0d9f17720a876dd31dece585d08c2e58e02/data/sample/images/car_butler1__16_04881.jpg")).convert("RGB")
results = model.predict(image)
for result in results:
    for box, keypoints, score in zip(result.boxes.xywh, result.keypoints.data, result.boxes.conf):
        print(box.tolist(), score.item(), len(keypoints))

The bundled example is a real CarFusion road image. Each detection has a box, confidence score, and 14 (x, y, visibility) keypoints. result.boxes.xywh uses center coordinates plus width and height.

Reported evaluation

CarFusion test split: 12,761 images, 42,706 predictions. OKS uses uniform sigmas 0.05; PCK uses a threshold of 0.05 of the bounding-box diagonal.

Model	OKS-mAP	OKS-mAP@50	PCK@0.05	Training note
YOLO26-pose	50.4%	70.4%	76.1%	30 epochs; corrected left/right flip labels
ViTPose-S reference	0.1%	0.4%	13.7%	15 epochs and val=train; under-trained

The reference run does not establish a fair comparison of architecture capacity. Values match the stored evaluation summary; model-index preserves the unrounded main-model results.

Synthetic-data companion

The synthetic-only 24-point model is a separate artifact trained on the UE5 dataset.

In the published matched experiments, adding synthetic frames reduced real-data OKS-mAP by 1.7 pp from a weak checkpoint and 9.5 pp from a strong one. The +2 pp acceptance criterion was not met for that recipe. See the protocol and error analysis.

Visual evidence

The second image contains training and validation curves. It is not a grid of predicted keypoints.

Data and limitations

CarFusion is a limited traffic-scene dataset with scene-level partitions. Its canonical schema covers wheels, lights, roof corners, exhaust, and a body-centre reference point. See dataset preparation and baseline details.

Research and education use. These artifacts have not been validated for safety-critical autonomous-driving use. The 3D companion has limited detector coverage and substantial domain shift.

Licenses and sources

The Hub metadata declares MIT for this published artifact; the project's code license is MIT. CarFusion is © Carnegie Mellon University and has its own dataset terms. Ultralytics and other dependencies have separate licenses.

• CarFusion paper and dataset
• Training recipes
• Serving and API