Traffic-JEPA — checkpoints

Checkpoints for Traffic-JEPA, the 1st place solution to the 10th AI City Challenge (2026), Track 2: Transportation Safety Understanding and Captioning (team Latent Painter – UTE, S2 60.0853, VQA accuracy 87.09 %).

Code: https://github.com/ThuongBuiRVC/Traffic-JEPA

Files

File	Stage	What
model_best.pt	VQA	the trained predictor head (5.83 M params) on top of frozen V-JEPA 2.1, Llama-3.2-1B and EmbeddingGemma
caption_lora/	Captioning, default	LoRA (rank 16, alpha 32) for Qwen/Qwen3-VL-8B-Instruct, trained text-only on SynWTS: facts -> caption. This is the configuration the paper reports.
caption_lora_mm/	Captioning, variant	the same LoRA shape trained multimodally — SynWTS facts and simulation frames -> caption. Used by the grounded variant, which also shows real frames at inference.

Usage

git clone https://github.com/ThuongBuiRVC/Traffic-JEPA
cd Traffic-JEPA
hf download ThuongBuiRVC/Traffic-JEPA --local-dir checkpoints/

bash scripts/05_submit_test.sh     # VQA     -> submissions/submission_final.json
bash scripts/06_caption.sh         # caption -> submissions/caption_submission.json
bash scripts/06_caption.sh mm      # grounded variant -> submissions/caption_submission_mm.json

The VQA model is rebuilt from run_args.json, which scripts/05_submit_test.sh writes next to model_best.pt from configs/train_args.json. A mismatched config changes the scores without raising anything, so do not swap it by hand.

Training data

The VQA predictor is trained on SynWTS simulation questions; the decoder tables are estimated from simulation counts. The caption LoRA is trained on SynWTS captions only. Neither uses test ground truth.

Citation

@inproceedings{bui2026trafficjepa,
  author    = {Nguyen Hoai Thuong Bui and Thanh Nguyen Vo and Trinh Tra Giang Nguyen and Ha Duc Bui},
  title     = {Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA},
  booktitle = {AI City Challenge Workshop},
  year      = {2026}
}