README.md

---
license: apache-2.0
tags:
 - robotics
 - autonomous-driving
 - computer-vision
 - end-to-end
---

# Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?

[**Paper**](https://arxiv.org/abs/2610.09695v1) | [**Project Page**](https://opendrivelab.com/ViRA/) | [**GitHub Repository**](https://github.com/OpenDriveLab/ViRA)

Visual foundation models (VFMs) are increasingly integrated into end-to-end autonomous driving, yet it remains unclear when these representations improve driving performance. **ViRA** is a planner-agnostic visual representation alignment framework: during training it aligns planner features with a frozen VFM, while the deployed planner architecture and inference cost stay unchanged.

![](https://ik.imagekit.io/zizizihao/ViRA_Diffusion/teaser.jpg?updatedAt=1791281459035)

## Findings

The study is organized around three empirical findings:

1. **Consistent driving gains:** VFM-guided visual representations improve driving across regression, diffusion, and scoring planners. The gains extend to zero-shot closed-loop evaluation.
2. **Target selection matters:** Different VFM targets yield different planning gains for the same planner. Alignment to another VFM can further benefit planners that already use a pre-trained VFM encoder.
3. **Supervision changes sensitivity:** Auxiliary perception supervision narrows the EPDMS spread across five VFM targets from 2.7 to 0.5 points, and can compensate for a less effective target.

## Performance

Guided by these findings, **ViRA-Diffusion** aligns a diffusion planner to DINOv3 and is trained without auxiliary perception supervision. It reaches **92.3 EPDMS** on NAVSIM v2 navtest, exceeding the recent methods in our comparison by at least 1.9 points.

The same alignment improves three camera-only planners. On NAVSIM v2 navtest, EPDMS rises by **+5.4** for TransFuser (83.6 → 89.0), **+7.4** for DiffusionDrive (84.5 → 91.9), and **+10.9** for Rap\* (72.8 → 83.7). On the challenging navhard split, Rap\*-ViRA gains **+16.4** EPDMS. Checkpoints trained only on NAVSIM also improve zero-shot closed-loop HD-Score on HUGSIM.

| Method | Decoder | NAVSIM v2 navtest EPDMS | NAVSIM v2 navhard EPDMS | HUGSIM HD-Score |
| --- | --- | --- | --- | --- |
| TransFuser | Regression | 83.6 | 27.6 | 22.1 |
| TransFuser-ViRA | Regression | **89.0** (+5.4) | **31.7** (+4.1) | **24.8** (+2.7) |
| DiffusionDrive | Diffusion | 84.5 | 30.5 | 22.2 |
| DiffusionDrive-ViRA | Diffusion | **91.9** (+7.4) | **35.8** (+5.3) | **28.6** (+6.4) |
| Rap\* | Scoring | 72.8 | 32.8 | 7.3 |
| Rap\*-ViRA | Scoring | **83.7** (+10.9) | **49.2** (+16.4) | **10.3** (+3.0) |

| Method | NAVSIM v2 navtest EPDMS |
| --- | --- |
| Epona | 85.1 |
| DiffusionDriveV2 | 87.5 |
| Latent-WAM | 89.3 |
| DriveFuture | 89.9 |
| SparseDriveV2 | 90.1 |
| Discrete-WAM | 90.4 |
| **ViRA-Diffusion** | **92.3** |

Camera-only. Rap\* is our reimplementation with a different backbone and without the original data augmentation. HUGSIM is zero-shot: planners are trained only on NAVSIM. ViRA-Diffusion is DiffusionDrive trained with DINOv3 alignment and without auxiliary perception supervision. DiffusionDrive-ViRA (91.9) and ViRA-Diffusion (92.3) are distinct configurations.

## License

All content in this repository is licensed under the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) license.


## HUGSIM Closed-loop Visualization

![](https://ik.imagekit.io/zizizihao/ViRA_Diffusion/p_027_with_trajectory.gif)

**(a) Car following: the lead vehicle brakes suddenly**

![](https://ik.imagekit.io/zizizihao/ViRA_Diffusion/w_200_hard.gif)

**(b) Oncoming traffic: other vehicles enter the current driving area**

## Citation

If you find ViRA useful in your research, please consider citing:

```bibtex
@article{zhang2026vira,
  title={Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?},
  author={Zhang, Zihao and Tian, Haochen and Li, Tianyu and Jing, Changhui and He, Jingliang and Ye, Naisheng and Pu, Ziyuan and Yang, Zhenjie},
  journal={arXiv preprint arXiv:2610.09695},
  year={2026},
  url={https://arxiv.org/abs/2610.09695}
}
```