| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - 3D |
| - Depth |
| - Pointmap |
| - Self-driving |
| --- |
| |
| ## 🔗 Links |
| - 📄 **Paper:** [arXiv](https://www.arxiv.org/abs/2602.05573) |
| - **Code**: [github](https://github.com/whesense/ViGT/tree/main) |
| - 🌐 **Project Page:** [whesense.github.io](https://whesense.github.io) |
|
|
|
|
| ## Model Description |
| We introduce the Visual Implicit Geometry Transformer (ViGT), an autonomous driving geometric model that estimates continuous 3D occupancy fields from surround-view camera rigs. ViGT represents a step towards foundational geometric models for autonomous driving, prioritizing scalability, architectural simplicity, and generalization across diverse sensor configurations. Our approach achieves this through a calibration-free architecture, enabling a single model to adapt to different sensor setups. Unlike general-purpose geometric foundational models that focus on pixel-aligned predictions, ViGT estimates a continuous 3D occupancy field in a bird's-eye-view (BEV) addressing domain-specific requirements. ViGT naturally infers geometry from multiple camera views into a single metric coordinate frame, providing a common representation for multiple geometric tasks. Unlike most existing occupancy models, we adopt a self-supervised training procedure that leverages synchronized image-LiDAR pairs, eliminating the need for costly manual annotations. We validate the scalability and generalizability of our approach by training our model on a mixture of five large-scale autonomous driving datasets (NuScenes, Waymo, NuPlan, ONCE, and Argoverse) and achieving state-of-the-art performance on the pointmap estimation task, with the best average rank across all evaluated baselines. We further evaluate ViGT on the Occ3D-nuScenes benchmark, where ViGT achieves comparable performance with supervised methods. |
|  |
|
|
| ## Visual Examples |
|
|
| ### Front-view |
|  |
|  |
|
|
| ### Bird's-eye view |
| <img src="./images/bev1.jpg" width="960" /> |
|
|
|
|
| Check out the project page for even more — you’ll find interactive visualizations there. |
|
|
