README.md

---
license: apache-2.0
language:
- en
metrics:
- accuracy
base_model:
- Qwen/Qwen3-VL-4B-Instruct
library_name: transformers
tags:
- vision-language-model
- depth-estimation
- 3d-vision
- multimodal
pipeline_tag: depth-estimation
---

<p align="center">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/652d3023cdb2a91205709b6a/G_n7TbVjIv8PYUjq0kxPP.png" width="160" />
</p>
<h2 align="center">
Unlocking Dense Metric Depth Estimation in VLMs
</h2>

<h4 align="center">
  <b>Project Page:</b> <a href="https://depthvlm.github.io/">depthvlm.github.io</a> | 
  <b>GitHub:</b> <a href="https://github.com/hanxunyu/DepthVLM">hanxunyu/DepthVLM</a> | 
  <b>arXiv:</b> <a href="https://arxiv.org/abs/2605.15876">2605.15876</a>
  <br><br>
  <a href="https://depthvlm.github.io/"><img src="https://img.shields.io/badge/Project-Home Page-green?logo=safari&logoColor=white" alt="Project Home Page"></a>
  <a href="https://github.com/hanxunyu/DepthVLM"><img src="https://img.shields.io/badge/GitHub-Repository-blue?logo=github" alt="GitHub Badge"></a>
  <a href="https://huggingface.co/datasets/JonnyYu828/DepthVLM-Bench"><img src="https://img.shields.io/badge/HuggingFace-Benchmark-yellow?logo=huggingface" alt="Hugging Face Benchmark"></a>
  <a href="https://arxiv.org/abs/2605.15876"><img src="https://img.shields.io/badge/arXiv-2605.15876-b31b1b.svg?logo=arxiv&logoColor=red" alt="arXiv"></a>
</h4>


---

## 📰 News

* **2026.05** — Released [DepthVLM-Bench](https://huggingface.co/datasets/JonnyYu828/DepthVLM-Bench).
* **2026.05** — Released [DepthVLM-4B](https://huggingface.co/JonnyYu828/DepthVLM-4B).

---

## 🌟 Model Overview

DepthVLM serves as **a unified foundation model for both low-level dense geometry prediction and high-level multimodal understanding**, while achieving substantially faster inference compared with existing VLM-based approaches such as DepthLM and Youtu-VL.

By attaching a lightweight depth head to the LLM backbone and adopting a two-stage supervision paradigm, DepthVLM transforms a single VLM into a native dense geometry predictor, while preserving its multimodal capabilities and enhancing its spatial reasoning.

## 🧠 Key Characteristics

- **Native Dense Metric Depth Estimation in VLMs**: Directly predicts geometry within the VLM framework.

- **Unified Multimodal Understanding and Geometry Prediction**: Generates full-resolution depth maps alongside language outputs in a single forward pass.

- **Efficient Inference**: Achieves higher efficiency compared to per-pixel query or coarse token-level outputs.  

- **Versatile Application**: Supports both indoor and outdoor metric depth estimation.

- **Improved 3D Spatial Reasoning**: Moving toward a truly unified foundation model.

---

## 🚀 Main Results

### Comparison with VLMs
| Benchmark  |      Ours-8B |      Ours-4B | DepthLM-12B | **Youtu-VL-4B** |
| ---------- | -----------: | -----------: | ----------: | --------------: |
| Argoverse2 | <u>0.798</u> |    **0.810** |       0.761 |           0.663 |
| Waymo      | <u>0.865</u> |    **0.879** |       0.588 |           0.473 |
| DDAD       | <u>0.813</u> |    **0.818** |       0.654 |           0.342 |
| NuScenes   |    **0.831** | <u>0.821</u> |       0.736 |           0.698 |
| ETH3D      |    **0.928** | <u>0.924</u> |       0.666 |           0.286 |
| ScanNet++  |    **0.901** | <u>0.861</u> |       0.756 |           0.522 |
| SUN RGB-D  |    **0.889** | <u>0.882</u> |       0.785 |           0.734 |
| IBims-1    |    **0.936** | <u>0.912</u> |       0.754 |           0.856 |
| NYUv2      |    **0.920** | <u>0.908</u> |       0.866 |           0.849 |
> **Bold** and <u>underlined</u> indicate the best and second-best results among the compared models.
> More details can be found in our paper.


## Citation

If you find DepthVLM useful for your research or applications, please consider citing our work using the following BibTeX:

```bibtex
@article{yu2026unlocking,
    title={Unlocking Dense Metric Depth Estimation in VLMs},
    author={Hanxun Yu and Xuan Qu and Yuxin Wang and Jianke Zhu and Lei Ke},
    journal={arXiv preprint arXiv:2605.15876},
    year={2026}
}
```