README.md

---
license: apache-2.0
pipeline_tag: depth-estimation
library_name: dinov3-depth-head
tags:
- depth-estimation
- monocular-depth-estimation
- dinov3
- anydepth
datasets:
- blanchon/dinodepth-dataset
---

# DinoDepth — SDT depth heads on a frozen DINOv3 backbone

Trained **Simple Depth Transformer (SDT)** decoder heads for zero-shot **affine-invariant
(relative) monocular depth**, reproducing **AnyDepth** (arXiv:2601.02760) on a **frozen DINOv3**
backbone. Only the small SDT decoder is trained; the DINOv3 encoder is frozen and loaded separately
from Meta's checkpoints. Two heads are provided in this repo:

| File | Backbone | Decoder params | Train |
|---|---|---|---|
| `sdt-vitl16.safetensors` | DINOv3 ViT-L/16 | 13.4 M | 5 epochs |
| `sdt-vits16.safetensors` | DINOv3 ViT-S/16 | 5.5 M | 10 epochs |

These are **decoder weights only** (~13/5 M params) — pair each with its matching frozen DINOv3
backbone (`facebook/dinov3-vitl16-pretrain-lvd1689m` / `-vits16-`).

## Zero-shot benchmark (our protocol)

AbsRel ↓ / δ1 ↑ on NYUv2 (Eigen 654) and KITTI (Eigen 652), scored with per-image least-squares
scale+shift alignment in disparity space, Eigen/Garg crop, 10 m / 80 m cap.

| Model | NYU AbsRel | NYU δ1 | KITTI AbsRel | KITTI δ1 |
|---|---|---|---|---|
| ViT-L/16 + SDT (this repo) | 0.068 | 0.955 | 0.093 | 0.911 |
| ViT-S/16 + SDT (this repo) | 0.091 | 0.917 | 0.115 | 0.852 |
| AnyDepth ViT-L (paper) | 0.060 | — | 0.086 | — |
| AnyDepth ViT-S (paper) | 0.082 | — | 0.102 | — |

A faithful reproduction — ~0.01 AbsRel behind the paper on each benchmark (consistent across both
backbones; plausibly the augmentation/data-filtering details AnyDepth underspecifies).

## Usage

```python
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from dinov3_depth.head import DepthModel, DepthModelConfig

# Frozen DINOv3 ViT-L/16 + (randomly-initialised) SDT head; default config matches the trained head
# (GroupNorm, fusion_channels=256).
model = DepthModel.from_pretrained(DepthModelConfig(backbone="vitl16"))
head = hf_hub_download("blanchon/dinodepth-model", "sdt-vitl16.safetensors")
model.head.load_state_dict(load_file(head))
model.eval()

# images: float [B, 3, H, W] in [0, 1], H and W multiples of 16. Returns affine-invariant disparity.
disparity = model(images)
```

(Use `backbone="vits16"` + `sdt-vits16.safetensors` for the small head.)

## Architecture & training

- **Encoder:** frozen DINOv3 ViT (LVD-1689M), patch 16; 4 intermediate layers tapped.
- **Decoder (SDT):** softmax-fuse the 4 tapped layers at the patch grid → depthwise detail enhancer →
  two learned DySample ×4 upsamplers → output conv. GroupNorm, fusion width 256.
- **Loss:** scale-and-shift-invariant + multi-scale gradient matching (1:2), on disparity.
- **Data:** the harmonized 369K-image corpus at `blanchon/dinodepth-dataset` (Hypersim, VKITTI2,
  BlendedMVS, IRS, TartanAir). 768² input, AdamW lr 1e-3, PolyLR.

## References
- AnyDepth (arXiv:2601.02760) · DINOv3 (arXiv:2508.10104)
- Dataset: [blanchon/dinodepth-dataset](https://huggingface.co/datasets/blanchon/dinodepth-dataset)