SemiSupMDE: Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation

Models for the paper Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation (CoLLAs 2024) by Amir El-Ghoussani, Julia Hornauer, Gustavo Carneiro and Vasileios Belagiannis. Code: github.com/AmirMaEl/SemiSupMDE.

A single U-Net (54.6M parameters) predicts depth from one RGB image. It is trained on synthetic Virtual KITTI with ground truth and adapted to real KITTI without any KITTI labels, using a pairwise loss on source images and a perturbation-consistency loss across RandAugment views of the unlabelled target images.

Files

File	Description
pretrained_vKITTI.pth	pretrained on Virtual KITTI 1.3.1 only (with CutMix, colour jitter and rotations); starting point of the adaptation
finetuned_KITTI.pth	after domain adaptation vKITTI → KITTI; the model reported in the paper (Tables 1 and 2)
model.py	the U-Net definition, to load the checkpoints without the full repository
SHA256SUMS	checksums of the checkpoints

Both checkpoints are PyTorch state dicts.

Results

KITTI Eigen test split (697 images), Garg crop, velodyne ground truth, no median scaling:

Model	Cap	AbsRel	SqRel	RMSE	RMSE log	δ<1.25	δ<1.25²	δ<1.25³
pretrained_vKITTI.pth	50 m	0.170	1.372	4.991	0.262	0.769	0.907	0.957
pretrained_vKITTI.pth	80 m	0.180	1.868	6.578	0.276	0.755	0.898	0.952
finetuned_KITTI.pth	50 m	0.161	1.029	4.449	0.237	0.785	0.918	0.968
finetuned_KITTI.pth	80 m	0.168	1.347	6.002	0.254	0.769	0.907	0.961

Usage

With the code repository installed as a package (it downloads the checkpoints from here on first use):

pip install git+https://github.com/AmirMaEl/SemiSupMDE.git

import semisupmde

model = semisupmde.load_model("finetuned")              # or "pretrained"
depth = semisupmde.predict_depth(model, "image.png")    # metres, at the resolution of the input

or from the command line: semisupmde predict image.png (writes a colourised depth map and the depth in metres as .npy). semisupmde eval reproduces the table above.

Without the package, only with this repository:

import importlib.util
import torch
import torchvision.transforms as T
from huggingface_hub import hf_hub_download
from PIL import Image

repo = "AmirMaEl/SemiSupMDE"
spec = importlib.util.spec_from_file_location("model", hf_hub_download(repo, "model.py"))
model_lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(model_lib)
model = model_lib.load_model(hf_hub_download(repo, "finetuned_KITTI.pth"), device="cuda")

img = Image.open("kitti_image.png").convert("RGB").resize((640, 192), Image.BICUBIC)
x = T.ToTensor()(img).unsqueeze(0).cuda() * 2 - 1           # input in [-1, 1]
with torch.no_grad():
    pred = model(x)[-1]                                     # finest output, in [-1, 1]
depth = (pred + 1) / 2 * model_lib.MAX_DEPTH                # metres, 192 x 640

The network expects 640×192 inputs; resize the prediction to the original image resolution for evaluation.

Training

• Pretraining: Virtual KITTI 1.3.1, depth clipped to 80 m, with CutMix (α = 0.5), colour jitter and random rotations.
• Adaptation: joint optimisation of the pairwise source loss on 12 vKITTI pairs and the consistency loss between 6 unlabelled KITTI images (right camera) and two RandAugment views of each (FixMatch pool, n = 1, m = 7, plus CutOut) per step. Adam, constant learning rate 4e-8, gradients accumulated over all steps, 941 steps per epoch. The released model is from epoch 8. Reproduce with ./run.sh finetune in the code repository.

Citation

@InProceedings{pmlr-v274-el-ghoussani25a,
  title = 	 {Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation},
  author =       {El-Ghoussani, Amir and Hornauer, Julia and Carneiro, Gustavo and Belagiannis, Vasileios},
  booktitle = 	 {Proceedings of The 3rd Conference on Lifelong Learning Agents},
  pages = 	 {577--596},
  year = 	 {2025},
  editor = 	 {Lomonaco, Vincenzo and Melacci, Stefano and Tuytelaars, Tinne and Chandar, Sarath and Pascanu, Razvan},
  volume = 	 {274},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {29 Jul--01 Aug},
  publisher =    {PMLR},
  pdf = 	 {https://raw.githubusercontent.com/mlresearch/v274/main/assets/el-ghoussani25a/el-ghoussani25a.pdf},
  url = 	 {https://proceedings.mlr.press/v274/el-ghoussani25a.html},
  abstract = 	 {In monocular depth estimation, unsupervised domain adaptation has recently been explored to relax the dependence on large annotated image-based depth datasets. However, this comes at the cost of training multiple models or requiring complex training protocols. We formulate unsupervised domain adaptation for monocular depth estimation as a consistency-based semi-supervised learning problem by assuming access only to the source domain ground truth labels. To this end, we introduce a pairwise loss function that regularises predictions on the source domain while enforcing perturbation consistency across multiple augmented views of the unlabelled target samples. Importantly, our approach is simple and effective, requiring only training of a single model in contrast to the prior work. In our experiments, we rely on the standard depth estimation benchmarks KITTI and NYUv2 to demonstrate state-of-the-art results compared to related approaches. Furthermore, we analyse the simplicity and effectiveness of our approach in a series of ablation studies. The code is available at \url{https://github.com/AmirMaEl/SemiSupMDE}.}
}