Depth Estimation
david-hall-csiro commited on
Commit
6e7b09c
verified
1 Parent(s): 34015ed

Updated README

Browse files
Files changed (1) hide show
  1. README.md +50 -23
README.md CHANGED
@@ -8,9 +8,10 @@ pipeline_tag: depth-estimation
8
  <div align="center">
9
  <h1>WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments</h1>
10
 
11
- [**Joshua Knights**](https://scholar.google.com/citations?user=RxbGr2EAAAAJ&hl=en)<sup>1,2</sup> 路 **Joseph Reid**<sup>1</sup> 路 [**Mark Cox**](https://scholar.google.com/citations?user=Bk3UD4EAAAAJ&hl=en)<sup>1</sup>
12
- <br>
13
- [**Kaushik Roy**](https://bit0123.github.io/)<sup>1</sup> 路 [**David Hall**](https://scholar.google.com/citations?user=dosODoQAAAAJ&hl=en)<sup>1</sup> 路 [**Peyman Moghadam**](https://scholar.google.com.au/citations?user=QAVcuWUAAAAJ&hl=en)<sup>1,2</sup>
 
14
 
15
  <sup>1</sup>DATA61, CSIRO&emsp;&emsp;&emsp;<sup>2</sup>Queensland University of Technology
16
  <br>
@@ -20,30 +21,29 @@ pipeline_tag: depth-estimation
20
  <a href='https://doi.org/10.25919/5fmy-yg37'><img src='https://img.shields.io/badge/Dataset_Download-WildCross-blue'></a>
21
  </div>
22
 
23
- This repository contains the pre-trained checkpoints for a variety of tasks on the WildCross benchmark.
24
 
25
  ![teaser](./teaser.png)
26
 
27
- If you find this repository useful or use the WildCross dataset in your work, please cite us using the following:
28
- ```
29
- @inproceedings{wildcross2026,
30
- title={{WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments}},
31
- author={Joshua Knights, Joseph Reid, Kaushik Roy, David Hall, Mark Cox, Peyman Moghadam},
32
- booktitle={Proceedings-IEEE International Conference on Robotics and Automation},
33
- pages={},
34
- year={2026}
35
- }
36
- ```
37
 
38
- ## Download Instructions
 
 
 
39
  Our dataset can be downloaded through the [**CSIRO Data Access Portal**](https://doi.org/10.25919/5fmy-yg37). Detailed instructions for downloading the dataset can be found in the README file provided on the data access portal page.
40
 
41
 
42
  ## Training and Benchmarking
43
- Here we provide pre-trained checkpoints for a variety of tasks on WildCross.
44
 
45
- **Visual Place Recognition**
46
- ### Checkpoints
 
 
47
  | Model | Checkpoint Folder|
48
  |------------|------------|
49
  | NetVlad | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/NetVLAD) |
@@ -51,20 +51,47 @@ Here we provide pre-trained checkpoints for a variety of tasks on WildCross.
51
  | SALAD | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/SALAD) |
52
  | BoQ | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/BoQ) |
53
 
54
- **Cross Modal Place Recognition**
55
- ### Checkpoints
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56
  | Model | Checkpoint Folder|
57
  |------------|------------|
58
  | Lip-Loc (ResNet50) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/resnet50) |
59
  | Lip-Loc (Dino-v2) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/dinov2) |
60
  | Lip-Loc (Dino-v3) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/dinov3) |
61
 
62
- **Metric Depth Estimation**
63
- ### Checkpoints
 
 
 
64
  | Model | Checkpoint Folder|
65
  |------------|------------|
66
  | DepthAnythingV2-vits | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vits.pth) |
67
  | DepthAnythingV2-vitb | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vitb.pth) |
68
  | DepthAnythingV2-vitl | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vitl.pth) |
69
 
70
- For instructions on how to use these checkpoints for training or evaluation, further instructions can be found on the [WildCross GitHub repository](https://github.com/csiro-robotics/WildCross).
 
 
 
 
 
 
 
 
 
 
 
8
  <div align="center">
9
  <h1>WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments</h1>
10
 
11
+ [**Joshua Knights**](https://scholar.google.com/citations?user=RxbGr2EAAAAJ&hl=en)<sup>1,2</sup> 路 **Joseph Reid**<sup>1</sup> 路 [**Kaushik Roy**](https://bit0123.github.io/)<sup>1</sup>
12
+ <br>
13
+ [**David Hall**](https://scholar.google.com/citations?user=dosODoQAAAAJ&hl=en)<sup>1</sup> 路 [**Mark Cox**](https://scholar.google.com/citations?user=Bk3UD4EAAAAJ&hl=en)<sup>1</sup>
14
+ 路 [**Peyman Moghadam**](https://scholar.google.com.au/citations?user=QAVcuWUAAAAJ&hl=en)<sup>1,2</sup>
15
 
16
  <sup>1</sup>DATA61, CSIRO&emsp;&emsp;&emsp;<sup>2</sup>Queensland University of Technology
17
  <br>
 
21
  <a href='https://doi.org/10.25919/5fmy-yg37'><img src='https://img.shields.io/badge/Dataset_Download-WildCross-blue'></a>
22
  </div>
23
 
24
+ This repository contains the pre-trained checkpoints for a variety of tasks on the **WildCross benchmark**.
25
 
26
  ![teaser](./teaser.png)
27
 
28
+ ## WildCross Overview
29
+ We introduced **WildCross**, a large-scale benchmark for cross-modal place recognition and metric depth estimation in natural environments. The dataset comprises over 476K sequential RGB frames with semi-dense depth and surface normal annotations, each aligned with accurate 6DoF poses and synchronized dense lidar submaps.
30
+
31
+ We conduct comprehensive experiments on visual, lidar, and cross-modal place recognition, as well as metric depth estimation, demonstrating the value of WildCross as a challenging benchmark for multi-modal robotic perception tasks.
 
 
 
 
 
 
32
 
33
+ This HuggingFace Repository contains the model weights for replicating all experiments outlined in the [**original paper**](https://arxiv.org/pdf/2603.01475).
34
+
35
+
36
+ ## Data Download Instructions
37
  Our dataset can be downloaded through the [**CSIRO Data Access Portal**](https://doi.org/10.25919/5fmy-yg37). Detailed instructions for downloading the dataset can be found in the README file provided on the data access portal page.
38
 
39
 
40
  ## Training and Benchmarking
41
+ Here we provide pre-trained checkpoints for a variety of tasks on WildCross. For instructions on how to use all checkpoints for training or evaluation, further instructions can be found on the [WildCross GitHub repository](https://github.com/csiro-robotics/WildCross).
42
 
43
+ ### Visual Place Recognition
44
+ WildCross supports visual relocalization with sequential RGB imagery across challenging revisits, including reverse-direction traversals and long-term appearance changes. The benchmark includes cross-fold train/test splits for robust evaluation of generalization and in-domain adaptation.
45
+ For each model below we provide the weights for the original pre-trained model as well as models fine-tuned on our different data splits.
46
+ #### Checkpoints
47
  | Model | Checkpoint Folder|
48
  |------------|------------|
49
  | NetVlad | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/NetVLAD) |
 
51
  | SALAD | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/SALAD) |
52
  | BoQ | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/VPR/BoQ) |
53
 
54
+ ### LiDAR Place Recognition
55
+ WildCross is an extension of the original [Wild-Places](https://csiro-robotics.github.io/Wild-Places/) dataset for LiDAR place recognition. WildCross extends it's evaluation setup using new splits of the original data. For LiDAR place recognition (LPR), code for training and evaluation can be found on a WildCross branch of the original [Wild-Places repository](https://github.com/csiro-robotics/Wild-Places/tree/WildCross_splits).
56
+
57
+ For each model below we provide model weights which have been fine-tuned on our new data splits.
58
+ #### Checkpoints
59
+ | Model | Checkpoint Folder|
60
+ |------------|------------|
61
+ | LoGG3D-Net | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/LPR/LoGG3DNet) |
62
+ | MinkLoc3Dv2 | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/LPR/MinkLoc3Dv2) |
63
+ | HOTFormerLoc | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/LPR/HotFormerLoc) |
64
+
65
+ ### Cross Modal Place Recognition
66
+ CMPR in WildCross evaluates retrieval across sensing modalities, such as image-to-lidar localization. The synchronized RGB frames, accurate poses, and dense lidar submaps provide a strong testbed for cross-modal representation learning.
67
+
68
+ Checkpoints below provide Lip-Loc CMPR model weights using different backbones, fine-tuned on our different data splits.
69
+ #### Checkpoints
70
  | Model | Checkpoint Folder|
71
  |------------|------------|
72
  | Lip-Loc (ResNet50) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/resnet50) |
73
  | Lip-Loc (Dino-v2) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/dinov2) |
74
  | Lip-Loc (Dino-v3) | [Link](https://huggingface.co/CSIRORobotics/WildCross/tree/main/crossmodal/dinov3) |
75
 
76
+ ### Metric Depth Estimation
77
+ WildCross provides semi-dense metric depth and surface normal annotations for every frame, generated from accumulated global point clouds, accurate camera poses, and visibility filtering to remove occluded points. This supports training and benchmarking depth models in natural environments where current methods face substantial domain-shift challenges.
78
+
79
+ Checkpoints below provide model weights for different DepthAnythingv2 models fine-tuned on WildCross data.
80
+ #### Checkpoints
81
  | Model | Checkpoint Folder|
82
  |------------|------------|
83
  | DepthAnythingV2-vits | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vits.pth) |
84
  | DepthAnythingV2-vitb | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vitb.pth) |
85
  | DepthAnythingV2-vitl | [Link](https://huggingface.co/CSIRORobotics/WildCross/resolve/main/DepthAnythingV2/finetuned/vitl.pth) |
86
 
87
+ ## BibTeX
88
+ If you find this repository useful or use the WildCross dataset in your work, please cite us using the following:
89
+ ```
90
+ @inproceedings{wildcross2026,
91
+ title={{WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments}},
92
+ author={Joshua Knights, Joseph Reid, Kaushik Roy, David Hall, Mark Cox, Peyman Moghadam},
93
+ booktitle={Proceedings-IEEE International Conference on Robotics and Automation},
94
+ pages={},
95
+ year={2026}
96
+ }
97
+ ```