Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- pose-estimation
|
| 5 |
+
- relative-pose-estimation
|
| 6 |
+
- multi-view
|
| 7 |
+
- computer-vision
|
| 8 |
+
- robotics
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# G2G: Exploiting Intra-Group Geometry for Inter-Group Pose Estimation
|
| 12 |
+
|
| 13 |
+
<p align="center">
|
| 14 |
+
<a href="https://weiyufei0217.github.io/G2G/"><b>๐ Project Page</b></a>
|
| 15 |
+
|
|
| 16 |
+
<a href="https://arxiv.org/abs/2606.08284"><b>๐ arXiv</b></a>
|
| 17 |
+
|
|
| 18 |
+
<a href="https://github.com/WeiYuFei0217/G2G"><b>๐ป Code (GitHub)</b></a>
|
| 19 |
+
</p>
|
| 20 |
+
|
| 21 |
+
> Recovering the relative 6-DoF pose between two image groups underlies cross-sequence relocalization, multi-camera rig odometry, and other multi-view tasks. Each group carries **known intra-group geometry** from a pre-built map, odometry, or rig calibration, and pretrained multi-view backbones already fuse such geometry into visual features. Yet current models treat all views as an unstructured set, leaving **cross-group reasoning** as the missing piece.
|
| 22 |
+
>
|
| 23 |
+
> G2G keeps a multi-view foundation model entirely frozen and adds three lightweight trainable modules (32M parameters, under 6% of the full model) to bridge the two groups: a perceiver resampler, a cross-group bridge with merged self-attention, and a multi-frame pose head. Supervised only by relative poses, G2G attains state-of-the-art accuracy on both tasks across four datasets.
|
| 24 |
+
|
| 25 |
+
This repository hosts the **released artifacts** for the paper. **Code, installation, and full usage live on GitHub:** https://github.com/WeiYuFei0217/G2G
|
| 26 |
+
|
| 27 |
+
## Contents
|
| 28 |
+
|
| 29 |
+
| Path | Description |
|
| 30 |
+
|------|-------------|
|
| 31 |
+
| `release_weights/*.pth` | 10 pretrained **G2G-only** weights (frozen MapAnything backbone excluded; each ~123 MB) |
|
| 32 |
+
| `examples.zip` | Sanity-check input bundles (`reloc/` + `rig/`); unzip into the repo root โ `examples/` |
|
| 33 |
+
| `eval_results.zip` | Paper-subset per-pair evaluation CSVs; unzip into the repo root โ `eval_results/` |
|
| 34 |
+
|
| 35 |
+
> The frozen MapAnything backbone (DINOv2-large / 1024-dim, MapAnything **v1.0.1**, commit `fde8425`) is **not** redistributed here. See the GitHub README, Installation step 2, for how to obtain the exact compatible backbone. The current `facebook/map-anything` Hugging Face weights are the newer giant/1536 variant and are **incompatible** with these G2G modules.
|
| 36 |
+
|
| 37 |
+
## Pretrained weights
|
| 38 |
+
|
| 39 |
+
| Weight | Task | Dataset |
|
| 40 |
+
|--------|------|---------|
|
| 41 |
+
| `HM3D-Reloc.pth` | Reloc | HM3D |
|
| 42 |
+
| `TartanGround-Reloc.pth` | Reloc | TartanGround |
|
| 43 |
+
| `NCLT-Reloc.pth` | Reloc | NCLT |
|
| 44 |
+
| `ZJH-Reloc.pth` | Reloc | ZJH |
|
| 45 |
+
| `HM3D-Rig-8.pth` | Rig | HM3D (8-cam) |
|
| 46 |
+
| `HM3D-Rig-4.pth` | Rig | HM3D (4-cam) |
|
| 47 |
+
| `TartanGround-Rig-4.pth` | Rig | TartanGround (4-cam) |
|
| 48 |
+
| `NCLT-Rig-Intra.pth` | Rig | NCLT intra-season (5-cam) |
|
| 49 |
+
| `NCLT-Rig-Cross.pth` | Rig | NCLT cross-season (5-cam) |
|
| 50 |
+
| `ZJH-Rig-4.pth` | Rig | ZJH (4-cam) |
|
| 51 |
+
|
| 52 |
+
These are G2G-only weights (frozen backbone excluded). The evaluation scripts in the GitHub repo automatically handle partial loading.
|
| 53 |
+
|
| 54 |
+
## Download
|
| 55 |
+
|
| 56 |
+
Grab a single weight:
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
from huggingface_hub import hf_hub_download
|
| 60 |
+
|
| 61 |
+
ckpt = hf_hub_download(
|
| 62 |
+
repo_id="feixue22/G2G",
|
| 63 |
+
filename="release_weights/HM3D-Reloc.pth",
|
| 64 |
+
)
|
| 65 |
+
print(ckpt)
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
Or pull everything (weights + example/eval bundles):
|
| 69 |
+
|
| 70 |
+
```python
|
| 71 |
+
from huggingface_hub import snapshot_download
|
| 72 |
+
|
| 73 |
+
local_dir = snapshot_download(repo_id="feixue22/G2G")
|
| 74 |
+
print(local_dir)
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
## Usage
|
| 78 |
+
|
| 79 |
+
These weights plug into the G2G code on GitHub. After cloning and installing
|
| 80 |
+
(https://github.com/WeiYuFei0217/G2G), run evaluation with the downloaded checkpoint:
|
| 81 |
+
|
| 82 |
+
```bash
|
| 83 |
+
python scripts/eval_reloc.py \
|
| 84 |
+
--config configs/reloc/hm3d.yaml \
|
| 85 |
+
--checkpoint release_weights/HM3D-Reloc.pth \
|
| 86 |
+
--output-dir outputs/eval_HM3D-Reloc \
|
| 87 |
+
--batch-size 16 --min-overlap 0.1
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
## License
|
| 91 |
+
|
| 92 |
+
[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/).
|
| 93 |
+
|
| 94 |
+
## Citation
|
| 95 |
+
|
| 96 |
+
```bibtex
|
| 97 |
+
@misc{wei2026g2gexploitingintragroupgeometry,
|
| 98 |
+
title={G2G: Exploiting Intra-Group Geometry for Inter-Group Pose Estimation},
|
| 99 |
+
author={Yufei Wei and Shuhao Ye and Chenxiao Hu and Yiyuan Pan and Dongyu Feng and Rong Xiong and Yue Wang and Yanmei Jiao},
|
| 100 |
+
year={2026},
|
| 101 |
+
eprint={2606.08284},
|
| 102 |
+
archivePrefix={arXiv},
|
| 103 |
+
primaryClass={cs.CV},
|
| 104 |
+
url={https://arxiv.org/abs/2606.08284},
|
| 105 |
+
}
|
| 106 |
+
```
|