MatchFormer: Interleaving Attention in Transformers for Feature Matching (ACCV 2022)

Official model checkpoints for MatchFormer: Interleaving Attention in Transformers for Feature Matching (ACCV 2022 Oral).

---

📌 Introduction

In this work, we propose a novel hierarchical extract-and-match transformer, termed MatchFormer. Inside each stage of the hierarchical encoder, we interleave self-attention for feature extraction and cross-attention for feature matching, enabling an intuitive and powerful extract-and-match scheme. MatchFormer establishes robust and accurate dense correspondences across diverse indoor and outdoor challenging scenes.

---

📂 Model Zoo & File Structure

InSAI-Lab/MatchFormer
├── indoor-large-SEA.ckpt    # Indoor large model with SEA (Spatial-Entropy Attention) (89.5 MB)
├── indoor-lite-LA.ckpt      # Indoor lightweight model with LA (Linear Attention) (77.4 MB)
├── outdoor-large-LA.ckpt    # Outdoor large model with LA (77.4 MB)
└── outdoor-lite-SEA.ckpt    # Outdoor lightweight model with SEA (103.5 MB)

---

📊 Benchmark Results

1. Indoor Relative Pose Estimation on ScanNet (AUC)

Model	Backbone	Resolution	AUC @ 5°	AUC @ 10°	AUC @ 20°	Weights
MatchFormer-Large	largesea	(8, 2)	43.3	63.4	77.1	indoor-large-SEA.ckpt
MatchFormer-Lite	litela	(8, 4)	38.8	59.2	74.0	indoor-lite-LA.ckpt

2. Outdoor Relative Pose Estimation on MegaDepth (AUC)

Model	Backbone	Resolution	AUC @ 5°	AUC @ 10°	AUC @ 20°	Weights
MatchFormer-Large	largela	(8, 2)	53.3	69.7	81.8	outdoor-large-LA.ckpt
MatchFormer-Lite	litesea	(8, 4)	48.2	65.1	78.4	outdoor-lite-SEA.ckpt

---

⚡ Quick Download Guide

Option 1: Hugging Face CLI

# Download all MatchFormer weights to model/weights/
hf download InSAI-Lab/MatchFormer --local-dir model/weights

# Or download a specific checkpoint
hf download InSAI-Lab/MatchFormer indoor-large-SEA.ckpt --local-dir model/weights

Option 2: Python huggingface_hub

from huggingface_hub import hf_hub_download

# Download indoor large SEA checkpoint
ckpt_path = hf_hub_download(
    repo_id="InSAI-Lab/MatchFormer",
    filename="indoor-large-SEA.ckpt",
    local_dir="model/weights"
)

---

📖 Citation

@inproceedings{wang2022matchformer,
  title={MatchFormer: Interleaving Attention in Transformers for Feature Matching},
  author={Wang, Qing and Zhang, Jiaming and Yang, Kailun and Peng, Kunyu and Stiefelhagen, Rainer},
  booktitle={Asian Conference on Computer Vision (ACCV)},
  year={2022}
}