README.md

---
license: mit
tags:
- slam
- monocular-slam
- visual-localization
- geometric-foundation-models
- pytorch
thumbnail: ./teaser.png
---

<p align="center">
  <h1 align="center">LeanGate</h1>
  <p align="center">
    Accelerating Transformer-Based Monocular SLAM via Geometric Utility Scoring
  </p>
</p>

<p align="center">
  <a href="https://lean-gate.github.io/">Project Page</a> |
  <a href="https://github.com/XinmiaoShawn/LeanGate-code">Code</a> |
  <a href="#">Paper Coming Soon</a>
</p>

<p align="center">
  <img src="./teaser.png" width="95%">
</p>

## Overview

LeanGate is a lightweight frame-gating model for transformer-based monocular SLAM.
It predicts the geometric utility of an incoming frame before expensive dense reconstruction,
allowing the system to skip redundant frames early and significantly reduce computation.

## Highlights

- Filters out more than 90% of redundant frames before heavy SLAM processing
- Reduces tracking FLOPs by more than 85%
- Achieves around 5x end-to-end throughput speedup
- Maintains competitive tracking and mapping accuracy

## Model

- Checkpoint: `model.pt`
- Framework: PyTorch
- Task: frame utility scoring for monocular SLAM
- Input: current frame and reference/keyframe features or paired image representation
- Output: geometric utility score used for frame gating

## Method

<p align="center">
  <img src="./system.png" width="95%">
</p>

## Quick Start

This release lets you download the public LeanGate checkpoint, run LeanGate on prepared `TUM`, `7SCENES`, or `EUROC` scenes, export sparse RGB manifests, and optionally launch MASt3R-SLAM on the filtered sequence.

### 1. Install
Use `python3` and install a PyTorch version matching your CUDA runtime first.

```bash
pip install -e .
pip install -e third_party/MASt3R-SLAM/thirdparty/mast3r
pip install -e third_party/MASt3R-SLAM/thirdparty/in3d
pip install --no-build-isolation -e third_party/MASt3R-SLAM
```

### 2. Download the released checkpoint
The public LeanGate checkpoint is hosted at:

- Repo: `ShawnX98/LeanGate`
- URL: `https://huggingface.co/ShawnX98/LeanGate`
- File: `leangate.pt`

Download it with:

```bash
python3 scripts/download_checkpoints.py --output-root checkpoints --repo-id ShawnX98/LeanGate
```

This will place the checkpoint at:

```text
checkpoints/leangate.pt
```

### 3. Run LeanGate on a prepared benchmark dataset
Example for `TUM`:

```bash
python3 scripts/generate_rgb_lists.py \
  --dataset-type TUM \
  --dataset-root /data/tum \
  --output-root outputs/predictions \
  --device cuda:0
```

Supported benchmark inputs:

- `TUM`
- `7SCENES`
- `EUROC`

Expected dataset layouts are documented in [`docs/dataset_layouts.md`](docs/dataset_layouts.md).

### 4. Run the plain RGB folder demo
For a simple folder of RGB frames:

```bash
./demo.sh \
  --folder /data/my_rgb_frames \
  --output-root outputs/demo \
  --device cuda:0
```

This processes frames in sorted filename order and writes the filtered manifest to:

```text
outputs/demo/leangate/<folder_name>.txt
```

### 5. Optional: launch MASt3R-SLAM on the sparse sequence
Single scene:

```bash
python3 scripts/run_slam_scene.py \
  --dataset-type TUM \
  --dataset-root /data/tum \
  --scene-id rgbd_dataset_freiburg1_desk \
  --predictions-root outputs/predictions \
  --output-root outputs/slam
```

Full dataset:

```bash
python3 scripts/run_slam_dataset.py \
  --dataset-type TUM \
  --dataset-root /data/tum \
  --predictions-root outputs/predictions \
  --output-root outputs/slam
```

### Outputs
LeanGate inference produces:

- `outputs/predictions/<dataset_slug>/leangate/<scene>.txt`
- `outputs/predictions/<dataset_slug>/leangate/scores/<scene>_scores.csv`

Optional MASt3R-SLAM outputs include:

- `outputs/slam/<dataset_slug>/leangate/<scene>/trajectory_keyframes.tum`
- `outputs/slam/<dataset_slug>/leangate/<scene>/reconstruction.ply`
- `outputs/slam/<dataset_slug>/leangate/<scene>/run_metadata.json`
- `outputs/slam/<dataset_slug>/leangate/summary.csv`
- `outputs/slam/<dataset_slug>/leangate/summary.json`