README.md

---
license: mit
language:
  - en
tags:
  - remote-sensing
  - semantic-segmentation
  - mamba
  - state-space-model
  - vmamba
  - mambavision
  - spatial-mamba
  - pytorch
  - benchmark
  - loveda
  - isprs-potsdam
  - domain-adaptation
datasets:
  - LoveDA
  - ISPRS-Potsdam
pipeline_tag: image-segmentation
---

# Mamba-Segmentation

**Controlled Visual State-Space Backbone Benchmark with Domain-Shift & Boundary Analysis for Remote-Sensing Segmentation**

> *Accepted at IGARSS 2026*

One pipeline. One decoder. One loss. One schedule. **Five backbone families.** The only variable is the encoder — so the results finally mean something.

---

## What Is This?

Remote-sensing segmentation papers routinely change the backbone *and* the decoder *and* the loss *and* the training schedule all at once. The numbers tell you who tuned harder, not which backbone is better.

This repo fixes that. **One shared pipeline — swap the backbone — read the truth.**

| Component | Status |
|---|---|
| Encoder backbone | 🔀 **Swapped** per experiment — the ONLY variable |
| Decoder | 🔒 Fixed (lightweight U-Net, 256ch, MambaBlock2d) |
| Loss | 🔒 Fixed (Lovász-Softmax + Focal + Boundary) |
| Training schedule | 🔒 Fixed (50k iters, AdamW, poly LR decay) |
| Augmentations | 🔒 Fixed (random crop, flip, colour jitter) |
| Input resolution | 🔒 Fixed (512×512) |
| Feature interface | 🔒 Fixed ({F1–F4} at strides {4, 8, 16, 32}) |

---

## Checkpoints in This Repository

All checkpoints are `best.pth` files (highest validation mIoU during training) stored with their original directory structure.

### LoveDA Experiments — `Comparison_Experiments/`

#### MambaVision (NVIDIA hybrid Mamba-Transformer)
| Checkpoint path | Training split |
|---|---|
| `Comparison_Experiments/mambavision_tiny_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/mambavision_tiny_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/mambavision_tiny_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/mambavision_tiny2_512/checkpoints/best.pth` | All→All (v2) |
| `Comparison_Experiments/mambavision_tiny2_ruraltrain_512/checkpoints/best.pth` | Rural→Urban (v2) |
| `Comparison_Experiments/mambavision_tiny2_urbantrain_512/checkpoints/best.pth` | Urban→Rural (v2) |
| `Comparison_Experiments/mambavision_small_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/mambavision_small_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/mambavision_small_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/mambavision_base_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/mambavision_base_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/mambavision_base_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/mambavision_large_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/mambavision_large_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/mambavision_large_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/mambavision_large2_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/mambavision_large2_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/mambavision_large2_urbantrain_512/checkpoints/best.pth` | Urban→Rural |

#### VMamba (cross-scan 2D selective SSM)
| Checkpoint path | Training split |
|---|---|
| `Comparison_Experiments/Vmamb_tiny_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/vmamba_tiny_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/vmamba_tiny_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/Vmamb_small_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/Vmamb_small_512_2/checkpoints/best.pth` | All→All (run 2) |
| `Comparison_Experiments/Vmamb_small_512_3/checkpoints/best.pth` | All→All (run 3) |
| `Comparison_Experiments/vmamba_small_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/vmamba_small_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/Vmamb_base_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/vmamba_base_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/vmamba_base_urbantrain_512/checkpoints/best.pth` | Urban→Rural |

#### VisionMamba / Vim (bidirectional Mamba)
| Checkpoint path | Training split |
|---|---|
| `Comparison_Experiments/VisionMamba_tiny_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/visionmamba_tiny_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/visionmamba_tiny_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/VisionMamba_small_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/visionmamba_small_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/visionmamba_small_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/VisionMamba_base_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/visionmamba_base_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/visionmamba_base_urbantrain_512/checkpoints/best.pth` | Urban→Rural |

#### Spatial-Mamba (spatially-aware SSM)
| Checkpoint path | Training split |
|---|---|
| `Comparison_Experiments/spatialmamba_tiny_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/spatialmamba_tiny_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/spatialmamba_tiny_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/spatialmamba_small_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/spatialmamba_small_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/spatialmamba_small_urbantrain_512/checkpoints/best.pth` | Urban→Rural |
| `Comparison_Experiments/spatialmamba_base_512/checkpoints/best.pth` | All→All |
| `Comparison_Experiments/spatialmamba_base_ruraltrain_512/checkpoints/best.pth` | Rural→Urban |
| `Comparison_Experiments/spatialmamba_base_urbantrain_512/checkpoints/best.pth` | Urban→Rural |

#### CNN & Transformer Baselines
| Checkpoint path | Model |
|---|---|
| `Comparison_Experiments/cnn_deeplabv3p_r50_512/checkpoints/best.pth` | DeepLabv3+ ResNet-50, All→All |
| `Comparison_Experiments/cnn_deeplabv3p_resnet50_ruraltrain_512/checkpoints/best.pth` | DeepLabv3+ ResNet-50, Rural→Urban |
| `Comparison_Experiments/cnn_deeplabv3p_resnet50_urbantrain_512/checkpoints/best.pth` | DeepLabv3+ ResNet-50, Urban→Rural |
| `Comparison_Experiments/cnn_unet_r50_512/checkpoints/best.pth` | U-Net ResNet-50, All→All |
| `Comparison_Experiments/transformer_unetformer_r18_512/checkpoints/best.pth` | UNetFormer ResNet-18, All→All |
| `Comparison_Experiments/transformerunetformer_resnet18_ruraltrain_512/checkpoints/best.pth` | UNetFormer ResNet-18, Rural→Urban |
| `Comparison_Experiments/transformerunetformer_resnet18_urbantrain_512/checkpoints/best.pth` | UNetFormer ResNet-18, Urban→Rural |

---

### ISPRS Potsdam Experiments — `Comparison_Experiments_ICPRS_potsdam/`

| Checkpoint path | Model |
|---|---|
| `Comparison_Experiments_ICPRS_potsdam/mambavision_tiny_512/checkpoints/best.pth` | MambaVision-Tiny |
| `Comparison_Experiments_ICPRS_potsdam/mambavision_tiny2_512/checkpoints/best.pth` | MambaVision-Tiny2 |
| `Comparison_Experiments_ICPRS_potsdam/mambavision_small_512/checkpoints/best.pth` | MambaVision-Small |
| `Comparison_Experiments_ICPRS_potsdam/mambavision_base_512/checkpoints/best.pth` | MambaVision-Base |
| `Comparison_Experiments_ICPRS_potsdam/mambavision_large_512/checkpoints/best.pth` | MambaVision-Large |
| `Comparison_Experiments_ICPRS_potsdam/mambavision_large2_512/checkpoints/best.pth` | MambaVision-Large2 |
| `Comparison_Experiments_ICPRS_potsdam/vmamba_tiny_512/checkpoints/best.pth` | VMamba-Tiny |
| `Comparison_Experiments_ICPRS_potsdam/vmamba_small_512/checkpoints/best.pth` | VMamba-Small |
| `Comparison_Experiments_ICPRS_potsdam/vmamba_base_512/checkpoints/best.pth` | VMamba-Base |
| `Comparison_Experiments_ICPRS_potsdam/spatialmamba_tiny_512/checkpoints/best.pth` | Spatial-Mamba-Tiny |
| `Comparison_Experiments_ICPRS_potsdam/spatialmamba_small_512/checkpoints/best.pth` | Spatial-Mamba-Small |
| `Comparison_Experiments_ICPRS_potsdam/spatialmamba_base_512/checkpoints/best.pth` | Spatial-Mamba-Base |
| `Comparison_Experiments_ICPRS_potsdam/cnn_deeplabv3p_r50_512/checkpoints/best.pth` | DeepLabv3+ ResNet-50 |
| `Comparison_Experiments_ICPRS_potsdam/transformer_unetformer_r18_512/checkpoints/best.pth` | UNetFormer ResNet-18 |

---

### ImageNet Backbone Weights — `weights/imagenet/`

| File | Description |
|---|---|
| `weights/imagenet/resnet50-11ad3fa6.pth` | ResNet-50 ImageNet-1K pretrained |
| `weights/imagenet/resnet18-f37072fd.pth` | ResNet-18 ImageNet-1K pretrained |

---

## Results Summary

Every row shares the same decoder, loss, optimizer, schedule, and data splits. **The only variable is the encoder.**

### LoveDA

| Backbone | mIoU (All→All) | mIoU (U→R) | mIoU (R→U) |
|---|---:|---:|---:|
| DeepLabv3+ ResNet-50 (CNN) | 43.01 | 30.36 | 39.98 |
| UNetFormer ResNet-18 (Transformer) | 48.61 | 34.56 | 44.84 |
| VMamba-Small **🥇** | **55.66** | **40.62** | 53.52 |
| MambaVision-Large | 55.25 | 38.53 | **54.01** |
| Spatial-Mamba-Base | 48.03 | 35.23 | 46.55 |

### ISPRS Potsdam

| Backbone | mIoU |
|---|---:|
| DeepLabv3+ ResNet-50 | 75.09 |
| UNetFormer ResNet-18 | 74.99 |
| VMamba-Small **🥇** | **77.59** |
| MambaVision-Large | 77.07 |
| Spatial-Mamba-Base | 70.00 |

**Key findings:**
- SSMs outperform CNNs and Transformers by a significant margin under identical conditions (+7–12 mIoU on LoveDA).
- Scaling the encoder past VMamba-Small yields diminishing returns under a fixed decoder.
- Domain transfer is asymmetric across all backbone families (Rural→Urban consistently outperforms Urban→Rural by 10–15 points) — a data distribution property, not a model property.
- Boundary accuracy collapses under domain shift while interior accuracy holds — every backbone, every family.

---

## How to Load a Checkpoint

```python
import torch

# Example: load MambaVision-Base best checkpoint for LoveDA All→All
ckpt = torch.load(
    "Comparison_Experiments/mambavision_base_512/checkpoints/best.pth",
    map_location="cpu"
)
# keys: 'model', 'optimizer', 'scheduler', 'iter', 'best_score'
model_state = ckpt["model"]
```

To build the full model and run inference, clone the code repository and follow the setup instructions:

```bash
git clone https://github.com/dineth18/Mamba-Segmentation
cd Mamba-Segmentation/MambaVision   # or VMamba/, spatial-mamba/, etc.
pip install -r requirements.txt

# Set your dataset path (no need to edit config files)
export LOVEDA_ROOT=/path/to/LoveDA
export POTSDAM_ROOT=/path/to/ISPRS_Potsdam

python eval.py --checkpoint path/to/best.pth
```

---

## Citation

If this benchmark is useful for your research, please cite:

```bibtex
@article{wasalathilaka2026controlledbenchmark,
  title={A Controlled Benchmark of Visual State-Space Backbones with
         Domain-Shift and Boundary Analysis for Remote-Sensing Segmentation},
  author={Wasalathilaka, Nichula and Perea, Dineth and Samarakoon, Oshadha
          and Wijenayake, Buddhi and Godaliyadda, Roshan and Herath, Vijitha
          and Ekanayake, Parakrama},
  journal={IGARSS 2026},
  year={2026}
}
```

---

## Acknowledgements

- [VMamba](https://github.com/MzeroMiko/VMamba) — Visual State Space Model
- [MambaVision](https://github.com/NVlabs/MambaVision) — NVIDIA hybrid Mamba-Transformer
- [Spatial-Mamba](https://github.com/EdwardChaworworrachat/SpatialMamba) — Spatially-aware Mamba
- [LoveDA](https://github.com/Junjue-Wang/LoveDA) — Land-cover domain adaptation dataset
- [ISPRS Potsdam](https://www.isprs.org/education/benchmarks/UrbanSemLab/) — Urban semantic labeling benchmark

Built at the **University of Peradeniya**.