README.md

---
license: mit
pipeline_tag: image-segmentation
tags:
  - remote-sensing
  - earth-observation
  - open-vocabulary
  - clip
  - sam3
  - semantic-segmentation
library_name: transformers
---

# SegEarth-OV: Unified Open-Vocabulary Segmentation for Remote Sensing

Unified repo for **SegEarth OV, OV-2, OV-3** — training-free open-vocabulary semantic segmentation. Each variant lives in a **self-contained subfolder** with its own `config.json`, pipeline, and weights.

## Structure

```
SegEarth-OV/
├── OV/           # CLIP (OpenAI ViT-B/16) + SimFeatUp
│   ├── config.json
│   ├── pipeline.py
│   ├── upsamplers.py
│   ├── prompts/
│   ├── configs/cls_*.txt
│   ├── weights/featup/      # SimFeatUp checkpoints
│   └── weights/backbone/    # OpenAI CLIP (clip-vit-base-patch16)
├── OV-2/         # AlignEarth (SAR) + SimFeatUp
│   ├── config.json
│   ├── pipeline.py
│   ├── weights/featup/
│   └── weights/backbone/    # AlignEarth-SAR-ViT-B-16
├── OV-3/         # SAM3 (no featup)
│   ├── config.json
│   ├── pipeline.py
│   ├── configs/
│   └── weights/backbone/   # facebook/sam3 (sam3.pt)
└── model_config.json
```

## Self-Contained (No Download)

All checkpoints are **included** in this repo. No additional download required.

| Variant | Backbone | Location |
|---------|----------|----------|
| OV | OpenAI CLIP ViT-B/16 | `OV/weights/backbone/clip-vit-base-patch16/` |
| OV-2 | AlignEarth-SAR-ViT-B-16 | `OV-2/weights/backbone/AlignEarth-SAR-ViT-B-16/` |
| OV-3 | SAM3 | `OV-3/weights/backbone/sam3/sam3.pt` |
| OV, OV-2 | SimFeatUp (jbu_one, etc.) | `OV/weights/featup/`, `OV-2/weights/featup/` |

## Usage

**From subfolder (self-contained):**

```python
# OV-2 with AlignEarth (SAR)
from pipeline import SegEarthPipeline
pipe = SegEarthPipeline()  # loads OV-2/config.json
seg = pipe(image)

# Or: cd OV-2 && python -c "from pipeline import load; pipe = load()"
```

**From repo root:**

```python
from pipeline import SegEarthPipeline

pipe = SegEarthPipeline(variant="OV-2")
pipe = SegEarthPipeline(variant="OV")
pipe = SegEarthPipeline(variant="OV-3")  # requires sam3 package
```

**Custom class config:**

```python
pipe = SegEarthPipeline(variant="OV-2", class_names_path="OV-2/configs/cls_yeseg_sar.txt")
```

## Variants

| Subfolder | Backbone | Model ID | FeatUp |
|-----------|----------|----------|--------|
| OV | CLIP | openai/clip-vit-base-patch16 | jbu_one |
| OV-2 | AlignEarth | BiliSakura/AlignEarth-SAR-ViT-B-16 | jbu_one |
| OV-3 | SAM3 | facebook/sam3 | None |

## Citation

```bibtex
@InProceedings{Li_2025_CVPR,
  author    = {Li, Kaiyu and Liu, Ruixun and Cao, Xiangyong and Bai, Xueru and Zhou, Feng and Meng, Deyu and Wang, Zhi},
  title     = {SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2025},
  pages     = {10545--10556}
}

@article{li2025segearthov2,
  title   = {Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images},
  author  = {Li, Kaiyu and Cao, Xiangyong and Liu, Ruixun and Wang, Shihong and Jiang, Zixuan and Wang, Zhi and Meng, Deyu},
  journal = {arXiv preprint arXiv:2508.18067},
  year    = {2025}
}

@article{li2025segearthov3,
  title   = {SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images},
  author  = {Li, Kaiyu and Zhang, Shengqi and Deng, Yupeng and Wang, Zhi and Meng, Deyu and Cao, Xiangyong},
  journal = {arXiv preprint arXiv:2512.08730},
  year    = {2025}
}
```