File size: 3,716 Bytes
fabc606 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | ---
license: mit
pipeline_tag: image-segmentation
tags:
- remote-sensing
- earth-observation
- open-vocabulary
- clip
- sam3
- semantic-segmentation
library_name: transformers
---
# SegEarth-OV: Unified Open-Vocabulary Segmentation for Remote Sensing
Unified repo for **SegEarth OV, OV-2, OV-3** — training-free open-vocabulary semantic segmentation. Each variant lives in a **self-contained subfolder** with its own `config.json`, pipeline, and weights.
## Structure
```
SegEarth-OV/
├── OV/ # CLIP (OpenAI ViT-B/16) + SimFeatUp
│ ├── config.json
│ ├── pipeline.py
│ ├── upsamplers.py
│ ├── prompts/
│ ├── configs/cls_*.txt
│ ├── weights/featup/ # SimFeatUp checkpoints
│ └── weights/backbone/ # OpenAI CLIP (clip-vit-base-patch16)
├── OV-2/ # AlignEarth (SAR) + SimFeatUp
│ ├── config.json
│ ├── pipeline.py
│ ├── weights/featup/
│ └── weights/backbone/ # AlignEarth-SAR-ViT-B-16
├── OV-3/ # SAM3 (no featup)
│ ├── config.json
│ ├── pipeline.py
│ ├── configs/
│ └── weights/backbone/ # facebook/sam3 (sam3.pt)
└── model_config.json
```
## Self-Contained (No Download)
All checkpoints are **included** in this repo. No additional download required.
| Variant | Backbone | Location |
|---------|----------|----------|
| OV | OpenAI CLIP ViT-B/16 | `OV/weights/backbone/clip-vit-base-patch16/` |
| OV-2 | AlignEarth-SAR-ViT-B-16 | `OV-2/weights/backbone/AlignEarth-SAR-ViT-B-16/` |
| OV-3 | SAM3 | `OV-3/weights/backbone/sam3/sam3.pt` |
| OV, OV-2 | SimFeatUp (jbu_one, etc.) | `OV/weights/featup/`, `OV-2/weights/featup/` |
## Usage
**From subfolder (self-contained):**
```python
# OV-2 with AlignEarth (SAR)
from pipeline import SegEarthPipeline
pipe = SegEarthPipeline() # loads OV-2/config.json
seg = pipe(image)
# Or: cd OV-2 && python -c "from pipeline import load; pipe = load()"
```
**From repo root:**
```python
from pipeline import SegEarthPipeline
pipe = SegEarthPipeline(variant="OV-2")
pipe = SegEarthPipeline(variant="OV")
pipe = SegEarthPipeline(variant="OV-3") # requires sam3 package
```
**Custom class config:**
```python
pipe = SegEarthPipeline(variant="OV-2", class_names_path="OV-2/configs/cls_yeseg_sar.txt")
```
## Variants
| Subfolder | Backbone | Model ID | FeatUp |
|-----------|----------|----------|--------|
| OV | CLIP | openai/clip-vit-base-patch16 | jbu_one |
| OV-2 | AlignEarth | BiliSakura/AlignEarth-SAR-ViT-B-16 | jbu_one |
| OV-3 | SAM3 | facebook/sam3 | None |
## Citation
```bibtex
@InProceedings{Li_2025_CVPR,
author = {Li, Kaiyu and Liu, Ruixun and Cao, Xiangyong and Bai, Xueru and Zhou, Feng and Meng, Deyu and Wang, Zhi},
title = {SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2025},
pages = {10545--10556}
}
@article{li2025segearthov2,
title = {Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images},
author = {Li, Kaiyu and Cao, Xiangyong and Liu, Ruixun and Wang, Shihong and Jiang, Zixuan and Wang, Zhi and Meng, Deyu},
journal = {arXiv preprint arXiv:2508.18067},
year = {2025}
}
@article{li2025segearthov3,
title = {SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images},
author = {Li, Kaiyu and Zhang, Shengqi and Deng, Yupeng and Wang, Zhi and Meng, Deyu and Cao, Xiangyong},
journal = {arXiv preprint arXiv:2512.08730},
year = {2025}
}
```
|