File size: 4,337 Bytes
96e3a14 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | ---
license: apache-2.0
language:
- en
tags:
- remote-sensing
- earth-observation
- self-supervised-learning
- semantic-segmentation
- feature-extraction
- vision
- s5
- s4p
- vit
- transformers
library_name: transformers
pipeline_tag: feature-extraction
---
# S5 Transformers Models
Hugging Face–compatible checkpoints converted from the official [S5](https://arxiv.org/abs/2508.12409) S4P pretrain weights. Each subfolder is a standalone model repo layout (`config.json`, `model.safetensors`, preprocessor, and remote code) for **encoder feature extraction** on optical remote sensing imagery.
## Model Description
These are ViT encoders pretrained with **S4P** (Semi-supervised Semantic Segmentation Pre-training) on [RS4P-1M](https://huggingface.co/datasets/lianglyu/R4P-1M). This collection currently bundles **2 converted backbone checkpoints**:
- **ViT-B:** ViT-Base/16, hidden size 768, 12 layers
- **ViT-L:** ViT-Large/16, hidden size 1024, 24 layers
Both checkpoints use `architecture: s4p_backbone` and expose the `s5-feature-extraction` pipeline. They are **encoder-only** weights (not UPerNet segmentation or MoE-MDF heads).
All folders ship self-contained remote code (`modeling_s5.py`, processor, pipeline) and load with `trust_remote_code=True`.
**Developed by:** [lianglyu / S5](https://huggingface.co/lianglyu/S5)
**Converted for Hugging Face by:** BiliSakura
**License (weights):** Apache 2.0
**Original paper:** [S5: Scalable Semi-Supervised Semantic Segmentation in Remote Sensing](https://arxiv.org/abs/2508.12409) (AAAI 2026 Oral)
## Available checkpoints
| Folder | Backbone | Hidden size | Layers | Heads | Patch | Image size | Original file |
|--------|----------|-------------|--------|-------|-------|------------|---------------|
| `ViT-B` | ViT-Base | 768 | 12 | 12 | 16 | 512 | `vit_b_s4p.pth` |
| `ViT-L` | ViT-Large | 1024 | 24 | 16 | 16 | 512 | `vit_l_s4p.pth` |
Original singular `.pth` files were converted and removed from this directory.
## Usage
Processors default to **`do_resize: false`**. Pass RGB images at native resolution; ImageNet mean/std normalization is applied when enabled.
```python
from transformers import pipeline
import numpy as np
REPO = "/path/to/S5-transformers"
pipe = pipeline(
task="s5-feature-extraction",
model=f"{REPO}/ViT-B",
trust_remote_code=True,
)
image = np.random.randint(0, 255, (512, 512, 3), dtype=np.uint8)
# Global pooled features
features = pipe(image, pool=True, return_tensors=True)
print(features.shape) # [1, 768] for ViT-B, [1, 1024] for ViT-L
# Dense feature map
featmap = pipe(image, pool=False, return_tensors=True)
print(featmap.shape) # [1, 768, 32, 32] for ViT-B, [1, 1024, 32, 32] for ViT-L
```
ViT-L:
```python
pipe = pipeline(
task="s5-feature-extraction",
model=f"{REPO}/ViT-L",
trust_remote_code=True,
)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape) # [1, 1024]
```
To force 512×512 resize:
```python
features = pipe(
image,
pool=True,
return_tensors=True,
image_processor_kwargs={"do_resize": True},
)
```
Load components directly:
```python
from transformers import AutoModel, AutoImageProcessor
model = AutoModel.from_pretrained(f"{REPO}/ViT-B", trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(f"{REPO}/ViT-B", trust_remote_code=True)
```
## Normalization
The bundled image processor applies ImageNet mean/std normalization by default (`do_normalize=True`, `rescale_factor=1/255`). Inputs should be RGB optical imagery.
## Conversion
Checkpoints were converted with [`scripts/convert_s5_checkpoint.py`](https://github.com/lianglyu/S5-transformers) from the official release:
```bash
python scripts/convert_s5_checkpoint.py \
--input-path /path/to/vit_b_s4p.pth \
--output-dir /path/to/ViT-B \
--clean-output
```
For semantic segmentation heads, convert `vit_*_s4p_upernet.pth` or `s5_vit_*_moe_mdf_seg.pth` instead (task: `s5-semantic-segmentation`).
## Dependencies
- `transformers>=4.45.0`
- `torch>=2.1.0`
- `safetensors`
- `Pillow`
- `numpy`
## Citation
```bibtex
@article{S5,
title={S5: Scalable Semi-Supervised Semantic Segmentation in Remote Sensing},
author={Liang Lv and Di Wang and Jing Zhang and Lefei Zhang},
journal={arXiv preprint arXiv:2508.12409},
year={2025}
}
```
|