README.md

---
license: mit
language:
- en
tags:
- remote-sensing
- earth-observation
- self-supervised-learning
- satellite
- multispectral
- multimodal
- feature-extraction
- anysat
- transformers
library_name: transformers
pipeline_tag: feature-extraction
---

# AnySat Transformers Models

Hugging Face–compatible checkpoints converted from the official [AnySat](https://arxiv.org/abs/2503.16990) GeoPlex-pretrained weights. Each subfolder is a standalone model repo (`config.json`, `model.safetensors`, processor, pipeline, and remote code) for multimodal Earth observation feature extraction.

## Model Description

[AnySat](https://github.com/gastruc/AnySat) is a scale-adaptive, multimodal Earth Observation foundation model supporting 11 sensor modalities (aerial, Sentinel-1/2, Landsat, MODIS, etc.) at variable resolutions. The `anysat-base` checkpoint is the published base encoder (768-dim, 6 layers, 12 heads).

All hyperparameters live in `config.json` and `preprocessor_config.json` — Python remote code does not embed global defaults.

**Developed by:** [AnySat Authors](https://github.com/gastruc/AnySat)  
**Converted for Hugging Face by:** BiliSakura  
**License (weights):** MIT  
**Original paper:** [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://arxiv.org/abs/2503.16990) (CVPR 2025 Highlight)

## Available checkpoints

| Folder | Size | Embed dim | Depth | Heads | Modalities |
|--------|------|-----------|-------|-------|------------|
| `anysat-base` | base | 768 | 6 | 12 | 11 |

Legacy source: `models/raw/AnySat_full.pth` (see [`conversion_manifest.json`](conversion_manifest.json)).

## Usage

Inputs must be **normalized** `(data - mean) / std` per modality when `normalize=True` in the processor. Set `normalization_stats` in `preprocessor_config.json` for your dataset statistics.

### Custom pipeline (recommended)

```python
from transformers import pipeline
import numpy as np

MODEL = "/path/to/AnySat-transformers/anysat-base"

pipe = pipeline(
    task="anysat-feature-extraction",
    model=MODEL,
    trust_remote_code=True,
)

# Sentinel-2 time series: (B, T, C, H, W)
s2 = np.random.randn(1, 4, 10, 32, 32).astype(np.float32)
dates = np.array([[100, 120, 140, 160]])

# Tile features (CLS token)
features = pipe(s2=s2, s2_dates=dates, patch_size=10, pool=True, return_tensors=True)
print(features.shape)  # torch.Size([1, 768])

# Patch features
patch_features = pipe(s2=s2, s2_dates=dates, patch_size=10, output="patch", return_tensors=True)
print(patch_features.shape)  # torch.Size([1, 1024, 768])
```

### Multimodal example

```python
pipe = pipeline(task="anysat-feature-extraction", model=MODEL, trust_remote_code=True)

s2 = np.random.randn(1, 4, 10, 32, 32).astype(np.float32)
s1 = np.random.randn(1, 4, 3, 32, 32).astype(np.float32)
features = pipe(
    s2=s2,
    s2_dates=np.array([[100, 120, 140, 160]]),
    s1=s1,
    s1_dates=np.array([[100, 120, 140, 160]]),
    patch_size=10,
    pool=True,
    return_tensors=True,
)
```

### Built-in ImageFeatureExtractionPipeline

`AnySatModel` returns `BaseModelOutputWithPooling`, so the registered `image-feature-extraction` task also works:

```python
pipe = pipeline(
    task="image-feature-extraction",
    model=MODEL,
    trust_remote_code=True,
)
```

## Supported modalities

| Key | Type | Channels | Resolution |
|-----|------|----------|------------|
| `aerial` | image | 4 | 0.2 m |
| `aerial-flair` | image | 5 | 0.2 m |
| `spot` | image | 3 | 1 m |
| `naip` | image | 4 | 1.25 m |
| `s2` | time series | 10 | 10 m |
| `s1-asc`, `s1` | time series | 2–3 | 10 m |
| `l7`, `l8` | time series | 6–11 | 10–30 m |
| `alos` | time series | 3 | 30 m |
| `modis` | time series | 7 | 250 m |

Time-series modalities require `{modality}_dates` (day-of-year, 0–364). Images must be square. `patch_size` is in **meters** (internally divided by 10).

## Output modes

| `output` | Shape | Description |
|----------|-------|-------------|
| `tile` | `[B, 768]` | CLS / global embedding (`pool=True`) |
| `patch` | `[B, H*W, 768]` | Patch tokens |
| `dense` | high-res map | Dense features for segmentation |
| `all` | all tokens | Full token sequence |