README.md

---
license: mit
language:
- en
tags:
- remote-sensing
- earth-observation
- vision
- feature-extraction
- dofa
- sentinel-2
- multimodal
library_name: transformers
pipeline_tag: feature-extraction
---

# DOFA Transformers Models

Self-contained HuggingFace model checkpoints for [DOFA](https://arxiv.org/abs/2403.15356).

Each checkpoint subfolder ships remote code for model, processor, and pipeline loading with `trust_remote_code=True`.

Sentinel-2 9-band defaults (`default_wavelengths`, `default_image_mean`, `default_image_std`) are baked into `config.json` and `preprocessor_config.json`.

## Available checkpoints

| Folder | Hidden size | Layers | Heads |
|--------|-------------|--------|-------|
| `dofa-base-patch16-224/` | 768 | 12 | 12 |
| `dofa-large-patch16-224/` | 1024 | 24 | 16 |

## Usage

Processors default to **`do_resize: false`**. Pass Sentinel-2 stacks at native `(H, W, C)`; the processor rescales values (typically `/255`) without changing spatial size.

```python
from transformers import pipeline

MODEL = "/path/to/DOFA-transformers/dofa-base-patch16-224"

pipe = pipeline(
    task="dofa-feature-extraction",
    model=MODEL,
    trust_remote_code=True,
)

# Native-resolution patch, e.g. 512×512×9 bands (uint8 or float)
features = pipe(image_array, pool=True, return_tensors=True)
```

Dense features:

```python
tokens = pipe(image_array, pool=False, return_tensors=True)
```

Opt in to 224×224 resize (original pretraining size):

```python
features = pipe(
    image_array,
    pool=True,
    return_tensors=True,
    image_processor_kwargs={"do_resize": True},
)
```

Override Sentinel-2 defaults for other sensors:

```python
features = pipe(
    image_array,
    wavelengths=[...],
    image_mean=[...],
    image_std=[...],
    pool=True,
    return_tensors=True,
)
```

## Test CLI

```bash
conda activate rsgen
python test_dofa.py
python test_dofa.py --model dofa-large-patch16-224
python test_dofa.py --model dofa-base-patch16-224 --no-pool
```

## Dependencies

- `transformers`
- `timm`
- `torch`
- `opencv-python` (only when resizing inputs with more than 4 channels)