---
license: cc-by-4.0
tags:
  - remote-sensing
  - earth-observation
  - self-supervised-learning
  - ssl4eo-s12
  - sentinel-2
  - vit-small
datasets:
  - wangyi111/SSL4EO-S12
base_model: wangyi111/SSL4EO-S12
---

# SSL4EO-S12 — ViT-S/16 pre-trained with Data2Vec

Pre-trained backbone from the [SSL4EO-S12](https://github.com/zhu-xlab/SSL4EO-S12) project.

| Property | Value |
|---|---|
| SSL method | **Data2Vec** |
| Architecture | ViT-S/16 |
| Input | S2-L1C 13 bands |
| Pre-training epochs | 100 |
| Normalisation | clip [0, 1] by dividing 10 000 |
| Checkpoint | `B13_vits16_data2vec_0099_ckpt.pth` |

## Load the backbone

```python
import torch
import timm

model = timm.create_model("vit_small_patch16_224", num_classes=0)
state = torch.load("B13_vits16_data2vec_0099_ckpt.pth", map_location="cpu")

# key may be "model", "state_dict", or "teacher" depending on the method
backbone_state = state.get("model", state.get("state_dict", state))
model.load_state_dict(backbone_state, strict=False)
model.eval()
```

## Citation

```bibtex
@article{wang2022ssl4eo,
  title={SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset
         for Self-Supervised Learning in Earth Observation},
  author={Wang, Yi and Braham, Nassim Ait Ali and Xiong, Zhitong and
           Liu, Chenying and Albrecht, Conrad M and Zhu, Xiao Xiang},
  journal={arXiv preprint arXiv:2211.07044},
  year={2022}
}
```