datasets:
- wangyi111/Copernicus-Pretrain
- wangyi111/Copernicus-Bench
license: cc-by-4.0
pipeline_tag: image-feature-extraction
tags:
- earth-observation
- remote-sensing
- sentinel
- foundation-model
library_name: transformers
Model Card for Copernicus-FM
Copernicus-FM is an extension of the DOFA foundation model, able to process any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding. The model is pretrained on the Copernicus-Pretrain dataset with masked image modeling and continual distillation. It was introduced in the paper Towards a Unified Copernicus Foundation Model for Earth Vision.
Model Description
Copernicus-FM is a unified foundation model designed for Earth Observation (EO). It takes a significant step towards next-generation EO foundation models by addressing limitations of existing efforts, such as fixed spectral sensors and overlooked metadata. The model leverages extended dynamic hypernetworks and flexible metadata encoding to process any spectral or non-spectral sensor modality, from the Earth's surface to its atmosphere.
This model is part of a larger framework that includes:
- Copernicus-Pretrain: A massive-scale pretraining dataset comprising 18.7 million aligned images from all major Copernicus Sentinel missions (S1-S5P).
- Copernicus-FM: The unified foundation model pretrained on Copernicus-Pretrain using masked image modeling and continual distillation.
- Copernicus-Bench: A systematic evaluation benchmark featuring 15 hierarchical downstream tasks across various Sentinel missions, ranging from preprocessing to specialized applications.
- Copernicus-Embed-025deg: A global embedding map derived from Copernicus-FM, providing highly compressed representations of satellite observations.
Uses
Copernicus-FM aims to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. Its capabilities greatly improve the scalability, versatility, and multimodal adaptability of Earth observation foundation models. Key uses include:
- Generic Feature Extraction: Providing robust features from diverse spectral and non-spectral satellite imagery.
- Downstream EO Applications: Supporting tasks within the Copernicus-Bench benchmark, such as land use/land cover classification and segmentation (e.g., EuroSAT, BigEarthNet, DFC2020), cloud segmentation, flood detection, and air quality regression.
- Connecting Earth Sciences: Facilitating new opportunities to connect Earth Observation, weather, and climate research by providing a unified model for analyzing various forms of satellite data.
Sample Usage
You can use Copernicus-FM for image feature extraction with the Hugging Face transformers library.
from transformers import AutoModel, AutoProcessor
from PIL import Image
import requests
import torch
# Load model and processor
model_name = "wangyi111/Copernicus-FM"
model = AutoModel.from_pretrained(model_name)
processor = AutoProcessor.from_pretrained(model_name)
# Example image (replace with your Sentinel image or local path)
# Using an image from the GitHub repo for demonstration purposes
image_url = "https://raw.githubusercontent.com/zhu-xlab/Copernicus-FM/main/assets/altogether-1.png"
image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")
# Prepare inputs
inputs = processor(images=image, return_tensors="pt")
# Get image features
with torch.no_grad():
outputs = model(**inputs)
# For image-feature-extraction models, the last_hidden_state often contains the features
image_features = outputs.last_hidden_state
print(f"Image features shape: {image_features.shape}")
# Example output shape: torch.Size([1, N, hidden_size]) where N is number of patches/tokens
Related Sources
- Repository: https://github.com/zhu-xlab/Copernicus-FM
- Paper: https://arxiv.org/abs/2503.11849
Citation
@misc{wang2025unifiedcopernicusfoundationmodel,
title={Towards a Unified Copernicus Foundation Model for Earth Vision},
author={Yi Wang and Zhitong Xiong and Chenying Liu and Adam J. Stewart and Thomas Dujardin and Nikolaos Ioannis Bountos and Angelos Zavras and Franziska Gerken and Ioannis Papoutsis and Laura Leal-Taixé and Xiao Xiang Zhu},
year={2025},
eprint={2503.11849},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.11849},
}