README.md

metadata

license: apache-2.0
language:
  - en
tags:
  - remote-sensing
  - earth-observation
  - self-supervised-learning
  - sentinel-2
  - multispectral
  - sar
  - feature-extraction
  - vision
  - softcon
  - vit
  - resnet
  - dinov2
  - transformers
library_name: transformers
pipeline_tag: feature-extraction
datasets:
  - wangyi111/SSL4EO-S12

SoftCon Transformers Models

Hugging Face–compatible checkpoints converted from the official SoftCon pretrained weights. Each subfolder is a standalone model repo layout (config.json, model.safetensors, preprocessor, and remote code) for feature extraction on Earth observation imagery.

Model Description

These models are encoders pretrained with multi-label guided soft contrastive learning on SSL4EO-S12 multispectral and SAR imagery, with DINOv2-style continual pretraining for ViT backbones.

This collection bundles 6 converted checkpoints:

• Architectures: ResNet-50, ViT-S/14, ViT-B/14 (DINOv2-style)
• Input modalities: S2-L1C 13-band (s2c), S1 SAR 2-band (s1)

All folders ship self-contained remote code (modeling_softcon.py, processor, pipeline) and load with trust_remote_code=True. ViT backbones reuse transformers' built-in Dinov2Model.

Developed by: zhu-xlab / SoftCon
Converted for Hugging Face by: BiliSakura
License (weights): Apache-2.0
Original paper: Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining

Available checkpoints (6 models)

Folder	Backbone	Modality	Channels	Embedding dim	Legacy file
softcon-resnet50-s2c	RN50	S2-L1C MS	13	2048	B13_rn50_softcon.pth
softcon-vit-small-patch14-s2c	ViT-S/14	S2-L1C MS	13	384	B13_vits14_softcon_enc.pth
softcon-vit-base-patch14-s2c	ViT-B/14	S2-L1C MS	13	768	B13_vitb14_softcon_enc.pth
softcon-resnet50-s1	RN50	S1 SAR	2	2048	B2_rn50_softcon.pth
softcon-vit-small-patch14-s1	ViT-S/14	S1 SAR	2	384	B2_vits14_softcon_enc.pth
softcon-vit-base-patch14-s1	ViT-B/14	S1 SAR	2	768	B2_vitb14_softcon_enc.pth

Usage

Processors default to do_resize: false. Pass patches at native (H, W, C); the processor rescales to [0, 1] without changing spatial size.

from transformers import pipeline
import numpy as np

REPO = "BiliSakura/SOFTCON-transformers"
SUBFOLDER = "softcon-vit-small-patch14-s2c"

pipe = pipeline(
    task="softcon-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)

# S2-L1C: 13 bands at native resolution (e.g. 512×512)
image = np.random.randint(0, 255, (512, 512, 13), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # [1, 384]

Dense token features:

tokens = pipe(image, pool=False, return_tensors=True)

Opt in to resize (patch-14 models were pretrained on 224×224):

features = pipe(image, pool=True, return_tensors=True, image_processor_kwargs={"do_resize": True})

Load components directly:

from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)

Normalization

By default, the bundled image processor rescales inputs to [0, 1] by dividing by 255. SoftCon recommends mapping each channel to uint8 using per-channel mean/std from SSL4EO-S12 or the target dataset before inference. Enable do_normalize=True with per-channel image_mean and image_std on SoftConImageProcessor when using normalized inputs.

Dependencies

• transformers, torch, torchvision, safetensors
• opencv-python (multispectral resize with more than 4 channels)

Citation

@misc{wang2024multilabel,
  title={Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining},
  author={Wang, Yi and Albrecht, Conrad M and Zhu, Xiao Xiang},
  journal={arXiv preprint arXiv:2405.20462},
  year={2024}
}