Sync updated code and configs (no weight re-upload)

README.md

@@ -0,0 +1,126 @@
+---
+license: mit
+language:
+- en
+tags:
+- remote-sensing
+- earth-observation
+- self-supervised-learning
+- satellite
+- multispectral
+- multimodal
+- feature-extraction
+- anysat
+- transformers
+library_name: transformers
+pipeline_tag: feature-extraction
+---
+
+# AnySat Transformers Models
+
+Hugging Face–compatible checkpoints converted from the official [AnySat](https://arxiv.org/abs/2503.16990) GeoPlex-pretrained weights. Each subfolder is a standalone model repo (`config.json`, `model.safetensors`, processor, pipeline, and remote code) for multimodal Earth observation feature extraction.
+
+## Model Description
+
+[AnySat](https://github.com/gastruc/AnySat) is a scale-adaptive, multimodal Earth Observation foundation model supporting 11 sensor modalities (aerial, Sentinel-1/2, Landsat, MODIS, etc.) at variable resolutions. The `anysat-base` checkpoint is the published base encoder (768-dim, 6 layers, 12 heads).
+
+All hyperparameters live in `config.json` and `preprocessor_config.json` — Python remote code does not embed global defaults.
+
+**Developed by:** [AnySat Authors](https://github.com/gastruc/AnySat)  
+**Converted for Hugging Face by:** BiliSakura  
+**License (weights):** MIT  
+**Original paper:** [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://arxiv.org/abs/2503.16990) (CVPR 2025 Highlight)
+
+## Available checkpoints
+
+| Folder | Size | Embed dim | Depth | Heads | Modalities |
+|--------|------|-----------|-------|-------|------------|
+| `anysat-base` | base | 768 | 6 | 12 | 11 |
+
+Legacy source: `models/raw/AnySat_full.pth` (see [`conversion_manifest.json`](conversion_manifest.json)).
+
+## Usage
+
+Inputs must be **normalized** `(data - mean) / std` per modality when `normalize=True` in the processor. Set `normalization_stats` in `preprocessor_config.json` for your dataset statistics.
+
+### Custom pipeline (recommended)
+
+```python
+from transformers import pipeline
+import numpy as np
+
+MODEL = "/path/to/AnySat-transformers/anysat-base"
+
+pipe = pipeline(
+    task="anysat-feature-extraction",
+    model=MODEL,
+    trust_remote_code=True,
+)
+
+# Sentinel-2 time series: (B, T, C, H, W)
+s2 = np.random.randn(1, 4, 10, 32, 32).astype(np.float32)
+dates = np.array([[100, 120, 140, 160]])
+
+# Tile features (CLS token)
+features = pipe(s2=s2, s2_dates=dates, patch_size=10, pool=True, return_tensors=True)
+print(features.shape)  # torch.Size([1, 768])
+
+# Patch features
+patch_features = pipe(s2=s2, s2_dates=dates, patch_size=10, output="patch", return_tensors=True)
+print(patch_features.shape)  # torch.Size([1, 1024, 768])
+```
+
+### Multimodal example
+
+```python
+pipe = pipeline(task="anysat-feature-extraction", model=MODEL, trust_remote_code=True)
+
+s2 = np.random.randn(1, 4, 10, 32, 32).astype(np.float32)
+s1 = np.random.randn(1, 4, 3, 32, 32).astype(np.float32)
+features = pipe(
+    s2=s2,
+    s2_dates=np.array([[100, 120, 140, 160]]),
+    s1=s1,
+    s1_dates=np.array([[100, 120, 140, 160]]),
+    patch_size=10,
+    pool=True,
+    return_tensors=True,
+)
+```
+
+### Built-in ImageFeatureExtractionPipeline
+
+`AnySatModel` returns `BaseModelOutputWithPooling`, so the registered `image-feature-extraction` task also works:
+
+```python
+pipe = pipeline(
+    task="image-feature-extraction",
+    model=MODEL,
+    trust_remote_code=True,
+)
+```
+
+## Supported modalities
+
+| Key | Type | Channels | Resolution |
+|-----|------|----------|------------|
+| `aerial` | image | 4 | 0.2 m |
+| `aerial-flair` | image | 5 | 0.2 m |
+| `spot` | image | 3 | 1 m |
+| `naip` | image | 4 | 1.25 m |
+| `s2` | time series | 10 | 10 m |
+| `s1-asc`, `s1` | time series | 2–3 | 10 m |
+| `l7`, `l8` | time series | 6–11 | 10–30 m |
+| `alos` | time series | 3 | 30 m |
+| `modis` | time series | 7 | 250 m |
+
+Time-series modalities require `{modality}_dates` (day-of-year, 0–364). Images must be square. `patch_size` is in **meters** (internally divided by 10).
+
+## Output modes
+
+| `output` | Shape | Description |
+|----------|-------|-------------|
+| `tile` | `[B, 768]` | CLS / global embedding (`pool=True`) |
+| `patch` | `[B, H*W, 768]` | Patch tokens |
+| `dense` | high-res map | Dense features for segmentation |
+| `all` | all tokens | Full token sequence |

anysat-base/config.json

@@ -0,0 +1,290 @@
+{
+  "architectures": [
+    "AnySatModel"
+  ],
+  "attn_drop_rate": 0.0,
+  "class_token": true,
+  "default_output": "patch",
+  "default_patch_size": 10,
+  "depth": 6,
+  "drop_path_rate": 0.0,
+  "drop_rate": 0.0,
+  "dtype": "float32",
+  "embed_dim": 768,
+  "flash_attn": false,
+  "hidden_size": 768,
+  "image_modalities": [
+    "aerial",
+    "aerial-flair",
+    "spot",
+    "naip"
+  ],
+  "mlp_ratio": 4.0,
+  "modalities": [
+    "aerial",
+    "aerial-flair",
+    "spot",
+    "naip",
+    "s2",
+    "s1-asc",
+    "s1",
+    "alos",
+    "l7",
+    "l8",
+    "modis"
+  ],
+  "modality_resolutions": {
+    "aerial": 0.2,
+    "aerial-flair": 0.2,
+    "alos": 30,
+    "l7": 30,
+    "l8": 10,
+    "modis": 250,
+    "naip": 1.25,
+    "s1": 10,
+    "s1-asc": 10,
+    "s1-des": 10,
+    "s2": 10,
+    "spot": 1.0
+  },
+  "model_size": "base",
+  "model_type": "anysat",
+  "normalization_stats": null,
+  "num_heads": 12,
+  "patch_drop_rate": 0.0,
+  "pre_norm": false,
+  "projectors": {
+    "aerial": {
+      "bias": false,
+      "embed_dim": 768,
+      "in_chans": 4,
+      "mlp": [
+        768,
+        1536,
+        768
+      ],
+      "patch_size": 10
+    },
+    "aerial-flair": {
+      "bias": false,
+      "embed_dim": 768,
+      "in_chans": 5,
+      "mlp": [
+        768,
+        1536,
+        768
+      ],
+      "patch_size": 10
+    },
+    "alos": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.2,
+      "in_channels": 3,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "reduce_scale": 3,
+      "return_att": false
+    },
+    "l7": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.2,
+      "in_channels": 6,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "reduce_scale": 3,
+      "return_att": false
+    },
+    "l8": {
+      "T": 366,
+      "d_k": 8,
+      "dropout": 0.0,
+      "in_channels": 11,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "return_att": false
+    },
+    "modis": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.2,
+      "in_channels": 7,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "reduce_scale": 12,
+      "return_att": false
+    },
+    "naip": {
+      "bias": false,
+      "embed_dim": 768,
+      "in_chans": 4,
+      "mlp": [
+        768,
+        1536,
+        768
+      ],
+      "patch_size": 8,
+      "resolution": 1.25
+    },
+    "s1": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.2,
+      "in_channels": 3,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "return_att": false
+    },
+    "s1-asc": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.2,
+      "in_channels": 2,
+      "in_norm": false,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "return_att": false
+    },
+    "s2": {
+      "T": 367,
+      "d_k": 8,
+      "dropout": 0.0,
+      "in_channels": 10,
+      "in_norm": true,
+      "mlp": [
+        768
+      ],
+      "mlp_in": [
+        96,
+        384,
+        768,
+        1536,
+        768
+      ],
+      "n_head": 16,
+      "positional_encoding": true,
+      "return_att": false
+    },
+    "spot": {
+      "bias": false,
+      "embed_dim": 768,
+      "in_chans": 3,
+      "mlp": [
+        768,
+        1536,
+        768
+      ],
+      "patch_size": 10,
+      "resolution": 1.0
+    }
+  },
+  "release": true,
+  "spatial_encoder_input_res": {
+    "aerial": 2,
+    "aerial-flair": 2,
+    "alos": 30,
+    "l7": 30,
+    "l8": 10,
+    "modis": 250,
+    "naip": 10,
+    "s1": 10,
+    "s1-asc": 10,
+    "s1-des": 10,
+    "s2": 10,
+    "spot": 10
+  },
+  "time_series_modalities": [
+    "s2",
+    "s1-asc",
+    "s1",
+    "alos",
+    "l7",
+    "l8",
+    "modis"
+  ],
+  "transformers_version": "5.0.0",
+  "auto_map": {
+    "AutoConfig": "modeling_anysat.AnySatConfig",
+    "AutoModel": "modeling_anysat.AnySatModel"
+  },
+  "custom_pipelines": {
+    "anysat-feature-extraction": {
+      "impl": "pipeline_anysat.AnySatImageFeatureExtractionPipeline",
+      "pt": [
+        "AutoModel"
+      ]
+    },
+    "image-feature-extraction": {
+      "impl": "pipeline_anysat.AnySatImageFeatureExtractionPipeline",
+      "pt": [
+        "AutoModel"
+      ]
+    }
+  }
+}

anysat-base/pipeline_anysat.py

@@ -0,0 +1,131 @@
+# Copyright 2025 AnySat Authors and The HuggingFace Inc. team.
+"""Self-contained AnySat feature extraction pipeline for trust_remote_code loading."""
+
+from __future__ import annotations
+
+from typing import Any, Union
+
+import torch
+
+from transformers.pipelines.base import GenericTensor, build_pipeline_init_args
+from transformers.pipelines.image_feature_extraction import ImageFeatureExtractionPipeline
+from transformers.utils import add_end_docstrings
+
+
+@add_end_docstrings(
+    build_pipeline_init_args(has_processor=True),
+    """
+        patch_size (`int`, *optional*, defaults to `10`):
+            Patch size in meters passed to the scale-adaptive encoder.
+        output (`str`, *optional*, defaults to `"patch"`):
+            Output mode. One of `"tile"`, `"patch"`, `"dense"`, or `"all"`.
+        pool (`bool`, *optional*, defaults to `False`):
+            Whether to return pooled tile features instead of patch hidden states.
+    """,
+)
+class AnySatImageFeatureExtractionPipeline(ImageFeatureExtractionPipeline):
+    _load_processor = True
+    _load_image_processor = False
+    _load_feature_extractor = False
+    _load_tokenizer = False
+
+    def _sanitize_parameters(
+        self,
+        processor_kwargs=None,
+        image_processor_kwargs=None,
+        return_tensors=None,
+        pool=None,
+        normalize=None,
+        patch_size=None,
+        output=None,
+        output_modality=None,
+        **kwargs,
+    ):
+        preprocess_params = {}
+        if processor_kwargs is not None:
+            preprocess_params.update(processor_kwargs)
+        if image_processor_kwargs is not None:
+            preprocess_params.update(image_processor_kwargs)
+
+        for modality in (
+            "aerial",
+            "aerial-flair",
+            "spot",
+            "naip",
+            "s2",
+            "s1-asc",
+            "s1",
+            "alos",
+            "l7",
+            "l8",
+            "modis",
+        ):
+            if modality in kwargs:
+                preprocess_params[modality] = kwargs.pop(modality)
+            dates_key = f"{modality}_dates"
+            if dates_key in kwargs:
+                preprocess_params[dates_key] = kwargs.pop(dates_key)
+
+        if normalize is not None:
+            preprocess_params["normalize"] = normalize
+        if patch_size is not None:
+            preprocess_params["patch_size"] = patch_size
+        if output is not None:
+            preprocess_params["output"] = output
+
+        forward_params = {}
+        if output_modality is not None:
+            forward_params["output_modality"] = output_modality
+
+        postprocess_params = {}
+        if pool is not None:
+            postprocess_params["pool"] = pool
+            if pool:
+                preprocess_params.setdefault("output", "tile")
+        if return_tensors is not None:
+            postprocess_params["return_tensors"] = return_tensors
+
+        return preprocess_params, forward_params, postprocess_params
+
+    def preprocess(self, image=None, timeout=None, **processor_kwargs) -> dict[str, GenericTensor]:
+        del image, timeout
+        model_inputs = self.processor(return_tensors="pt", **processor_kwargs)
+        if hasattr(model_inputs, "to"):
+            model_inputs = model_inputs.to(self.device)
+        model_inputs = model_inputs.to(self.dtype)
+        return model_inputs
+
+    def _forward(self, model_inputs, output_modality: str = ""):
+        patch_size = model_inputs.pop("patch_size", None)
+        output = model_inputs.pop("output", None)
+        if torch.is_tensor(patch_size):
+            patch_size = int(patch_size.item())
+        return self.model(
+            patch_size=patch_size,
+            output=output,
+            output_modality=output_modality,
+            **model_inputs,
+        )
+
+    def postprocess(self, model_outputs, pool=None, return_tensors=False):
+        if pool:
+            outputs = model_outputs.pooler_output
+            if outputs is None and model_outputs.last_hidden_state is not None:
+                outputs = model_outputs.last_hidden_state
+                if outputs.ndim > 2:
+                    outputs = outputs[:, 0]
+        else:
+            outputs = model_outputs.last_hidden_state
+            if outputs is None:
+                outputs = model_outputs[0]
+        if return_tensors:
+            return outputs
+        return outputs.detach().cpu().tolist()
+
+    def __call__(self, *args: Union[Any, list[Any]], **kwargs: Any) -> list[Any]:
+        if not args:
+            args = (None,)
+        return super().__call__(*args, **kwargs)
+
+
+__all__ = ["AnySatImageFeatureExtractionPipeline"]

anysat-base/preprocessor_config.json

@@ -0,0 +1,23 @@
+{
+  "processor_class": "AnySatProcessor",
+  "normalize": true,
+  "patch_size": 10,
+  "output": "patch",
+  "modalities": [
+    "aerial",
+    "aerial-flair",
+    "spot",
+    "naip",
+    "s2",
+    "s1-asc",
+    "s1",
+    "alos",
+    "l7",
+    "l8",
+    "modis"
+  ],
+  "normalization_stats": null,
+  "auto_map": {
+    "AutoProcessor": "processing_anysat.AnySatProcessor"
+  }
+}

anysat-base/processing_anysat.py

@@ -0,0 +1,100 @@
+# Copyright 2025 AnySat Authors and The HuggingFace Inc. team.
+"""Self-contained AnySat processor for trust_remote_code loading."""
+
+from __future__ import annotations
+
+from typing import Any, Optional, Union
+
+import numpy as np
+import torch
+
+from transformers.feature_extraction_utils import BatchFeature
+from transformers.processing_utils import ProcessorMixin
+from transformers.utils import TensorType
+
+
+def _to_numpy(value):
+    if torch.is_tensor(value):
+        return value.detach().cpu().numpy()
+    return np.asarray(value)
+
+
+def _normalize_modality(array, mean, std):
+    if mean is None or std is None:
+        return array.astype(np.float32)
+    mean_arr = np.asarray(mean, dtype=np.float32)
+    std_arr = np.asarray(std, dtype=np.float32)
+    if array.ndim >= 3 and array.shape[-3] <= mean_arr.shape[0]:
+        channels = array.shape[-3]
+        mean_arr = mean_arr[:channels]
+        std_arr = std_arr[:channels]
+        reshape = (1,) * (array.ndim - 3) + (channels,) + (1,) * 2
+        mean_arr = mean_arr.reshape(reshape)
+        std_arr = std_arr.reshape(reshape)
+    return ((array - mean_arr) / std_arr).astype(np.float32)
+
+
+class AnySatProcessor(ProcessorMixin):
+    attributes = []
+
+    def __init__(
+        self,
+        normalize=None,
+        patch_size=None,
+        output=None,
+        normalization_stats=None,
+        modalities=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        self.normalize = normalize
+        self.patch_size = patch_size
+        self.output = output
+        self.normalization_stats = normalization_stats
+        self.modalities = modalities
+
+    def __call__(
+        self,
+        return_tensors: Optional[Union[str, TensorType]] = None,
+        normalize: Optional[bool] = None,
+        patch_size: Optional[int] = None,
+        output: Optional[str] = None,
+        **kwargs: Any,
+    ) -> BatchFeature:
+        normalize = self.normalize if normalize is None else normalize
+        patch_size = patch_size if patch_size is not None else self.patch_size
+        output = output if output is not None else self.output
+        data = {"patch_size": patch_size, "output": output}
+
+        for key, value in kwargs.items():
+            if value is None:
+                continue
+            if key.endswith("_dates"):
+                dates = _to_numpy(value)
+                if dates.ndim == 1:
+                    dates = dates[np.newaxis, ...]
+                data[key] = dates.astype(np.int64)
+                continue
+            if self.modalities and key in self.modalities:
+                array = _to_numpy(value).astype(np.float32)
+                if array.ndim == 3:
+                    array = array[np.newaxis, ...]
+                if normalize and self.normalization_stats and key in self.normalization_stats:
+                    stats = self.normalization_stats[key]
+                    array = _normalize_modality(array, stats.get("mean"), stats.get("std"))
+                data[key] = array
+                continue
+            data[key] = value
+
+        if return_tensors == TensorType.PYTORCH:
+            for key, value in list(data.items()):
+                if key in {"patch_size", "output"}:
+                    continue
+                if not torch.is_tensor(value):
+                    dtype = torch.long if key.endswith("_dates") else torch.float32
+                    data[key] = torch.as_tensor(value, dtype=dtype)
+
+        return BatchFeature(data=data, tensor_type=return_tensors)
+
+
+__all__ = ["AnySatProcessor"]