TSRDA / PASTIS /main_method /code /TESSERA_UPSTREAM_MODEL_CARD.md
Dhruv1000's picture
Move PASTIS work into PASTIS/ (3518/3518)
0ad6e78 verified
|
Raw History Blame Contribute Delete
12.8 kB
metadata
language:
  - en
library_name: pytorch
license: cc0-1.0
license_name: cc0
license_link: https://creativecommons.org/publicdomain/zero/1.0/
pipeline_tag: image-feature-extraction
tags:
  - geospatial
  - earth-observation
  - remote-sensing
  - foundation-model
  - sentinel-1
  - sentinel-2
  - sar
  - multispectral
  - temporal
  - pixel-embedding
  - self-supervised-learning
  - barlow-twins
  - knowledge-distillation
  - matryoshka
provider: University of Cambridge / ucam-eo
funder: >-
  UKRI MR/Z505456/1; AIRR DAWN; AMD; Vultr; AIRR Isambard AI; STFC Durham DiRAC
  HPC Facility; Microsoft AI For Good Lab; Tarides; Jane Street; Google; Dr.
  Robert Sansom; John Bernstein
embedding_spatial_types:
  - pixel
embedding_temporal_type:
  - multi-date
embedding_spatial_context: spatial context determined by embedding spatial type
embedding_temporal_context: >-
  annual multi-temporal context from valid Sentinel-1 and Sentinel-2
  observations for the same pixel
embedding_dimension: 128
description: >-
  TESSERA v2 Small (7.11M parameters) is a compact pixel-wise geospatial
  foundation encoder for multimodal Sentinel-1/2 time series, distilled from the
  2.06-billion-parameter TESSERA v2 2B teacher
  (geotessera/TESSERA-V-2.0-2B-Teacher). It emits a 128-dimensional Matryoshka
  embedding whose first K coordinates are independently usable for K in {16, 32,
  64, 128}, so storage can be traded against accuracy without retraining.
compression: >-
  Native output is fp32 128-d. Two orthogonal compression paths are supported:
  (1) Matryoshka truncation -- keep the first 16/32/64 dimensions with no
  retraining; (2) linear int8 quantization via the bundled quantize.py, storing
  an int8 array plus a float32 per-pixel scale, reconstructed as
  int8.astype(float32) * scales[..., None].
intention: >-
  Scientific research and downstream geospatial analysis, including
  environmental monitoring, conservation, habitat mapping, carbon and biomass
  estimation, agricultural monitoring, crop mapping, food security analysis,
  climate change research, and sustainable land-use applications.
cautions: >-
  TESSERA embeddings are annual, 10m spectral-temporal representations, not raw
  imagery and not real-time monitoring data. Accuracy may degrade in regions or
  years with very few valid observations, and users should validate performance
  for their own task and geography. The Sentinel-2 input channel order used by
  this model is NOT the conventional ascending-wavelength order -- see the Input
  contract section. Benchmark scores reported in the TESSERA v2 preprint were
  measured on students distilled from the 1B teacher and do not describe these
  checkpoints; numbers for the 2B-distilled students will be published
  separately.
precomputed_embeddings: 'no'
publication_link: https://arxiv.org/abs/2607.03949
model_architecture: >-
  Two per-modality backbones (Sentinel-2 and merged Sentinel-1), each an MLP
  band embedding plus sinusoidal day-of-year positional encoding, a 4-layer
  post-LN Transformer encoder (d_model=256, 4 heads, FFN=1024, ReLU) and
  single-head softmax attention pooling over time; concat fusion into an MLP
  dim_reducer ending in a non-affine LayerNorm that yields a scale-locked 128-d
  output. QK-norm is disabled.
distillation:
  teacher: >-
    TESSERA v2 2B pixel-wise teacher (2,064,266,242 parameters, 1024-d output) β€”
    geotessera/TESSERA-V-2.0-2B-Teacher
  objective: >-
    Matryoshka distillation -- a separate linear head per prefix length K in
    {16, 32, 64, 128} reconstructs the full frozen teacher embedding from the
    student's first K coordinates, which imposes an ordering on the student's
    coordinates that self-supervision alone cannot supply.
  projector: training-only; discarded at inference
  student_parameters: 7112322
inference:
  data_types:
    - multispectral
    - SAR
  product_names:
    - Sentinel-2 L2A
    - Sentinel-1 RTC
  patch_size: 1
  temporal_context: >-
    annual multi-date time series; every valid observation per pixel is used and
    the observation count is bucketized to the nearest bin in 8, 16, ..., 256

TESSERA v2 β€” Small pixel student (7.11 M)

A compact per-pixel encoder for Sentinel-2 + Sentinel-1 satellite time series, distilled from the TESSERA v2 2B teacher. It maps one pixel's annual observation history to a 128-dimensional Matryoshka embedding: the first K dimensions are independently usable for K ∈ {16, 32, 64, 128}, so you can truncate to 16/32/64 dims with no retraining and no separate checkpoint.

Part of the TESSERA v2 release described in TESSERA v2: Scaling Pixel-wise Earth Foundation Models.

Where this model comes from

The teacher β€” geotessera/TESSERA-V-2.0-2B-Teacher β€” is a 2,064,266,242-parameter pixel-wise encoder producing 1024-d representations. Evaluating 2.06 B parameters per pixel makes it impractical to deploy at tile or global scale, so it is released mainly as a distillation target. This student is one of four compact encoders distilled from it for actual use.

Distillation uses a Matryoshka objective: a separate linear head per prefix length K ∈ {16, 32, 64, 128} reconstructs the full frozen teacher embedding from the student's first K coordinates. That is what imposes an ordering on the student's coordinates β€” a redundancy-reduction objective like Barlow Twins identifies subspaces only up to a rotation, so self-supervised prefix losses alone cannot produce a usable nesting. The projection heads are training-only and are discarded at inference; this checkpoint contains the encoder alone.

Note on normalization. The students z-score Sentinel-1 ascending and descending with their own per-source statistics before merging the two into one stream. The teacher instead merges them in raw units and applies a single set of pooled statistics. If you move data between the two, do not carry the normalization across.

The family

Variant Repo Parameters latent_dim d_model Layers FFN
Nano geotessera/TESSERA-V-2.0-2B-N 1.07 M 36 144 2 384
Small geotessera/TESSERA-V-2.0-2B-S 7.11 M 64 256 4 1024
Medium geotessera/TESSERA-V-2.0-2B-M 21.03 M 110 440 4 1792
Large geotessera/TESSERA-V-2.0-2B-L 43.83 M 160 640 4 2560

All four share identical inference code and the same 128-d Matryoshka output contract β€” only capacity differs. Nano targets edge/on-device use, Medium is the balanced default, Large is intended for provider-side global inference.

Architecture

latent_dim d_model layers heads FFN output parameters
64 256 4 4 1024 128 (Matryoshka) 7,112,322

Two per-modality backbones (Sentinel-2, and ascending+descending Sentinel-1 merged into one stream). Each backbone is an MLP band embedding plus a sinusoidal day-of-year positional encoding, a post-LN Transformer encoder (ReLU FFN, QK-norm off), and single-head softmax attention pooling over time. The two pooled vectors are concatenated and passed through an MLP dim_reducer that ends in a non-affine LayerNorm β€” so every output embedding is exactly mean 0 / std 1 across its 128 dimensions.

Input contract

The Sentinel-2 channel order is not the conventional ascending-wavelength order. Feeding bands in the usual B02β†’B12 order will silently produce garbage embeddings. The model expects exactly:

B04  B02  B03  B08  B8A  B05  B06  B07  B11  B12

Sentinel-1 is VV VH, ascending and descending concatenated along time.

  • Sentinel-2: (T, 10) per pixel, raw L2A reflectance (unscaled DN), plus a (T,) day-of-year array and a (T,) validity mask (1 = clear, 0 = cloud).
  • Sentinel-1: (T, 2) per pixel, raw RTC values, plus day-of-year. All-zero timesteps are treated as missing.
  • Day-of-year is passed as a raw integer 1–365 (not normalized).

infer.py performs standardization internally (standardize=True, the default): Sentinel-2 uses per-band statistics, and Sentinel-1 ascending and descending each use their own statistics, applied before the two are merged into a single stream. Pass standardize=False only if your arrays are already z-scored with those exact constants (they are in model.py).

Per pixel, the number of valid observations is bucketized to the nearest bin in {8, 16, 24, ..., 256}, then padded or subsampled to that bin size β€” matching the training-time procedure. Pixels sharing a bin are batched together.

Usage

from huggingface_hub import snapshot_download
import sys, torch, numpy as np

path = snapshot_download("geotessera/TESSERA-V-2.0-2B-S")
sys.path.insert(0, path)

from model import load_model
from infer import encode_tile, encode_pixels
from quantize import quantize as quantize_int8

model = load_model(f"{path}/ckpt/student_small.pt", torch.device("cuda"))

# One tile -> (H, W, 128) embedding map.
#   s2_bands (T,H,W,10) raw reflectance, s2_doys (T,), s2_masks (T,H,W) 1=valid
#   s1_asc / s1_desc  (T,H,W,2) raw + their own doys
emb = encode_tile(
    model, s2_bands, s2_doys, s2_masks=s2_masks,
    s1_asc_bands=s1_asc, s1_asc_doys=s1_asc_doys,
    s1_desc_bands=s1_desc, s1_desc_doys=s1_desc_doys,
    batch_pixels=4096, device=torch.device("cuda"),
)                                    # -> (H, W, 128) float32

emb16 = emb[..., :16]                # Matryoshka truncation, 1/8 the storage

code, scale = quantize_int8(emb)     # int8 (H,W,128) + per-pixel float32 scale
recon = code.astype("float32") * scale[..., None]

For per-pixel time series rather than a gridded tile, call encode_pixels() on (N, T, C) arrays β€” same keyword arguments, returns (N, 128).

Files

TESSERA-V-2.0-2B-S/
β”œβ”€β”€ model.py          PixelStudent + load_model()  (torch + numpy only)
β”œβ”€β”€ infer.py          encode_pixels() / encode_tile()
β”œβ”€β”€ quantize.py       linear int8 quantize() / dequantize()
β”œβ”€β”€ __init__.py       package-style re-exports
β”œβ”€β”€ requirements.txt  torch>=2.0, numpy>=1.21
└── ckpt/student_small.pt

The checkpoint is a plain torch.save payload with two keys, model (the state_dict) and args (the architecture config load_model reads). No custom classes are pickled, and nothing outside torch and numpy is needed.

Intended use and limitations

Intended for scientific research and downstream geospatial analysis: environmental monitoring, conservation and habitat mapping, biomass and carbon estimation, crop mapping and agricultural monitoring, land-use and land-cover work.

Limitations:

  • Embeddings are annual, 10 m spectral-temporal representations β€” not raw imagery, and not a real-time monitoring product.
  • Quality degrades where a pixel has very few valid observations (persistent cloud, sparse revisit).
  • Validate on your own task and geography before operational use.
  • No benchmark scores are quoted here. The results in the TESSERA v2 preprint were measured on students distilled from the 1B teacher; these checkpoints are distilled from the 2B teacher and their evaluation will be published separately.

License

Released under CC0 1.0 (public domain dedication), matching the rest of the TESSERA release. Use of TESSERA is additionally governed by the project's Acceptable Use Policy.

Citation

@article{tessera_v2_2026,
  title   = {TESSERA v2: Scaling Pixel-wise Earth Foundation Models},
  author  = {Feng, Zhengpeng and Jaffer, Sadiq and Shokar, Ira and
             Knezevic, Jovana and Elvers, Mark and Atzberger, Clement and
             Young, Robin and Naik, Aneesh and Robinson, Niall and
             Blake, Andrew and Coomes, David and Madhavapeddy, Anil and
             Keshav, Srinivasan},
  journal = {arXiv preprint arXiv:2607.03949},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.03949}
}

Links