TSRDA / PASTIS /main_method /code /TESSERA_UPSTREAM_MODEL_CARD.md
Dhruv1000's picture
Move PASTIS work into PASTIS/ (3518/3518)
0ad6e78 verified
|
Raw History Blame Contribute Delete
12.8 kB
---
language:
- en
library_name: pytorch
license: cc0-1.0
license_name: cc0
license_link: https://creativecommons.org/publicdomain/zero/1.0/
pipeline_tag: image-feature-extraction
tags:
- geospatial
- earth-observation
- remote-sensing
- foundation-model
- sentinel-1
- sentinel-2
- sar
- multispectral
- temporal
- pixel-embedding
- self-supervised-learning
- barlow-twins
- knowledge-distillation
- matryoshka
provider: University of Cambridge / ucam-eo
funder: UKRI MR/Z505456/1; AIRR DAWN; AMD; Vultr; AIRR Isambard AI; STFC Durham DiRAC
HPC Facility; Microsoft AI For Good Lab; Tarides; Jane Street; Google; Dr. Robert
Sansom; John Bernstein
embedding_spatial_types:
- pixel
embedding_temporal_type:
- multi-date
embedding_spatial_context: spatial context determined by embedding spatial type
embedding_temporal_context: annual multi-temporal context from valid Sentinel-1 and
Sentinel-2 observations for the same pixel
embedding_dimension: 128
description: TESSERA v2 Small (7.11M parameters) is a compact pixel-wise geospatial
foundation encoder for multimodal Sentinel-1/2 time series, distilled from the 2.06-billion-parameter
TESSERA v2 2B teacher (geotessera/TESSERA-V-2.0-2B-Teacher). It emits a 128-dimensional
Matryoshka embedding whose first K coordinates are independently usable for K in
{16, 32, 64, 128}, so storage can be traded against accuracy without retraining.
compression: 'Native output is fp32 128-d. Two orthogonal compression paths are supported:
(1) Matryoshka truncation -- keep the first 16/32/64 dimensions with no retraining;
(2) linear int8 quantization via the bundled quantize.py, storing an int8 array
plus a float32 per-pixel scale, reconstructed as int8.astype(float32) * scales[...,
None].'
intention: Scientific research and downstream geospatial analysis, including environmental
monitoring, conservation, habitat mapping, carbon and biomass estimation, agricultural
monitoring, crop mapping, food security analysis, climate change research, and sustainable
land-use applications.
cautions: TESSERA embeddings are annual, 10m spectral-temporal representations, not
raw imagery and not real-time monitoring data. Accuracy may degrade in regions or
years with very few valid observations, and users should validate performance for
their own task and geography. The Sentinel-2 input channel order used by this model
is NOT the conventional ascending-wavelength order -- see the Input contract section.
Benchmark scores reported in the TESSERA v2 preprint were measured on students distilled
from the 1B teacher and do not describe these checkpoints; numbers for the 2B-distilled
students will be published separately.
precomputed_embeddings: 'no'
publication_link: https://arxiv.org/abs/2607.03949
model_architecture: Two per-modality backbones (Sentinel-2 and merged Sentinel-1),
each an MLP band embedding plus sinusoidal day-of-year positional encoding, a 4-layer
post-LN Transformer encoder (d_model=256, 4 heads, FFN=1024, ReLU) and single-head
softmax attention pooling over time; concat fusion into an MLP dim_reducer ending
in a non-affine LayerNorm that yields a scale-locked 128-d output. QK-norm is disabled.
distillation:
teacher: TESSERA v2 2B pixel-wise teacher (2,064,266,242 parameters, 1024-d output)
β€” geotessera/TESSERA-V-2.0-2B-Teacher
objective: Matryoshka distillation -- a separate linear head per prefix length K
in {16, 32, 64, 128} reconstructs the full frozen teacher embedding from the student's
first K coordinates, which imposes an ordering on the student's coordinates that
self-supervision alone cannot supply.
projector: training-only; discarded at inference
student_parameters: 7112322
inference:
data_types:
- multispectral
- SAR
product_names:
- Sentinel-2 L2A
- Sentinel-1 RTC
patch_size: 1
temporal_context: annual multi-date time series; every valid observation per pixel
is used and the observation count is bucketized to the nearest bin in 8, 16, ...,
256
---
# TESSERA v2 β€” Small pixel student (7.11 M)
A compact per-pixel encoder for Sentinel-2 + Sentinel-1 satellite time series,
**distilled from the [TESSERA v2 2B teacher](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher)**. It maps one
pixel's annual observation history to a **128-dimensional Matryoshka
embedding**: the first K dimensions are independently usable for
K ∈ {16, 32, 64, 128}, so you can truncate to 16/32/64 dims with no
retraining and no separate checkpoint.
Part of the TESSERA v2 release described in
[*TESSERA v2: Scaling Pixel-wise Earth Foundation Models*](https://arxiv.org/abs/2607.03949).
## Where this model comes from
The teacher β€” [`geotessera/TESSERA-V-2.0-2B-Teacher`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher) β€” is a
2,064,266,242-parameter pixel-wise encoder producing 1024-d representations.
Evaluating 2.06 B parameters per pixel makes it impractical to deploy at tile
or global scale, so it is released mainly as a distillation target. This
student is one of four compact encoders distilled from it for actual use.
Distillation uses a **Matryoshka objective**: a separate linear head per prefix
length K ∈ {16, 32, 64, 128} reconstructs the *full* frozen teacher embedding
from the student's first K coordinates. That is what imposes an ordering on the
student's coordinates β€” a redundancy-reduction objective like Barlow Twins
identifies subspaces only up to a rotation, so self-supervised prefix losses
alone cannot produce a usable nesting. The projection heads are training-only
and are discarded at inference; this checkpoint contains the encoder alone.
> **Note on normalization.** The students z-score Sentinel-1 ascending and
> descending with **their own per-source statistics before merging** the two
> into one stream. The teacher instead merges them in raw units and applies a
> single set of pooled statistics. If you move data between the two, do not
> carry the normalization across.
## The family
| Variant | Repo | Parameters | `latent_dim` | `d_model` | Layers | FFN |
|---|---|---|---|---|---|---|
| Nano | [`geotessera/TESSERA-V-2.0-2B-N`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-N) | 1.07 M | 36 | 144 | 2 | 384 |
| Small | [`geotessera/TESSERA-V-2.0-2B-S`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-S) | 7.11 M | 64 | 256 | 4 | 1024 |
| Medium | [`geotessera/TESSERA-V-2.0-2B-M`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-M) | 21.03 M | 110 | 440 | 4 | 1792 |
| Large | [`geotessera/TESSERA-V-2.0-2B-L`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-L) | 43.83 M | 160 | 640 | 4 | 2560 |
All four share identical inference code and the same 128-d Matryoshka output
contract β€” only capacity differs. Nano targets edge/on-device use, Medium is
the balanced default, Large is intended for provider-side global inference.
## Architecture
| `latent_dim` | `d_model` | layers | heads | FFN | output | parameters |
|---|---|---|---|---|---|---|
| 64 | 256 | 4 | 4 | 1024 | 128 (Matryoshka) | **7,112,322** |
Two per-modality backbones (Sentinel-2, and ascending+descending Sentinel-1
merged into one stream). Each backbone is an MLP band embedding plus a
sinusoidal day-of-year positional encoding, a post-LN Transformer encoder
(ReLU FFN, QK-norm **off**), and single-head softmax attention pooling over
time. The two pooled vectors are concatenated and passed through an MLP
`dim_reducer` that ends in a **non-affine LayerNorm** β€” so every output
embedding is exactly mean 0 / std 1 across its 128 dimensions.
## Input contract
> [!IMPORTANT]
> **The Sentinel-2 channel order is not the conventional ascending-wavelength
> order.** Feeding bands in the usual B02β†’B12 order will silently produce
> garbage embeddings. The model expects exactly:
>
> ```
> B04 B02 B03 B08 B8A B05 B06 B07 B11 B12
> ```
>
> Sentinel-1 is `VV VH`, ascending and descending concatenated along time.
- **Sentinel-2**: `(T, 10)` per pixel, raw L2A reflectance (unscaled DN), plus a
`(T,)` day-of-year array and a `(T,)` validity mask (1 = clear, 0 = cloud).
- **Sentinel-1**: `(T, 2)` per pixel, raw RTC values, plus day-of-year.
All-zero timesteps are treated as missing.
- Day-of-year is passed as a **raw integer 1–365** (not normalized).
`infer.py` performs standardization internally (`standardize=True`, the
default): Sentinel-2 uses per-band statistics, and Sentinel-1 ascending and
descending each use **their own** statistics, applied *before* the two are
merged into a single stream. Pass `standardize=False` only if your arrays are
already z-scored with those exact constants (they are in `model.py`).
Per pixel, the number of valid observations is bucketized to the nearest bin in
`{8, 16, 24, ..., 256}`, then padded or subsampled to that bin size β€” matching
the training-time procedure. Pixels sharing a bin are batched together.
## Usage
```python
from huggingface_hub import snapshot_download
import sys, torch, numpy as np
path = snapshot_download("geotessera/TESSERA-V-2.0-2B-S")
sys.path.insert(0, path)
from model import load_model
from infer import encode_tile, encode_pixels
from quantize import quantize as quantize_int8
model = load_model(f"{path}/ckpt/student_small.pt", torch.device("cuda"))
# One tile -> (H, W, 128) embedding map.
# s2_bands (T,H,W,10) raw reflectance, s2_doys (T,), s2_masks (T,H,W) 1=valid
# s1_asc / s1_desc (T,H,W,2) raw + their own doys
emb = encode_tile(
model, s2_bands, s2_doys, s2_masks=s2_masks,
s1_asc_bands=s1_asc, s1_asc_doys=s1_asc_doys,
s1_desc_bands=s1_desc, s1_desc_doys=s1_desc_doys,
batch_pixels=4096, device=torch.device("cuda"),
) # -> (H, W, 128) float32
emb16 = emb[..., :16] # Matryoshka truncation, 1/8 the storage
code, scale = quantize_int8(emb) # int8 (H,W,128) + per-pixel float32 scale
recon = code.astype("float32") * scale[..., None]
```
For per-pixel time series rather than a gridded tile, call `encode_pixels()` on
`(N, T, C)` arrays β€” same keyword arguments, returns `(N, 128)`.
## Files
```
TESSERA-V-2.0-2B-S/
β”œβ”€β”€ model.py PixelStudent + load_model() (torch + numpy only)
β”œβ”€β”€ infer.py encode_pixels() / encode_tile()
β”œβ”€β”€ quantize.py linear int8 quantize() / dequantize()
β”œβ”€β”€ __init__.py package-style re-exports
β”œβ”€β”€ requirements.txt torch>=2.0, numpy>=1.21
└── ckpt/student_small.pt
```
The checkpoint is a plain `torch.save` payload with two keys, `model` (the
`state_dict`) and `args` (the architecture config `load_model` reads). No
custom classes are pickled, and nothing outside `torch` and `numpy` is needed.
## Intended use and limitations
Intended for scientific research and downstream geospatial analysis:
environmental monitoring, conservation and habitat mapping, biomass and carbon
estimation, crop mapping and agricultural monitoring, land-use and land-cover
work.
Limitations:
- Embeddings are **annual, 10 m spectral-temporal representations** β€” not raw
imagery, and not a real-time monitoring product.
- Quality degrades where a pixel has very few valid observations (persistent
cloud, sparse revisit).
- Validate on your own task and geography before operational use.
- **No benchmark scores are quoted here.** The results in the TESSERA v2
preprint were measured on students distilled from the **1B** teacher; these
checkpoints are distilled from the **2B** teacher and their evaluation will
be published separately.
## License
Released under **CC0 1.0** (public domain dedication), matching the rest of the
TESSERA release. Use of TESSERA is additionally governed by the project's
[Acceptable Use Policy](https://github.com/ucam-eo/tessera/blob/master/aup.md).
## Citation
```bibtex
@article{tessera_v2_2026,
title = {TESSERA v2: Scaling Pixel-wise Earth Foundation Models},
author = {Feng, Zhengpeng and Jaffer, Sadiq and Shokar, Ira and
Knezevic, Jovana and Elvers, Mark and Atzberger, Clement and
Young, Robin and Naik, Aneesh and Robinson, Niall and
Blake, Andrew and Coomes, David and Madhavapeddy, Anil and
Keshav, Srinivasan},
journal = {arXiv preprint arXiv:2607.03949},
year = {2026},
url = {https://arxiv.org/abs/2607.03949}
}
```
## Links
- Teacher model: [`geotessera/TESSERA-V-2.0-2B-Teacher`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher)
- Code and releases: https://github.com/ucam-eo/tessera
- Preprint: https://arxiv.org/abs/2607.03949
- Project website: https://geotessera.org/
- Earlier releases: [`TESSERA-V-1.0`](https://huggingface.co/geotessera/TESSERA-V-1.0), [`TESSERA-V-1.1`](https://huggingface.co/geotessera/TESSERA-V-1.1)