|
Download PASTIS/main_method/code/TESSERA_UPSTREAM_MODEL_CARD.md from Dhruv1000/TSRDA: direct link, hf CLI and curl.
- Browser
- Download file 12.8 kB
-
https://huggingface.co/Dhruv1000/TSRDA/resolve/main/PASTIS/main_method/code/TESSERA_UPSTREAM_MODEL_CARD.md
- Command line
-
hf download hf://Dhruv1000/TSRDA/PASTIS/main_method/code/TESSERA_UPSTREAM_MODEL_CARD.md
-
curl -L -o TESSERA_UPSTREAM_MODEL_CARD.md https://huggingface.co/Dhruv1000/TSRDA/resolve/main/PASTIS/main_method/code/TESSERA_UPSTREAM_MODEL_CARD.md
12.8 kB
| language: | |
| - en | |
| library_name: pytorch | |
| license: cc0-1.0 | |
| license_name: cc0 | |
| license_link: https://creativecommons.org/publicdomain/zero/1.0/ | |
| pipeline_tag: image-feature-extraction | |
| tags: | |
| - geospatial | |
| - earth-observation | |
| - remote-sensing | |
| - foundation-model | |
| - sentinel-1 | |
| - sentinel-2 | |
| - sar | |
| - multispectral | |
| - temporal | |
| - pixel-embedding | |
| - self-supervised-learning | |
| - barlow-twins | |
| - knowledge-distillation | |
| - matryoshka | |
| provider: University of Cambridge / ucam-eo | |
| funder: UKRI MR/Z505456/1; AIRR DAWN; AMD; Vultr; AIRR Isambard AI; STFC Durham DiRAC | |
| HPC Facility; Microsoft AI For Good Lab; Tarides; Jane Street; Google; Dr. Robert | |
| Sansom; John Bernstein | |
| embedding_spatial_types: | |
| - pixel | |
| embedding_temporal_type: | |
| - multi-date | |
| embedding_spatial_context: spatial context determined by embedding spatial type | |
| embedding_temporal_context: annual multi-temporal context from valid Sentinel-1 and | |
| Sentinel-2 observations for the same pixel | |
| embedding_dimension: 128 | |
| description: TESSERA v2 Small (7.11M parameters) is a compact pixel-wise geospatial | |
| foundation encoder for multimodal Sentinel-1/2 time series, distilled from the 2.06-billion-parameter | |
| TESSERA v2 2B teacher (geotessera/TESSERA-V-2.0-2B-Teacher). It emits a 128-dimensional | |
| Matryoshka embedding whose first K coordinates are independently usable for K in | |
| {16, 32, 64, 128}, so storage can be traded against accuracy without retraining. | |
| compression: 'Native output is fp32 128-d. Two orthogonal compression paths are supported: | |
| (1) Matryoshka truncation -- keep the first 16/32/64 dimensions with no retraining; | |
| (2) linear int8 quantization via the bundled quantize.py, storing an int8 array | |
| plus a float32 per-pixel scale, reconstructed as int8.astype(float32) * scales[..., | |
| None].' | |
| intention: Scientific research and downstream geospatial analysis, including environmental | |
| monitoring, conservation, habitat mapping, carbon and biomass estimation, agricultural | |
| monitoring, crop mapping, food security analysis, climate change research, and sustainable | |
| land-use applications. | |
| cautions: TESSERA embeddings are annual, 10m spectral-temporal representations, not | |
| raw imagery and not real-time monitoring data. Accuracy may degrade in regions or | |
| years with very few valid observations, and users should validate performance for | |
| their own task and geography. The Sentinel-2 input channel order used by this model | |
| is NOT the conventional ascending-wavelength order -- see the Input contract section. | |
| Benchmark scores reported in the TESSERA v2 preprint were measured on students distilled | |
| from the 1B teacher and do not describe these checkpoints; numbers for the 2B-distilled | |
| students will be published separately. | |
| precomputed_embeddings: 'no' | |
| publication_link: https://arxiv.org/abs/2607.03949 | |
| model_architecture: Two per-modality backbones (Sentinel-2 and merged Sentinel-1), | |
| each an MLP band embedding plus sinusoidal day-of-year positional encoding, a 4-layer | |
| post-LN Transformer encoder (d_model=256, 4 heads, FFN=1024, ReLU) and single-head | |
| softmax attention pooling over time; concat fusion into an MLP dim_reducer ending | |
| in a non-affine LayerNorm that yields a scale-locked 128-d output. QK-norm is disabled. | |
| distillation: | |
| teacher: TESSERA v2 2B pixel-wise teacher (2,064,266,242 parameters, 1024-d output) | |
| β geotessera/TESSERA-V-2.0-2B-Teacher | |
| objective: Matryoshka distillation -- a separate linear head per prefix length K | |
| in {16, 32, 64, 128} reconstructs the full frozen teacher embedding from the student's | |
| first K coordinates, which imposes an ordering on the student's coordinates that | |
| self-supervision alone cannot supply. | |
| projector: training-only; discarded at inference | |
| student_parameters: 7112322 | |
| inference: | |
| data_types: | |
| - multispectral | |
| - SAR | |
| product_names: | |
| - Sentinel-2 L2A | |
| - Sentinel-1 RTC | |
| patch_size: 1 | |
| temporal_context: annual multi-date time series; every valid observation per pixel | |
| is used and the observation count is bucketized to the nearest bin in 8, 16, ..., | |
| 256 | |
| # TESSERA v2 β Small pixel student (7.11 M) | |
| A compact per-pixel encoder for Sentinel-2 + Sentinel-1 satellite time series, | |
| **distilled from the [TESSERA v2 2B teacher](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher)**. It maps one | |
| pixel's annual observation history to a **128-dimensional Matryoshka | |
| embedding**: the first K dimensions are independently usable for | |
| K β {16, 32, 64, 128}, so you can truncate to 16/32/64 dims with no | |
| retraining and no separate checkpoint. | |
| Part of the TESSERA v2 release described in | |
| [*TESSERA v2: Scaling Pixel-wise Earth Foundation Models*](https://arxiv.org/abs/2607.03949). | |
| ## Where this model comes from | |
| The teacher β [`geotessera/TESSERA-V-2.0-2B-Teacher`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher) β is a | |
| 2,064,266,242-parameter pixel-wise encoder producing 1024-d representations. | |
| Evaluating 2.06 B parameters per pixel makes it impractical to deploy at tile | |
| or global scale, so it is released mainly as a distillation target. This | |
| student is one of four compact encoders distilled from it for actual use. | |
| Distillation uses a **Matryoshka objective**: a separate linear head per prefix | |
| length K β {16, 32, 64, 128} reconstructs the *full* frozen teacher embedding | |
| from the student's first K coordinates. That is what imposes an ordering on the | |
| student's coordinates β a redundancy-reduction objective like Barlow Twins | |
| identifies subspaces only up to a rotation, so self-supervised prefix losses | |
| alone cannot produce a usable nesting. The projection heads are training-only | |
| and are discarded at inference; this checkpoint contains the encoder alone. | |
| > **Note on normalization.** The students z-score Sentinel-1 ascending and | |
| > descending with **their own per-source statistics before merging** the two | |
| > into one stream. The teacher instead merges them in raw units and applies a | |
| > single set of pooled statistics. If you move data between the two, do not | |
| > carry the normalization across. | |
| ## The family | |
| | Variant | Repo | Parameters | `latent_dim` | `d_model` | Layers | FFN | | |
| |---|---|---|---|---|---|---| | |
| | Nano | [`geotessera/TESSERA-V-2.0-2B-N`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-N) | 1.07 M | 36 | 144 | 2 | 384 | | |
| | Small | [`geotessera/TESSERA-V-2.0-2B-S`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-S) | 7.11 M | 64 | 256 | 4 | 1024 | | |
| | Medium | [`geotessera/TESSERA-V-2.0-2B-M`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-M) | 21.03 M | 110 | 440 | 4 | 1792 | | |
| | Large | [`geotessera/TESSERA-V-2.0-2B-L`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-L) | 43.83 M | 160 | 640 | 4 | 2560 | | |
| All four share identical inference code and the same 128-d Matryoshka output | |
| contract β only capacity differs. Nano targets edge/on-device use, Medium is | |
| the balanced default, Large is intended for provider-side global inference. | |
| ## Architecture | |
| | `latent_dim` | `d_model` | layers | heads | FFN | output | parameters | | |
| |---|---|---|---|---|---|---| | |
| | 64 | 256 | 4 | 4 | 1024 | 128 (Matryoshka) | **7,112,322** | | |
| Two per-modality backbones (Sentinel-2, and ascending+descending Sentinel-1 | |
| merged into one stream). Each backbone is an MLP band embedding plus a | |
| sinusoidal day-of-year positional encoding, a post-LN Transformer encoder | |
| (ReLU FFN, QK-norm **off**), and single-head softmax attention pooling over | |
| time. The two pooled vectors are concatenated and passed through an MLP | |
| `dim_reducer` that ends in a **non-affine LayerNorm** β so every output | |
| embedding is exactly mean 0 / std 1 across its 128 dimensions. | |
| ## Input contract | |
| > [!IMPORTANT] | |
| > **The Sentinel-2 channel order is not the conventional ascending-wavelength | |
| > order.** Feeding bands in the usual B02βB12 order will silently produce | |
| > garbage embeddings. The model expects exactly: | |
| > | |
| > ``` | |
| > B04 B02 B03 B08 B8A B05 B06 B07 B11 B12 | |
| > ``` | |
| > | |
| > Sentinel-1 is `VV VH`, ascending and descending concatenated along time. | |
| - **Sentinel-2**: `(T, 10)` per pixel, raw L2A reflectance (unscaled DN), plus a | |
| `(T,)` day-of-year array and a `(T,)` validity mask (1 = clear, 0 = cloud). | |
| - **Sentinel-1**: `(T, 2)` per pixel, raw RTC values, plus day-of-year. | |
| All-zero timesteps are treated as missing. | |
| - Day-of-year is passed as a **raw integer 1β365** (not normalized). | |
| `infer.py` performs standardization internally (`standardize=True`, the | |
| default): Sentinel-2 uses per-band statistics, and Sentinel-1 ascending and | |
| descending each use **their own** statistics, applied *before* the two are | |
| merged into a single stream. Pass `standardize=False` only if your arrays are | |
| already z-scored with those exact constants (they are in `model.py`). | |
| Per pixel, the number of valid observations is bucketized to the nearest bin in | |
| `{8, 16, 24, ..., 256}`, then padded or subsampled to that bin size β matching | |
| the training-time procedure. Pixels sharing a bin are batched together. | |
| ## Usage | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| import sys, torch, numpy as np | |
| path = snapshot_download("geotessera/TESSERA-V-2.0-2B-S") | |
| sys.path.insert(0, path) | |
| from model import load_model | |
| from infer import encode_tile, encode_pixels | |
| from quantize import quantize as quantize_int8 | |
| model = load_model(f"{path}/ckpt/student_small.pt", torch.device("cuda")) | |
| # One tile -> (H, W, 128) embedding map. | |
| # s2_bands (T,H,W,10) raw reflectance, s2_doys (T,), s2_masks (T,H,W) 1=valid | |
| # s1_asc / s1_desc (T,H,W,2) raw + their own doys | |
| emb = encode_tile( | |
| model, s2_bands, s2_doys, s2_masks=s2_masks, | |
| s1_asc_bands=s1_asc, s1_asc_doys=s1_asc_doys, | |
| s1_desc_bands=s1_desc, s1_desc_doys=s1_desc_doys, | |
| batch_pixels=4096, device=torch.device("cuda"), | |
| ) # -> (H, W, 128) float32 | |
| emb16 = emb[..., :16] # Matryoshka truncation, 1/8 the storage | |
| code, scale = quantize_int8(emb) # int8 (H,W,128) + per-pixel float32 scale | |
| recon = code.astype("float32") * scale[..., None] | |
| ``` | |
| For per-pixel time series rather than a gridded tile, call `encode_pixels()` on | |
| `(N, T, C)` arrays β same keyword arguments, returns `(N, 128)`. | |
| ## Files | |
| ``` | |
| TESSERA-V-2.0-2B-S/ | |
| βββ model.py PixelStudent + load_model() (torch + numpy only) | |
| βββ infer.py encode_pixels() / encode_tile() | |
| βββ quantize.py linear int8 quantize() / dequantize() | |
| βββ __init__.py package-style re-exports | |
| βββ requirements.txt torch>=2.0, numpy>=1.21 | |
| βββ ckpt/student_small.pt | |
| ``` | |
| The checkpoint is a plain `torch.save` payload with two keys, `model` (the | |
| `state_dict`) and `args` (the architecture config `load_model` reads). No | |
| custom classes are pickled, and nothing outside `torch` and `numpy` is needed. | |
| ## Intended use and limitations | |
| Intended for scientific research and downstream geospatial analysis: | |
| environmental monitoring, conservation and habitat mapping, biomass and carbon | |
| estimation, crop mapping and agricultural monitoring, land-use and land-cover | |
| work. | |
| Limitations: | |
| - Embeddings are **annual, 10 m spectral-temporal representations** β not raw | |
| imagery, and not a real-time monitoring product. | |
| - Quality degrades where a pixel has very few valid observations (persistent | |
| cloud, sparse revisit). | |
| - Validate on your own task and geography before operational use. | |
| - **No benchmark scores are quoted here.** The results in the TESSERA v2 | |
| preprint were measured on students distilled from the **1B** teacher; these | |
| checkpoints are distilled from the **2B** teacher and their evaluation will | |
| be published separately. | |
| ## License | |
| Released under **CC0 1.0** (public domain dedication), matching the rest of the | |
| TESSERA release. Use of TESSERA is additionally governed by the project's | |
| [Acceptable Use Policy](https://github.com/ucam-eo/tessera/blob/master/aup.md). | |
| ## Citation | |
| ```bibtex | |
| @article{tessera_v2_2026, | |
| title = {TESSERA v2: Scaling Pixel-wise Earth Foundation Models}, | |
| author = {Feng, Zhengpeng and Jaffer, Sadiq and Shokar, Ira and | |
| Knezevic, Jovana and Elvers, Mark and Atzberger, Clement and | |
| Young, Robin and Naik, Aneesh and Robinson, Niall and | |
| Blake, Andrew and Coomes, David and Madhavapeddy, Anil and | |
| Keshav, Srinivasan}, | |
| journal = {arXiv preprint arXiv:2607.03949}, | |
| year = {2026}, | |
| url = {https://arxiv.org/abs/2607.03949} | |
| } | |
| ``` | |
| ## Links | |
| - Teacher model: [`geotessera/TESSERA-V-2.0-2B-Teacher`](https://huggingface.co/geotessera/TESSERA-V-2.0-2B-Teacher) | |
| - Code and releases: https://github.com/ucam-eo/tessera | |
| - Preprint: https://arxiv.org/abs/2607.03949 | |
| - Project website: https://geotessera.org/ | |
| - Earlier releases: [`TESSERA-V-1.0`](https://huggingface.co/geotessera/TESSERA-V-1.0), [`TESSERA-V-1.1`](https://huggingface.co/geotessera/TESSERA-V-1.1) |