๐ฅ Introducing BetaEarth - your own Earth embedding emulator [๐๐ซ๐-๐๐๐ฅ๐๐๐ฌ๐]
The past year has brought many notable embedding products, like AlphaEarth, TESSERA or OlmoEarth. We are entering a phase where embeddings begin to act as a substitute for real observation data.
BetaEarth is an attempt to explore how much one can learn from a model based on its embeddings alone, and whether those embeddings can serve as a useful training target for other models. Huge credit to the AlphaEarth team for releasing the embedding archive openly โ it's what made this kind of community-built extension possible.
BetaEarth is a flexible (and relatively lightweight) emulator of the AlphaEarth annual product. It doesn't reproduce AlphaEarth's exact outputs, nor the product, but it reaches ~0.87 cosine similarity on held-out data and retains 97% of downstream land-cover classification accuracy. It only took 1-2 days to train.
It can encode any combination (including multi-temporal) of: - Sentinel-2 L1C - Sentinel-2 L2A - Sentinel-1 RTC - COP-DEM 30 product
The model weights are open, just like its training data (built exclusively using Major TOM). The GitHub repository provides a script for automated generation of embeddings across any footprint. You can also try the workflow over small bounding boxes on the free Hugging Face web app!
This is a tiny (about 6 TB of data, but only 62,489 grid cells of ~100 sqkm) prototype dataset that allows to instantly connect existing Major TOM data with AlphaEarth embeddings.
I curated it to support several relevant research projects, but I figured it could help more people in the community to experiment and explore new applications of AlphaEarth.
๐๐ข๐ซ๐๐๐ญ๐ข๐จ๐ง๐ฌ ๐๐จ๐ซ ๐๐ฌ๐ Each embedding sample comes from the original annual dataset produced by Google DeepMind. It means that, unlike samples from Sentinel-2 or Sentinel-1, it contains aggregated annual information from a particular year and is not linked to one particular observation. The existing Major TOM samples from physical sensors provide information potentially (and likely) contained in the AlphaEarth embedding sample, but they miss the temporal component represented within AEF embedding fields.
For more information, please check the dataset card on HuggingFace.
Today we release a prototype of COP-GEN - a universal generative model for Copernicus data. ๐๐๐-๐๐๐-๐๐๐ญ๐ is a model trained globally on the thumbnails of the Major TOM Core datasets, including Sentinel-2 L1C, Sentinel-2 L2A, Sentinel-1 RTC, and COP-DEM GLO-30.
How is it universal? COP-GEN learns a joint generative process of all modalities, which means that it can reconstruct data from any subset of present observations. ๐๐ข๐ญ๐ก๐จ๐ฎ๐ญ ๐ญ๐ซ๐๐ข๐ง๐ข๐ง๐ ๐ฌ๐ฉ๐๐๐ข๐๐ข๐๐๐ฅ๐ฅ๐ฒ to perform any of these tasks it can be used to approximate:
โ Sentinel-1 to Sentinel-2 translation
โ Elevation estimation from Sentinel-2 or Sentinel-1
โ Atmospheric Correction (L1C to L2A pipeline)
โ Atmospheric Generation (L2A to L1C)
โ ...and any other task involving translation between the supported modalities
On its own, the model can be used as a useful prior for estimating the data likelihood distribution for Copernicus data. COP-GEN-Beta learns joint, conditional, and marginal distributions within a single unified backbone, allowing to flexibly sample any modality given any condition.
Why is it Beta? Because thumbnails are a low-cost representation of the data that scales well and we managed to develop this prototype quite fast. We are currently developing the more costly COP-GEN model that supports the original data. For now, we wanted to showcase the prototype and make it available to the community for a test!
๐๐๐๐ ๐๏ธ ๐๐๐ฑ๐ญ-๐๐๐ฌ๐๐ ๐ญ๐๐ซ๐ซ๐๐ข๐ง ๐ ๐๐ง๐๐ซ๐๐ญ๐ข๐จ๐ง ๐ฆ๐จ๐๐๐ฅ MESA is a novel generative model based on latent denoising diffusion capable of generating 2.5D representations (co-registered colour and depth maps) of terrains based on text prompt conditioning.
Work developed by Paul BorneโPons (@NewtNewt) during his joint internship at Adobe & ESA, and in collaboration with asterisk labs.
๐งฑ ๐๐จ๐ฆ๐ฆ๐ฎ๐ง๐ข๐ญ๐ฒ ๐๐ซ๐จ๐ฐ๐ญ๐ก: our community continues to grow! To coordinate the upcoming expansions as well as use cases of the open data, we will organise a meet up on 23 April, you can ๐ซ๐๐ ๐ข๐ฌ๐ญ๐๐ซ ๐ฒ๐จ๐ฎ๐ซ ๐ข๐ง๐ญ๐๐ซ๐๐ฌ๐ญ here: https://forms.gle/eBj8JvibJx9b6PLf9
๐ ๐๐ฉ๐๐ง ๐๐๐ญ๐ ๐๐จ๐ซ ๐๐ฉ๐๐ง ๐๐จ๐๐๐ฅ๐ฌ: Major-TOM Core dataset is currently supporting several strands of ongoing research within and outwith our lab and we are looking forward to the time when we can release models that take advantage of that data!
๐ ๐๐จ๐ฌ๐ญ๐๐ซ ๐๐ญ ๐๐๐๐๐๐: We will present Major TOM project as a poster at IGARSS in Athens (July) - come talk to us if you're there! You can access the paper here: Major TOM: Expandable Datasets for Earth Observation (2402.12095)
๐ Developed at European Space Agency ฮฆ-lab in partnership with Hugging Face
๐บ Major TOM: Expandable Datasets for Earth Observation
๐จ RECORD-BREAKING EO DATASET: the largest ever ML-ready Sentinel-2 dataset! It covers almost every single point on Earth captured by the Copernicus Sentinel-2 satellite. @mikonvergence and I are thrilled to finally announce the release of Major-TOM/Core-S2L2A and Major-TOM/Core-S2L1C
๐ About half of the entire planet is covered. That's 2,245,886 patches of 1068 x 1068 pixels, available in both L1C and L2A. At 10 m resolution, we've got 256 million square km with over 2.5 trillion pixels. It's all yours with a few lines of code. See the paper linked below ๐ฝ for more info!
๐งฑ And this is just the beginning. We are currently preparing more datasets from different satellites for the Major TOM org. TOM stands for Terrestrial Observation Metaset - a simple set of rules for building an ecosystem of ML-ready EO datasets, which can be seamlessly combined as if they were Lego bricks.
๐ดโโ๏ธ Want to take the dataset for a spin? We have a viewer app on spaces that lets you go anywhere on Earth and shows you the data, if its available Major-TOM/MajorTOM-Core-Viewer