Model Card for ESFM/ESFM_s_wm_modis_pre

Intermediate ESFM small checkpoint finetuned for six-hour forecasting of sparse MODIS total precipitable-water retrievals, with ERA5 temperature and specific humidity as auxiliary inputs.

Checkpoint selection: Use to reproduce or inspect the MODIS specialization before its final 10,000-step continuation. Prefer ESFM_s_wm_modis for the final released MODIS checkpoint.

Model Details

Developed by: The ESFM research team, with the full contributor and author lists linked below.
Shared by: ESFM on Hugging Face
Model type: Intermediate deterministic sparse-satellite checkpoint; modified 3D Swin-UNet encoder-decoder
Model size: Approximately 115 million parameters
Masking protocol: Variable, pressure-level, and spatial masking
Ensemble members: 1
Forecast lead time: 6 hours
License: MIT
Repository: https://huggingface.co/ESFM/ESFM_s_wm_modis_pre

Model Sources

Code: https://github.com/swiss-ai/ESFM
Paper: https://arxiv.org/abs/2605.00850
Project page: https://swiss-ai.github.io/ESFM/

The paper is currently available as an arXiv preprint.

Uses

Direct Use

Use to reproduce or inspect the MODIS specialization before its final 10,000-step fine-tuning. Use ESFM_s_wm_modis for the final released MODIS checkpoint.

Downstream Use

Research on sparse-swath satellite forecasting and adaptation to other satellite-derived Earth-system variables.

Out-of-Scope Use

Not a generic ERA5 forecaster, not valid for arbitrary satellite instruments without finetuning, and not evidence that MODIS-derived fields are closer to atmospheric truth than reanalysis.

Bias, Risks, and Limitations

Predictions can inherit MODIS retrieval biases and product-specific artifacts. Input occupancy is extremely sparse, and performance depends on the supplied auxiliary ERA5 fields and preprocessing. This intermediate checkpoint is not separately evaluated in the manuscript.

All ESFM checkpoints are research artifacts. Validate forecasts for the target variables, stations or regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.

How to Get Started

The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct ESFM with the matching config and load the state dictionary. The released notebook demonstrates checkpoint download and architecture construction.

from huggingface_hub import hf_hub_download

model_name = "ESFM_s_wm_modis_pre"
weights_path = hf_hub_download(
    repo_id=f"ESFM/{model_name}",
    filename=f"{model_name}.safetensors",
)
print(weights_path)

The public ERA5 notebook can download this checkpoint but cannot supply valid MODIS inputs. Use configs/config_ESFM_s_wm_modis_pre.yaml with locally preprocessed MODIS data following the released preprocessing scripts.

Clone the implementation first:

git clone https://github.com/swiss-ai/ESFM.git
cd ESFM

Training Details

Training Data

Hourly MODIS Terra MOD05 and Aqua MYD05 infrared and near-infrared total precipitable-water products, gridded to 0.25 degrees, plus ERA5 temperature and specific humidity. The manuscript uses 2014-2020 for training.

Preprocessing and the exact variable registry are documented in the ESFM repository, preprocessing repository, and preprint.

Training Procedure

Initialized from the masked ERA5 lineage and finetuned on MODIS for 50,000 steps on 16 GPUs using the main ESFM masking protocol. The completed MODIS lineage totals 60,000 steps.

Nominal architecture: ESFM small, approximately 115M parameters
Software environment: PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
Training regime: Lightning precision="32-true" with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through torch.autocast(dtype=torch.bfloat16).

Evaluation

The manuscript evaluates the completed MODIS lineage against held-out 2023-2024 MODIS retrievals using relative MAE and Pearson correlation and studies autoregressive stability. This intermediate release is not separately tabulated.

Detailed numerical results are intentionally not copied into this card.

Technical Specifications

ESFM uses variable-specific tokenization, axial attention across variables, perceiver aggregation, a 3D Swin-UNet backbone, and a decoder queried at target pressure levels. Missing patches are represented by learnable NaN tokens. Resolution-specific tokenizers and station mapping are enabled for the relevant sparse-data configs. The ensemble checkpoint additionally applies member-conditioned AdaLN-Zero after the backbone.

Environmental Impact

Hardware type: NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
Total training time: 57 hours
Compute location: Training used CSCS Alps infrastructure.

Citation

@misc{ozdemir2026esfm,
  title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
  author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
  year={2026},
  eprint={2605.00850},
  archivePrefix={arXiv},
  primaryClass={physics.ao-ph},
  url={https://arxiv.org/abs/2605.00850}
}