Model Card for ESFM/ESFM_s_wm

Default deterministic ESFM small checkpoint for six-hour global forecasting with heterogeneous or incomplete ERA5-style inputs. It was trained with variable, pressure-level, and spatial masking and represents missing patches with learnable NaN tokens.

Checkpoint selection: Six-hour deterministic global forecasting when inputs may have missing variables, pressure levels, or spatial regions. Complete ERA5-style inputs are also supported.

Model Details

Developed by: The ESFM research team, with the full contributor and author lists linked below.
Shared by: ESFM on Hugging Face
Model type: Final deterministic masked ERA5 checkpoint (ESFM_s,kd); modified 3D Swin-UNet encoder-decoder
Model size: Approximately 115 million parameters
Masking protocol: Variable, pressure-level, and spatial masking
Forecast lead time: 6 hours
License: MIT
Repository: https://huggingface.co/ESFM/ESFM_s_wm

Model Sources

Code: https://github.com/swiss-ai/ESFM
Paper: https://arxiv.org/abs/2605.00850
Project page: https://swiss-ai.github.io/ESFM/

The paper is currently available as an arXiv preprint.

Uses

Direct Use

Six-hour deterministic global forecasting when inputs may have missing variables, pressure levels, or spatial regions. Complete ERA5-style inputs are also supported.

Downstream Use

Starting point for adaptation to new gridded, satellite, or station datasets with the released ESFM training code.

Out-of-Scope Use

Not intended as an operational weather-warning system, a replacement for numerical weather prediction, or a guarantee of accuracy under arbitrary missingness outside the evaluated regimes.

Bias, Risks, and Limitations

Accuracy on clean dense reanalysis is modestly lower than the no-masking counterpart, and entire missing pressure levels are the hardest evaluated condition. The checkpoint inherits ERA5 biases and lacks extensive rollout-focused finetuning.

All ESFM checkpoints are research artifacts. Users should validate forecasts for their variables, regions, seasons, lead times, missingness pattern, and decision context. Do not use the model as the sole basis for safety-critical decisions.

How to Get Started

The checkpoint is not packaged as a Hugging Face Transformers from_pretrained model. Construct the ESFM architecture with the matching repository config, then load the state dictionary. The released notebook contains the complete download, model-construction, normalization, and inference workflow.

git clone https://github.com/swiss-ai/ESFM.git
cd ESFM
# Open notebooks/inference_ESFMs_on_ERA5.ipynb

In the notebook, set:

EXPERIMENT_NAME = "ESFM_s_wm"

To download the weights directly:

from huggingface_hub import hf_hub_download

model_name = "ESFM_s_wm"
weights_path = hf_hub_download(
    repo_id=f"ESFM/{model_name}",
    filename=f"{model_name}.safetensors",
)
print(weights_path)

Set EXPERIMENT_NAME = "ESFM_s_wm" in notebooks/inference_ESFMs_on_ERA5.ipynb, or run python inference.py --config configs/config_ESFM_s_wm.yaml after configuring dataset paths.

Training Details

Training Data

WeatherBench2 ERA5 at 0.25-degree resolution, trained on 1979 through 2020 and evaluated in the manuscript on held-out windows in 2023 and 2024.

Dataset preprocessing and the exact variable registry are documented in the ESFM repository and preprint.

Training Procedure

Initialized through Aurora-to-ESFM encoder knowledge distillation and 100,000-step masked ERA5 training as ESFM_s_wm_pre, then continued for 10,000 steps as ESFM_s_wm. This experiment was run on 16 GPUs. Detailed masking probabilities are in the preprint and masking_config.yaml.

Training objective: Six-hour forecast learning, as specified above
Nominal architecture: ESFM small, approximately 115M parameters
Software environment: PyTorch/Lightning in the released NVIDIA PhysicsNeMo 25.03 container; lightning==2.5.1 is pinned in the Dockerfile
Training regime: Lightning precision="32-true" with FP32 parameters and optimizer state; selected model forward operations use CUDA BF16 autocasting through torch.autocast(dtype=torch.bfloat16).

Evaluation

The manuscript evaluates latitude-weighted MAE and Pearson correlation on dense inputs and when regions, variables, or levels are withheld. It reports graceful degradation and physically plausible relationships, while retaining detailed results in the linked preprint.

The manuscript uses held-out temporal data and reports task-appropriate metrics: latitude-weighted MAE and Pearson correlation for gridded deterministic forecasts, relative MAE for MODIS comparisons, station metrics for station models, and CRPS for ensembles. Detailed values are intentionally not copied into this card.

Technical Specifications

ESFM retains Aurora's 3D Swin-UNet backbone and adds variable-specific tokenization, axial attention across variables, perceiver aggregation across variables and pressure levels, learnable NaN tokens for missing patches, resolution-specific tokenizers where configured, and a decoder queried at target pressure levels. The small configuration uses a 256-dimensional embedding and approximately 115M parameters.

Environmental Impact

Hardware type: NVIDIA GH200 systems with four GPUs per node. This experiment was run on four nodes, totaling 16 GPUs.
Total training time: 21 hours
Compute location: Training used CSCS Alps infrastructure.

Citation

@misc{ozdemir2026esfm,
  title={Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting},
  author={Firat Ozdemir and Yun Cheng and Salman Mohebi and Fanny Lehmann and Simon Adamov and Zhenyi Zhang and Leonardo Trentini and Dana Grund and Oliver Fuhrer and Torsten Hoefler and Siddhartha Mishra and Sebastian Schemm and Benedikt Soja and Mathieu Salzmann},
  year={2026},
  eprint={2605.00850},
  archivePrefix={arXiv},
  primaryClass={physics.ao-ph},
  url={https://arxiv.org/abs/2605.00850}
}