README.md

---
license: mit
language:
- en
- zh
tags:
- OneScience
- Earth Science
- Weather Forecast
- ERA5
- foundation-model
frameworks: PyTorch
datasets:
  - OneScience/ERA5
---

<p align="center"><strong><span style="font-size: 30px;">AURORA</span></strong></p>

# Model Introduction

AURORA is a foundation model of the Earth system developed by Microsoft Research. It addresses a range of Earth system prediction tasks, including global weather forecasting and air pollution prediction. The paper was published at ICML 2024.

Paper: Aurora: A Foundation Model of the Atmosphere

https://arxiv.org/abs/2405.13063

# Model Description

Aurora is a deep learning model with 1.3 billion parameters, composed of a 3D Perceiver encoder, a 3D Swin Transformer processor, and a 3D Perceiver decoder.

# Use Cases

| Scenario | Description |
| :---: | :--- |
| Weather Forecast Training | Train AURORA using ERA5 HDF5 data |
| Local Quick Validation | Use synthetic data to verify data loading, model training, fine-tuning, inference, and inference result visualization. |
| ModelScope / OneCode Execution | Download as a standalone model package, install dependencies, and run scripts directly. |
| Multi-GPU Training | Launch multi-process training via `torchrun`. |

# Usage Guide

## 1. OneCode Usage

Experience intelligent one-click AI4S programming through the OneCode online environment:

[Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)

## 2. Manual Installation and Usage

**Hardware Requirements**

- A GPU or DCU is recommended.
- CPU can be used for import and small-scale connectivity verification; full training and inference will be slow.
- DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended.

### Download the Model Package

```bash
hf download OneScience-Group/AURORA --local-dir ./AURORA
cd AURORA
```

### Install the Runtime Environment

**DCU Environment**

```bash
# Please activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

**GPU Environment**

```bash
# Please activate CONDA first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
# uv installation is supported
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

### Training Data Introduction

The OneScience community provides ERA5 data for training (due to file size limits, the current repository contains a slice of the full dataset). Users can download it with the command below and confirm that the data path in `conf/config.yaml` is set correctly:

```bash
hf download --repo-type dataset OneScience-Group/ERA5 --local-dir ./data
```

### Generate Synthetic Data for Pipeline Validation

Synthetic data is only used to verify the data protocol and end-to-end pipeline; it does not represent forecast quality:

```bash
python scripts/fake_data.py
```

### Training

Single GPU:

```bash
python scripts/train.py
```

Multi-GPU:

The command below launches 2 training processes on one machine, each using a single device.

```bash
torchrun --nproc_per_node=2 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py
```

### Fine-tuning

The configuration points `training.finetune.checkpoint` to `data/checkpoint/model_bak.pt` produced during training, so fine-tuning uses the trained model by default.

```bash
python scripts/finetune.py
```

### Training Weights

This repository provides weights trained on ERA5 reanalysis data in the `weights/` folder. The weight files will be uploaded soon and are expected to be available in the near future.

### Inference

Inference reads `data/checkpoint/model_finetune.pt` saved by fine-tuning by default.

```bash
python scripts/inference.py
```

### Evaluation and Visualization

```bash
python scripts/result.py
```

# OneScience Official Information

| Platform | OneScience Main Repository | Skills Repository |
| --- | --- | --- |
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |

# Citation & License

- Aurora paper: https://arxiv.org/abs/2405.13063.
- This repository is the OneScience reproduction of the original Aurora paper. For citation or commercial use, please contact AIWeatherClimate@microsoft.com by email; see the official requirements for details: https://microsoft.github.io/aurora/intro.html.
- Copyright (c) Microsoft Corporation. Licensed under the MIT license.