README.md

---
license: cc-by-4.0
language:
- en
tags:
- OneScience
- Earth Science
- Causal Networks
- Climate Model Evaluation
frameworks: PyTorch
---

<p align="center">
  <strong><span style="font-size: 30px;">CausalModelEvaluation</span></strong>
</p>

# Model Introduction

CausalModelEvaluation (CME) addresses the limitations of climate-model evaluations that rely mainly on mean-state errors and do not determine whether key process relationships are represented correctly. It is primarily intended to construct causal climate fingerprints, identify dependencies among models with shared development backgrounds, evaluate precipitation simulation skill, and use historical process skill to constrain uncertainty in future precipitation change.

Paper: Causal networks for climate model evaluation and constrained projections  
https://doi.org/10.1038/s41467-020-15195-y

# Model Description

Causal Model Evaluation was proposed by research teams from Imperial College London, the German Aerospace Center, the University of Bremen, and the University of East Anglia. The paper evaluates CMIP5 sea-level-pressure and precipitation simulations with NCEP-NCAR and ERA-Interim reanalyses and CRU TS v4.02 precipitation observations. The model supports causal climate-fingerprint reconstruction, process-oriented climate-model evaluation, and constrained precipitation-change projection.

# Use Cases

| Use Case | Description |
| :---: | :--- |
| Causal fingerprinting | Estimate a directed, signed, lagged network from seasonal 50-node time series. |
| Climate model evaluation | Compute reference-oriented asymmetric F1 with direction, sign, and lag tolerance. |
| Precipitation skill | Compute latitude-area-weighted Taylor S-score and pattern correlation. |
| Constrained projection | Fit an RBF plus white-noise GP between CME F1 and precipitation change. |
| ModelScope/OneCode | Validate data, fitting, checkpoint, inference, evaluation, and task-figure workflows. |

# Usage Instructions

## 1.OneCode

Experience intelligent, one-click AI4S programming through the OneCode online environment:

[Try intelligent, one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)

## 2. Download and Installation

```bash
hf download OneScience-Group/CausalModelEvaluation --local-dir ./CausalModelEvaluation
cd CausalModelEvaluation
```

### Environment Dependencies

**Hardware Requirements**

- A GPU or DCU is recommended.
- A CPU can be used for connectivity validation with the default small-sample configuration.
- DCU users must install DTK in advance. DTK 25.04.2 or later, or the OneScience-recommended version matching the current cluster, is recommended.

**DCU Environment**

```bash
# Activate DTK and Conda first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
```

**GPU Environment**

```bash
# Activate Conda first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
```

### Training Data

```bash
python scripts/fake_data.py
```

This repository uses a small number of structured synthetic samples to validate the engineering workflow. The data contains seasonal sea-level-pressure mode time series, distinct lagged causal relationships across models, and precipitation fields and changes related to model process skill. It preserves the paper's node, time, lag, and spatial dimensions and validates causal-network construction, model comparison, and constrained projection only; it does not represent the real climate-data distribution or paper performance.

### Training

For single-device training, use:

```bash
python scripts/train.py
```

For multi-GPU training, use:

```bash
torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py
```

Training results include reference and model causal networks, model-comparison scores, and the precipitation-constraint relationship, with a checkpoint and training metrics saved for subsequent inference.

```text
result/checkpoints/causalmodelevaluation.pt
result/training/metrics.json
```

### Trained Weights

No weights are bundled under `weight/`. The paper method produces no conventional neural-network weights; the engineering checkpoint stores statistical networks and GP state and is not claimed to be compatible with external weights.

### Inference

```bash
python scripts/inference.py
```

Inference reloads the checkpoint and exports complete model and four-season `edges/pvalues/mci`, complete reference networks, reference/model precipitation fields, CME F1, precipitation changes, GP means and 95% intervals, and network/projection metadata to `result/output/inference.npz`.

### Evaluation and Visualization

```bash
python scripts/result.py
```

Evaluation generates structured results for climate-model causal-network comparison, precipitation simulation skill, and constrained projection, and saves them to `result/evaluation/metrics.json`. It also creates the model-skill and precipitation-change comparison figure `result/evaluation/cme_task.png`. Synthetic-data results are only for engineering-workflow validation and do not represent paper performance on the real test set.

# Official OneScience Information

| Platform | OneScience Main Repository | Skills Repository |
| --- | --- | --- |
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |

# Citation and License

This repository is an independent engineering reproduction of the public Causal Model Evaluation paper specifications.

Use of this repository's code, official model weights, and data remains subject to the licenses and terms of their respective projects.