README.md

---
license: cc-by-4.0
language:
- en
tags:
- scikit-learn
- xgboost
- random-forest
- regression
- weather-radar
- quantitative-precipitation-estimation
- dual-polarization-radar
- geoscience
- taiwan
---
# Tree-Based Radar Quantitative Precipitation Estimation Models

## Overview

This archive contains trained Random Forest (RF) and Extreme Gradient Boosting
(XGB) models used to evaluate how radar predictor representation affects
quantitative precipitation estimation (QPE). Models are provided for 10-minute
and 1-hour rainfall estimation and for Experiments A-G described below.

The RF models were trained with scikit-learn, and the XGB models were trained
with XGBoost. The restricted radar, rain-gauge, training, and validation data
are not included in this archive.

## File Naming

Model filenames follow this pattern:

```text
{model}_{timescale}_Exp{experiment}[_{KDP_variant}].joblib
```

- `model`: `RF` or `XGB`
- `timescale`: `10min` or `1h`
- `experiment`: `A` through `G`
- `KDP_variant`: present only for Experiments F and G

Examples:

- `RF_10min_ExpA.joblib`: 10-minute RF model for Experiment A
- `XGB_1h_ExpE.joblib`: 1-hour XGB model for Experiment E
- `RF_10min_ExpF_3d.joblib`: 10-minute RF model using the formal
  three-dimensional KDP representation in Experiment F
- `XGB_1h_ExpG_3d_plus_max.joblib`: 1-hour XGB model using the vertical KDP
  profile together with maximum KDP in Experiment G

## Experiments

| Experiment | Predictor representation |
| --- | --- |
| A | Composite reflectivity: two-dimensional maximum dBZ |
| B | Experiment A plus latitude, longitude, and elevation |
| C | Vertical reflectivity profiles sampled from CAPPI data |
| D | Experiment C plus maximum dBZ, 18-dBZ echo top, 45-dBZ echo top, and vertically integrated liquid (VIL) |
| E | Experiment D plus latitude, longitude, and elevation |
| F | Experiment D plus a KDP representation |
| G | Experiment E plus a KDP representation |

The formal Experiments F and G use the vertical KDP profile (`3d`). Additional
F/G files are sensitivity experiments that use alternative KDP
representations:

| Suffix | KDP representation |
| --- | --- |
| `3d` | Vertical KDP profile |
| `low` | Low-level KDP representation |
| `max` | Maximum KDP representation |
| `3d_plus_max` | Vertical KDP profile plus maximum KDP |

The reflectivity profile contains 21 CAPPI levels from 1.0 to 17.0 km. The KDP
profile contains 34 levels from 0.5 to 17.0 km. Radar predictors use seven
temporal lags from `t-60` to `t0`, matched to the nearest radar observation
within 5 minutes.

The formal A-G representations contain 7, 10, 147, 175, 178, 413, and 416
predictors, respectively. Sensitivity variants of F and G may have different
predictor counts.

## Prediction Targets

- `10min`: rainfall accumulated over 10 minutes and expressed as an
  hourly-equivalent intensity in mm h^-1. It is not a 1-hour accumulation.
- `1h`: hourly rainfall intensity in mm h^-1.

Model predictions therefore use units of mm h^-1 for both time scales.

## Joblib Contents

Each joblib file contains a dictionary with these entries:

| Key | Meaning |
| --- | --- |
| `model` | Trained RF or XGB model |
| `features` | Ordered feature names required by the model |
| `feature_importances` | Stored feature-importance table |
| `importance_type` | `impurity` for RF or `gain` for XGB |
| `model_type` | `RF` or `XGB` |
| `target` | `10min` or `1h` |
| `experiment` | Experiment identifier |
| `kdp_shape` | KDP representation, where applicable |
| `backend` | Model implementation recorded during export |
| `versions` | Relevant package versions recorded during export |

Always prepare input columns in the exact order stored in `features`. A model
should not be applied to data with a different preprocessing procedure, feature
definition, vertical level, temporal lag, unit, or column order.

## Loading and Prediction

The following example loads an artifact and produces predictions from a pandas
DataFrame named `frame`:

```python
import joblib
import numpy as np
import xgboost as xgb

artifact = joblib.load("RF_10min_ExpA.joblib")
model = artifact["model"]
feature_names = artifact["features"]

X = frame.loc[:, feature_names].to_numpy(dtype=np.float32)

if artifact["model_type"] == "XGB":
    predictions = model.predict(xgb.DMatrix(X, feature_names=feature_names))
else:
    predictions = model.predict(X)
```

The required Python packages include `joblib`, `numpy`, `scikit-learn`, and
`xgboost`. Consult `artifact["versions"]` for the package versions recorded for
an individual model. Matching those versions as closely as possible is
recommended for reproducible loading and prediction.

## Feature Importance

Feature importance is available in `artifact["feature_importances"]`:

- RF importance is the scikit-learn impurity-based importance.
- XGB importance is the XGBoost gain importance.

These definitions measure different quantities. Importance values should be
interpreted within the context of each model and should not be treated as
directly equivalent between RF and XGB. Correlated radar predictors can also
share or redistribute importance.

## Validation Design

The models were developed using observations from 2020-2023 at 252 stations in
the training region. Spatial validation used 30 held-out stations from the same
period, and temporal validation used 2024 observations from the training-region
stations. Validation data and results are not part of this model-only archive.

## Data Availability and Limitations

The source radar products, including three-dimensional CAPPI reflectivity were 
provided by the Central Weather Administration and the National Science and 
Technology Center for Disaster Reduction. Redistribution restrictions prevent 
their inclusion. Rain-gauge observations and derived training or validation 
samples are also not included here.

Reusing these models requires independently obtained data with matching
variables and identical preprocessing. The archive alone is not sufficient to
reconstruct the restricted input dataset or reproduce model training from raw
observations.

## Security Notice

Joblib files use Python's pickle-based serialization. Load these files only
from a trusted source, because loading an untrusted pickle or joblib file can
execute arbitrary code.