franch commited on
Commit
19a5da9
·
verified ·
1 Parent(s): 510f74a

Restore model card as README

Browse files
Files changed (1) hide show
  1. README.md +52 -248
README.md CHANGED
@@ -1,278 +1,82 @@
1
- <div align="center">
2
-
3
- # ConvGRU-Ensemble
4
-
5
- **Ensemble precipitation nowcasting using Convolutional GRU networks**
6
-
7
- *Pretrained model for Italy:* ***IRENE*** — **I**talian **R**adar **E**nsemble **N**owcasting **E**xperiment
8
-
9
- [![CI](https://github.com/DSIP-FBK/ConvGRU-Ensemble/actions/workflows/ci.yml/badge.svg)](https://github.com/DSIP-FBK/ConvGRU-Ensemble/actions)
10
- [![License: BSD-2](https://img.shields.io/badge/license-BSD--2-blue.svg)](LICENSE)
11
- [![Python 3.13+](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://python.org)
12
- [![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97-Model-yellow)](https://huggingface.co/it4lia/irene)
 
 
13
 
14
- <br>
15
 
16
- <a href="https://www.fbk.eu"><img src="https://webvalley.fbk.eu/static/img/logos/fbk-logo-blue.png" height="55" alt="Fondazione Bruno Kessler"></a>
17
- &nbsp;&nbsp;&nbsp;&nbsp;
18
- <a href="https://it4lia-aifactory.eu"><img src="https://it4lia-aifactory.eu/wp-content/uploads/2025/05/logo-IT4LIA-AI-factory.svg" height="55" alt="IT4LIA AI-Factory"></a>
19
- &nbsp;&nbsp;&nbsp;&nbsp;
20
- <a href="https://www.italiameteo.eu"><img src="https://it4lia-aifactory.eu/wp-content/uploads/2025/08/logo-italiameteo.svg" height="55" alt="ItaliaMeteo"></a>
21
 
22
- <br><br>
23
 
24
- The model encodes past radar frames into multi-scale hidden states and decodes them into an **ensemble of probabilistic forecasts** by running the decoder multiple times with different noise inputs, trained with **CRPS loss**.
 
 
 
 
25
 
26
- </div>
27
 
28
- ---
29
 
30
- ## Quick Start
 
 
 
31
 
32
- <details open>
33
- <summary><b>Load from HuggingFace Hub</b></summary>
34
 
35
  ```python
36
  from convgru_ensemble import RadarLightningModel
37
 
 
38
  model = RadarLightningModel.from_pretrained("it4lia/irene")
39
 
 
40
  import numpy as np
41
- past = np.load("past_radar.npy") # rain rate in mm/h, shape (T_past, H, W)
42
  forecasts = model.predict(past, forecast_steps=12, ensemble_size=10)
43
- # forecasts.shape = (10, 12, H, W) — 10 members, 12 future steps, mm/h
44
- ```
45
-
46
- </details>
47
-
48
- <details>
49
- <summary><b>CLI Inference</b></summary>
50
-
51
- ```bash
52
- convgru-ensemble predict \
53
- --input examples/sample_data.nc \
54
- --hub-repo it4lia/irene \
55
- --forecast-steps 12 \
56
- --ensemble-size 10 \
57
- --output predictions.nc
58
- ```
59
-
60
- </details>
61
-
62
- <details>
63
- <summary><b>Serve via API</b></summary>
64
-
65
- ```bash
66
- # With Docker
67
- docker compose up
68
-
69
- # Or directly
70
- pip install convgru-ensemble[serve]
71
- convgru-ensemble serve --hub-repo it4lia/irene --port 8000
72
- ```
73
-
74
- **Submit a forecast request:**
75
-
76
- ```bash
77
- # 4-hour forecast (4 steps × 1h) with 5 ensemble members
78
- curl -X POST "http://localhost:8000/predict?forecast_steps=4&ensemble_size=5" \
79
- -F "file=@examples/sample_data.nc" \
80
- -o predictions.nc
81
-
82
- # Use default settings (12 steps, 10 members)
83
- curl -X POST http://localhost:8000/predict \
84
- -F "file=@examples/sample_data.nc" -o predictions.nc
85
- ```
86
-
87
- **Read the predictions:**
88
-
89
- ```python
90
- import xarray as xr
91
-
92
- ds = xr.open_dataset("predictions.nc")
93
- print(ds.precipitation_forecast.shape)
94
- # (5, 4, 1400, 1200) — ensemble_member, forecast_step, y, x
95
- ```
96
-
97
- | Endpoint | Method | Description |
98
- |---|---|---|
99
- | `/health` | GET | Health check |
100
- | `/model/info` | GET | Model metadata and hyperparameters |
101
- | `/predict` | POST | Upload NetCDF, get ensemble forecast as NetCDF |
102
-
103
- **`/predict` query parameters:**
104
-
105
- | Parameter | Default | Description |
106
- |---|---|---|
107
- | `variable` | `RR` | Name of the rain rate variable in the NetCDF |
108
- | `forecast_steps` | `12` | Number of future 5-min steps (1–48, i.e. max 4h) |
109
- | `ensemble_size` | `10` | Number of ensemble members (1–10) |
110
-
111
- The input NetCDF must contain a 3D variable `(T, H, W)` with rain rate in mm/h and at least 2 timesteps.
112
-
113
- </details>
114
-
115
- <details>
116
- <summary><b>Fine-tune on your data</b></summary>
117
-
118
- ```bash
119
- pip install convgru-ensemble
120
- # See "Training" section below
121
- ```
122
-
123
- </details>
124
-
125
- ## Setup
126
-
127
- Requires Python >= 3.13. Uses [uv](https://github.com/astral-sh/uv) for dependency management.
128
-
129
- ```bash
130
- uv sync # core dependencies
131
- uv sync --extra serve # + FastAPI serving
132
- ```
133
-
134
- ## Data Preparation
135
-
136
- The training pipeline expects a Zarr dataset with a rain rate variable `RR` indexed by `(time, x, y)`.
137
-
138
- <details>
139
- <summary><b>1. Filter valid datacubes</b></summary>
140
-
141
- Scan the Zarr and find all space-time datacubes with fewer than `n_nan` NaN values:
142
-
143
- ```bash
144
- cd importance_sampler
145
- uv run python filter_nan.py path/to/dataset.zarr \
146
- --start_date 2021-01-01 --end_date 2025-12-11 \
147
- --Dt 24 --w 256 --h 256 \
148
- --step_T 3 --step_X 16 --step_Y 16 \
149
- --n_nan 10000 --n_workers 8
150
  ```
151
 
152
- </details>
153
 
154
- <details>
155
- <summary><b>2. Importance sampling</b></summary>
156
 
157
- Sample valid datacubes with higher probability for rainier events:
158
 
159
- ```bash
160
- uv run python sample_valid_datacubes.py path/to/dataset.zarr valid_datacubes_*.csv \
161
- --q_min 1e-4 --m 0.1 --n_workers 8
162
- ```
163
-
164
- A pre-sampled CSV is provided in [`importance_sampler/output/`](importance_sampler/output/).
165
-
166
- </details>
167
-
168
- ## Training
169
-
170
- Training is configured via [Fiddle](https://github.com/google/fiddle). Run with defaults:
171
 
172
- ```bash
173
- uv run python -m convgru_ensemble.train
174
- ```
175
-
176
- Override parameters from the command line:
177
-
178
- ```bash
179
- uv run python -m convgru_ensemble.train \
180
- --config config:experiment \
181
- --config set:model.num_blocks=5 \
182
- --config set:model.forecast_steps=12 \
183
- --config set:model.loss_class=crps \
184
- --config set:model.ensemble_size=2 \
185
- --config set:datamodule.batch_size=16 \
186
- --config set:trainer.max_epochs=100
187
- ```
188
 
189
- Monitor with TensorBoard: `uv run tensorboard --logdir logs/`
190
-
191
- | Parameter | Description | Default |
192
- |---|---|---|
193
- | `model.num_blocks` | Encoder/decoder depth | `5` |
194
- | `model.forecast_steps` | Future steps to predict | `12` |
195
- | `model.ensemble_size` | Ensemble members during training | `2` |
196
- | `model.loss_class` | Loss function (`mse`, `mae`, `crps`, `afcrps`) | `crps` |
197
- | `model.masked_loss` | Mask NaN regions in loss | `True` |
198
- | `datamodule.steps` | Total timesteps per sample (past + future) | `18` |
199
- | `datamodule.batch_size` | Batch size | `16` |
200
-
201
- ## Architecture
202
-
203
- ```
204
- Input (B, T_past, 1, H, W)
205
- |
206
- v
207
- +--------------------------+
208
- | Encoder | ConvGRU + PixelUnshuffle (x num_blocks)
209
- | Spatial dims halve at | Channels: 1 -> 4 -> 16 -> 64 -> 256 -> 1024
210
- | each block |
211
- +----------+---------------+
212
- | hidden states
213
- v
214
- +--------------------------+
215
- | Decoder | ConvGRU + PixelShuffle (x num_blocks)
216
- | Noise input (x M runs) | Each run produces one ensemble member
217
- | for ensemble generation |
218
- +----------+---------------+
219
- |
220
- v
221
- Output (B, T_future, M, H, W)
222
- ```
223
-
224
- ## Docker
225
-
226
- ```bash
227
- docker build -t convgru-ensemble .
228
-
229
- # Run with local checkpoint
230
- docker run -p 8000:8000 -v ./checkpoints:/app/checkpoints \
231
- -e MODEL_CHECKPOINT=/app/checkpoints/model.ckpt convgru-ensemble
232
-
233
- # Run with HuggingFace Hub
234
- docker run -p 8000:8000 -e HF_REPO_ID=it4lia/irene convgru-ensemble
235
- ```
236
-
237
- ## Project Structure
238
-
239
- ```
240
- ConvGRU-Ensemble/
241
- +-- convgru_ensemble/ # Python package
242
- | +-- model.py # ConvGRU encoder-decoder architecture
243
- | +-- losses.py # CRPS, afCRPS, masked loss wrappers
244
- | +-- lightning_model.py # PyTorch Lightning training module
245
- | +-- datamodule.py # Dataset and data loading
246
- | +-- train.py # Training entry point (Fiddle config)
247
- | +-- utils.py # Rain rate <-> reflectivity conversions
248
- | +-- hub.py # HuggingFace Hub upload/download
249
- | +-- cli.py # CLI for inference and serving
250
- | +-- serve.py # FastAPI inference server
251
- +-- examples/ # Sample data for testing
252
- +-- importance_sampler/ # Data preparation scripts
253
- +-- notebooks/ # Example notebooks
254
- +-- scripts/ # Utility scripts (e.g., upload to Hub)
255
- +-- tests/ # Test suite
256
- +-- Dockerfile # Container for serving API
257
- +-- MODEL_CARD.md # HuggingFace model card template
258
- ```
259
 
260
  ## Acknowledgements
261
 
262
- <div align="center">
263
-
264
- Developed at **Fondazione Bruno Kessler (FBK)**, Trento, Italy, as part of the **Italian AI-Factory (IT4LIA)**, an EU-funded initiative supporting AI adoption across SMEs, academia, and public/private sectors. This work showcases capabilities in the **Earth (weather and climate) vertical domain**.
265
-
266
- <br>
267
-
268
- <a href="https://www.fbk.eu"><img src="https://webvalley.fbk.eu/static/img/logos/fbk-logo-blue.png" height="45" alt="Fondazione Bruno Kessler"></a>
269
- &nbsp;&nbsp;&nbsp;&nbsp;
270
- <a href="https://it4lia-aifactory.eu"><img src="https://it4lia-aifactory.eu/wp-content/uploads/2025/05/logo-IT4LIA-AI-factory.svg" height="45" alt="IT4LIA AI-Factory"></a>
271
- &nbsp;&nbsp;&nbsp;&nbsp;
272
- <a href="https://www.italiameteo.eu"><img src="https://it4lia-aifactory.eu/wp-content/uploads/2025/08/logo-italiameteo.svg" height="45" alt="ItaliaMeteo"></a>
273
 
274
- </div>
275
 
276
  ## License
277
 
278
- BSD 2-Clause — see [LICENSE](LICENSE).
 
1
+ ---
2
+ license: bsd-2-clause
3
+ language: en
4
+ tags:
5
+ - weather
6
+ - nowcasting
7
+ - radar
8
+ - precipitation
9
+ - ensemble-forecasting
10
+ - convgru
11
+ - earth-observation
12
+ library_name: pytorch
13
+ pipeline_tag: image-to-image
14
+ ---
15
 
16
+ # IRENE — Italian Radar Ensemble Nowcasting Experiment
17
 
18
+ **IRENE** is a ConvGRU encoder-decoder model for short-term precipitation forecasting (nowcasting) from radar data. The model generates probabilistic ensemble forecasts, producing multiple plausible future scenarios from a single input sequence.
 
 
 
 
19
 
20
+ ## Model Description
21
 
22
+ - **Architecture**: ConvGRU encoder-decoder with PixelShuffle/PixelUnshuffle for spatial scaling
23
+ - **Input**: Sequence of past radar rain rate fields (T, H, W) in mm/h
24
+ - **Output**: Ensemble of future rain rate forecasts (E, T, H, W) in mm/h
25
+ - **Temporal resolution**: 5 minutes per timestep
26
+ - **Training loss**: Continuous Ranked Probability Score (CRPS) with temporal consistency regularization
27
 
28
+ The model encodes past radar observations into multi-scale hidden states using stacked ConvGRU blocks with PixelUnshuffle downsampling. The decoder generates forecasts by unrolling with different random noise inputs, producing diverse ensemble members that capture forecast uncertainty.
29
 
30
+ ## Intended Uses
31
 
32
+ - Short-term precipitation forecasting (0-60 min ahead) from radar reflectivity data
33
+ - Probabilistic nowcasting with uncertainty quantification via ensemble spread
34
+ - Research on deep learning for weather prediction
35
+ - Fine-tuning on regional radar datasets
36
 
37
+ ## How to Use
 
38
 
39
  ```python
40
  from convgru_ensemble import RadarLightningModel
41
 
42
+ # Load from HuggingFace Hub
43
  model = RadarLightningModel.from_pretrained("it4lia/irene")
44
 
45
+ # Run inference on past radar data (rain rate in mm/h)
46
  import numpy as np
47
+ past = np.random.rand(6, 256, 256).astype(np.float32) # 6 past timesteps
48
  forecasts = model.predict(past, forecast_steps=12, ensemble_size=10)
49
+ # forecasts.shape = (10, 12, 256, 256) — 10 ensemble members, 12 future steps
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  ```
51
 
52
+ ## Training Data
53
 
54
+ Trained on the Italian DPC (Dipartimento della Protezione Civile) radar mosaic surface rain intensity (SRI) dataset, covering the Italian territory at ~1 km resolution with 5-minute temporal resolution.
 
55
 
56
+ ## Training Procedure
57
 
58
+ - **Optimizer**: Adam (lr=1e-4)
59
+ - **Loss**: CRPS with temporal consistency penalty (lambda=0.01)
60
+ - **Batch size**: 16
61
+ - **Ensemble size during training**: 2 members
62
+ - **Input window**: 6 past timesteps (30 min)
63
+ - **Forecast horizon**: 12 future timesteps (60 min)
64
+ - **Data augmentation**: Random rotations and flips
65
+ - **NaN handling**: Masked loss for missing radar data
 
 
 
 
66
 
67
+ ## Limitations
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
+ - Trained on Italian radar data; performance may degrade on other domains without fine-tuning
70
+ - 5-minute temporal resolution only
71
+ - Best suited for convective and stratiform precipitation; extreme events may be underrepresented
72
+ - Ensemble spread is generated via noisy decoder inputs, not a full Bayesian approach
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
 
74
  ## Acknowledgements
75
 
76
+ This model was developed as part of the **Italian AI-Factory** (IT4LIA), an EU-funded initiative supporting the adoption of AI across SMEs, academia, and public/private sectors. The AI-Factory provides free HPC compute, consultancy, and AI-ready datasets. This work showcases capabilities in the **Earth (weather and climate) vertical domain**.
 
 
 
 
 
 
 
 
 
 
77
 
78
+ Developed at **Fondazione Bruno Kessler (FBK)**, Trento, Italy.
79
 
80
  ## License
81
 
82
+ BSD 2-Clause License