Pluvium-500 / NETWORK_CARD.md
Sigmandndnns's picture
Pluvium-500 release
1e9995d verified
|
Raw History Blame Contribute Delete
6.56 kB
metadata
license: apache-2.0
library_name: pluvium
tags:
  - weather
  - precipitation
  - nowcasting
  - caribbean
  - probabilistic-forecasting
  - ihelix

Pluvium-500

Hour-by-hour rain forecasts for every town and village in the Caribbean, three days ahead: the chance of rain in each hour, when rain is most likely to start (in local time) and how long it lasts, how much may fall, and what is causing it (cold front or shear line, tropical wave, tropical cyclone, afternoon storms, trade-wind showers).

This is a research model, not an official forecast. Follow your national meteorological service.

  • 67.3M parameters, the NatureV1 / iHELIX engine: geodesic attention on a variable-resolution Caribbean mesh (6,075 points, about 11 km over land), index attention, and a gated delta-rule recurrence along the forecast's frames (12 blocks of width 512).
  • Inputs: driving-model fields (ERA5 in training; GFS, ECMWF IFS and ECMWF AIFS live) with convective indices, rain already falling (CMORPH in training, GOES-East live), the last three hours of airport reports, terrain, coasts, sun and season.
  • Truth: CMORPH 8 km satellite rain (half-hourly, NOAA CDR) and hourly airport weather reports (NOAA ISD, read for intensity, drizzle/rain/showers and thunder).
  • Outputs per town and hour: chance of dry/light/moderate/heavy rain, thunder, drizzle/rain/showers, expected rain; start, heaviest and stop times; totals and the wettest 1, 3 and 6 hours; flood risk; the cause.

Experimental components

These have not been tested in published work; each can be switched off for an ablation, and pluvium bench compares the full version 2 engine with version 1 on held-out data.

  1. An intensity chain. For every hour and place the network writes a transition matrix between dry, light, moderate and heavy rain. Hourly chances, start and stop times, the hour rain turns heavy, the chance of any rain in a window and the totals all come from that one chain, so they agree with each other.
  2. Truth seen through its instrument. Training scores the observed classes through a hidden Markov model in which the satellite and the airports each have their own learned confusion matrix.
  3. Thunder and drizzle/rain/showers heads, trained on what airports report (METAR groups such as -SHRA, +TSRA, DZ).
  4. Airport assimilation. The last three hours of reports are read into the mesh by cross-attention, with dropout so the model works with or without them.
  5. Advected rain. The rain falling now is carried along the steering wind for the first 12 hours (the idea of radar nowcasting, with satellite rain).
  6. Convective indices among the inputs: K-index, total totals, theta-e difference, lifted index, 700 hPa humidity, moisture-flux convergence, 850-200 hPa shear, 200 hPa divergence.
  7. A differentiable orographic layer. Smith-Barstad linear theory of rain over mountains, solved by FFT on 2 km tiles every forecast hour, with learned parameters.
  8. Island-heating convergence, front and shear-line arrival times, per-place daily cycles, and forecast-error emulation during reanalysis training.
  9. A mixture of chains live, one per driving model, so disagreement widens the windows.
  10. Online corrections. Forecasts are checked against later airport reports and recalibrated per parish.
  11. Theoretical physics inputs (version 3): parcel CAPE/CIN/LCL with exact moist thermodynamics, column relative humidity and the Bretherton law, moisture-flux convergence, the Saharan air layer, tropical-wave arrival, a parametric cyclone rain prior, sea-breeze front timing, diurnal clocks, forced ascent and blocking.
  12. A deeper decoder (version 3): terrain patches, lead-time modulation and shrinkage towards climatology, attention along each point's future and among neighbouring points, a regime mixture of experts, and head dropout sampled at forecast time.
  13. Rare-event training (version 3): a curriculum that oversamples heavy rain and thunder with tempered importance weights, averaged weights, and physical augmentation.

Training

  • Stage A: ERA5 years 2015-2020, perfect prognosis with emulated forecast errors.
  • Stage B: model output statistics on real GFS runs.
  • Calibration: isotonic, per lead-time band, fitted on half of 2021; scores below are on the other half.

Scores on held-out data (2021)

Test Hourly Brier Climatology Skill vs climatology Any rain in 24 h: skill Start hour within 3 h 24 h total CRPS skill Intensity (RPS) skill Heavy hours skill
Reanalysis-driven (upper bound) 0.0352 0.0411 +14.5% +61.2% 53% +59.0% +12.8% -2.7%
Emulated forecast errors 0.0360 0.0411 +12.5% +60.7% 52% +58.6% +11.0% -3.7%
Driven by real GFS runs (live situation) 0.0491 0.0537 +8.5% +55.9% 54% +47.4% +6.9% -9.9%
  • Brier score: mean squared error of the hourly chance of rain (lower is better). Skill: improvement over the climatological chance for that place, month and hour of day.
  • "Reanalysis-driven" uses the reanalysis as if it were a perfect forecast, so it is an upper bound on live skill. "Emulated forecast errors" degrades it the way forecasts are degraded; "driven by real GFS runs" is the live situation (when stage B ran).
  • A wet hour is at least 0.2 mm. Rain-start error is for places dry at the start of the forecast where rain began within 24 hours.

Limitations

  • Satellite rain (CMORPH) misses some light showers and is smoothed at 8 km; the model's truth inherits that.
  • Live, the rain already falling comes from GOES-East infrared when its rain-rate product is unavailable; that estimate is cruder than the CMORPH used in training.
  • Tropical cyclones are rare in any training period; rain in landfalling storms is less reliable than everyday rain.
  • Towns are forecast at their coordinates; a town spread over a hillside can see very different rain at each end.

Use

pip install pluvium
pluvium forecast --repo CorxLabs/Pluvium-500          # every town, tables in ./pluvium-forecast
pluvium place "Mandeville" --country JM  # hour by hour for one place

Data and attribution

ERA5 (Copernicus Climate Change Service), CMORPH CDR and ISD (NOAA NCEI), GFS (NOAA NCEP), ECMWF open data (CC-BY-4.0), GOES-East (NOAA), GeoNames (CC-BY-4.0), geoBoundaries (CC-BY-4.0), Terrarium elevation tiles (Mapzen and its sources). Code: Apache-2.0, CorX Labs.