Persistence reference forecast

README.md

@@ -0,0 +1,51 @@
+---
+license: mit
+tags:
+  - weather
+  - forecasting
+  - baseline
+---
+
+# Persistence baseline
+
+A reference forecast for 2 metre temperature: the analysis at T+0, carried
+forward unchanged to every lead hour.
+
+Persistence is the standard trivial reference in weather forecasting. It has no
+parameters and no training data. Its purpose is comparison — a forecast system
+is only demonstrating skill once it beats persistence, and the margin by which
+it does so at each lead time is a more meaningful number than an error score
+quoted on its own.
+
+## Behaviour
+
+Skill decays in a predictable way, which is what makes it useful as a control:
+
+| Lead | Typical behaviour |
+|---|---|
+| +6 h | Hard to beat by much — the atmosphere is strongly autocorrelated at this range |
+| +24 h | Clearly beaten by any competent forecast |
+| +120 h | Error approaches that of climatology |
+
+Error that does **not** grow with lead time indicates a problem in the
+evaluation, not in the forecast.
+
+## Inputs
+
+Reads `2m_temperature` from a gridded analysis in netCDF, with dimensions
+`(time, latitude, longitude)`, latitude descending from +90 to −90, and
+longitude ascending on a 0–360 convention. It uses the most recent timestep.
+
+## Outputs
+
+`temperature_2m` with dimensions `(lead_hour, latitude, longitude)` on the input
+grid, in Kelvin (the unit of the source field).
+
+## Files
+
+- `predict.py` — the model
+- `earthboi.yaml` — declared variables, lead hours and grid
+
+## Licence
+
+MIT.

earthboi.yaml

@@ -0,0 +1,22 @@
+# What this model forecasts. The platform verifies every field against what the
+# model actually writes, and displays this on the leaderboard beside the score.
+entrypoint: predict.py
+
+variables:
+  - temperature_2m
+
+# 6-hourly to +168h (7 days).
+#
+# Matched to AIFS's own 6-hourly instantaneous steps so the two are scored at
+# the same instants against the same observations. A model may declare any lead
+# set it likes -- the platform pools whatever exists rather than forcing a
+# package -- but a baseline is only useful if it lines up with what it is a
+# baseline FOR.
+lead_hours: [6, 12, 18, 24, 30, 36, 42, 48, 54, 60, 66, 72, 78, 84, 90, 96, 102, 108, 114, 120, 126, 132, 138, 144, 150, 156, 162, 168]
+
+grid:
+  resolution: 0.25
+  layout: global
+
+display_name: Persistence baseline
+license: mit

predict.py

@@ -0,0 +1,84 @@
+"""Persistence: tomorrow is the same as today.
+
+The simplest forecast there is. It copies the T+0 initial condition forward,
+unchanged, to every declared lead hour.
+
+This is not a joke entry. Persistence is the standard reference forecast in
+meteorology, and on this platform it does two jobs:
+
+1. **A skill floor.** Any model worth ranking must beat it beyond the first step
+   or two. A submission that cannot is not forecasting, whatever it costs to
+   run.
+
+2. **A pipeline oracle.** Its error curve is known in advance -- small at +6h,
+   climbing steadily, bad by +120h. That makes it the one model whose score can
+   be checked rather than merely believed, which is why it is what the platform
+   is proven with. Weather forecasts fail plausibly: a latitude flip, a
+   Kelvin/Celsius slip, or an off-by-one in lead hours all produce output that
+   still looks like weather. Against persistence those show up immediately:
+
+       flat error across leads   -> lead hours are not being applied
+       error around 273          -> a unit conversion was missed
+       large error at +6h        -> grid sampling is wrong (lat or lon)
+       beats AIFS at long leads  -> scoring or window alignment is broken
+
+Keeping it permanently on the leaderboard means any future regression of that
+kind is visible on the board itself rather than needing a diagnostic run.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+
+# The unit the arrays below are in.
+#
+# The harness reads this to convert once, at the boundary. Persistence returns a
+# slice of the IC untouched, and the IC stores 2m temperature in Kelvin, so that
+# is what this says. Getting it wrong is the "confident RMSE of 273" failure --
+# which is precisely the failure this model exists to make visible, so it would
+# be poor form to commit it here.
+OUTPUT_UNITS = {"temperature_2m": "K"}
+
+# Index of T+0 within the IC's time axis.
+#
+# The IC carries two timesteps, T-6h and T+0, in that order (`INPUT_STEP_HOURS =
+# (-6, 0)` in the spec). Persistence forecasts from the most recent one, so it
+# is the last, not the first. Using index 0 would forecast from six hours before
+# the init and quietly lose that much skill at every lead.
+T_ZERO = -1
+
+
+def run(inputs, declaration, init_time):
+    """Copy the analysis forward to every lead hour.
+
+    Args:
+        inputs: IC fields as numpy arrays, keyed by ERA5 name. Surface fields
+            are (time, lat, lon).
+        declaration: the parsed earthboi.yaml -- lead_hours, grid_shape, etc.
+        init_time: the init as an aware datetime (unused; persistence has no
+            time dependence beyond the analysis it starts from).
+
+    Returns:
+        {"temperature_2m": (lead, lat, lon) float32}
+    """
+    analysis = np.asarray(inputs["2m_temperature"])[T_ZERO]
+
+    rows, cols = declaration["grid_shape"]
+    if analysis.shape != (rows, cols):
+        # The IC is 0.25 degrees; a 1 degree declaration would need subsampling.
+        # Persistence does not implement that -- a wrong-shaped array would be
+        # caught by the harness anyway, but failing here names the actual reason.
+        raise ValueError(
+            f"IC grid {analysis.shape} does not match the declared grid "
+            f"{(rows, cols)}. This baseline only runs at the IC's native resolution."
+        )
+
+    leads = len(declaration["lead_hours"])
+    # `broadcast_to` returns a read-only view over one grid rather than N copies;
+    # `np.array` then makes the one contiguous array the harness writes. At 0.25
+    # degrees and 28 leads that is ~116 MB, which is worth not doubling.
+    return {
+        "temperature_2m": np.array(
+            np.broadcast_to(analysis, (leads, rows, cols)), dtype="float32"
+        )
+    }