File size: 4,114 Bytes
d5e211a
5803a25
 
 
 
 
 
d5e211a
5803a25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
d5e211a
5803a25
 
d5e211a
5803a25
d5e211a
 
 
 
 
5803a25
d5e211a
 
 
 
5803a25
d5e211a
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
# Adding a model family

The benchmark is model-agnostic below the adapter. Adding a family should touch only the following, with
no changes to scoring, result storage, caching or app conditionals.

1. `weather_lab.models.FAMILIES` – capabilities (below) and `status`.
2. `weather_lab.models.CHECKPOINTS` – name, repo, full revision, inference files, SHA-256.
3. An `Adapter` subclass (its own module), registration in `weather_lab.evaluate.ADAPTERS` for frozen configs, and construction in `weather_lab.forecast_config.build_adapter` for composable configurations. Use a verified retrieval index when retrieval is supported; storage verification and device lifetime stay with callers.
4. Configs: a new config or suite version listing the new system ids.
5. `src/weather_lab_app/labels.py`: readable name, description and colour family for every new id
   (enforced by `test_labels_cover_every_configured_system`).

## Adapter contract (`weather_lab.forecast.Adapter`)

- **Input** is a `CaseBatch`:
  - `history` (N, V, T) at or before each origin, NaN = missing;
  - known-future covariates: calendar from `future_times`, issued AROME in `arome`;
  - optional archived AROME history with provenance.
  - Targets are never in the batch.
- **Output** is a `ForecastBatch` with `point` (N, V, H) in benchmark units (°C, hPa, m/s).
  - Probabilistic systems add `quantiles` (N, V, H, Q) on `QUANTILE_LEVELS`, the 21 levels 0.01, 0.05,
    0.10 … 0.95, 0.99.
  - `validate()` rejects other grids. A family with different native levels must map them in the adapter
    and record the method in `info["quantile_mapping"]`; levels are never relabelled or invented.
  - Deterministic systems return `output="point"` and get no probabilistic scores.
- **Declarations.** `inputs` lists what the adapter consumes (`history`, `calendar`, `arome`), and `family`
  names its `FAMILIES` entry.
- **Preprocessing** belongs to the adapter: scaling, context trimming, covariate encoding. Do not reuse
  Timestral's choices unless they are right for the family.
- **Caching.** Load weights through `weather_lab.models` (`prefetch` for files; one loaded object per
  revision/device/dtype, held in an LRU like `ModelCache`). Keep request data out of the loaded object.
- **Same cases.** A new family is evaluated on the frozen cases of an existing suite through a config with
  that suite and manifest. The scorer's common valid set then forces every compared system onto the same
  (case, lead) pairs; a family that cannot forecast some cases reduces the common set visibly in the
  availability table, never silently.

`FAMILIES` fields: `past_covariates`, `known_covariates`, `missing_values`, `native_quantiles`,
`max_context`, `max_horizon`, `runtime`, `status` (`integrated` | `planned`). Forecast configuration validation uses them to tell **unsupported input** (the family cannot take an input setup) from **not run** (it could,
but no result is saved) and **not integrated yet** (`status: planned`).

## Existing native families

Timestral and TFC t0-gamma are integrated. Read `forecast.Timestral`, `t0_adapter.T0`,
`rag_adapters.RetrievedTimestral` / `RetrievedT0`, and `forecast_config.build_adapter` before adding another path.
The factory preserves each family's native covariate semantics. `development.predict_cases` owns case
batching, result validation, split metadata and single-case failure isolation. Device allocation and CUDA
release belong outside that worker.

Checkpoint identities, hashes and download sizes are declared in `models.CHECKPOINTS`; package versions
are locked in `uv.lock`. t0 weights exceed the initialization download budget and remain an explicit
prefetch rather than a bootstrap default. Do not change precision, batching, preprocessing or checkpoint
revisions as part of structural refactoring.

Parameterize genuinely shared adapter contracts, retain family-specific covariate and negative tests,
and agree a small real-device check before downloading or running new weights. Scoring, independent
saved-file rescoring and recorded-system mapping must remain covered separately.