yx21e commited on
Commit
f716dab
·
verified ·
1 Parent(s): 36cc968

Update integration handoff and CONUS retraining notes

Browse files
Files changed (1) hide show
  1. docs/hugh_handoff_status.md +49 -0
docs/hugh_handoff_status.md ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Integration Handoff Status
2
+
3
+ This note answers the implementation questions raised during web/inference
4
+ integration and points to the public files that should be used by downstream
5
+ clients. It is intentionally scoped to the released FireWx-FM repository.
6
+
7
+ ## What is fully specified now
8
+
9
+ | Question | Status | Where to look |
10
+ |---|---|---|
11
+ | Which input channel maps to which dataset? | Specified in human-readable and machine-readable form. Channel order matters. | `data_sources/DATA_SOURCES.md`, `models/wildfire_fm/input_channels.json` |
12
+ | Is value normalization applied? | Released California checkpoints use native source units with no mean/std or min/max normalization. The training pipeline now supports optional train-split z-score normalization for retraining. | `models/wildfire_fm/input_channels.json`, `training/train_cold_tiled_mainline.py` |
13
+ | Which CAPE variable is used? | HRRR surface instantaneous CAPE, selected with `typeOfLevel=surface` and `stepType=instant`; not most-unstable or layer CAPE. | `models/wildfire_fm/input_channels.json`, `data_sources/DATA_SOURCES.md` |
14
+ | What do the validity masks mean? | `firewx_valid` is a cache-level dynamic input-presence mask and is 1.0 everywhere in the released California cache. `static_valid` is the fraction of four static layers valid after reprojection. | `models/wildfire_fm/README.md`, `models/wildfire_fm/input_channels.json` |
15
+ | How are static 30 m rasters resampled to the 5 km grid? | LANDFIRE fuel and canopy use nearest-neighbor resampling; WRC housing density and LandScan population use bilinear resampling. | `training/configs/stage1_cache_regional_hrrr_ca_5km_l12_template.json`, `training/build_phase1_cache_regional_hrrr.py` |
16
+ | Can outputs be shown at county or sub-county granularity? | Yes, by aggregating native 5 km grid probabilities after inference. This does not require changing the checkpoint. | `spatial_serving/README.md`, `spatial_serving/grid_to_polygons.py` |
17
+ | Can HRRR be downloaded automatically? | Yes. A direct NOAA HRRR public-archive downloader is included and has been smoke-tested with an `.idx` sidecar download. | `data_downloader/hrrr_downloader.py`, `data_downloader/README.md` |
18
+ | Is the released checkpoint nationwide-trained? | No. The public checkpoint is California regional. CONUS cache and training templates are included, but no CONUS checkpoint is released yet. | `models/wildfire_fm/training_scope.json`, `training/NATIONWIDE_RETRAINING.md` |
19
+
20
+ ## Current model-serving boundary
21
+
22
+ The released checkpoints are the paper-aligned California regional checkpoints.
23
+ They are valid for reproducing the reported regional model behavior and for
24
+ inspecting the tensor contract. They should not be described as nationwide
25
+ weights.
26
+
27
+ For gridded inference, use full-map inference when possible. If tiled inference
28
+ is required, use overlapping tiles with halo cropping or blending. Independent
29
+ non-overlapping 32-by-32 tiles are not the intended serving mode because they can
30
+ introduce tile-position and edge artifacts. The helper
31
+ `models/wildfire_fm/tiled_inference.py` implements the overlap/halo pattern.
32
+
33
+ For CONUS or client-facing deployment, the current path is:
34
+
35
+ 1. Build a local CONUS cache from provider-hosted HRRR, FIRMS, LANDFIRE, WRC,
36
+ and LandScan inputs.
37
+ 2. Retrain using the CONUS template with random-containing positive tile
38
+ placement and optional continuous-channel z-score normalization.
39
+ 3. Serve native-grid probabilities with full-map or overlap/halo inference.
40
+ 4. Aggregate native-grid probabilities to county or sub-county polygons using
41
+ the spatial serving adapter.
42
+
43
+ ## What is not being uploaded as solved
44
+
45
+ A preliminary CONUS normalized/jittered retraining run completed, but seed-level
46
+ behavior was not stable enough to release as a corrected nationwide checkpoint.
47
+ For that reason, the repository exposes the corrected training and serving
48
+ hooks, but it does not publish a new CONUS checkpoint or claim that the CONUS
49
+ serving behavior is fully resolved.