CREST_runner / server.py

Commit History

16 days ago

  1. Health-check fix batch (CREST_AI 3dca271)

    Verified signature
    2ac66f1

  1. admin archive: owner-only (ADMIN_USERS) view of EVERY stored 2-D inundation result (listed + rolled-off) — /api/admin/archive + Archive panel; open any result on the map (shared backend)

    Verified signature
    f2a65be

  1. Risk view (map_visual lens for 2-D inundation): /api/riskprobe per-PIXEL depth/speed/flow-dir + DEFRA hazard class + WorldPop 2026-2030 population (resampled to the solver grid) + OSM infrastructure values; Dockerfile CRESTIMAP_REF -> e8652781 (maxspeed/flowdir rasters)

    Verified signature
    a8f20b9

  1. health: watch MRMS Pass2 / PET / TEMP archive freshness

    Verified signature
    7f30fae

  1. event_run cold=1: full cold re-run of a listed episode (no warm-start frame; spec 'cold' stamp makes workers drop resident sessions) — basin re-run of the list

    Verified signature
    ff0fe9c
  2. /api/event_run: episode=<id> re-simulates an existing listed episode in place (basin-domain re-run of the list)

    Verified signature
    ecaaa3e

  1. V30.1: wall-clock budget for the demoted CPU solve rung + tick wedge detection (a 39-h local solve had starved the hourly event tick)

    Verified signature
    e9f5960

  1. V30 operations: episode final products (consolidated stats + event-scoped nowcast verification replace the running event on end), CPU rung demoted to last resort behind worker claims, provenance stamps in manifests, /api/health self-monitoring, runner-split plumbing (EVENT_TICK / EVENT_RUNNER_URL), event-score labels + no Model-options panel in nowcast mode

    Verified signature
    3490275
  2. V29 fixed episode windows: anchored at trigger-12h (no moving window), USGS-obs-based episode end, frame carry-forward on republish, warm-start IC from previous depth frame, in-flight queue shown in the event panel

    Verified signature
    15fc593
  3. event ladder: delegated worker hand-off frees the runner (no more result-wait/CPU while claimed); new detections outrank episode re-sims; fresh claims survive re-enqueue; event_run gains t0 backdating + defer for refills

    Verified signature
    e2681d6
  4. Nowcast MRMS rain-radar overlay (24h/48h/7d anim) + persistent 2D depth legend + queue worker wake-ping

    Verified signature
    e66a060

  1. events API: full runner log (diagnosis)

    Verified signature
    1c6dcc2
  2. V25: hourly event tick + continuous parquet archive

    Verified signature
    548b5a3
  3. V25: nowcast-triggered 2-D inundation events (CREST-iMAP v2)

    Verified signature
    c6461c0

  1. serve ungauged nowcasts from store; drop live routed EF5 path

    Verified signature
    ff2f718
  2. V24 foundation: two-phase incremental routed nowcast (routednow.compute) + 10-day state retention (prune_states) + temp /api/_rn_test verify endpoint

    Verified signature
    bb285a7

  1. V23.1: ungauged nowcast shows 7-day routed history (no obs to anchor) + 12h forecast

    Verified signature
    41584ad
  2. V23: ungauged-point nowcast via routed upstream injection — route each upstream cut gauge's observed+DI-LSTM-nowcast flow through EF5 speed scheme to the ungauged point; future precip zero-filled; hydrograph splits at t0 (observed vs predicted)

    Verified signature
    2f88bc3
  3. V22.1: rename to 'ungauged points'; 10-day warm-up for ungauged runs (upstream injection carries the river)

    Verified signature
    b1bc469
  4. V22: virtual points — 2,676 ungauged HydroBASINS lev07 outlets fill CONUS gauge gaps; hindcast via upstream-gauge injection (speed scheme forced) + nearest-gauge parameter regionalization

    Verified signature
    19ff563

  1. hotspots: fully dynamic count (0..N, no cap) + helpful zero-risk answer

    Verified signature
    2d2b961
  2. chat: flood-risk hotspot routing — agent scans clustered risk map, hotspot action zooms there

    Verified signature
    32a4b61
  3. hydrograph times in the gauge local time zone (UTC toggle for emergency users)

    Verified signature
    01c67fc
  4. UX: gauge locate ping on tab click (both modes) + upstream HydroRIVERS network in nowcast

    Verified signature
    4d51491

  1. V21.3: nowcast flood-risk map — 10-yr LP3 thresholds, red density layer zoomed out, red pins zoomed in

    Verified signature
    1d9bd0f
  2. V21: Nowcast mode — precomputed DI-LSTM +6h predictions, Nowcast/Hindcast toggle

    Verified signature
    6c9967b

  1. Fix SSE stream breaking on NaN discharge values

    Verified signature
    fcca414

  1. V19: AI nowcast wiring — after each successful gauge run the dashboard asks the companion CREST_nowcast Space (DI-LSTM, ZeroGPU) for a +6h corrected tail and draws it as a dotted orange trace on the hydrograph; graceful no-op while the model is untrained or the Space is asleep

    Verified signature
    a2b5ce1
  2. V18.8: error watchdog records everything — browser error beacon (window.onerror/unhandledrejection -> /api/clienterror, rate-limited 30/min), threading.excepthook backstop, SSE stream generator capture, and captures at the silent fallbacks (LLM parse, BC obs, result-cache save, cal LLM propose, snow clips, OAuth callback, history save, compactor). errors.jsonl stays ephemeral by design.

    Verified signature
    ae3019a

  1. V18.6: reports are a registered-user benefit — signed-in downloads are copied to a per-user library (cap 20) that persists to the private dataset and shows under 👤 My reports (download/delete); guests' reports stay ephemeral: discarded via a pagehide beacon when they close the app, with a 6-h janitor TTL backstop

    Verified signature
    2df811d
  2. V18.5: downloadable PDF report — AQUAH's report-writer agent (ARW) integrated with its verbatim prompts: figures rendered from the live job (hydrograph+precip, peak 2-D streamflow, DEM/FACC), vision LLM figure analysis, simulation summary (incl. KGE), report-writer agent -> Markdown -> pypandoc/xelatex PDF; '⬇ PDF report' button in the right panel after each gauge finishes; cached per (sim,gauge); md fallback when pandoc/LLM unavailable

    Verified signature
    2b36d56
  3. V18.3: durable cache — states (priority, as zstd+byte-stream-split parquet), results, calibrated params and user profiles now mirror to the private dataset vincewin/CREST_state; restored at boot, incrementally synced every 10 min and right after each run/calibration/profile write; janitor evictions mirrored so the repo tracks the local caps; GET/POST /api/persist for status + manual sync. Frames/forcing stay ephemeral (re-derivable); logs excluded.

    Verified signature
    1bfa7d0
  4. V18.2: adaptive calibration depth — if the best NSE is still < 0.3 after the standard 12 trials, one extended stage of 5 rounds x 4 LLM-proposed candidates (+20 runs) is appended automatically (CREST_CAL_EXT_ROUNDS/EXT_K/EXT_NSE Space vars); extension announced as a chat card

    Verified signature
    e9efca8
  5. V18.1: calibration hygiene — each of the 13 calibration runs (1 baseline + 4 rounds x 3 candidates) gets its own workdir deleted the moment the run ends; candidate runs no longer save trial-parameter state grids into the shared state cache (warm-start only); CREST_CAL_ROUNDS / CREST_CAL_K Space variables to raise the iteration budget without a redeploy

    Verified signature
    c9e5dbb
  6. V18: registered-user focused basins — up to 5 favorite gauges saved to the account profile via the right-panel star after a simulation or a typed USGS number in the profile page; favorites show as gold star pins at every zoom (click = zoom in + select), GET/POST /api/favorites with catalog validation + cap

    Verified signature
    67f0001

  1. V16.2: raise the simulation-window cap 90 d -> 366 d (a 3-month test run was being truncated); CREST_MAX_SIM_HOURS=0 now disables the cap entirely

    Verified signature
    1012c59

  1. V14: Speed run vs Full run at the Simulate button — speed run truncates the domain at cut-set boundary gauges (obs injected as inflow), with an obs-coverage gate and fallback to full

    Verified signature
    90e23e6
  2. V13: upstream-gauge boundary conditions (multi-gauge control + EF5 DA), cancel/supersede running sims + Stop button, auto re-run after calibration

    Verified signature
    4891009
  3. V10: clip-store + retry (fixes calibration RasterioIOError), AI error explainer, expert-panel gate, wording

    Verified signature
    7232d71
  4. V9: flexible timestep (integer + unit) + frontend cache-busting (versioned assets, no-cache index)

    Verified signature
    9029784
  5. V8: test-user feedback widget (durable HF-dataset sink activates when HF_TOKEN secret is set)

    Verified signature
    c440315
  6. V7: error-watchdog recorder (Sentry-SDK compatible) + data manager (shared forcing store, overlap merge, cache janitor)

    Verified signature
    249c752
  7. V6: HUC8 basin guide layer, stall-based watchdog, per-user simulation history, agent date fix

    Verified signature
    305b5b1
  8. V5: conversational chat agent, reattachable background runs, stuck-run watchdog

    Verified signature
    049574e
  9. V4: run lock + Simulate guard, current-step progress, 2-D frame cache/replay, robust time parsing + SET/CLEAR override

    Verified signature
    1e08c08

  1. V3 interface: map-first gauge pins, full-width chat + 3/4 map, gauge tabs, AI info feed (progress bars + event news), HF OAuth + profile, streamflow legend/toggle/autozoom, AI calibration (AI_cali rules) + persistent param store, spline hydrograph, time-range fallback prompt

    Verified signature
    9aef03e
  2. fix build: python3-dev for gdal bindings; wire timestep/warmup/start knobs end-to-end

    Verified signature
    6d371d4
  3. V2: FastAPI+Leaflet map dashboard, Docker SDK (EF5 fork + Arrow), live hydrograph + 2-D streamflow, warmup/state/cache, SNOW17, western CREST

    Verified signature
    2fe3948