README.md

metadata

title: La Eure / La Risle Hydrometric GNN
emoji: 📈
colorFrom: indigo
colorTo: indigo
sdk: gradio
sdk_version: 5.47.0
app_file: src/gradio_app.py
pinned: true

La Eure / La Risle Hydrometric GNN

A physics-informed graph neural network that predicts streamflow (discharge, water level) at gauged and ungauged points along two Normandy rivers, La Eure and La Risle — each modeled as a real reach-based network (confluences, braided splits/rejoins, ~4,500 nodes per basin including virtual infill points), not a single chain of gauges, with covariates pulled from nine independent data sources.

flowchart LR
    hubeau["Hub'Eau<br/>discharge · water level<br/>catchment area"]
    ades["ADES<br/>groundwater levels"]
    era5["Copernicus ERA5<br/>climate reanalysis"]
    otd["Open Topo Data<br/>station elevation"]
    brgm["BRGM<br/>IDPR · BD Charm-50 geology"]
    bdtopo["IGN BD TOPO<br/>real reach topology + catchment polygons"]
    bdcav["Géorisques<br/>BDCavités (sinkholes)"]
    wc["ESA WorldCover<br/>landcover · NDVI"]

    bdtopo --> brg["build_reach_graph.py<br/>real confluences, splits/rejoins,<br/>gauge snapping"]
    brg --> brgs["build_reach_graphs.py<br/>~4,500 nodes/basin"]

    hubeau --> nf
    ades --> nf
    era5 --> nf
    otd --> nf
    brgm --> nf
    bdcav --> nf
    wc --> nf
    brgs --> nf["node_features.py /<br/>enrich_reach_graph.py<br/>date-filtered 2013-2026"]

    bdtopo --> cc["compute_cumulative_catchment.py<br/>graph-wide catchment area"]
    cc --> nf

    nf --> pyg["build_pyg_graph<br/>x_static / x_dynamic split"]
    pyg --> phys["physics_losses.py<br/>confluence · split-rejoin ·<br/>routing · water balance"]

    pyg --> app["src/app.py<br/>Streamlit explorer +<br/>network validation view"]
    pyg --> testsuite["test_build_graph.py<br/>validation"]

---

1. Repository layout

PoC_v1/
├── scripts/                     # one-off download / extraction / build scripts
│   ├── download_hubeau.py           # discharge + water level, Hub'Eau API v2
│   ├── download_elevation.py        # point elevations, Open Topo Data
│   ├── download_era5_sample.py      # ERA5 sanity-check pull (Jan 2020 only)
│   ├── download_era5_full.py        # ERA5 1960–2026, split instant/accum vars
│   ├── extract_era5.py              # unzips CDS API's zipped NetCDF output
│   ├── download_catchment.py        # Hub'Eau referentiel/sites -> surface_bv
│   ├── download_bdtopo_hydro.py     # IGN WFS -> tronçons, surfaces, catchments
│   ├── analyze_bdtopo_hydro.py      # centerline export + karst check
│   ├── run_bdtopo_checks.py         # karst + catchment cross-check, one shot
│   ├── cross_check_catchments.py    # spatial join: station -> containing polygon
│   ├── build_reach_graphs.py        # real reach-based topology, both basins
│   ├── enrich_reach_graph.py        # runs node_features.py against the reach graph
│   ├── compute_cumulative_catchment.py  # graph-wide catchment area from BD TOPO polygons
│   ├── diagnose_confluences.py      # verify real vs. artifact confluences
│   ├── build_dynamic_tensors.py     # genuine [n_nodes, T] tensors, wired into physics_losses.py
│   ├── download_bdcavites.py        # Géorisques BDCavités (sinkhole/cavity inventory)
│   ├── download_bdcharm.py          # BRGM BD Charm-50 harmonized geology, per department
│   ├── fetch_landcover.py           # ESA WorldCover landcover class, real gauges
│   └── fetch_worldcover_ndvi.py     # ESA WorldCover NDVI percentile composite
│
├── src/
│   ├── app.py                       # Streamlit river explorer + network validation view
│   ├── generate_plots.py            # batch plot generation across all loaders
│   ├── test_build_graph.py          # graph-construction test/validation suite
│   ├── extract_river_centerline.py  # digitizes a traced map image into a centerline
│   │
│   ├── data/
│   │   ├── loaders/
│   │   │   ├── base.py                  # BaseDataLoader — shared load()/get_metadata()
│   │   │   ├── hydrometric.py           # discharge & water level (Hub'Eau)
│   │   │   ├── ades.py                  # groundwater levels (ADES)
│   │   │   ├── safran.py                # ERA5 reanalysis, vectorized station interpolation
│   │   │   ├── idpr.py                  # infiltration/runoff tendency (BRGM)
│   │   │   ├── catchment.py             # per-station catchment area (Hub'Eau)
│   │   │   ├── bdtopo_hydro.py          # IGN BD TOPO hydrography (GeoJSON)
│   │   │   ├── shapefile.py             # watershed boundary polygon
│   │   │   └── station_elevations.py    # station coordinates + elevation
│   │   │
│   │   ├── river_graph.py           # basin assignment, elevation ordering, edges
│   │   ├── river_line.py            # straight-line interpolation between gauges
│   │   └── river_centerline.py      # real-centerline interpolation + gauge snapping
│   │
│   └── graph/
│       ├── build_graph.py           # PyG conversion: x_static/x_dynamic split, structural columns
│       ├── build_reach_graph.py     # real reach topology: confluences, splits/rejoins, MultiDiGraph
│       ├── node_features.py         # pulls every loader into one feature table (static, one row/node)
│       ├── dynamic_features.py      # genuine [n_nodes, T] series: discharge, groundwater, climate
│       └── physics_losses.py        # confluence/split-rejoin/routing/water-balance loss terms, NaN-masked
│
├── datasets/                    # not checked in; populated by the scripts above
│   ├── station_list.csv             # raw station roster (X, Y, names, INSEE, etc.)
│   ├── station_elevations.csv       # station_code, lat, lon, elevation_m
│   ├── idpr.csv
│   ├── catchment_area.csv
│   ├── ades/
│   ├── hydrometric/
│   ├── safran/
│   ├── bdtopo_hydro/
│   ├── bdcavites/
│   ├── bdcharm50/
│   ├── centerlines/
│   └── reach_graph/                 # {eure,risle}_{nodes,edges}.csv, _nodes_enriched.csv

scripts/ talks to the outside world (APIs, WFS, S3); src/ doesn't — nothing under src/ makes a network call, and a script under src/ that wants one is a bug. Most of src/data/loaders/ predates the graph work — general-purpose readers/plotters for each dataset, with node_features.py stitching them together afterward rather than the other way around.

---

2. The graph

This is the part everything else in the repo exists to feed. Two graphs, one per river — H4xx… stations feed the La Eure graph, H6xx… feed La Risle — built with no edge between them, because there's no surface connection between the two basins to model.

The graph is now built from real reach topology, not a single ordered chain of gauges. build_reach_graph.py constructs it directly from BD TOPO's own tronçon-to-node linkage (lien_vers_noeud_hydrographique_ini/fin) — the NEXT_DOWN-equivalent approach — rather than inferring station order from position along a digitized line. That means real branching, real confluences, and real braided-channel structure fall directly out of the data instead of needing to be modeled separately.

2.1 Node types

Four kinds of node, not one:

Type	What it is	Column
Real gauge	one of the 27 hydrometric stations	is_gauged
Real confluence	a genuinely different, independently-sourced river joins	is_confluence
Split / rejoin	a channel divides and later recombines (braiding, an anabranch) — same water, no new mass	is_split_point / is_rejoin_point, paired via braid_id
Virtual (infill)	inserted along long confluence-free stretches so "predict at any point" has real spatial resolution	none of the above

A confluence requires more than a shared node with in-degree ≥ 2 — BD TOPO's fine tronçon segmentation produces plenty of same-river multi-inflow points with no real branching involved (confirmed against real data: incoming-edge distances as short as 4.6 m at some falsely-flagged "confluences"). The real test (find_real_confluences in build_reach_graph.py) requires (a) more than one distinct normalized river name among the incoming edges — river-name normalization strips articles, parenthetical qualifiers, and "bras de/du/d'" (arm-of) prefixes, since a named secondary channel of the same river ("Bras de la Charentonne") isn't a different river — and (b) that those branches don't trace back to a common upstream split within 15 km, which would mean it's a rejoin, not a confluence. Splits themselves need no such disambiguation: out-degree ≥ 2 is an unambiguous physical definition on its own, since a split by construction has exactly one thing flowing in.

Real branching topology also meant the underlying graph had to move from a plain DiGraph to a MultiDiGraph — two distinct tronçons directly connecting the same two hydrographic nodes (exactly the shape a short braid takes) is real data, not a collision, and a plain DiGraph was silently overwriting the second such edge's data on add_edge rather than keeping both. Confirmed as a real bug with real impact, not just a synthetic-test concern: fixing it recovered dozens of previously-invisible parallel edges per basin on the actual data.

2.2 Node and edge features

The feature set now spans several independent sources, each merged onto the node table by node_features.py's add_*_features functions. Every column lands in exactly one of four places once build_pyg_graph processes it:

flowchart TD
    raw["Enriched node table<br/>(node_features.py)"]

    raw --> struct{"structural /<br/>graph-role column?"}
    struct -->|"is_gauged, is_confluence,<br/>is_split_point, is_rejoin_point,<br/>braid_id, snap_distance_km"| structout["data.is_gauged, data.is_confluence, ...<br/>own Data attribute — never in x"]

    raw --> tgt{"target_* column?"}
    tgt -->|"target_discharge_m3s_*<br/>target_waterlevel_mm_*"| y["data.y<br/>never in x — label leakage otherwise"]

    raw --> feat{"real model input"}
    feat -->|"static: elevation_m, idpr_*,<br/>catchment_area_km2, landcover_*,<br/>geology_*, cavites distance/count"| xstatic["data.x_static"]
    feat -->|"dynamic: climate_*,<br/>avg_groundwater_*, ndvi_*<br/>(period-aggregate, not a real series yet)"| xdynamic["data.x_dynamic"]

    xstatic --> x["data.x — full combined tensor,<br/>z-scored"]
    xdynamic --> x

    edges["Edge table<br/>(build_reach_graph_tables)"] --> eattr{"numeric edge<br/>attribute?"}
    eattr -->|"distance_km,<br/>elevation_drop_m,<br/>verified_continuous"| edgeattr["data.edge_attr<br/>[n_edges, 3]"]
    eattr -->|"toponym, cleabs<br/>(diagnostic metadata)"| meta["not used by build_pyg_graph —<br/>stays in edges_df only"]

Node features:

Feature	Source	Coverage
latitude, longitude, elevation_m	station coords / real BD TOPO tronçon Z	every node
idpr_value, idpr_nearest_point_distance	BRGM IDPR	every node (spatial fallback for non-gauge codes)
catchment_area_km2	Hub'Eau, cumulative, real gauges only	27 stations
cumulative_catchment_area_km2	BD TOPO incremental polygons, summed upstream via real graph topology	graph-wide (~98% of nodes)
landcover_* (one-hot)	ESA WorldCover 10 m classification	real gauges only, for now
ndvi_p10, ndvi_p50, ndvi_p90	ESA WorldCover NDVI percentile composite	real gauges only, for now
geology_* (one-hot)	BRGM BD Charm-50, point-in-polygon	real gauges only, for now
distance_to_nearest_cavity_km, n_cavities_within_20km	Géorisques BDCavités, KD-tree + haversine	real gauges only, for now
avg_groundwater_level_m, avg_groundwater_depth_m, n_nearby_wells	ADES, radius-averaged, KD-tree + exact haversine	every node
climate_* (temp/wind/solar/precip/evap/snow/runoff)	ERA5, vectorized station interpolation	every node (needs safran_path)
{col}__was_missing	auto-generated	any feature column with real gaps

Edge features — three numeric attributes per edge, from build_reach_graph.py's build_reach_graph_tables:

Feature	Meaning
distance_km	along-river distance between the two endpoint nodes
elevation_drop_m	elevation difference, upstream minus downstream — negated on the reverse edge when bidirectional=True
verified_continuous	False for any edge deliberately flagged via known_losing_reaches (the bétoire stretch — §3.7)

toponym and cleabs also live on the real edges table (the tronçon's river name and unique BD TOPO ID) but are diagnostic metadata, not model input — build_pyg_graph selects edge_attr columns by explicit name, so extra columns like these pass through harmlessly rather than needing to be stripped out first.

Structural columns never enter x. is_gauged, is_confluence, is_split_point, is_rejoin_point, snap_distance_km, braid_id describe node role, not a physical covariate — build_pyg_graph's auto-detection excludes them explicitly (confirmed as a real, not hypothetical, bug once: pandas treats bool as a numeric dtype, so without this exclusion these columns were being silently z-scored and fed to the model as if they were elevation or precipitation). They're still attached to the returned Data object as their own typed attributes, for masking supervised loss to gauged nodes and for the physics-loss index builders.

Landcover and geology are one-hot, not a raw class code. Both are nominal categories (10 = Tree cover, 50 = Built-up; a geological formation code), not an ordered quantity — leaving either as a raw integer would let auto-detection z-score it as if one category were numerically "more" than another, the same class of error as the structural-column bug, just subtler since these are meant to be real model input.

Targets are not features. target_discharge_m3s_mean/std/count and target_waterlevel_mm_mean/std/count exist on the enriched table but never enter x — they're pulled out into data.y separately, and attach only to real gauge rows (verified: gauge codes, BD TOPO hydrographic node IDs, and virtual-node marker strings occupy structurally distinct namespaces, so a left-merge on station_code can never mislabel a confluence or virtual node).

2.3 Static vs. dynamic features — and a real temporal pipeline

build_pyg_graph splits every feature by physical temporal nature:

• data.x_static / data.static_feature_names — genuinely time-invariant: elevation, IDPR, catchment area, landcover, geology, cavité proximity, coordinates.
• data.x_dynamic / data.dynamic_feature_names — physically time-varying quantities, still as a single period-aggregated number here (mean/sum over the whole date range, or a latest well reading) — this tensor is a static snapshot of dynamic-natured quantities, not a real series. data.x remains the full combined tensor unchanged; the split is additional, not a replacement.

A genuine [n_nodes, T] series exists separately, in src/graph/dynamic_features.py — build_discharge_timeseries, build_groundwater_timeseries, build_climate_timeseries — built specifically because physics_losses.py's routing_consistency_loss needs a real time dimension and had nothing to consume before this existed. Same loaders as everywhere else, no re-fetching; the only difference is that these functions pivot to wide [date x station_code] form instead of collapsing to one aggregate the way node_features.py's add_*_features do.

Two real challenges, not incidental engineering:

• Groundwater reports on wildly irregular schedules (confirmed: 13 different "latest dates" among 18 real wells within 20 km of one station). Each well is resampled to a common daily grid via forward-fill (a water table changes slowly — carrying the last known reading forward is standard practice, not an invented shortcut) before spatial averaging, not after — averaging raw irregular readings per exact calendar date is exactly what made the static version undercount real coverage by 5–10x before that was fixed (§3.3). The spatial neighbor-set per node is computed once, reused across every date — verified fast at real reach-graph scale (21.6s for 2,900 nodes × 14 years daily, real 272k-row ADES data).
• Discharge deliberately does not get forward-filled the way groundwater does — a missing daily reading stays missing, since discharge genuinely changes day to day and papering over a gap with yesterday's value would misrepresent it.

scripts/build_dynamic_tensors.py is the actual wiring: builds these tensors for a basin, saves them, and feeds discharge directly into routing_consistency_loss alongside build_routing_index — the real integration point, not just parallel unconnected pieces.

Climate is untested against real data — no real ERA5/safran_path files were available to validate build_climate_timeseries against in this project's development environment; the logic mirrors the already-tested discharge pivot directly, but verify the real output before trusting it.

2.4 Date-range filtering

build_node_features/enrich_reach_graph.py accept a date_range applied to every time-varying source (groundwater, climate, hydrometric targets) together, so all three describe the same period rather than each silently aggregating over its own full, differently-shaped history (ADES wells reporting from the 1970s to 2026 on wildly different schedules; ERA5 spanning 1960–2026; hydrometric records with their own per-station ranges entirely).

Default: 2013-01-01 to 2026-12-31 — computed, not guessed, via a brute-force interval-overlap check across all 8 discharge-gauged stations' real date ranges. This is the window that maximizes simultaneous station coverage: 6 of 8 stations, 8,923 real, quality-filtered observations (code_qualification >= 16, the same threshold HydrometricLoader itself applies — a naive raw count that skips this filter gives 13,084, which is what an earlier pass at this analysis originally reported before the discrepancy was traced and corrected). Two stations (H403301101: 1969–1985, H605022010: 1970–1980) are permanently excluded by any reasonable window — a ~35–40 year dead gap separates them from every other station's record, so including them would mean spanning six mostly-empty decades, not a genuine improvement.

2.5 Physics-informed loss terms (physics_losses.py)

Four constraints, each tied to real graph structure, not generic:

Term	Constraint	Applies to
confluence_mass_balance_loss	Q_confluence ≈ sum(Q_upstream_branches) — new mass genuinely enters	is_confluence nodes
split_rejoin_conservation_loss	Q_split ≈ Q_rejoin — same water, no new mass	paired braid_id nodes
routing_consistency_loss	Q_downstream[t] ≈ Q_upstream[t - lag], lag from real distance_km/slope	every edge, real [n_nodes, T] via dynamic_features.py (§2.3)
water_balance_loss	P - ET - Q - ΔS ≈ 0 in volume terms	nodes with cumulative_catchment_area_km2

Confluence and split/rejoin are deliberately different constraints, not one generic "conserve mass everywhere" rule — a model that only learned "sum the inflows" would get a split/rejoin wrong, since a rejoin's two branches together should equal the split's value, not add something new on top. All four apply graph-wide, not just at the 27 labeled gauges — that's the actual mechanism by which sparse supervision generalizes to the ~4,500 ungauged nodes, not an incidental detail. ΔS (storage change) defaults to zero, a named steady-state approximation — this project has no direct basin-wide storage measurement, only sparse well levels, which aren't the same thing.

All four are NaN-masked, not just tolerant of complete data. Real ground- truth Q is ~93.5% NaN by construction (only real gauges with real observations ever have a value — confirmed against the real discharge tensor) — that's the normal shape of the data, not a rare edge case. The shared _mse helper every loss function uses previously computed a plain mean, so a single NaN anywhere in a residual silently poisoned the entire loss to NaN — confirmed as a real, not hypothetical, failure: calling routing_consistency_loss directly on the real discharge tensor returned NaN before this was fixed. _mse now masks NaN out before averaging (returning NaN only if truly nothing usable exists at all, which is a real "no data" signal worth keeping, not silently averaging to a misleading 0) — verified with the exact real scenario that first exposed the bug: routing_consistency_loss on the real discharge tensor now returns a real number instead of NaN.

This also means these functions are directly usable as a diagnostic against real historical data alone, independent of any trained model — e.g. "does real observed discharge at two connected gauges actually satisfy the routing physics" — a genuine, model-free sanity check on both the physics math and the graph topology, not just a training-time loss term.

2.6 Two graphs, not one

build_pyg_graphs_per_basin() returns {0: eure_graph, 1: risle_graph}, each with its own local 0..n-1 node indexing, rather than one merged Data object with two disconnected components. La Eure and La Risle are distinct hydrographic systems with nothing connecting them at the surface, and PyTorch Geometric's own batching (Batch.from_data_list) expects a list of separate small graphs — building two graphs from the start matches that convention directly.

2.7 What it looks like

The Streamlit explorer (src/app.py) has two views. "Explore" renders the original click-to-read interface over real course geometry. "Network validation" renders the full reach graph — confluences as diamonds, gauges as elevation-colored circles, every edge as one line trace regardless of edge count (verified fast at real scale: 0.29s to build a figure for ~2,900 edges) — specifically for visually confirming the topology looks like a real river network before trusting it as model input.

---

3. Datasets

Every dataset here has its own quirks, and in a couple of cases the quirks materially affect what the data means.

3.1 Station roster (station_list.csv, station_elevations.csv)

27 stations across the two basins, spanning three French departments — verified directly against the real roster: 12 in Eure (27), 10 in Eure-et-Loir (28, the Eure's southern tributaries near Chartres/Dreux — Voise, Drouette, and others), 2 in Orne (61). Split roughly by Hub'Eau code prefix (H4xx… for La Eure, H6xx… for La Risle — a heuristic based on observed codes, not a documented rule). Elevation comes from Open Topo Data's eudem25m endpoint (scripts/download_elevation.py), queried per station coordinate — a point lookup, not a raster, so there's no slope or catchment information hiding in it.

The row order in station_list.csv does not follow the river's course — verified directly, it jumps around in both latitude and elevation. Anything that needs upstream/downstream ordering has to derive it from elevation, latitude, or real centerline/graph position; never from file order.

Not every station in this list is actively gauged. Cross-referencing station names against the hydrometric data turned up three categories worth knowing about:

• Manual "observateur" stations — read by a person, not telemetered, so there's no digital time series to have. Four of these in the current roster.
• Partner-network ("SEBV") stations — operated outside the standard Hub'Eau telemetry network, likely need a different data source entirely if you want their readings. H431021010 is one of these.
• Everything else with no data is unexplained from the name alone and worth a direct check on Hub'Eau's site before assuming it's just a gap.

3.2 Hydrometric data (hydrometric/, via scripts/download_hubeau.py)

Discharge and water level from Hub'Eau's obs_elab endpoint. One thing that trips up a naive read: both files contain a mix of grandeur_hydro_elab codes, not just the variable implied by the filename. discharge_observations.csv has HIXnJ/HIXM (water-level codes) sitting right alongside QmnJ (discharge) rows for the same stations — HydrometricLoader filters each file down to its intended grandeur code explicitly (QmnJ from the discharge file, HIXnJ from the water-level file) rather than trusting the filename.

HydrometricLoader also filters on Hub'Eau's own code_qualification field, keeping only >= 16 (their "acceptable"/"good" threshold) and dropping lower- quality/provisional readings. This is real and meaningful, not a rounding detail — traced directly against the raw discharge file for the 2013–2026 window (§2.4): 13,084 raw QmnJ rows in range, of which 4,022 have code_qualification == 12 (below the threshold) and get correctly excluded, leaving 8,923. Any manual read of the raw CSVs that skips this filter will overcount real usable observations by close to a third.

Only 8 of the 27 stations have any QmnJ (daily mean discharge) rows at all. Several others report water level only. This isn't evenly distributed and matters a lot for anything downstream that assumes "gauged" means "has both variables" — it usually doesn't.

3.3 Groundwater (ades/)

ADES piezometer data: 113 wells in the watershed extract, 102 of them with actual level readings, going back as far as 1967 for some wells and to 2026-07-05 for the most recent reading at time of writing. ADESLoader.load() merges the levels file against the stations file on code_bss and renames x/y to lon/lat — those columns are already in degrees in this dataset, not a projected CRS, so no reprojection happens or is needed.

node_features.py's add_groundwater_features does not use ADESLoader.aggregate_to_stations — that method loops per station and does a full haversine .apply() over the entire groundwater dataframe for each one. At 27 stations against ~272k readings that's slow but tolerable; at the reach graph's ~4,500 nodes it's over a billion row-wise Python calls, confirmed as a genuine, not hypothetical, multi-hour hang. The fix (reduce to each well's latest reading first, then a KD-tree coarse prefilter + exact haversine on the small candidate set) turned out to also fix a real accuracy bug: the old method required wells to share the exact same reporting date before averaging, but real wells report on wildly different schedules (18 real wells within 20 km of one station spanned 13 different "latest dates," one from 1972) — silently discarding most real coverage every time.

Groundwater is used as a station-level input covariate, not as a graph edge. Well proximity alone isn't sufficient grounds for a subsurface/karst connectivity edge — that would need either correlated well hydrographs over time or a shared BDLISA aquifer-unit code (groundwater_stations.csv has a codes_bdlisa column available for exactly this kind of check; still unused).

Well coverage is not uniform across the two basins. The Eure's southern reach (south of roughly 48.68°N, toward Chartres) has essentially zero wells within range in this extract.

3.4 Climate (safran/, via download_era5_sample.py / download_era5_full.py)

Despite the safran naming throughout this codebase (a holdover from an earlier plan to use Météo-France's SAFRAN reanalysis), the actual data is ERA5 from Copernicus's Climate Data Store, pulled via cdsapi. ERA5 splits instantaneous variables (temperature, wind) from accumulated ones (precipitation, evaporation, radiation, snowfall, runoff) at the API level — download_era5_full.py downloads each set separately per year and merges them, because the CDS API rejects mixed requests. The full pull spans 1960–2026 and is genuinely slow.

SAFRANLoader interpolates the ERA5 grid to every station in one vectorized xarray call per file, not one .sel() + .to_dataframe() call per station — the per-station loop version does real per-call work (an index lookup, then a full DataFrame conversion) that's tolerable at 27 stations (1,800 calls across ~67 year-files) but was confirmed to actually hang at the reach graph's ~4,500 nodes (193,000 calls). Vectorized indexing with DataArray indexers sharing a station dimension does every station in one call per file instead.

3.5 IDPR (idpr.csv)

BRGM's Indice de Développement et de Persistance des Réseaux — an infiltration-vs-runoff tendency index, and the closest thing this project has to a real soil/drainage covariate. It's an integrated hydrological behavior indicator (infiltration tendency), not raw soil texture data, but arguably more directly useful for a streamflow model than a texture map would be on its own — paired with BD Charm-50 geology (§3.9) for the broader hydrological context soil data would otherwise provide. The file used here is already one row per station (station_id matching station_code exactly, verified 1:1 against all 27 stations), so node_features.py does a direct ID join when possible rather than nearest-neighbor search, falling back to spatial nearest-neighbor for any station code that isn't an exact match (every non-gauge reach-graph node, and — a real, minor precision trade-off worth knowing — every gauge too, once the table also contains non-gauge codes, since the exact-match path requires the entire table to match IDPR's station list).

3.6 Catchment area — two independent sources

Hub'Eau (catchment_area.csv, via scripts/download_catchment.py): published on the site referentiel, not the station referentiel — surface_bv on hydrometrie/referentiel/sites, in km². Since one site can have several stations, the download script does two passes: station → code_site, then code_site → surface_bv. 16 of 27 stations have a value. This number is cumulative — the total catchment area draining to that point, all the way to the source.

BD TOPO, graph-wide (cumulative_catchment_area_km2, via scripts/compute_cumulative_catchment.py): sums BD TOPO's incremental catchment polygons upstream of any node, via the real graph topology — distinct polygons counted once even when many nodes/edges share the same coarse polygon (verified with a hand-computed test case specifically checking this). Covers ~98% of nodes graph-wide, not just the 27 gauges — the actual fix for the "confluences and virtual nodes have no catchment area at all" gap.

Cross-checked against Hub'Eau's real values on real gauges — and there's a real, identified bias, not a clean match. Ratio (BD-TOPO-summed ÷ Hub'Eau) runs from about 0.75 to 1.25 for smaller catchments (< ~800 km², plausibly normal polygon-boundary/digitization precision) but drops to 0.75–0.89 for the largest catchments (> ~3,500 km²) — a clean, monotonic pattern, not noise. Most likely cause: bounding-box truncation — the original BD TOPO pull bbox had only a 9.6 km margin on its southern edge (the tightest of all four directions, and south is exactly where the Eure's longest upstream tributaries run, toward Chartres/Dreux), not a safe margin for real watershed extent. The bbox in download_bdtopo_hydro.py was widened afterward (from (0.3, 48.3, 1.7, 49.5) to (-0.1, 47.7, 2.1, 49.9), ~2.9x the area) — the full download_bdtopo_hydro.py → build_reach_graphs.py → enrich_reach_graph.py → compute_cumulative_catchment.py chain needs re-running against the wider box to actually resolve this, which had not yet happened as of the last verified run in this project.

3.7 BD TOPO hydrography (bdtopo_hydro/, via scripts/download_bdtopo_hydro.py)

IGN's BD TOPO / BD TOPAGE hydrographic network, pulled from the Geoplateforme WFS (https://data.geopf.fr/wfs) rather than downloaded as a national bulk file — the download script queries a bounding box around the two basins instead (see §3.6 for why that box was widened). Three layers, all scoped to that bbox:

• troncon_hydrographique.geojson — river centerline reaches, now the primary source for graph topology too (§2), via lien_vers_noeud_ hydrographique_ini/fin and sens_de_l_ecoulement. Real per-vertex altitude data doubles as a fine-grained elevation profile, denser than anything derivable from the 27 gauge points alone.
• surface_hydrographique.geojson — hydrographic surfaces, including a Nature attribute that's supposed to flag karst-influenced reaches. IGN documents this attribute as provisional and incomplete.
• bassin_versant_topographique.geojson — catchment polygons, incremental (see §3.6).

WFS axis order: when a BBOX parameter's CRS is given via the URN form, the OGC spec requires latitude, longitude axis order — the opposite of the lon,lat order most GIS tools use by default. Getting this backwards doesn't raise an error; it silently matches zero real features. download_bdtopo_hydro.py and scripts/download_bdcavites.py both try lon,lat first and automatically retry with the axes swapped if that comes back empty.

Real branching topology fixed a naive assumption. Filtering 30,045 tronçons down to a single named river and building a graph from their endpoints does not give one connected line — for "Risle" alone, 1,195 name-matched tronçons split into 132 disconnected components. Broadening the name filter to include known tributaries (§2.1) initially made this worse (487/214 components), traced to short/generic tributary names ("Bec", "Avre") matching unrelated streams elsewhere within the ~100×130 km bbox — fixed by requiring every name-matched tronçon to also fall within a real distance of a known gauge (load_troncons_for_basin's anchor_radius_km), and by selecting the connected component actually containing the most real gauges rather than the component with the most raw tronçons (best_component_for_stations) — proven to matter, not just theoretically: a synthetic adversarial test showed the naive "biggest component" approach picking a larger but entirely unrelated decoy network over the real one.

The bétoire finding: two stations in the roster are explicitly named "[amont bétoire]" and "[aval bétoire]" in Hub'Eau's own site names — bétoire being the Normandy dialect term for a karst swallow-hole. Three edges spanning that stretch on La Risle (H605641101 → H605022010 → H605641401 → H605641201) are flagged verified_continuous=False. BD TOPO's own karst attribute doesn't currently confirm it (see the provisional- attribute note above) — scripts/download_bdcavites.py (§3.8) exists specifically to get an independent, purpose-built second check on this, rather than relying only on naming inference.

3.8 BDCavités (bdcavites/, via scripts/download_bdcavites.py)

BRGM's national underground cavity inventory (sinkholes, quarries, natural cavities), via Géorisques' WFS (georisques.gouv.fr/services, typeName CAVITE_LOCALISEE, confirmed live and GeoJSON-capable directly against the real service). Built specifically as an independent check on the bétoire finding (§3.7) — a purpose-built cavity dataset, not inference from station naming or a provisional BD TOPO attribute. One real caveat: departments 75/78/91/92/93/94/95 (Paris region, unrelated to this project) are excluded from BDCavités entirely, and the Eure department's own inventory was among the later batches of the national 2001–2013 completion program — worth checking coverage density before treating a sparse result as a negative finding rather than incomplete data.

3.9 Geology (bdcharm50/, via scripts/download_bdcharm.py)

BRGM's BD Charm-50, harmonized 1:50,000 geological maps — free, open (Licence Ouverte), no authentication, direct per-department ZIP download from InfoTerre (a genuinely different access pattern than the WFS sources elsewhere in this project: fixed URL per department, no bbox query, no axis- order ambiguity). Departments 27 (Eure), 28 (Eure-et-Loir), 61 (Orne) — verified directly against the real, complete station roster (§3.1), not guessed. A separate, CIGAL-membership-gated distribution of similar data exists for at least one other French region; this project only uses the free InfoTerre path.

3.10 Landcover and NDVI (scripts/fetch_landcover.py, scripts/fetch_worldcover_ndvi.py)

ESA WorldCover, sampled at real gauge points from the public AWS S3 Cloud- Optimized GeoTIFFs.

Landcover classification uses the product's 3°×3° tile grid; every real station coordinate falls inside exactly one tile (N48E000), verified directly against all 27 real coordinates. NDVI uses the annual composites' 1°×1° tile grid instead — genuinely different from the classification grid, looked up per-station via VITO's own authoritative tile-index grid file (esa_worldcover_grid_composites.fgb) rather than a hand-guessed S3 key pattern. Both need AWS_NO_SIGN_REQUEST=YES for s3://-scheme tile URLs specifically — a plain HTTPS URL to the same public bucket needs no signing at all.

Both currently cover only the 27 real gauges (exact station_code match), same limitation as Hub'Eau's catchment_area_km2 before the cumulative-BD- TOPO fix (§3.6) — extending either script to the full reach graph is unstarted work, not a design decision.

3.11 Centerline generation

Only relevant to the older single-chain pipeline (build_surface_edges, still available for direct comparison/debugging) — the reach graph (§2) derives its topology directly from BD TOPO's own node linkage and doesn't use these centerline files at all.

centerlines/eure_centerline.csv and centerlines/risle_centerline.csv — the geometry build_surface_edges orders stations against — are generated by scripts/analyze_bdtopo_hydro.py --export-centerline. It filters troncon_hydrographique.geojson (§3.7) down to the named river, builds a graph from the tronçon endpoints, and walks the longest path through it via double-BFS shortest-path to get one continuous, correctly-ordered sequence of real coordinates. This is the accurate method: real BD TOPO vector geometry snaps stations to within 0.05 km on average.

scripts/extract_river_centerline.py is a separate, standalone technique for deriving a centerline directly from a traced map image, for a river or region without BD TOPO coverage: color-threshold the image to isolate a traced route, skeletonize it, walk the end-to-end path the same double-BFS way, then georeference by fitting a least-squares affine transform from a handful of manually-read reference-point pixel positions to their known coordinates. This produces a reliable shape, but the absolute position is only as good as the georeferencing step — residuals at the reference points run to a few kilometers with a handful of manually-read points, giving roughly a 0.88 km average snap distance rather than 0.05 km. It's the fallback when a real vector source isn't available, not the method used for the current centerlines/ files.

---

4. Applications

src/app.py (Streamlit) has two views, selected by a radio at the top:

Explore — the original click-to-read UI over real course geometry: pick a river, click (or slide) along its course, see interpolated elevation, estimated groundwater level, and — for whichever real gauge is nearest that point — water level, discharge, and rating-curve plots pulled directly from HydrometricLoader's own plotting methods rather than reimplemented.

Click support uses Streamlit's native chart-selection (st.plotly_chart(..., on_select="rerun")), not a third-party click-handling package. The click handler and the position slider share a single source of truth by design: Streamlit only honors a slider's value= argument the first time that widget is created, and on every later rerun returns whatever's stored under that widget's own session-state key — so the click handler writes directly into the slider's own key before it's instantiated, rather than a separate key. It also de-duplicates incoming click events, since Streamlit's chart-selection state persists across reruns caused by other widgets and would otherwise re-fire on every unrelated interaction.

Network validation — renders the full reach graph (§2): every edge as one Plotly line trace regardless of edge count (a trace-per-edge approach doesn't hold up at ~5,000+ edges; verified fast at real scale — 0.29s to build a figure for ~2,900 edges), real confluences as diamond markers, real gauges as elevation-colored circles. Metrics card reports node/edge/confluence/gauge counts and, when available, IDPR and cumulative-catchment coverage. Virtual infill nodes are deliberately not drawn individually — at ~2,400 per basin, markers for each would bury the actual validation signal (do confluences sit where a tributary visibly joins the line? do gauges sit on the network, not offset from it?) rather than help it. Reads directly from reach_graph/ {basin}_nodes_enriched.csv, keyed on file modification time so a re-run of build_reach_graphs.py/enrich_reach_graph.py is picked up automatically — st.cache_data otherwise keys purely on function arguments, not file contents, and this was confirmed to actually cause stale numbers once during development, not just a theoretical risk.

---

5. Testing (src/test_build_graph.py)

Not a unit test suite in the pytest sense — a script with two independent sections, both run from main().

run_checks — the original single-chain pipeline: runs node_features → build_surface_edges → build_pyg_graph(s) against real data and checks the result is sane — no NaN/Inf in the feature tensor, no accidental cross-basin edges, targets genuinely excluded from the model input, edge indices within bounds, bidirectional edge count exactly double the directed count, per-basin node counts summing to the combined total, standardized features actually landing near zero mean / unit variance, the known_losing_reaches flag actually taking effect, and mean/max snap_distance_km per basin against whatever centerline is currently in centerlines/.

run_reach_graph_checks — the reach graph pipeline, gracefully skipped (not a failure) if reach_graph/ doesn't exist yet. Mostly regression tests for three bugs found and fixed during development, kept here specifically so they can't silently reintroduce themselves:

• structural columns (is_gauged/is_confluence/etc.) never leak into feature_names, but remain accessible as their own Data attributes
• target values never attach to a non-gauge node, and target coverage never exceeds the real gauge count
• edge_attr stays exactly 3 columns despite extra edge metadata (toponym, cleabs) sitting on the real edges table
• physics_losses.py's build_confluence_index/build_braid_index produce counts matching is_confluence/is_rejoin_point sums, with every index within node bounds and every confluence having ≥ 2 upstream branches
• IDPR and cumulative_catchment_area_km2 presence/coverage are reported explicitly (the latter compared against the Hub'Eau-only baseline it's meant to exceed)

Exits 0 on a clean pass across both sections, 1 otherwise — usable as a pre-commit or CI gate if that's ever set up.

---

6. Known limitations and open questions

• cumulative_catchment_area_km2 underestimates the largest catchments by up to ~25%, traced to the BD TOPO pull's original bounding box having an insufficient southern margin. The bbox has been widened in download_bdtopo_hydro.py; the full re-pull-and-rebuild chain needs re-running for this to actually resolve. Treat the largest catchments' values as approximate until then.
• Climate's genuine time series (build_climate_timeseries) is untested against real data — discharge and groundwater's equivalents are; verify climate's real output before relying on it.
• Landcover and NDVI only cover the 27 real gauges, not the full reach graph — same scope catchment_area_km2 had before its cumulative-BD-TOPO extension.
• No model exists yet. This repo builds the graph and the physics-loss substrate a model would train against; there is no architecture, forward pass, or training loop here.
• The karst losing-reach flag rests on naming evidence and a BDCavités cross-check (§3.7–3.8), not a fully confirmed BD TOPO classification.
• The groundwater-well BDLISA aquifer-unit field is unused. First place to look if a subsurface connectivity edge is ever justified with real evidence rather than proximity.

---

7. Running things

Data acquisition (from repo root, in roughly dependency order):

python -m scripts.download_hubeau
python -m scripts.download_elevation
python -m scripts.download_era5_full        # slow; download_era5_sample.py first if just testing
python -m scripts.extract_era5
python -m scripts.download_catchment
python -m scripts.download_bdtopo_hydro --check   # verify typeNames before the real pull
python -m scripts.download_bdtopo_hydro
python -m scripts.download_bdcavites --check
python -m scripts.download_bdcavites
python -m scripts.download_bdcharm

Build and validate the reach graph:

python -m scripts.build_reach_graphs --data-root datasets
python -m scripts.enrich_reach_graph --data-root datasets   # --skip-climate if that step hangs
python -m scripts.compute_cumulative_catchment --data-root datasets
python -m scripts.diagnose_confluences --data-root datasets --basin eure
python -m scripts.diagnose_confluences --data-root datasets --basin risle

Build genuine [n_nodes, T] dynamic tensors and verify the physics-loss wiring:

python -m scripts.build_dynamic_tensors --data-root datasets --basin risle
python -m scripts.build_dynamic_tensors --data-root datasets --basin eure

Landcover / NDVI, real gauges only (needs rasterio, and geopandas for NDVI's tile lookup):

python -m scripts.fetch_landcover --check
python -m scripts.fetch_landcover
python -m scripts.fetch_worldcover_ndvi --check
python -m scripts.fetch_worldcover_ndvi

Validate everything against whatever's actually in datasets/:

python -m src.test_build_graph --data-root datasets

Run the explorer:

streamlit run src/app.py -- --data-root datasets