Spaces:
Running on Zero
Running on Zero
Upload folder using huggingface_hub
Browse files- .gitattributes +2 -0
- README.md +131 -72
- datasets/README.md +320 -0
- docs/images/eure_reach_graph.png +3 -0
- docs/images/explore_view.png +3 -0
- docs/images/risle_reach_graph.png +0 -0
.gitattributes
CHANGED
|
@@ -213,3 +213,5 @@ datasets/bdcharm50/dept_061/GEO050K_HARM_061_P_STRUCT_2154.dbf filter=lfs diff=l
|
|
| 213 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.dbf filter=lfs diff=lfs merge=lfs -text
|
| 214 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.lyr filter=lfs diff=lfs merge=lfs -text
|
| 215 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.shp filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 213 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.dbf filter=lfs diff=lfs merge=lfs -text
|
| 214 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.lyr filter=lfs diff=lfs merge=lfs -text
|
| 215 |
datasets/bdcharm50/dept_061/GEO050K_HARM_061_S_FGEOL_2154.shp filter=lfs diff=lfs merge=lfs -text
|
| 216 |
+
docs/images/eure_reach_graph.png filter=lfs diff=lfs merge=lfs -text
|
| 217 |
+
docs/images/explore_view.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -72,9 +72,9 @@ PoC_v1/
|
|
| 72 |
│ ├── enrich_reach_graph.py # runs node_features.py against the reach graph
|
| 73 |
│ ├── compute_cumulative_catchment.py # graph-wide catchment area from BD TOPO polygons
|
| 74 |
│ ├── diagnose_confluences.py # verify real vs. artifact confluences
|
|
|
|
| 75 |
│ ├── download_bdcavites.py # Géorisques BDCavités (sinkhole/cavity inventory)
|
| 76 |
│ ├── download_bdcharm.py # BRGM BD Charm-50 harmonized geology, per department
|
| 77 |
-
│ ├── fetch_idpr_brgm.py # (unused — see §3.5) live IDPR re-fetch attempt
|
| 78 |
│ ├── fetch_landcover.py # ESA WorldCover landcover class, real gauges
|
| 79 |
│ └── fetch_worldcover_ndvi.py # ESA WorldCover NDVI percentile composite
|
| 80 |
│
|
|
@@ -103,8 +103,9 @@ PoC_v1/
|
|
| 103 |
│ └── graph/
|
| 104 |
│ ├── build_graph.py # PyG conversion: x_static/x_dynamic split, structural columns
|
| 105 |
│ ├── build_reach_graph.py # real reach topology: confluences, splits/rejoins, MultiDiGraph
|
| 106 |
-
│ ├── node_features.py # pulls every loader into one feature table
|
| 107 |
-
│
|
|
|
|
| 108 |
│
|
| 109 |
├── datasets/ # not checked in; populated by the scripts above
|
| 110 |
│ ├── station_list.csv # raw station roster (X, Y, names, INSEE, etc.)
|
|
@@ -263,24 +264,55 @@ real gauge rows (verified: gauge codes, BD TOPO hydrographic node IDs, and
|
|
| 263 |
virtual-node marker strings occupy structurally distinct namespaces, so a
|
| 264 |
left-merge on `station_code` can never mislabel a confluence or virtual node).
|
| 265 |
|
| 266 |
-
### 2.3 Static vs. dynamic features
|
| 267 |
|
| 268 |
-
`build_pyg_graph`
|
| 269 |
|
| 270 |
- **`data.x_static`** / **`data.static_feature_names`** — genuinely
|
| 271 |
-
time-invariant: elevation, IDPR, catchment area, landcover,
|
|
|
|
| 272 |
- **`data.x_dynamic`** / **`data.dynamic_feature_names`** — physically
|
| 273 |
-
time-varying quantities
|
| 274 |
-
|
| 275 |
-
|
| 276 |
-
|
| 277 |
-
|
| 278 |
-
|
| 279 |
-
|
| 280 |
-
|
| 281 |
-
|
| 282 |
-
|
| 283 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 284 |
|
| 285 |
### 2.4 Date-range filtering
|
| 286 |
|
|
@@ -312,7 +344,7 @@ Four constraints, each tied to real graph structure, not generic:
|
|
| 312 |
|---|---|---|
|
| 313 |
| `confluence_mass_balance_loss` | `Q_confluence ≈ sum(Q_upstream_branches)` — new mass genuinely enters | `is_confluence` nodes |
|
| 314 |
| `split_rejoin_conservation_loss` | `Q_split ≈ Q_rejoin` — same water, no new mass | paired `braid_id` nodes |
|
| 315 |
-
| `routing_consistency_loss` | `Q_downstream[t] ≈ Q_upstream[t - lag]`, lag from real `distance_km`/slope | every edge
|
| 316 |
| `water_balance_loss` | `P - ET - Q - ΔS ≈ 0` in volume terms | nodes with `cumulative_catchment_area_km2` |
|
| 317 |
|
| 318 |
Confluence and split/rejoin are deliberately different constraints, not one
|
|
@@ -326,6 +358,27 @@ zero, a named steady-state approximation — this project has no direct
|
|
| 326 |
basin-wide storage measurement, only sparse well *levels*, which aren't the
|
| 327 |
same thing.
|
| 328 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 329 |
### 2.6 Two graphs, not one
|
| 330 |
|
| 331 |
`build_pyg_graphs_per_basin()` returns `{0: eure_graph, 1: risle_graph}`,
|
|
@@ -346,6 +399,14 @@ count (verified fast at real scale: 0.29s to build a figure for ~2,900
|
|
| 346 |
edges) — specifically for visually confirming the topology looks like a real
|
| 347 |
river network before trusting it as model input.
|
| 348 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 349 |
---
|
| 350 |
|
| 351 |
## 3. Datasets
|
|
@@ -382,10 +443,7 @@ about:
|
|
| 382 |
- Everything else with no data is unexplained from the name alone and worth a
|
| 383 |
direct check on Hub'Eau's site before assuming it's just a gap.
|
| 384 |
|
| 385 |
-
|
| 386 |
-
observations within the project's 2013–2026 date window** (§2.4) — worth
|
| 387 |
-
knowing before assuming "8 gauged stations" translates directly into training
|
| 388 |
-
examples.
|
| 389 |
|
| 390 |
### 3.2 Hydrometric data (`hydrometric/`, via `scripts/download_hubeau.py`)
|
| 391 |
|
|
@@ -467,24 +525,19 @@ call per file instead.
|
|
| 467 |
|
| 468 |
BRGM's *Indice de Développement et de Persistance des Réseaux* — an
|
| 469 |
infiltration-vs-runoff tendency index, and the closest thing this project has
|
| 470 |
-
to a real soil/drainage covariate
|
| 471 |
-
|
| 472 |
-
|
| 473 |
-
|
| 474 |
-
|
| 475 |
-
|
| 476 |
-
|
| 477 |
-
|
| 478 |
-
|
| 479 |
-
|
| 480 |
-
|
| 481 |
-
|
| 482 |
-
|
| 483 |
-
`IDPR_50M` raster layer) was attempted, to get fresher values and eventually
|
| 484 |
-
cover the full reach graph rather than just the 27 gauges. **It failed** in
|
| 485 |
-
testing and the project is currently using the original, already-uploaded
|
| 486 |
-
`idpr.csv` instead — not resolved further, since the existing file already
|
| 487 |
-
gives real, usable IDPR coverage for every gauge.
|
| 488 |
|
| 489 |
### 3.6 Catchment area — two independent sources
|
| 490 |
|
|
@@ -600,50 +653,24 @@ InfoTerre path.
|
|
| 600 |
### 3.10 Landcover and NDVI (`scripts/fetch_landcover.py`, `scripts/fetch_worldcover_ndvi.py`)
|
| 601 |
|
| 602 |
ESA WorldCover, sampled at real gauge points from the public AWS S3 Cloud-
|
| 603 |
-
Optimized GeoTIFFs
|
| 604 |
-
the last month of this project's active development reported actively resets
|
| 605 |
-
connections from non-browser HTTP clients (TLS fingerprinting, confirmed
|
| 606 |
-
across multiple tools and User-Agents, not a coding problem to work around),
|
| 607 |
-
and separately was slated for full phase-out already past by the time this
|
| 608 |
-
was checked.
|
| 609 |
|
| 610 |
Landcover classification uses the product's 3°×3° tile grid; every real
|
| 611 |
station coordinate falls inside exactly one tile (`N48E000`), verified
|
| 612 |
directly against all 27 real coordinates. NDVI uses the *annual composites'*
|
| 613 |
1°×1° tile grid instead — genuinely different from the classification grid,
|
| 614 |
looked up per-station via VITO's own authoritative tile-index grid file
|
| 615 |
-
(`esa_worldcover_grid_composites.fgb`) rather than a
|
| 616 |
-
|
| 617 |
specifically — a plain HTTPS URL to the same public bucket needs no signing
|
| 618 |
-
at all
|
| 619 |
-
the NDVI grid's returned URLs (`s3://`) initially failed on AWS credential
|
| 620 |
-
errors despite the bucket being fully public.
|
| 621 |
|
| 622 |
Both currently cover only the 27 real gauges (exact `station_code` match),
|
| 623 |
same limitation as Hub'Eau's `catchment_area_km2` before the cumulative-BD-
|
| 624 |
TOPO fix (§3.6) — extending either script to the full reach graph is
|
| 625 |
unstarted work, not a design decision.
|
| 626 |
|
| 627 |
-
### 3.11
|
| 628 |
-
|
| 629 |
-
**SoilGrids (ISRIC)** — confirmed non-functional directly, not from a stale
|
| 630 |
-
search result: even a bare `lon`/`lat` query to the live REST API returned
|
| 631 |
-
`422` consistently, consistent with ISRIC's own currently-posted "temporarily
|
| 632 |
-
paused" service notice.
|
| 633 |
-
|
| 634 |
-
**INRAE's national soil survey (RRP/BDGSF)** — not a uniformly-accessible
|
| 635 |
-
source at all. Access is explicitly described as depending on regional/
|
| 636 |
-
departmental "référents" (varies by department, sometimes needs a formal
|
| 637 |
-
agreement with the regional chamber of agriculture), and INRAE's own
|
| 638 |
-
documentation states outright that the more detailed scale "n'est pas encore
|
| 639 |
-
en accès libre." IDPR (§3.5) and BD Charm-50 geology (§3.9) are this
|
| 640 |
-
project's actual substitutes for the hydrologically-relevant part of what
|
| 641 |
-
soil data would otherwise provide — not literal soil texture data, but IDPR
|
| 642 |
-
specifically is an *integrated hydrological behavior* indicator (infiltration
|
| 643 |
-
tendency), arguably more directly useful for a streamflow model than a raw
|
| 644 |
-
soil property map would be on its own.
|
| 645 |
-
|
| 646 |
-
### 3.12 Centerline generation
|
| 647 |
|
| 648 |
**Only relevant to the older single-chain pipeline** (`build_surface_edges`,
|
| 649 |
still available for direct comparison/debugging) — the reach graph (§2)
|
|
@@ -751,7 +778,32 @@ pre-commit or CI gate if that's ever set up.
|
|
| 751 |
|
| 752 |
---
|
| 753 |
|
| 754 |
-
## 6.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 755 |
|
| 756 |
Data acquisition (from repo root, in roughly dependency order):
|
| 757 |
|
|
@@ -778,6 +830,13 @@ python -m scripts.diagnose_confluences --data-root datasets --basin eure
|
|
| 778 |
python -m scripts.diagnose_confluences --data-root datasets --basin risle
|
| 779 |
```
|
| 780 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 781 |
Landcover / NDVI, real gauges only (needs `rasterio`, and `geopandas` for NDVI's
|
| 782 |
tile lookup):
|
| 783 |
|
|
|
|
| 72 |
│ ├── enrich_reach_graph.py # runs node_features.py against the reach graph
|
| 73 |
│ ├── compute_cumulative_catchment.py # graph-wide catchment area from BD TOPO polygons
|
| 74 |
│ ├── diagnose_confluences.py # verify real vs. artifact confluences
|
| 75 |
+
│ ├── build_dynamic_tensors.py # genuine [n_nodes, T] tensors, wired into physics_losses.py
|
| 76 |
│ ├── download_bdcavites.py # Géorisques BDCavités (sinkhole/cavity inventory)
|
| 77 |
│ ├── download_bdcharm.py # BRGM BD Charm-50 harmonized geology, per department
|
|
|
|
| 78 |
│ ├── fetch_landcover.py # ESA WorldCover landcover class, real gauges
|
| 79 |
│ └── fetch_worldcover_ndvi.py # ESA WorldCover NDVI percentile composite
|
| 80 |
│
|
|
|
|
| 103 |
│ └── graph/
|
| 104 |
│ ├── build_graph.py # PyG conversion: x_static/x_dynamic split, structural columns
|
| 105 |
│ ├── build_reach_graph.py # real reach topology: confluences, splits/rejoins, MultiDiGraph
|
| 106 |
+
│ ├── node_features.py # pulls every loader into one feature table (static, one row/node)
|
| 107 |
+
│ ├── dynamic_features.py # genuine [n_nodes, T] series: discharge, groundwater, climate
|
| 108 |
+
│ └── physics_losses.py # confluence/split-rejoin/routing/water-balance loss terms, NaN-masked
|
| 109 |
│
|
| 110 |
├── datasets/ # not checked in; populated by the scripts above
|
| 111 |
│ ├── station_list.csv # raw station roster (X, Y, names, INSEE, etc.)
|
|
|
|
| 264 |
virtual-node marker strings occupy structurally distinct namespaces, so a
|
| 265 |
left-merge on `station_code` can never mislabel a confluence or virtual node).
|
| 266 |
|
| 267 |
+
### 2.3 Static vs. dynamic features — and a real temporal pipeline
|
| 268 |
|
| 269 |
+
`build_pyg_graph` splits every feature by physical temporal nature:
|
| 270 |
|
| 271 |
- **`data.x_static`** / **`data.static_feature_names`** — genuinely
|
| 272 |
+
time-invariant: elevation, IDPR, catchment area, landcover, geology,
|
| 273 |
+
cavité proximity, coordinates.
|
| 274 |
- **`data.x_dynamic`** / **`data.dynamic_feature_names`** — physically
|
| 275 |
+
time-varying quantities, still as a single period-aggregated number here
|
| 276 |
+
(mean/sum over the whole date range, or a latest well reading) — this
|
| 277 |
+
tensor is a *static snapshot* of dynamic-natured quantities, not a real
|
| 278 |
+
series. `data.x` remains the full combined tensor unchanged; the split is
|
| 279 |
+
additional, not a replacement.
|
| 280 |
+
|
| 281 |
+
**A genuine `[n_nodes, T]` series exists separately**, in
|
| 282 |
+
`src/graph/dynamic_features.py` — `build_discharge_timeseries`,
|
| 283 |
+
`build_groundwater_timeseries`, `build_climate_timeseries` — built
|
| 284 |
+
specifically because `physics_losses.py`'s `routing_consistency_loss` needs a
|
| 285 |
+
real time dimension and had nothing to consume before this existed. Same
|
| 286 |
+
loaders as everywhere else, no re-fetching; the only difference is that these
|
| 287 |
+
functions pivot to wide `[date x station_code]` form instead of collapsing to
|
| 288 |
+
one aggregate the way `node_features.py`'s `add_*_features` do.
|
| 289 |
+
|
| 290 |
+
Two real challenges, not incidental engineering:
|
| 291 |
+
|
| 292 |
+
- **Groundwater** reports on wildly irregular schedules (confirmed: 13
|
| 293 |
+
different "latest dates" among 18 real wells within 20 km of one station).
|
| 294 |
+
Each well is resampled to a common daily grid via forward-fill (a water
|
| 295 |
+
table changes slowly — carrying the last known reading forward is standard
|
| 296 |
+
practice, not an invented shortcut) *before* spatial averaging, not after —
|
| 297 |
+
averaging raw irregular readings per exact calendar date is exactly what
|
| 298 |
+
made the static version undercount real coverage by 5–10x before that was
|
| 299 |
+
fixed (§3.3). The spatial neighbor-set per node is computed once, reused
|
| 300 |
+
across every date — verified fast at real reach-graph scale (21.6s for
|
| 301 |
+
2,900 nodes × 14 years daily, real 272k-row ADES data).
|
| 302 |
+
- **Discharge** deliberately does *not* get forward-filled the way
|
| 303 |
+
groundwater does — a missing daily reading stays missing, since discharge
|
| 304 |
+
genuinely changes day to day and papering over a gap with yesterday's
|
| 305 |
+
value would misrepresent it.
|
| 306 |
+
|
| 307 |
+
`scripts/build_dynamic_tensors.py` is the actual wiring: builds these tensors
|
| 308 |
+
for a basin, saves them, and feeds discharge directly into
|
| 309 |
+
`routing_consistency_loss` alongside `build_routing_index` — the real
|
| 310 |
+
integration point, not just parallel unconnected pieces.
|
| 311 |
+
|
| 312 |
+
**Climate is untested against real data** — no real ERA5/`safran_path` files
|
| 313 |
+
were available to validate `build_climate_timeseries` against in this
|
| 314 |
+
project's development environment; the logic mirrors the already-tested
|
| 315 |
+
discharge pivot directly, but verify the real output before trusting it.
|
| 316 |
|
| 317 |
### 2.4 Date-range filtering
|
| 318 |
|
|
|
|
| 344 |
|---|---|---|
|
| 345 |
| `confluence_mass_balance_loss` | `Q_confluence ≈ sum(Q_upstream_branches)` — new mass genuinely enters | `is_confluence` nodes |
|
| 346 |
| `split_rejoin_conservation_loss` | `Q_split ≈ Q_rejoin` — same water, no new mass | paired `braid_id` nodes |
|
| 347 |
+
| `routing_consistency_loss` | `Q_downstream[t] ≈ Q_upstream[t - lag]`, lag from real `distance_km`/slope | every edge, real `[n_nodes, T]` via `dynamic_features.py` (§2.3) |
|
| 348 |
| `water_balance_loss` | `P - ET - Q - ΔS ≈ 0` in volume terms | nodes with `cumulative_catchment_area_km2` |
|
| 349 |
|
| 350 |
Confluence and split/rejoin are deliberately different constraints, not one
|
|
|
|
| 358 |
basin-wide storage measurement, only sparse well *levels*, which aren't the
|
| 359 |
same thing.
|
| 360 |
|
| 361 |
+
**All four are NaN-masked, not just tolerant of complete data.** Real ground-
|
| 362 |
+
truth Q is ~93.5% `NaN` by construction (only real gauges with real
|
| 363 |
+
observations ever have a value — confirmed against the real discharge
|
| 364 |
+
tensor) — that's the normal shape of the data, not a rare edge case. The
|
| 365 |
+
shared `_mse` helper every loss function uses previously computed a plain
|
| 366 |
+
mean, so a single `NaN` anywhere in a residual silently poisoned the *entire*
|
| 367 |
+
loss to `NaN` — confirmed as a real, not hypothetical, failure: calling
|
| 368 |
+
`routing_consistency_loss` directly on the real discharge tensor returned
|
| 369 |
+
`NaN` before this was fixed. `_mse` now masks `NaN` out before averaging
|
| 370 |
+
(returning `NaN` only if truly nothing usable exists at all, which is a
|
| 371 |
+
real "no data" signal worth keeping, not silently averaging to a misleading
|
| 372 |
+
`0`) — verified with the exact real scenario that first exposed the bug:
|
| 373 |
+
`routing_consistency_loss` on the real discharge tensor now returns a real
|
| 374 |
+
number instead of `NaN`.
|
| 375 |
+
|
| 376 |
+
This also means these functions are directly usable as a diagnostic against
|
| 377 |
+
real historical data alone, independent of any trained model — e.g.
|
| 378 |
+
"does real observed discharge at two connected gauges actually satisfy the
|
| 379 |
+
routing physics" — a genuine, model-free sanity check on both the physics
|
| 380 |
+
math and the graph topology, not just a training-time loss term.
|
| 381 |
+
|
| 382 |
### 2.6 Two graphs, not one
|
| 383 |
|
| 384 |
`build_pyg_graphs_per_basin()` returns `{0: eure_graph, 1: risle_graph}`,
|
|
|
|
| 399 |
edges) — specifically for visually confirming the topology looks like a real
|
| 400 |
river network before trusting it as model input.
|
| 401 |
|
| 402 |
+

|
| 403 |
+
|
| 404 |
+
|
| 405 |
+

|
| 406 |
+
|
| 407 |
+
|
| 408 |
+

|
| 409 |
+
|
| 410 |
---
|
| 411 |
|
| 412 |
## 3. Datasets
|
|
|
|
| 443 |
- Everything else with no data is unexplained from the name alone and worth a
|
| 444 |
direct check on Hub'Eau's site before assuming it's just a gap.
|
| 445 |
|
| 446 |
+
|
|
|
|
|
|
|
|
|
|
| 447 |
|
| 448 |
### 3.2 Hydrometric data (`hydrometric/`, via `scripts/download_hubeau.py`)
|
| 449 |
|
|
|
|
| 525 |
|
| 526 |
BRGM's *Indice de Développement et de Persistance des Réseaux* — an
|
| 527 |
infiltration-vs-runoff tendency index, and the closest thing this project has
|
| 528 |
+
to a real soil/drainage covariate. It's an *integrated hydrological behavior*
|
| 529 |
+
indicator (infiltration tendency), not raw soil texture data, but arguably
|
| 530 |
+
more directly useful for a streamflow model than a texture map would be on
|
| 531 |
+
its own — paired with BD Charm-50 geology (§3.9) for the broader hydrological
|
| 532 |
+
context soil data would otherwise provide. The file used here is already one
|
| 533 |
+
row per station (`station_id` matching `station_code` exactly, verified 1:1
|
| 534 |
+
against all 27 stations), so `node_features.py` does a direct ID join when
|
| 535 |
+
possible rather than nearest-neighbor search, falling back to spatial
|
| 536 |
+
nearest-neighbor for any station code that isn't an exact match (every
|
| 537 |
+
non-gauge reach-graph node, and — a real, minor precision trade-off worth
|
| 538 |
+
knowing — every gauge too, once the table also contains non-gauge codes,
|
| 539 |
+
since the exact-match path requires the *entire* table to match IDPR's
|
| 540 |
+
station list).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 541 |
|
| 542 |
### 3.6 Catchment area — two independent sources
|
| 543 |
|
|
|
|
| 653 |
### 3.10 Landcover and NDVI (`scripts/fetch_landcover.py`, `scripts/fetch_worldcover_ndvi.py`)
|
| 654 |
|
| 655 |
ESA WorldCover, sampled at real gauge points from the public AWS S3 Cloud-
|
| 656 |
+
Optimized GeoTIFFs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 657 |
|
| 658 |
Landcover classification uses the product's 3°×3° tile grid; every real
|
| 659 |
station coordinate falls inside exactly one tile (`N48E000`), verified
|
| 660 |
directly against all 27 real coordinates. NDVI uses the *annual composites'*
|
| 661 |
1°×1° tile grid instead — genuinely different from the classification grid,
|
| 662 |
looked up per-station via VITO's own authoritative tile-index grid file
|
| 663 |
+
(`esa_worldcover_grid_composites.fgb`) rather than a hand-guessed S3 key
|
| 664 |
+
pattern. Both need `AWS_NO_SIGN_REQUEST=YES` for `s3://`-scheme tile URLs
|
| 665 |
specifically — a plain HTTPS URL to the same public bucket needs no signing
|
| 666 |
+
at all.
|
|
|
|
|
|
|
| 667 |
|
| 668 |
Both currently cover only the 27 real gauges (exact `station_code` match),
|
| 669 |
same limitation as Hub'Eau's `catchment_area_km2` before the cumulative-BD-
|
| 670 |
TOPO fix (§3.6) — extending either script to the full reach graph is
|
| 671 |
unstarted work, not a design decision.
|
| 672 |
|
| 673 |
+
### 3.11 Centerline generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 674 |
|
| 675 |
**Only relevant to the older single-chain pipeline** (`build_surface_edges`,
|
| 676 |
still available for direct comparison/debugging) — the reach graph (§2)
|
|
|
|
| 778 |
|
| 779 |
---
|
| 780 |
|
| 781 |
+
## 6. Known limitations and open questions
|
| 782 |
+
|
| 783 |
+
- **`cumulative_catchment_area_km2` underestimates the largest catchments**
|
| 784 |
+
by up to ~25%, traced to the BD TOPO pull's original bounding box having an
|
| 785 |
+
insufficient southern margin. The bbox has been widened in
|
| 786 |
+
`download_bdtopo_hydro.py`; the full re-pull-and-rebuild chain needs
|
| 787 |
+
re-running for this to actually resolve. Treat the largest catchments'
|
| 788 |
+
values as approximate until then.
|
| 789 |
+
- **Climate's genuine time series (`build_climate_timeseries`) is untested
|
| 790 |
+
against real data** — discharge and groundwater's equivalents are; verify
|
| 791 |
+
climate's real output before relying on it.
|
| 792 |
+
- **Landcover and NDVI only cover the 27 real gauges**, not the full reach
|
| 793 |
+
graph — same scope `catchment_area_km2` had before its cumulative-BD-TOPO
|
| 794 |
+
extension.
|
| 795 |
+
- **No model exists yet.** This repo builds the graph and the physics-loss
|
| 796 |
+
substrate a model would train against; there is no architecture, forward
|
| 797 |
+
pass, or training loop here.
|
| 798 |
+
- **The karst losing-reach flag rests on naming evidence and a BDCavités
|
| 799 |
+
cross-check** (§3.7–3.8), not a fully confirmed BD TOPO classification.
|
| 800 |
+
- **The groundwater-well BDLISA aquifer-unit field is unused.** First place
|
| 801 |
+
to look if a subsurface connectivity edge is ever justified with real
|
| 802 |
+
evidence rather than proximity.
|
| 803 |
+
|
| 804 |
+
---
|
| 805 |
+
|
| 806 |
+
## 7. Running things
|
| 807 |
|
| 808 |
Data acquisition (from repo root, in roughly dependency order):
|
| 809 |
|
|
|
|
| 830 |
python -m scripts.diagnose_confluences --data-root datasets --basin risle
|
| 831 |
```
|
| 832 |
|
| 833 |
+
Build genuine `[n_nodes, T]` dynamic tensors and verify the physics-loss wiring:
|
| 834 |
+
|
| 835 |
+
```bash
|
| 836 |
+
python -m scripts.build_dynamic_tensors --data-root datasets --basin risle
|
| 837 |
+
python -m scripts.build_dynamic_tensors --data-root datasets --basin eure
|
| 838 |
+
```
|
| 839 |
+
|
| 840 |
Landcover / NDVI, real gauges only (needs `rasterio`, and `geopandas` for NDVI's
|
| 841 |
tile lookup):
|
| 842 |
|
datasets/README.md
ADDED
|
@@ -0,0 +1,320 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Datasets Documentation
|
| 2 |
+
|
| 3 |
+
**Project:** Physics-Informed KARST-Aware GNN for Streamflow Prediction
|
| 4 |
+
**Study Area:** La Risle and La Eure River Systems, Seine Basin, France
|
| 5 |
+
**Period:** 1960-2026 (historical data)
|
| 6 |
+
|
| 7 |
+
## Directory Structure
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
datasets/
|
| 11 |
+
├── station_list.csv # 27 stations metadata (coordinates, names)
|
| 12 |
+
├── selected_stations.csv # 27 selected station codes
|
| 13 |
+
├── IDPR_values.csv # BRGM infiltration/runoff index (27 stations)
|
| 14 |
+
├── ADES/ # Groundwater data (KARST connectivity)
|
| 15 |
+
│ ├── groundwater_stations.csv # 114 groundwater monitoring wells
|
| 16 |
+
│ ├── groundwater_levels_watershed.csv# 272,379 groundwater level measurements
|
| 17 |
+
│ ├── stations_metadata.csv # 10,001 stations (full ADES database)
|
| 18 |
+
│ └── ADES_data_fetching.py # Data collection script
|
| 19 |
+
├── Watersehd_risle/ # La Risle watershed boundary
|
| 20 |
+
│ └── Watershed_Risle.shp # Shapefile (EPSG:2154)
|
| 21 |
+
├── Watershed_Eure/ # La Eure watershed boundary
|
| 22 |
+
│ └── Watershed_Eure.shp # Shapefile (EPSG:2154)
|
| 23 |
+
├── Risle_Eure_watershed/ # Merged watershed boundary
|
| 24 |
+
│ └── Risle_Eure_Watershed.shp # Combined shapefile (EPSG:2154)
|
| 25 |
+
└── River Images/ # Reference maps
|
| 26 |
+
├── Carte-de-la_Seine-390x305.png
|
| 27 |
+
├── Eure_(rivière).png
|
| 28 |
+
└── Risle.png
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
## Available Data
|
| 32 |
+
|
| 33 |
+
### 1. Hydrometric Stations (`station_list.csv`)
|
| 34 |
+
|
| 35 |
+
**File:** `station_list.csv`
|
| 36 |
+
**Rows:** 27 stations
|
| 37 |
+
**Columns:**
|
| 38 |
+
- `X`, `Y` - Lambert 93 coordinates (EPSG:2154)
|
| 39 |
+
- `station_code` - Unique station identifier (e.g., H602021010)
|
| 40 |
+
- `region_name` - NORMANDIE
|
| 41 |
+
- `departament_name` - ORNE or EURE
|
| 42 |
+
- `station_name` - Full station name
|
| 43 |
+
- `municipality` - Municipality name
|
| 44 |
+
- `INSEE` - INSEE code
|
| 45 |
+
- `id_root` - Root identifier
|
| 46 |
+
- `lon`, `lat` - WGS84 coordinates
|
| 47 |
+
- `lon-lat-ID` - Combined identifier
|
| 48 |
+
|
| 49 |
+
**Basin Distribution:**
|
| 50 |
+
- **La Risle:** 12 stations (e.g., La Risle à Rai, La Risle à Brionne, etc.)
|
| 51 |
+
- **La Eure:** 13 stations (e.g., L'Eure à Chartres, L'Eure à Louviers, etc.)
|
| 52 |
+
- **Tributary:** 1 station (Le bras de l'Eure à Lormaye)
|
| 53 |
+
|
| 54 |
+
**Sample Stations:**
|
| 55 |
+
```
|
| 56 |
+
H602021010 - La Risle à Rai (48.749°N, 0.582°E)
|
| 57 |
+
H605641401 - La Risle à Barquet (49.034°N, 0.808°E)
|
| 58 |
+
H620021010 - La Risle à Brionne
|
| 59 |
+
H4202010 - L'Eure à Chartres
|
| 60 |
+
H4522010 - L'Eure à Louviers
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
### 2. Selected Stations (`selected_stations.csv`)
|
| 64 |
+
|
| 65 |
+
**File:** `selected_stations.csv`
|
| 66 |
+
**Rows:** 27 stations
|
| 67 |
+
**Content:** List of selected station codes for analysis
|
| 68 |
+
|
| 69 |
+
### 3. IDPR Data (`IDPR_values.csv`)
|
| 70 |
+
|
| 71 |
+
**File:** `IDPR_values.csv`
|
| 72 |
+
**Source:** BRGM (Bureau de Recherches Géologiques et Minières)
|
| 73 |
+
**Rows:** 27 stations
|
| 74 |
+
**Purpose:** Infiltration vs runoff tendency (KARST indicator)
|
| 75 |
+
|
| 76 |
+
**Columns:**
|
| 77 |
+
- `fid` - Feature ID
|
| 78 |
+
- `X`, `Y` - Original coordinates
|
| 79 |
+
- `X_lambert`, `Y_lambert` - Lambert 93 coordinates
|
| 80 |
+
- `station_id` - Station identifier
|
| 81 |
+
- `basin_name` - "La Risle" or "La Eure"
|
| 82 |
+
- `latitude`, `longitude` - WGS84 coordinates
|
| 83 |
+
- `IDPR` - **Infiltration Dominance vs Precipitation Runoff index**
|
| 84 |
+
- Higher values → More infiltration (karst behavior)
|
| 85 |
+
- Lower values → More surface runoff
|
| 86 |
+
|
| 87 |
+
**IDPR Range in Dataset:**
|
| 88 |
+
- La Risle: 885 - 2000
|
| 89 |
+
- Indicates varying karst influence across the basin
|
| 90 |
+
|
| 91 |
+
### 4. ADES Groundwater Data
|
| 92 |
+
|
| 93 |
+
**Source:** ADES (Accès aux Données sur les Eaux Souterraines)
|
| 94 |
+
**Purpose:** Karst groundwater connectivity, subsurface flow pathways
|
| 95 |
+
|
| 96 |
+
#### 4.1 Groundwater Stations (`ADES/groundwater_stations.csv`)
|
| 97 |
+
|
| 98 |
+
**Rows:** 114 groundwater monitoring wells in watershed
|
| 99 |
+
**Columns:**
|
| 100 |
+
- `code_bss` - BSS code (unique well identifier)
|
| 101 |
+
- `urn_bss` - URN identifier
|
| 102 |
+
- `date_debut_mesure`, `date_fin_mesure` - Measurement period
|
| 103 |
+
- `code_commune_insee`, `nom_commune` - Municipality
|
| 104 |
+
- `x`, `y` - Coordinates
|
| 105 |
+
- `bss_id` - Short BSS ID
|
| 106 |
+
- `altitude_station` - Well altitude (m)
|
| 107 |
+
- `nb_mesures_piezo` - Number of piezometric measurements
|
| 108 |
+
- `code_departement`, `nom_departement` - Department (61=Orne, 27=Eure)
|
| 109 |
+
- `profondeur_investigation` - Investigation depth (m)
|
| 110 |
+
|
| 111 |
+
**Sample Wells:**
|
| 112 |
+
- BSS000TTRG (Le Mage, Orne) - 137 measurements, depth 38m, altitude 231m
|
| 113 |
+
- BSS000TTRB (Longny les Villages) - 9 measurements, depth 55m, altitude 235m
|
| 114 |
+
|
| 115 |
+
#### 4.2 Groundwater Levels (`ADES/groundwater_levels_watershed.csv`)
|
| 116 |
+
|
| 117 |
+
**Rows:** 272,379 piezometric measurements
|
| 118 |
+
**Period:** Various (e.g., 2013-present for some wells)
|
| 119 |
+
**Columns:**
|
| 120 |
+
- `code_bss`, `bss_id`, `urn_bss` - Well identifiers
|
| 121 |
+
- `date_mesure` - Measurement date
|
| 122 |
+
- `timestamp_mesure` - Unix timestamp
|
| 123 |
+
- `niveau_nappe_eau` - Groundwater level (m above reference)
|
| 124 |
+
- `profondeur_nappe` - Depth to water table (m below surface)
|
| 125 |
+
- `mode_obtention` - Measurement method
|
| 126 |
+
- `statut` - Data status ("Donnée contrôlée niveau 2")
|
| 127 |
+
- `qualification` - Data quality ("Correcte", etc.)
|
| 128 |
+
- `code_producteur`, `nom_producteur` - Data producer
|
| 129 |
+
|
| 130 |
+
**Note:** This is time series data for groundwater levels, crucial for understanding subsurface karst connectivity.
|
| 131 |
+
|
| 132 |
+
#### 4.3 ADES Metadata (`ADES/stations_metadata.csv`)
|
| 133 |
+
|
| 134 |
+
**Rows:** 10,001 stations (full ADES database, not filtered)
|
| 135 |
+
**Purpose:** Reference metadata for all groundwater stations in France
|
| 136 |
+
|
| 137 |
+
### 5. Watershed Shapefiles
|
| 138 |
+
|
| 139 |
+
#### 5.1 La Risle Watershed (`Watersehd_risle/Watershed_Risle.shp`)
|
| 140 |
+
|
| 141 |
+
**CRS:** EPSG:2154 (Lambert 93 - French national projection)
|
| 142 |
+
**Features:** 1 multipolygon
|
| 143 |
+
**Attributes:**
|
| 144 |
+
- `ID_INTERNE` - Internal ID (e.g., Sav21_226)
|
| 145 |
+
- `ID_FONCTIO` - Functional ID
|
| 146 |
+
- `NOM` - Watershed name
|
| 147 |
+
- `SITUATION_` - Situation/status
|
| 148 |
+
|
| 149 |
+
#### 5.2 La Eure Watershed (`Watershed_Eure/Watershed_Eure.shp`)
|
| 150 |
+
|
| 151 |
+
**CRS:** EPSG:2154 (Lambert 93)
|
| 152 |
+
**Features:** 1 multipolygon
|
| 153 |
+
**Attributes:** Same as La Risle
|
| 154 |
+
|
| 155 |
+
#### 5.3 Merged Watershed (`Risle_Eure_watershed/Risle_Eure_Watershed.shp`)
|
| 156 |
+
|
| 157 |
+
**CRS:** EPSG:2154 (Lambert 93)
|
| 158 |
+
**Features:** 1 multipolygon (merged boundary)
|
| 159 |
+
**Attributes:**
|
| 160 |
+
- `FID` - Feature ID
|
| 161 |
+
- `geometry` - Geometry
|
| 162 |
+
|
| 163 |
+
**Note:** This merged shapefile covers both La Risle and La Eure basins for basin-wide analysis.
|
| 164 |
+
|
| 165 |
+
### Hydrometric Data (Hub'Eau API)
|
| 166 |
+
|
| 167 |
+
**Status:** Downloaded and available datasets/hydrometric
|
| 168 |
+
**Expected:** JSON and CSV files for 27 stations (1960-2026)
|
| 169 |
+
|
| 170 |
+
**Historical Statistics (Daily/Monthly):**
|
| 171 |
+
- `QmnJ` / `QmM` - Mean discharge (m³/s)
|
| 172 |
+
- `QIXnJ` / `QIXM` - Maximum instantaneous discharge (m³/s)
|
| 173 |
+
- `QINnJ` / `QINM` - Minimum instantaneous discharge (m³/s)
|
| 174 |
+
- `HIXnJ` / `HIXM` - Maximum water level (m)
|
| 175 |
+
|
| 176 |
+
**Real-Time Observations (5-min interval, ~1 month):**
|
| 177 |
+
- `H` - Water level (m)
|
| 178 |
+
- `Q` - River discharge (m³/s)
|
| 179 |
+
|
| 180 |
+
### Meteorological Data (SAFRAN Reanalysis)
|
| 181 |
+
|
| 182 |
+
**Status:** datasets/safran
|
| 183 |
+
**Expected:** JSON and CSV files for 27 stations (1960-2026)
|
| 184 |
+
|
| 185 |
+
**Variables (12):**
|
| 186 |
+
- `DLI_Q` - Atmospheric radiation (W/m²)
|
| 187 |
+
- `DRAINC_Q` - Drainage (mm)
|
| 188 |
+
- `ETP_Q` - Potential evapotranspiration (mm)
|
| 189 |
+
- `FF_Q` - Wind speed (m/s)
|
| 190 |
+
- `HU_Q` - Relative humidity (%)
|
| 191 |
+
- `PRELIQ_Q` - Liquid precipitation (mm)
|
| 192 |
+
- `PRENEI_Q` - Solid precipitation (mm)
|
| 193 |
+
- `RESR_NEIGE_Q` - Snowpack water equivalent (mm)
|
| 194 |
+
- `RUNC_Q` - Runoff (mm)
|
| 195 |
+
- `SSI_Q` - Visible radiation (W/m²)
|
| 196 |
+
- `SWI_Q` - Soil moisture index
|
| 197 |
+
- `T_Q` - Average air temperature (°C)
|
| 198 |
+
|
| 199 |
+
### DEM and Topographic Data
|
| 200 |
+
|
| 201 |
+
**Status:** datasets/station_elevations.csv
|
| 202 |
+
**Required for:**
|
| 203 |
+
- Elevation statistics (mean, min, max)
|
| 204 |
+
- Basin slope calculation
|
| 205 |
+
- Drainage density
|
| 206 |
+
- Topographic Wetness Index (TWI)
|
| 207 |
+
- River network topology extraction (QGIS hydrology tools)
|
| 208 |
+
|
| 209 |
+
**Recommended Source:**
|
| 210 |
+
- IGN RGE ALTI (French national DEM, 1m or 5m resolution)
|
| 211 |
+
- EU-DEM (25m resolution)
|
| 212 |
+
|
| 213 |
+
### River Network Topology
|
| 214 |
+
|
| 215 |
+
**Status:** ❌ Not yet in repository
|
| 216 |
+
**Required:** `station_edges.csv` for graph construction
|
| 217 |
+
|
| 218 |
+
**Expected Format:**
|
| 219 |
+
```csv
|
| 220 |
+
source,target,distance,elevation_gradient,basin_id
|
| 221 |
+
H602021010,H605641401,15.2,10.0,0
|
| 222 |
+
H605641401,H620021010,12.5,-5.0,0
|
| 223 |
+
...
|
| 224 |
+
```
|
| 225 |
+
|
| 226 |
+
**Generation Method:**
|
| 227 |
+
- Use QGIS DEM hydrology processing tools
|
| 228 |
+
- Extract upstream-downstream relationships
|
| 229 |
+
- Calculate river distances and elevation gradients
|
| 230 |
+
- Ensure no cross-basin edges (La Risle ≠ La Eure)
|
| 231 |
+
|
| 232 |
+
### Basin Attributes (To Be Extracted)
|
| 233 |
+
|
| 234 |
+
**Status:** ❌ Not yet extracted from shapefiles/DEM
|
| 235 |
+
|
| 236 |
+
**Topological:**
|
| 237 |
+
- Elevation, slope, drainage density, TWI, basin area
|
| 238 |
+
|
| 239 |
+
**Soil Properties:**
|
| 240 |
+
- Soil type, texture, hydraulic conductivity, porosity, depth
|
| 241 |
+
|
| 242 |
+
**Land Use:**
|
| 243 |
+
- Forest/agricultural/urban cover %, NDVI
|
| 244 |
+
|
| 245 |
+
**Geological/KARST:**
|
| 246 |
+
- Aquifer characteristics, lithology, permeability, sinkhole density
|
| 247 |
+
|
| 248 |
+
## Data Processing Notes
|
| 249 |
+
|
| 250 |
+
### Basin Assignment
|
| 251 |
+
|
| 252 |
+
Based on station names in `station_list.csv`:
|
| 253 |
+
|
| 254 |
+
**La Risle Stations (basin_id = 0):**
|
| 255 |
+
- 12 stations with names starting with "La Risle"
|
| 256 |
+
- Station codes: H60*, H62*
|
| 257 |
+
|
| 258 |
+
**La Eure Stations (basin_id = 1):**
|
| 259 |
+
- 13 stations with names starting with "L'Eure"
|
| 260 |
+
- Station codes: H42*, H45*, H46*, H47*
|
| 261 |
+
- Includes tributary: "Le bras de l'Eure"
|
| 262 |
+
|
| 263 |
+
### Coordinate Systems
|
| 264 |
+
|
| 265 |
+
**All geospatial data uses:**
|
| 266 |
+
- **Projected:** EPSG:2154 (Lambert 93) for distances/areas
|
| 267 |
+
- **Geographic:** EPSG:4326 (WGS84) for lat/lon
|
| 268 |
+
|
| 269 |
+
**Conversion needed for:**
|
| 270 |
+
- Station coordinates (already in both systems in `station_list.csv`)
|
| 271 |
+
- Shapefiles (already in EPSG:2154)
|
| 272 |
+
|
| 273 |
+
### Data Size Summary
|
| 274 |
+
|
| 275 |
+
| File | Rows | Size | Notes |
|
| 276 |
+
|------|------|------|-------|
|
| 277 |
+
| `station_list.csv` | 27 | 4.1 KB | Station metadata |
|
| 278 |
+
| `selected_stations.csv` | 27 | 338 B | Station codes only |
|
| 279 |
+
| `IDPR_values.csv` | 27 | 2.6 KB | KARST infiltration index |
|
| 280 |
+
| `ADES/groundwater_stations.csv` | 114 | ~50 KB | Groundwater wells |
|
| 281 |
+
| `ADES/groundwater_levels_watershed.csv` | 272,379 | ~50 MB | **Large!** Time series |
|
| 282 |
+
| `ADES/stations_metadata.csv` | 10,001 | ~2 MB | Full ADES database |
|
| 283 |
+
|
| 284 |
+
**⚠️ Note:** `groundwater_levels_watershed.csv` is the largest file (272K rows). Use efficient processing (chunking, filtering by date range).
|
| 285 |
+
|
| 286 |
+
## Next Steps
|
| 287 |
+
|
| 288 |
+
1. **Collect Hydrometric Data**
|
| 289 |
+
- Use Hub'Eau API to download discharge/water level data
|
| 290 |
+
- Store in `datasets/hydrometric/` (JSON and CSV)
|
| 291 |
+
|
| 292 |
+
2. **Collect SAFRAN Data**
|
| 293 |
+
- Download meteorological reanalysis for 27 station coordinates
|
| 294 |
+
- Store in `datasets/safran/` (JSON and CSV)
|
| 295 |
+
|
| 296 |
+
3. **Acquire DEM**
|
| 297 |
+
- Download IGN RGE ALTI or EU-DEM for study area
|
| 298 |
+
- Store in `datasets/DEM/`
|
| 299 |
+
|
| 300 |
+
4. **Extract River Topology**
|
| 301 |
+
- Use QGIS DEM hydrology tools
|
| 302 |
+
- Generate `datasets/station_edges.csv`
|
| 303 |
+
|
| 304 |
+
5. **Extract Basin Attributes**
|
| 305 |
+
- Use DEM to calculate topological features
|
| 306 |
+
- Use soil/land use databases for additional features
|
| 307 |
+
- Store in `datasets/basin_features/`
|
| 308 |
+
|
| 309 |
+
6. **Process Data Through Pipeline**
|
| 310 |
+
- Run `example_pipeline.py` once all data is available
|
| 311 |
+
- Generate fused datasets, graphs, and tensors
|
| 312 |
+
|
| 313 |
+
## Data Sources & References
|
| 314 |
+
|
| 315 |
+
- **Hub'Eau API:** https://hubeau.eaufrance.fr/
|
| 316 |
+
- **SAFRAN Reanalysis:** Météo-France
|
| 317 |
+
- **ADES:** https://ades.eaufrance.fr/ (groundwater data)
|
| 318 |
+
- **BRGM IDPR:** Bureau de Recherches Géologiques et Minières
|
| 319 |
+
- **IGN:** https://geoservices.ign.fr/ (DEM, elevation data)
|
| 320 |
+
- **Watersheds:** Digitized from official French hydrographic boundaries
|
docs/images/eure_reach_graph.png
ADDED
|
Git LFS Details
|
docs/images/explore_view.png
ADDED
|
Git LFS Details
|
docs/images/risle_reach_graph.png
ADDED
|