Delete datasets/README.md

datasets/README.md

@@ -1,320 +0,0 @@
-# Datasets Documentation
-
-**Project:** Physics-Informed KARST-Aware GNN for Streamflow Prediction
-**Study Area:** La Risle and La Eure River Systems, Seine Basin, France
-**Period:** 1960-2026 (historical data)
-
-## Directory Structure
-
-```
-datasets/
-├── station_list.csv                    # 27 stations metadata (coordinates, names)
-├── selected_stations.csv               # 27 selected station codes
-├── IDPR_values.csv                     # BRGM infiltration/runoff index (27 stations)
-├── ADES/                               # Groundwater data (KARST connectivity)
-│   ├── groundwater_stations.csv        # 114 groundwater monitoring wells
-│   ├── groundwater_levels_watershed.csv# 272,379 groundwater level measurements
-│   ├── stations_metadata.csv          # 10,001 stations (full ADES database)
-│   └── ADES_data_fetching.py          # Data collection script
-├── Watersehd_risle/                    # La Risle watershed boundary
-│   └── Watershed_Risle.shp            # Shapefile (EPSG:2154)
-├── Watershed_Eure/                     # La Eure watershed boundary
-│   └── Watershed_Eure.shp             # Shapefile (EPSG:2154)
-├── Risle_Eure_watershed/              # Merged watershed boundary
-│   └── Risle_Eure_Watershed.shp       # Combined shapefile (EPSG:2154)
-└── River Images/                       # Reference maps
-    ├── Carte-de-la_Seine-390x305.png
-    ├── Eure_(rivière).png
-    └── Risle.png
-```
-
-## Available Data
-
-### 1. Hydrometric Stations (`station_list.csv`)
-
-**File:** `station_list.csv`
-**Rows:** 27 stations
-**Columns:**
-- `X`, `Y` - Lambert 93 coordinates (EPSG:2154)
-- `station_code` - Unique station identifier (e.g., H602021010)
-- `region_name` - NORMANDIE
-- `departament_name` - ORNE or EURE
-- `station_name` - Full station name
-- `municipality` - Municipality name
-- `INSEE` - INSEE code
-- `id_root` - Root identifier
-- `lon`, `lat` - WGS84 coordinates
-- `lon-lat-ID` - Combined identifier
-
-**Basin Distribution:**
-- **La Risle:** 12 stations (e.g., La Risle à Rai, La Risle à Brionne, etc.)
-- **La Eure:** 13 stations (e.g., L'Eure à Chartres, L'Eure à Louviers, etc.)
-- **Tributary:** 1 station (Le bras de l'Eure à Lormaye)
-
-**Sample Stations:**
-```
-H602021010 - La Risle à Rai (48.749°N, 0.582°E)
-H605641401 - La Risle à Barquet (49.034°N, 0.808°E)
-H620021010 - La Risle à Brionne
-H4202010  - L'Eure à Chartres
-H4522010  - L'Eure à Louviers
-```
-
-### 2. Selected Stations (`selected_stations.csv`)
-
-**File:** `selected_stations.csv`
-**Rows:** 27 stations
-**Content:** List of selected station codes for analysis
-
-### 3. IDPR Data (`IDPR_values.csv`)
-
-**File:** `IDPR_values.csv`
-**Source:** BRGM (Bureau de Recherches Géologiques et Minières)
-**Rows:** 27 stations
-**Purpose:** Infiltration vs runoff tendency (KARST indicator)
-
-**Columns:**
-- `fid` - Feature ID
-- `X`, `Y` - Original coordinates
-- `X_lambert`, `Y_lambert` - Lambert 93 coordinates
-- `station_id` - Station identifier
-- `basin_name` - "La Risle" or "La Eure"
-- `latitude`, `longitude` - WGS84 coordinates
-- `IDPR` - **Infiltration Dominance vs Precipitation Runoff index**
-  - Higher values → More infiltration (karst behavior)
-  - Lower values → More surface runoff
-
-**IDPR Range in Dataset:**
-- La Risle: 885 - 2000
-- Indicates varying karst influence across the basin
-
-### 4. ADES Groundwater Data
-
-**Source:** ADES (Accès aux Données sur les Eaux Souterraines)
-**Purpose:** Karst groundwater connectivity, subsurface flow pathways
-
-#### 4.1 Groundwater Stations (`ADES/groundwater_stations.csv`)
-
-**Rows:** 114 groundwater monitoring wells in watershed
-**Columns:**
-- `code_bss` - BSS code (unique well identifier)
-- `urn_bss` - URN identifier
-- `date_debut_mesure`, `date_fin_mesure` - Measurement period
-- `code_commune_insee`, `nom_commune` - Municipality
-- `x`, `y` - Coordinates
-- `bss_id` - Short BSS ID
-- `altitude_station` - Well altitude (m)
-- `nb_mesures_piezo` - Number of piezometric measurements
-- `code_departement`, `nom_departement` - Department (61=Orne, 27=Eure)
-- `profondeur_investigation` - Investigation depth (m)
-
-**Sample Wells:**
-- BSS000TTRG (Le Mage, Orne) - 137 measurements, depth 38m, altitude 231m
-- BSS000TTRB (Longny les Villages) - 9 measurements, depth 55m, altitude 235m
-
-#### 4.2 Groundwater Levels (`ADES/groundwater_levels_watershed.csv`)
-
-**Rows:** 272,379 piezometric measurements
-**Period:** Various (e.g., 2013-present for some wells)
-**Columns:**
-- `code_bss`, `bss_id`, `urn_bss` - Well identifiers
-- `date_mesure` - Measurement date
-- `timestamp_mesure` - Unix timestamp
-- `niveau_nappe_eau` - Groundwater level (m above reference)
-- `profondeur_nappe` - Depth to water table (m below surface)
-- `mode_obtention` - Measurement method
-- `statut` - Data status ("Donnée contrôlée niveau 2")
-- `qualification` - Data quality ("Correcte", etc.)
-- `code_producteur`, `nom_producteur` - Data producer
-
-**Note:** This is time series data for groundwater levels, crucial for understanding subsurface karst connectivity.
-
-#### 4.3 ADES Metadata (`ADES/stations_metadata.csv`)
-
-**Rows:** 10,001 stations (full ADES database, not filtered)
-**Purpose:** Reference metadata for all groundwater stations in France
-
-### 5. Watershed Shapefiles
-
-#### 5.1 La Risle Watershed (`Watersehd_risle/Watershed_Risle.shp`)
-
-**CRS:** EPSG:2154 (Lambert 93 - French national projection)
-**Features:** 1 multipolygon
-**Attributes:**
-- `ID_INTERNE` - Internal ID (e.g., Sav21_226)
-- `ID_FONCTIO` - Functional ID
-- `NOM` - Watershed name
-- `SITUATION_` - Situation/status
-
-#### 5.2 La Eure Watershed (`Watershed_Eure/Watershed_Eure.shp`)
-
-**CRS:** EPSG:2154 (Lambert 93)
-**Features:** 1 multipolygon
-**Attributes:** Same as La Risle
-
-#### 5.3 Merged Watershed (`Risle_Eure_watershed/Risle_Eure_Watershed.shp`)
-
-**CRS:** EPSG:2154 (Lambert 93)
-**Features:** 1 multipolygon (merged boundary)
-**Attributes:**
-- `FID` - Feature ID
-- `geometry` - Geometry
-
-**Note:** This merged shapefile covers both La Risle and La Eure basins for basin-wide analysis.
-
-### Hydrometric Data (Hub'Eau API)
-
-**Status:** Downloaded and available datasets/hydrometric
-**Expected:** JSON and CSV files for 27 stations (1960-2026)
-
-**Historical Statistics (Daily/Monthly):**
-- `QmnJ` / `QmM` - Mean discharge (m³/s)
-- `QIXnJ` / `QIXM` - Maximum instantaneous discharge (m³/s)
-- `QINnJ` / `QINM` - Minimum instantaneous discharge (m³/s)
-- `HIXnJ` / `HIXM` - Maximum water level (m)
-
-**Real-Time Observations (5-min interval, ~1 month):**
-- `H` - Water level (m)
-- `Q` - River discharge (m³/s)
-
-### Meteorological Data (SAFRAN Reanalysis)
-
-**Status:** datasets/safran
-**Expected:** JSON and CSV files for 27 stations (1960-2026)
-
-**Variables (12):**
-- `DLI_Q` - Atmospheric radiation (W/m²)
-- `DRAINC_Q` - Drainage (mm)
-- `ETP_Q` - Potential evapotranspiration (mm)
-- `FF_Q` - Wind speed (m/s)
-- `HU_Q` - Relative humidity (%)
-- `PRELIQ_Q` - Liquid precipitation (mm)
-- `PRENEI_Q` - Solid precipitation (mm)
-- `RESR_NEIGE_Q` - Snowpack water equivalent (mm)
-- `RUNC_Q` - Runoff (mm)
-- `SSI_Q` - Visible radiation (W/m²)
-- `SWI_Q` - Soil moisture index
-- `T_Q` - Average air temperature (°C)
-
-### DEM and Topographic Data
-
-**Status:** datasets/station_elevations.csv
-**Required for:**
-- Elevation statistics (mean, min, max)
-- Basin slope calculation
-- Drainage density
-- Topographic Wetness Index (TWI)
-- River network topology extraction (QGIS hydrology tools)
-
-**Recommended Source:**
-- IGN RGE ALTI (French national DEM, 1m or 5m resolution)
-- EU-DEM (25m resolution)
-
-### River Network Topology
-
-**Status:** ❌ Not yet in repository
-**Required:** `station_edges.csv` for graph construction
-
-**Expected Format:**
-```csv
-source,target,distance,elevation_gradient,basin_id
-H602021010,H605641401,15.2,10.0,0
-H605641401,H620021010,12.5,-5.0,0
-...
-```
-
-**Generation Method:**
-- Use QGIS DEM hydrology processing tools
-- Extract upstream-downstream relationships
-- Calculate river distances and elevation gradients
-- Ensure no cross-basin edges (La Risle ≠ La Eure)
-
-### Basin Attributes (To Be Extracted)
-
-**Status:** ❌ Not yet extracted from shapefiles/DEM
-
-**Topological:**
-- Elevation, slope, drainage density, TWI, basin area
-
-**Soil Properties:**
-- Soil type, texture, hydraulic conductivity, porosity, depth
-
-**Land Use:**
-- Forest/agricultural/urban cover %, NDVI
-
-**Geological/KARST:**
-- Aquifer characteristics, lithology, permeability, sinkhole density
-
-## Data Processing Notes
-
-### Basin Assignment
-
-Based on station names in `station_list.csv`:
-
-**La Risle Stations (basin_id = 0):**
-- 12 stations with names starting with "La Risle"
-- Station codes: H60*, H62*
-
-**La Eure Stations (basin_id = 1):**
-- 13 stations with names starting with "L'Eure"
-- Station codes: H42*, H45*, H46*, H47*
-- Includes tributary: "Le bras de l'Eure"
-
-### Coordinate Systems
-
-**All geospatial data uses:**
-- **Projected:** EPSG:2154 (Lambert 93) for distances/areas
-- **Geographic:** EPSG:4326 (WGS84) for lat/lon
-
-**Conversion needed for:**
-- Station coordinates (already in both systems in `station_list.csv`)
-- Shapefiles (already in EPSG:2154)
-
-### Data Size Summary
-
-| File | Rows | Size | Notes |
-|------|------|------|-------|
-| `station_list.csv` | 27 | 4.1 KB | Station metadata |
-| `selected_stations.csv` | 27 | 338 B | Station codes only |
-| `IDPR_values.csv` | 27 | 2.6 KB | KARST infiltration index |
-| `ADES/groundwater_stations.csv` | 114 | ~50 KB | Groundwater wells |
-| `ADES/groundwater_levels_watershed.csv` | 272,379 | ~50 MB | **Large!** Time series |
-| `ADES/stations_metadata.csv` | 10,001 | ~2 MB | Full ADES database |
-
-**⚠️ Note:** `groundwater_levels_watershed.csv` is the largest file (272K rows). Use efficient processing (chunking, filtering by date range).
-
-## Next Steps
-
-1. **Collect Hydrometric Data**
-   - Use Hub'Eau API to download discharge/water level data
-   - Store in `datasets/hydrometric/` (JSON and CSV)
-
-2. **Collect SAFRAN Data**
-   - Download meteorological reanalysis for 27 station coordinates
-   - Store in `datasets/safran/` (JSON and CSV)
-
-3. **Acquire DEM**
-   - Download IGN RGE ALTI or EU-DEM for study area
-   - Store in `datasets/DEM/`
-
-4. **Extract River Topology**
-   - Use QGIS DEM hydrology tools
-   - Generate `datasets/station_edges.csv`
-
-5. **Extract Basin Attributes**
-   - Use DEM to calculate topological features
-   - Use soil/land use databases for additional features
-   - Store in `datasets/basin_features/`
-
-6. **Process Data Through Pipeline**
-   - Run `example_pipeline.py` once all data is available
-   - Generate fused datasets, graphs, and tensors
-
-## Data Sources & References
-
-- **Hub'Eau API:** https://hubeau.eaufrance.fr/
-- **SAFRAN Reanalysis:** Météo-France
-- **ADES:** https://ades.eaufrance.fr/ (groundwater data)
-- **BRGM IDPR:** Bureau de Recherches Géologiques et Minières
-- **IGN:** https://geoservices.ign.fr/ (DEM, elevation data)
-- **Watersheds:** Digitized from official French hydrographic boundaries