patch_dataset/README.md

patch_dataset/ — RockGlacierNet data collection & patches

Self-contained pipeline. Every script and every output lives in this folder. All tunables are in config.py.

Reference method: Liu, Xing & Yao, Remote Sens. 2025 (Hunza Basin) — region-grid tiling of a mosaic, not tiny per-object crops.

Why region tiling

The old per-glacier crops (200 m buffer) were median 83 px, 86 % smaller than 128 px → a 128 patch was mostly padding (the 64 px IoU regression). We instead tile the whole-Sikkim AOI (bbox of all inventory polygons + margin ≈ 7130 km², 84×84 km) into 512 px download tiles, so every glacier sits in real terrain with proper background / hard negatives, and we can re-tile locally to any patch size.

Key fix: GEE's interactive download trips "User memory limit exceeded" on a raw 3-year median. A scene-level CLOUDY_PIXEL_PERCENTAGE < S2_CLOUD_PCT filter keeps the composite light enough to download; a sub-tiling fallback handles any tile that still exceeds the limit.

Two stages (both in patch_dataset/)

Stage	Script	Needs	Output
1. Collect	patch_dataset/collect_region.py	ee,geemap,geopandas,rasterio	patch_dataset/raw_region/{features,masks}/tile_RRRR_CCCC.tif
2. Build	patch_dataset/build_patches.py	rasterio	patch_dataset/patches_{size}/{split}/{features,masks}/

Env: use brats (or geoground/vlm) — they have ee+geemap+geopandas+rasterio.

Stage 1 — collect whole-Sikkim tiles

python patch_dataset/collect_region.py                # sampled: glacier tiles + REGION_BG_KEEP background
python patch_dataset/collect_region.py --mode dense   # every AOI tile (full mosaic, multi-GB)
python patch_dataset/collect_region.py --limit 5      # smoke test

Per tile: downloads the 12-band stack (B2 B3 B4 B8 B11 B12 NDVI NDWI NDSI Elev Slope Aspect) and rasterizes all intersecting polygons onto the tile's exact grid → feature and mask are co-registered by construction. Resumable (skips existing tiles).

Scope (256 px grid over the AOI): 1122 cells total, ~228 contain a glacier.

• sampled (default): ~228 glacier + 25 % background ≈ 450 tiles, ~1.3 GB.
• dense: all 1122 tiles, ~3.3 GB (use only for region-wide inference mapping).

Stage 2 — build patches at any size

python patch_dataset/build_patches.py                 # DEFAULT_PATCH = 128
python patch_dataset/build_patches.py --patch-size 64
python patch_dataset/build_patches.py --patch-size 256

Splits by source tile (no patch leakage), tiles with 50 % overlap, keeps every glacier patch + BACKGROUND_KEEP_RATIO of background patches, writes georeferenced GeoTIFF patches feature_*.tif / mask_*.tif + metadata/. Each size in its own folder so they coexist:

patch_dataset/
  config.py  collect_region.py  build_patches.py  README.md
  raw_region/{features,masks}/tile_RRRR_CCCC.tif
  patches_64/   {train,val,test}/{features,masks}/  metadata/
  patches_128/  {train,val,test}/{features,masks}/  metadata/
  patches_256/  {train,val,test}/{features,masks}/  metadata/

Training (drop-in)

Patches are named feature_*.tif / mask_*.tif, so RockGlacierDataGenerator trains on them unchanged — repoint the paths in train.py:

TRAIN_FEAT = "patch_dataset/patches_128/train/features"
TRAIN_MASK = "patch_dataset/patches_128/train/masks"
VAL_FEAT   = "patch_dataset/patches_128/val/features"
VAL_MASK   = "patch_dataset/patches_128/val/masks"
INPUT_SHAPE = (128, 128, 12)

Band order matches data_generator.GLOBAL_MEAN/STD. Region tiles are background-heavy, so consider raising the positive-class loss weight / lowering BACKGROUND_KEEP_RATIO if the kept set is too imbalanced (see the build's diagnostics_report.txt).

Changing size later

Edit DEFAULT_PATCH / PATCH_SIZES in config.py or pass --patch-size. No re-download needed (raw 512 px tiles hold up to 256 px patches). For >512 px tiles or a different AOI, re-run Stage 1.

Legacy per-glacier scripts (preprocess/collect_features.py, preprocess/build_patches.py) still exist but are superseded by this region pipeline.