| # patch_dataset/ — RockGlacierNet data collection & patches |
| |
| Self-contained pipeline. **Every script and every output lives in this folder.** |
| All tunables are in [`config.py`](config.py). |
| |
| Reference method: Liu, Xing & Yao, *Remote Sens.* 2025 (Hunza Basin) — region-grid |
| tiling of a mosaic, not tiny per-object crops. |
| |
| ## Why region tiling |
| |
| The old per-glacier crops (200 m buffer) were median 83 px, 86 % smaller than 128 px |
| → a 128 patch was mostly padding (the 64 px IoU regression). We instead tile the |
| **whole-Sikkim AOI** (bbox of all inventory polygons + margin ≈ 7130 km², 84×84 km) |
| into 512 px download tiles, so every glacier sits in real terrain with proper |
| background / hard negatives, and we can re-tile locally to any patch size. |
| |
| Key fix: GEE's interactive download trips **"User memory limit exceeded"** on a raw |
| 3-year median. A scene-level `CLOUDY_PIXEL_PERCENTAGE < S2_CLOUD_PCT` filter keeps the |
| composite light enough to download; a sub-tiling fallback handles any tile that still |
| exceeds the limit. |
| |
| ## Two stages (both in patch_dataset/) |
|
|
| | Stage | Script | Needs | Output | |
| |---|---|---|---| |
| | 1. Collect | `patch_dataset/collect_region.py` | `ee`,`geemap`,`geopandas`,`rasterio` | `patch_dataset/raw_region/{features,masks}/tile_RRRR_CCCC.tif` | |
| | 2. Build | `patch_dataset/build_patches.py` | `rasterio` | `patch_dataset/patches_{size}/{split}/{features,masks}/` | |
|
|
| Env: use `brats` (or `geoground`/`vlm`) — they have `ee`+`geemap`+`geopandas`+`rasterio`. |
|
|
| ### Stage 1 — collect whole-Sikkim tiles |
| ```bash |
| python patch_dataset/collect_region.py # sampled: glacier tiles + REGION_BG_KEEP background |
| python patch_dataset/collect_region.py --mode dense # every AOI tile (full mosaic, multi-GB) |
| python patch_dataset/collect_region.py --limit 5 # smoke test |
| ``` |
| Per tile: downloads the 12-band stack (`B2 B3 B4 B8 B11 B12 NDVI NDWI NDSI Elev Slope |
| Aspect`) and rasterizes all intersecting polygons onto the tile's exact grid → feature |
| and mask are co-registered by construction. Resumable (skips existing tiles). |
|
|
| Scope (256 px grid over the AOI): 1122 cells total, ~228 contain a glacier. |
| - `sampled` (default): ~228 glacier + 25 % background ≈ 450 tiles, ~1.3 GB. |
| - `dense`: all 1122 tiles, ~3.3 GB (use only for region-wide inference mapping). |
|
|
| ### Stage 2 — build patches at any size |
| ```bash |
| python patch_dataset/build_patches.py # DEFAULT_PATCH = 128 |
| python patch_dataset/build_patches.py --patch-size 64 |
| python patch_dataset/build_patches.py --patch-size 256 |
| ``` |
| Splits **by source tile** (no patch leakage), tiles with 50 % overlap, keeps every |
| glacier patch + `BACKGROUND_KEEP_RATIO` of background patches, writes georeferenced |
| GeoTIFF patches `feature_*.tif` / `mask_*.tif` + `metadata/`. Each size in its own |
| folder so they coexist: |
| ``` |
| patch_dataset/ |
| config.py collect_region.py build_patches.py README.md |
| raw_region/{features,masks}/tile_RRRR_CCCC.tif |
| patches_64/ {train,val,test}/{features,masks}/ metadata/ |
| patches_128/ {train,val,test}/{features,masks}/ metadata/ |
| patches_256/ {train,val,test}/{features,masks}/ metadata/ |
| ``` |
|
|
| ## Training (drop-in) |
|
|
| Patches are named `feature_*.tif` / `mask_*.tif`, so `RockGlacierDataGenerator` |
| trains on them unchanged — repoint the paths in `train.py`: |
| ```python |
| TRAIN_FEAT = "patch_dataset/patches_128/train/features" |
| TRAIN_MASK = "patch_dataset/patches_128/train/masks" |
| VAL_FEAT = "patch_dataset/patches_128/val/features" |
| VAL_MASK = "patch_dataset/patches_128/val/masks" |
| INPUT_SHAPE = (128, 128, 12) |
| ``` |
| Band order matches `data_generator.GLOBAL_MEAN/STD`. Region tiles are background-heavy, |
| so consider raising the positive-class loss weight / lowering `BACKGROUND_KEEP_RATIO` |
| if the kept set is too imbalanced (see the build's `diagnostics_report.txt`). |
|
|
| ## Changing size later |
| Edit `DEFAULT_PATCH` / `PATCH_SIZES` in `config.py` or pass `--patch-size`. No |
| re-download needed (raw 512 px tiles hold up to 256 px patches). For >512 px tiles or a |
| different AOI, re-run Stage 1. |
|
|
| > Legacy per-glacier scripts (`preprocess/collect_features.py`, `preprocess/build_patches.py`) |
| > still exist but are superseded by this region pipeline. |
|
|
