RockGlacier_Project

Project Overview

This project focuses on the automated detection and segmentation of rock glaciers using deep learning techniques. By leveraging multi-modal remote sensing data—including Sentinel-2 optical imagery, Digital Elevation Models (DEM), Land Surface Temperature (LST), and surface velocity data—we aim to accurately map rock glaciers to better understand their distribution, dynamics, and response to climate change.

Repository Structure

raw_data/: Contains the original, unprocessed data files from various sources.
- shapefiles/: Rock glacier polygons in vector format (e.g., .shp).
- sentinel2/: Optical satellite imagery from Sentinel-2.
- dem/: Digital Elevation Model data.
- lst/: Land Surface Temperature data.
- velocity/: Surface velocity maps.
features/: Stores intermediate extracted features derived from the raw data sources.
masks/: Contains binary masks generated from the shapefiles, where 0 indicates background and 1 indicates a rock glacier.
stacks/: Holds stacked feature arrays combining multiple data modalities (Sentinel-2, DEM, LST, velocity) into single multi-band files for model input.
dataset/: Contains the final extracted patches organized for model training, validation, and testing.
- train/: Data used to train the model, containing images/ and corresponding masks/.
- val/: Data used to validate the model during training and tune hyperparameters.
- test/: Unseen data used to evaluate the final model's generalization performance.
models/: Contains the PyTorch/TensorFlow implementations of various deep learning architectures (e.g., U-Net, Swin-Unet, SegFormer).
outputs/: Stores the artifacts generated during model training and evaluation.
- checkpoints/: Saved model weights from training.
- logs/: Training metrics, loss logs, and TensorBoard data.
- predictions/: Output segmentation masks generated by the models on the test set.

Dataset Creation Pipeline

The dataset is created through a systematic processing pipeline from raw data to machine-learning-ready patches:

Rock Glacier Polygons (.shp) ↓ Rasterization ↓ Binary Mask (0=Background, 1=Rock Glacier)

Sentinel-2 + DEM + LST + Velocity ↓ Feature Extraction ↓ Feature Stack

Feature Stack + Mask ↓ Patch Extraction ↓ Train / Val / Test Dataset

Train Dataset ↓ U-Net / Swin-Unet / SegFormer

Google Earth Engine Setup & Feature Extraction

To generate the multi-band feature stacks for the dataset, we rely on Google Earth Engine (GEE). The extraction pipeline requires a local GEE Python environment.

1. Environment Setup

A dedicated Conda environment is used to run the extraction scripts without dependency conflicts:

conda create -y -n rockglacier python=3.11
conda activate rockglacier
pip install earthengine-api geemap geopandas

2. Authentication

Because the script runs locally, you must authenticate your Google Account via the terminal (this is a one-time process):

earthengine authenticate

This command will open a web browser. Select the Google account registered for Earth Engine, grant permissions, and copy the provided Authorization Code back into your terminal.

3. Feature Extraction (Task 02)

Once authenticated, the feature extraction pipeline will:

Read the generated .geojson polygon bounds.
Query Sentinel-2 Surface Reflectance (Summer 2023) and Copernicus DEM.
Calculate indices (NDVI, NDWI, NDSI, Slope, Aspect).
Stack them into a 12-band GeoTIFF at exactly 10m/px resolution with a 200m buffer.
Download them directly to the features/ directory to perfectly align with the masks/.

See task_02.md for full technical specifications.

Future Workflow

Data Preprocessing Pipeline: Implement scripts to automate the extraction, rasterization, and stacking of features from raw_data into features and stacks.
Patch Generation: Develop a sliding window approach to extract appropriately sized patches from the feature stacks and masks, saving them into the respective dataset splits.
Model Implementation: Complete the architecture definitions in the models/ directory for unet.py, swin_unet.py, and segformer.py.
Training & Evaluation: Implement train.py and evaluate.py to streamline model training, hyperparameter tuning, and comprehensive evaluation on the test set.
Inference & Visualization: Set up automated prediction on new regions using the best trained model and visualize the resulting predictions from outputs/predictions.