| \documentclass[12pt,a4paper]{article} |
|
|
| |
| \usepackage[top=1.45in,bottom=1in,left=1in,right=1in,headheight=70pt,headsep=0.05in]{geometry} |
| \usepackage{times} |
| \usepackage{setspace} |
| \usepackage{graphicx} |
| \usepackage{caption} |
| \usepackage{amsmath} |
| \usepackage{amssymb} |
| \usepackage{multirow} |
| \usepackage{array} |
| \usepackage{xcolor} |
| \usepackage{hyperref} |
| \hypersetup{colorlinks=true, urlcolor=blue, citecolor=black, linkcolor=blue} |
| \usepackage{enumitem} |
| \usepackage{titlesec} |
| \usepackage{fancyhdr} |
| \usepackage{booktabs} |
| \usepackage{tikz} |
| \usetikzlibrary{shapes.geometric, arrows.meta, positioning, calc, fit, backgrounds} |
|
|
| |
| \pagestyle{fancy} |
| \fancyhf{} |
| \renewcommand{\headrulewidth}{0pt} |
| \fancyhead[C]{\includegraphics[width=\textwidth]{image.png}} |
|
|
| |
| \titleformat{\section}{\fontsize{12}{14}\bfseries}{}{0pt}{} |
| \titlespacing*{\section}{0pt}{12pt}{6pt} |
|
|
| \setlength{\parindent}{0pt} |
| \setlength{\parskip}{8pt} |
|
|
| |
| \tikzset{ |
| src/.style = {cylinder, shape border rotate=90, draw, fill=orange!25, |
| minimum width=1.1cm, minimum height=1.0cm, align=center, font=\scriptsize}, |
| proc/.style = {trapezium, trapezium left angle=70, trapezium right angle=110, draw, |
| fill=yellow!35, align=center, font=\scriptsize, minimum height=0.7cm}, |
| band/.style = {rectangle, draw, fill=white, align=center, font=\scriptsize, |
| minimum width=1.6cm, minimum height=0.42cm}, |
| stack/.style = {rectangle, draw, fill=cyan!18, align=center, font=\scriptsize, |
| minimum width=2.1cm, minimum height=0.7cm}, |
| outp/.style = {rectangle, draw, fill=green!18, align=center, font=\scriptsize, |
| minimum width=1.7cm, minimum height=0.6cm}, |
| ar/.style = {-{Latex[length=2mm]}, thick} |
| } |
|
|
| \begin{document} |
|
|
| |
| \begin{center} |
| {\fontsize{14}{17}\selectfont\bfseries |
| A Multi-Modal Deep-Learning Benchmark for Rock Glacier Mapping\\ |
| in the Sikkim Himalaya at 10\,m Resolution} |
| \end{center} |
|
|
| \vspace{6pt} |
|
|
| \begin{center} |
| {\fontsize{12}{14}\selectfont Nasiruddin Kabir$^{1*}$} |
| \end{center} |
|
|
| \vspace{6pt} |
|
|
| \begin{center} |
| {\fontsize{11}{13}\selectfont |
| $^{1}$Department of Civil Engineering, Indian Institute of Technology Bombay, Mumbai, India\par} |
| \end{center} |
|
|
| \vspace{6pt} |
|
|
| \begin{center} |
| {\fontsize{11}{13}\selectfont \href{mailto:nasiruddinkabir0123@gmail.com}{nasiruddinkabir0123@gmail.com}\quad(*Corresponding author)} |
| \end{center} |
|
|
| \vspace{10pt} |
|
|
| |
| \noindent |
| {\fontsize{12}{15}\selectfont\bfseries\itshape Abstract:}\enspace |
| {\fontsize{11}{15}\selectfont\itshape |
| Rock glaciers are climatically resilient stores of ground ice and key indicators of mountain |
| permafrost, yet mapping them at scale is hampered by their spectral similarity to surrounding talus and |
| debris. We present an end-to-end deep-learning segmentation workflow for rock glaciers in the Sikkim |
| Himalayas from freely available 10\,m multi-modal satellite data, structured around \textbf{three |
| contributions}. \textbf{First}, a data-collection and preparation pipeline: driven by a recent InSAR-derived |
| rock-glacier inventory for Sikkim (626 polygons; Kumar et al., 2026), it composites Sentinel-2 optical |
| imagery over a cloud-free post-monsoon window, adds Copernicus DEM terrain derivatives and a Sentinel-1 |
| SAR band, and clips a multi-band feature stack for each glacier; the polygons are simultaneously rasterised into binary masks, so every glacier yields a |
| paired feature--mask patch for supervised segmentation. Cloud-probability masking over this window |
| removes the cloud/snow no-data that contaminates naive annual composites --- the single largest source |
| of accuracy gain in our study. \textbf{Second}, a separability-guided band-selection study that |
| identifies a compact, high-performing band subset and shows that \emph{terrain} morphology contributes |
| roughly twice the discriminative power of optical reflectance. \textbf{Third}, a controlled benchmark of |
| four architectures (U-Net, Attention U-Net, SegFormer, Swin-UNet) against a classical Random-Forest |
| baseline under an identical protocol, with test-time augmentation, ensembling, and cross-validation. |
| The Swin-UNet is the best single model (Dice~0.653, IoU~0.485), exceeding the Random Forest by |
| +0.052~Dice and U-Net by +0.043~Dice; a three-model ensemble reaches Dice~0.668 and IoU~0.501, and |
| 5-fold cross-validation confirms stability at Dice~$0.583\pm0.014$. Notably, the Random Forest matches |
| the convolutional baselines, showing that deep learning helps here only when paired with the right |
| inductive bias. The workflow and curated 10\,m Himalayan rock-glacier dataset provide a reproducible, |
| transferable basis for periglacial inventory mapping.} |
|
|
| \vspace{6pt} |
|
|
| \noindent |
| {\fontsize{12}{14}\selectfont\itshape Keywords:}\quad |
| {\fontsize{12}{14}\selectfont\itshape Rock-glaciers; Segmentation; Permafrost; Himalaya; Sentinel; Swin-UNet; Multi-modal; Deep-learning} |
|
|
| \vspace{10pt} |
|
|
| |
| \fontsize{12}{18}\selectfont |
|
|
| \section*{1.\quad Introduction} |
|
|
| Rock glaciers are lobate or tongue-shaped accumulations of angular debris with interstitial ice, |
| predominantly found in the periglacial zones of high-altitude mountain ranges. They act as long-term |
| freshwater reservoirs and are sensitive indicators of permafrost distribution and climate change |
| (Haeberli et al., 2006). In the Hindu Kush--Himalayan region, where glacial and periglacial landforms |
| supply water to millions downstream, accurate mapping of rock glaciers is essential for hydrological |
| modelling, hazard assessment, and climate-impact studies (Bolch et al., 2012). |
|
|
| Traditional rock-glacier inventories rely on manual digitisation from optical imagery---a |
| labour-intensive, subjective process that is difficult to scale. The availability of free 10\,m |
| Sentinel imagery and modern deep-learning segmentation methods opens the way to automation. However, |
| rock glaciers are difficult targets: their spectral signatures overlap with talus, debris-covered ice, |
| and moraine, and their sizes range from small cirque lobes ($\sim$0.4\,km) to valley-filling tongues |
| ($>$2\,km). Purely spectral classifiers therefore struggle, and the quality of the input data |
| (seasonal snow, cloud contamination) strongly conditions achievable accuracy. |
|
|
| This paper makes \textbf{three contributions}. \textbf{(i)}~A reproducible, cloud-free multi-modal |
| data-collection and preparation pipeline that pairs, for each glacier, a 28-band 10\,m feature stack |
| with a rasterised binary mask and eliminates the no-data contamination of naive composites. |
| \textbf{(ii)}~A rigorous band-selection study, based on Cohen's~$d$ separability and cumulative |
| retraining, that identifies an optimal 20-band subset and quantifies the contribution of each feature |
| modality. \textbf{(iii)}~A controlled benchmark of four architectures (U-Net, Attention U-Net, |
| SegFormer, Swin-UNet) and a classical Random-Forest baseline under an identical protocol, with honest |
| held-out test evaluation, test-time augmentation, model ensembling, and 5-fold cross-validation. |
|
|
| \section*{2.\quad Study Area and Reference Data} |
|
|
| The study area encompasses the Sikkim Himalayas, India, spanning approximately |
| 88.07\textdegree--88.95\textdegree\,E and 27.32\textdegree--28.08\textdegree\,N. The terrain is rugged |
| and high-altitude; rock glaciers occur predominantly above 4{,}500\,m on north- and east-facing cirques |
| and valleys. A polygon inventory of 626 active rock glaciers---produced for Sikkim by SBAS-InSAR |
| time-series detection with Google Earth Pro optical refinement (Kumar et al., 2026)---was used as |
| reference. After removing invalid geometries, the inventory was rasterised at 10\,m into binary masks |
| in UTM Zone~45N |
| (EPSG:32645). One 128$\times$128\,px patch (1.28\,km $\times$ 1.28\,km) was extracted per glacier, |
| centred on the glacier centroid, plus a small fraction of glacier-free background patches. The resulting |
| dataset contains 671 patches (613 glacier-bearing, 58 background), split at the patch level into |
| 469 training, 101 validation, and 101 test patches (seed~=~42). Crucially, because all glacier masks are |
| rasterised jointly, neighbouring glaciers within one patch are labelled correctly rather than being |
| treated as background. |
|
|
| \section*{3.\quad Data-Collection Pipeline} |
|
|
| All features were generated on Google Earth Engine and downloaded directly as per-glacier |
| NumPy patches. The key design choice is the \emph{temporal window}: instead of an all-season |
| multi-year median (which retains cloud shadow, haze and seasonal snow, leaving $\sim$20\% of optical |
| pixels as no-data at these altitudes), we composite only October--November scenes from 2020--2023 and |
| mask with the Sentinel-2 cloud-probability product ($p_{\text{cloud}}<35\%$). This post-monsoon, |
| pre-winter window yields a cloud-free, snow-minimised surface with \emph{0\%} optical no-data in the |
| final patches. Terrain derivatives are computed from the Copernicus GLO-30 DEM at native resolution and |
| resampled to 10\,m; a Sentinel-1 VV backscatter composite over the same window supplies a |
| weather-independent structural cue. Figure~1 summarises the pipeline. |
|
|
| \begin{figure}[h] |
| \centering |
| \resizebox{\textwidth}{!}{ |
| \begin{tikzpicture}[ |
| every node/.style={font=\scriptsize}, |
| src/.style ={cylinder, shape border rotate=90, draw, fill=orange!25, align=center, |
| minimum width=1.05cm, minimum height=0.7cm, aspect=0.28}, |
| win/.style ={rectangle, rounded corners=2pt, draw, very thick, fill=violet!18, align=center, |
| minimum width=2.9cm, minimum height=1.0cm}, |
| proc/.style={trapezium, trapezium left angle=74, trapezium right angle=106, draw, |
| fill=yellow!35, align=center, minimum height=0.85cm, text width=3.0cm, inner sep=2.5pt}, |
| feat/.style={rectangle, rounded corners=1pt, draw, fill=cyan!22, align=center, |
| minimum width=2.1cm, minimum height=1.0cm}, |
| mask/.style={rectangle, rounded corners=1pt, draw, fill=red!16, align=center, |
| minimum width=2.1cm, minimum height=1.0cm}, |
| pair/.style={rectangle, rounded corners=1pt, draw, fill=gray!18, align=center, |
| minimum width=2.1cm, minimum height=1.1cm}, |
| outp/.style={rectangle, draw, fill=green!22, align=center, minimum width=1.5cm, minimum height=0.6cm}, |
| ar/.style ={-{Latex[length=2mm]}, thick}] |
|
|
| |
| \node[src] (poly) at (0, 2.1) {Glacier polygon\\inventory\\(626, shapefile)}; |
| \node[win] (win) at (4.1, 2.1) {Per-glacier $128{\times}128$ window\\(centred on centroid;\\neighbouring polygons kept)}; |
|
|
| |
| \node[src] (s2) at (0, 0) {Sentinel-2\\MSI}; |
| \node[src] (dem) at (0,-2.3) {Copernicus\\GLO-30 DEM}; |
| \node[src] (s1) at (0,-4.6) {Sentinel-1\\GRD (VV)}; |
| \node[proc] (f2) at (4.1, 0) {Oct--Nov cloud-prob\\composite $\rightarrow$ 10 optical\\$+$ 7 indices}; |
| \node[proc] (fd) at (4.1,-2.3) {Terrain derivatives\\(slope, TPI, TRI, TWI,\\rough., curv., hillshade)\\$\rightarrow$ 10 terrain}; |
| \node[proc] (fs) at (4.1,-4.6) {Oct--Nov VV\\median $\rightarrow$ 1 SAR}; |
| \node[feat] (feat) at (7.9,-2.3) {28-band feature\\patch\\$128^2\times28$}; |
|
|
| |
| \node[proc] (raster) at (7.9, 2.1) {Rasterise all polygons\\within window}; |
| \node[mask] (mask) at (11.3, 2.1) {Binary mask\\patch\\$128^2\times1$}; |
|
|
| |
| \node[pair] (pair) at (11.3,-2.3) {671 paired\\(feature, mask)\\patches}; |
| \node[outp] (train) at (14.2,-1.4) {Train (469)}; |
| \node[outp] (val) at (14.2,-2.3) {Val (101)}; |
| \node[outp] (test) at (14.2,-3.2) {Test (101)}; |
|
|
| |
| \draw[ar](poly)--(win); |
| \draw[ar](win.south) to[out=-90,in=90] (feat.north); |
| \draw[ar](win.east)--(raster.west); |
| \draw[ar](s2)--(f2); \draw[ar](dem)--(fd); \draw[ar](s1)--(fs); |
| \draw[ar](f2.east)--(feat.north west); |
| \draw[ar](fd.east)--(feat.west); |
| \draw[ar](fs.east)--(feat.south west); |
| \draw[ar](raster)--(mask); |
| \draw[ar](feat.east)--(pair.west); |
| \draw[ar](mask.south)--(pair.north); |
| \draw[ar](pair.east)--(train.west); |
| \draw[ar](pair.east)--(val.west); |
| \draw[ar](pair.east)--(test.west); |
|
|
| |
| \draw[orange!75, dashed, very thick, rounded corners] |
| (-1.5, 3.7) rectangle (15.7, -5.9); |
| \node[anchor=west, font=\normalsize\bfseries, red!70!black] at (-1.35, 3.3) {Data preparation}; |
|
|
| \end{tikzpicture}} |
| \caption{Data-collection pipeline. The glacier polygon inventory (626 polygons) drives the process: |
| each glacier defines a $128\times128$ (1.28\,km) window centred on its centroid, within which |
| \emph{neighbouring} polygons are retained. For every window, (i)~a cloud-free 28-band feature patch is |
| extracted from three sensors --- Sentinel-2 (post-monsoon Oct--Nov, cloud-probability-masked; 10 optical |
| $+$ 7 index bands), Copernicus GLO-30 DEM (10 terrain bands), and Sentinel-1 SAR (1 band) --- and |
| (ii)~all polygons within the window are rasterised into a binary mask. The paired |
| (feature, mask) patches ($n=671$) are split at the patch level into train, validation, and test sets.} |
| \end{figure} |
|
|
| The 28 bands comprise: \emph{ten optical} (B2, B3, B4, B8, B8A, B11, B12, and red-edge B5/B6/B7); |
| \emph{seven indices} (NDVI, NDWI, NDSI, NDMI, BSI, Tasselled-Cap Brightness, Iron-Oxide ratio); |
| \emph{ten terrain} derivatives (Elevation, Slope, Aspect, Plan/Profile Curvature, TPI, TRI, TWI, |
| Surface Roughness, Hillshade); and \emph{one SAR} band (Sentinel-1 VV). All bands were normalised |
| per channel using z-score statistics computed on the training patches only. |
|
|
| \section*{4.\quad Band Selection} |
|
|
| \subsection*{4.1\quad Separability ranking} |
|
|
| To rank the 28 candidate bands we computed Cohen's~$d$ effect size between glacier and background |
| pixel populations for each band across the training and validation patches: |
| $d = |\mu_{g}-\mu_{b}| / \sigma_{\text{pooled}}$. Table~1 reports the ranking. The four highest-ranked |
| bands are all \emph{terrain} descriptors (Surface Roughness, Slope, TPI, TRI), confirming that rock |
| glaciers are, first and foremost, a morphological rather than a spectral target. Sentinel-1 SAR and |
| Elevation also rank highly, while several optical bands and the curvature terms carry very little |
| class-separating information ($d<0.05$). |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Band separability by Cohen's~$d$ (glacier vs.\ background), measured on patches\_v2.} |
| \small |
| \begin{tabular}{clc@{\hskip 1.2cm}clc} |
| \toprule |
| \textbf{Rank} & \textbf{Band} & \textbf{Cohen's $d$} & \textbf{Rank} & \textbf{Band} & \textbf{Cohen's $d$} \\ |
| \midrule |
| 1 & Surface Roughness & 0.496 & 15 & Red-Edge 3 & 0.161 \\ |
| 2 & Slope & 0.489 & 16 & Hillshade & 0.154 \\ |
| 3 & TPI & 0.424 & 17 & Red-Edge 2 & 0.141 \\ |
| 4 & TRI & 0.361 & 18 & B11 (SWIR-1) & 0.123 \\ |
| 5 & NDVI & 0.304 & 19 & Brightness (TC) & 0.081 \\ |
| 6 & B12 (SWIR-2) & 0.275 & 20 & Red-Edge 1 & 0.080 \\ |
| 7 & TWI & 0.261 & 21 & Aspect & 0.077 \\ |
| 8 & S1\_VV (SAR) & 0.237 & 22 & B2 (Blue) & 0.076 \\ |
| 9 & Elevation & 0.237 & 23 & B3 (Green) & 0.064 \\ |
| 10 & NDWI & 0.234 & 24 & B4 (Red) & 0.051 \\ |
| 11 & NDMI & 0.228 & 25 & NDSI & 0.031 \\ |
| 12 & BSI & 0.212 & 26 & Iron-Oxide & 0.008 \\ |
| 13 & B8A (Narrow-NIR) & 0.184 & 27 & Plan Curvature & 0.008 \\ |
| 14 & B8 (NIR) & 0.167 & 28 & Profile Curvature & 0.005 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection*{4.2\quad Cumulative band-count experiment} |
|
|
| Starting from the Cohen's~$d$ ranking, we trained an Attention U-Net (identical protocol) on the |
| top-$k$ bands for $k \in \{8,12,16,18,20,22,24,28\}$ and recorded validation Dice (Table~2). Performance |
| rises with more bands up to $k{=}20$, then \emph{declines}: adding the four lowest-ranked |
| bands ($k{=}24,28$) injects noise and lowers Dice. The 20-band subset is therefore adopted for all |
| subsequent experiments. It comprises the ten terrain derivatives excluding the two curvature terms |
| and Aspect, the SAR band, and the higher-ranked optical bands and indices. |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Effect of band count (Attention U-Net, validation Dice). Best at 20 bands.} |
| \small |
| \begin{tabular}{lcccccccc} |
| \toprule |
| \textbf{\# Bands} & 8 & 12 & 16 & 18 & \textbf{20} & 22 & 24 & 28 \\ |
| \midrule |
| \textbf{Val Dice} & 0.5288 & 0.5438 & 0.5448 & 0.5355 & \textbf{0.5700} & 0.5540 & 0.5497 & 0.5514 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection*{4.3\quad Comprehensive band-configuration study} |
|
|
| Beyond the monotone $k$-sweep, we conducted a total of \textbf{thirteen} band-configuration |
| experiments across two rounds to ensure the 20-band choice is robust and not an artefact of a single |
| selection strategy. Round~1 varied the number of top-ranked bands ($k = 8$ to $28$); Round~2 tested |
| targeted hypotheses: dropping the SAR band (``Top-20~$-$~SAR''), optical-only and terrain-only subsets, |
| a compact terrain$+$SAR set, and a hand-curated 18-band set that removes the three lowest-ranked |
| terrain terms (Aspect, Plan/Profile Curvature). All experiments used the identical Attention U-Net |
| training protocol; Table~3 ranks them by validation Dice. The 20-band configuration is the global |
| optimum across every metric except recall, and no alternative selection strategy surpasses it. |
|
|
| \begin{table}[h] |
| \centering |
| \caption{All thirteen band-configuration experiments (Attention U-Net, validation set), ranked by |
| Dice. The top-20 subset is the overall optimum.} |
| \small |
| \begin{tabular}{clccccc} |
| \toprule |
| \textbf{Rank} & \textbf{Configuration} & \textbf{\# Bands} & \textbf{Dice} & \textbf{IoU} & \textbf{Prec.} & \textbf{Rec.} \\ |
| \midrule |
| 1 & \textbf{Top-20 (terrain+optical+SAR)} & \textbf{20} & \textbf{0.5700} & \textbf{0.4001} & 0.5029 & 0.6642 \\ |
| 2 & Top-22 & 22 & 0.5540 & 0.3855 & 0.4605 & 0.6984 \\ |
| 3 & All bands & 28 & 0.5514 & 0.3835 & 0.4636 & 0.6863 \\ |
| 4 & Top-24 & 24 & 0.5497 & 0.3807 & 0.5010 & 0.6135 \\ |
| 5 & Top-16 & 16 & 0.5448 & 0.3765 & 0.4366 & 0.7272 \\ |
| 6 & Top-12 & 12 & 0.5438 & 0.3748 & 0.5258 & 0.5744 \\ |
| 7 & Top-18 & 18 & 0.5355 & 0.3667 & 0.4795 & 0.6178 \\ |
| 8 & Top-20 $-$ SAR & 19 & 0.5341 & 0.3670 & 0.4565 & 0.6583 \\ |
| 9 & Top-8 & 8 & 0.5288 & 0.3616 & 0.4716 & 0.6096 \\ |
| 10 & Curated-18 (no curvature/aspect) & 18 & 0.5143 & 0.3488 & 0.4854 & 0.5484 \\ |
| 11 & Terrain + SAR & 8 & 0.5086 & 0.3466 & 0.5129 & 0.5104 \\ |
| 12 & Terrain only & 10 & 0.4738 & 0.3113 & 0.4285 & 0.5494 \\ |
| 13 & Optical + indices only & 12 & 0.4465 & 0.2883 & 0.4483 & 0.4468 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection*{4.4\quad Feature-modality ablation} |
|
|
| Isolating the modality-only rows of Table~3 quantifies the contribution of each data source (Table~4). |
| Terrain-only already reaches Dice~0.474, whereas optical-only reaches only 0.447; adding optical to the |
| full stack contributes +0.062~Dice while terrain contributes +0.124 --- roughly twice the discriminative |
| power. The SAR band adds a further +0.036. This confirms that rock-glacier segmentation is dominated by |
| morphological signal, with spectral and radar features providing complementary refinement. |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Feature-modality ablation (validation Dice) and marginal contribution to the 20-band model.} |
| \small |
| \begin{tabular}{lcc} |
| \toprule |
| \textbf{Feature subset} & \textbf{Val Dice} & \textbf{Marginal contribution} \\ |
| \midrule |
| Optical + indices only (12 bands) & 0.4465 & --- \\ |
| Terrain only (10 bands) & 0.4738 & --- \\ |
| Terrain + SAR (8 bands) & 0.5086 & --- \\ |
| \midrule |
| Full top-20 (terrain + optical + SAR) & \textbf{0.5700} & terrain +0.124 / optical +0.062 / SAR +0.036 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \section*{5.\quad Segmentation Models and Training} |
|
|
| As a non-deep reference, a \textbf{Random-Forest} pixel classifier (300 trees, max depth 20, |
| balanced class weights) was trained on the same 20 bands using a class-balanced sample of |
| $\sim$1.2M training pixels. Four deep architectures were then compared under an identical protocol on |
| the 20-band input: |
| \textbf{(i)}~a plain U-Net (Ronneberger et al., 2015) with a 64--128--256--512 encoder; |
| \textbf{(ii)}~an Attention U-Net (Oktay et al., 2018) adding soft attention gates on each skip |
| connection; \textbf{(iii)}~SegFormer (Xie et al., 2021), a hierarchical transformer encoder (MiT-B0) |
| with a lightweight all-MLP decoder; and \textbf{(iv)}~a Swin-UNet (Liu et al., 2021; Cao et al., 2022), |
| a U-shaped network with shifted-window self-attention blocks that capture long-range context while |
| retaining locality. The Swin-UNet architecture (Figure~2) follows a residual encoder--decoder: four |
| residual-convolution encoder stages (16, 32, 64, 128 channels) progressively halve the spatial |
| resolution to an $8\times8$ bottleneck, where two shifted-window self-attention blocks operate over the |
| 64 spatial tokens to model long-range context; a symmetric decoder with skip connections then restores |
| full resolution, and a $1\times1$ convolution with sigmoid activation produces the glacier probability |
| map. |
|
|
| \begin{figure}[h] |
| \centering |
| \begin{tikzpicture}[ |
| enc/.style ={rectangle, draw, fill=blue!12, align=center, font=\scriptsize, |
| minimum width=1.7cm, minimum height=0.62cm}, |
| dec/.style ={rectangle, draw, fill=green!14, align=center, font=\scriptsize, |
| minimum width=1.7cm, minimum height=0.62cm}, |
| bn/.style ={rectangle, rounded corners, draw, fill=orange!22, align=center, |
| font=\scriptsize, minimum width=3.0cm, minimum height=0.95cm}, |
| io/.style ={rectangle, draw, fill=gray!14, align=center, font=\scriptsize, |
| minimum width=1.6cm, minimum height=0.6cm}, |
| dn/.style ={-{Latex[length=1.7mm]}, thick}, |
| sk/.style ={-{Latex[length=1.7mm]}, dashed, gray!70}] |
|
|
| \node[io] (in) at (0,4.5) {Input\\$128^2\times20$}; |
| \node[enc] (e1) at (0,3.55) {Res-Conv 16\\$128^2$}; |
| \node[enc] (e2) at (0,2.55) {Res-Conv 32\\$64^2$}; |
| \node[enc] (e3) at (0,1.55) {Res-Conv 64\\$32^2$}; |
| \node[enc] (e4) at (0,0.55) {Res-Conv 128\\$16^2$}; |
| \node[bn] (bn) at (2.9,-0.55){Res-Conv 256 $+\;2\times$Swin\\$8^2$ (64 tokens)}; |
| \node[dec] (d4) at (5.8,0.55) {Res-Conv 128\\$16^2$}; |
| \node[dec] (d3) at (5.8,1.55) {Res-Conv 64\\$32^2$}; |
| \node[dec] (d2) at (5.8,2.55) {Res-Conv 32\\$64^2$}; |
| \node[dec] (d1) at (5.8,3.55) {Res-Conv 16\\$128^2$}; |
| \node[io] (out) at (5.8,4.5) {Output\\$128^2\times1$}; |
|
|
| \draw[dn] (in)--(e1); |
| \draw[dn] (e1)--(e2) node[midway,left,font=\tiny]{pool $\downarrow$2}; |
| \draw[dn] (e2)--(e3) node[midway,left,font=\tiny]{pool $\downarrow$2}; |
| \draw[dn] (e3)--(e4) node[midway,left,font=\tiny]{pool $\downarrow$2}; |
| \draw[dn] (e4)-|(bn); |
| \draw[dn] (bn)|-(d4); |
| \draw[dn] (d4)--(d3) node[midway,right,font=\tiny]{up $\uparrow$2}; |
| \draw[dn] (d3)--(d2) node[midway,right,font=\tiny]{up $\uparrow$2}; |
| \draw[dn] (d2)--(d1) node[midway,right,font=\tiny]{up $\uparrow$2}; |
| \draw[dn] (d1)--(out); |
| \draw[sk] (e1)--(d1) node[midway,above,font=\tiny,black]{skip}; |
| \draw[sk] (e2)--(d2); |
| \draw[sk] (e3)--(d3); |
| \draw[sk] (e4)--(d4); |
| \end{tikzpicture} |
| \caption{Swin-UNet architecture. A residual convolutional encoder (blue) downsamples the 20-band |
| $128\times128$ input to an $8\times8$ bottleneck (orange), where two shifted-window self-attention |
| (Swin) blocks model long-range context over the 64 spatial tokens. A symmetric decoder (green) with |
| skip connections (dashed) restores full resolution; a final $1\times1$ convolution yields the glacier |
| probability map.} |
| \end{figure} |
|
|
| All models were trained with the AdamW optimiser (weight decay $10^{-4}$), batch size~32, for 250 |
| epochs, using a cosine-annealing learning-rate schedule decaying from $10^{-4}$ to $10^{-6}$. Cosine |
| annealing was found to outperform step-decay scheduling by allowing the model to keep improving late in |
| training. The loss combines a label-smoothed binary cross-entropy with a Focal-Tversky term: |
| \[ |
| \mathcal{L} = 0.5\,\mathcal{L}_{\text{BCE}} |
| + \left(1 - \frac{TP}{TP + \alpha\,FP + \beta\,FN}\right)^{\gamma}, |
| \] |
| with $\alpha{=}0.4$, $\beta{=}0.6$ (mild recall bias), and $\gamma{=}4/3$. Online augmentation |
| comprised random flips, 90\textdegree\ rotations, and radiometric jitter. At inference, the final |
| predictor is an \emph{ensemble} of three independently trained Swin-UNet models, each evaluated with |
| 8-way test-time augmentation (TTA; four rotations $\times$ two flips); the resulting probability maps |
| are averaged and thresholded (Figure~3). The operating threshold was selected by sweeping 0.30--0.70 on |
| the validation set. All metrics are computed over valid (non-padded) pixels. |
|
|
| \begin{figure}[h] |
| \centering |
| \begin{tikzpicture}[node distance=0.45cm and 0.85cm, |
| mdl/.style={rectangle, rounded corners, draw, fill=cyan!15, align=center, |
| font=\scriptsize, minimum width=1.9cm, minimum height=0.8cm}, |
| op/.style ={rectangle, draw, fill=yellow!30, align=center, font=\scriptsize, |
| minimum width=1.5cm, minimum height=0.7cm}, |
| io/.style ={rectangle, draw, fill=green!15, align=center, font=\scriptsize, |
| minimum width=1.7cm, minimum height=0.7cm}, |
| a/.style ={-{Latex[length=2mm]}, thick}] |
|
|
| \node[io] (in) {20-band\\128$\times$128\\patch}; |
| \node[mdl, right=of in, yshift=1.1cm] (m1) {Swin-UNet\\(seed 1)}; |
| \node[mdl, right=of in] (m2) {Swin-UNet\\(seed 2)}; |
| \node[mdl, right=of in, yshift=-1.1cm] (m3) {Swin-UNet\\(seed 3)}; |
| \node[op, right=1.0cm of m1] (t1) {8-way\\TTA}; |
| \node[op, right=1.0cm of m2] (t2) {8-way\\TTA}; |
| \node[op, right=1.0cm of m3] (t3) {8-way\\TTA}; |
| \node[op, right=of t2] (avg) {average\\prob.}; |
| \node[op, right=of avg] (th) {threshold\\$t^{*}$}; |
| \node[io, right=of th] (out) {glacier\\mask}; |
|
|
| \draw[a] (in) -- (m1); \draw[a] (in) -- (m2); \draw[a] (in) -- (m3); |
| \draw[a] (m1) -- (t1); \draw[a] (m2) -- (t2); \draw[a] (m3) -- (t3); |
| \draw[a] (t1) -- (avg); \draw[a] (t2) -- (avg); \draw[a] (t3) -- (avg); |
| \draw[a] (avg) -- (th); \draw[a] (th) -- (out); |
| \end{tikzpicture} |
| \caption{Ensemble inference pipeline. A 20-band patch is passed through three independently trained |
| Swin-UNet models; each applies 8-way test-time augmentation (four rotations $\times$ two flips). The |
| resulting probability maps are averaged and thresholded at the validation-optimal $t^{*}$ to yield the |
| final binary glacier mask.} |
| \end{figure} |
|
|
| \section*{6.\quad Results} |
|
|
| \subsection*{6.1\quad Architecture comparison (held-out test)} |
|
|
| Table~5 reports test-set performance for a classical Random-Forest baseline and all four deep |
| architectures under identical inputs (20 bands). The Random Forest---a per-pixel classifier trained on |
| the same features---reaches Dice~0.601, which is already \emph{competitive with the convolutional |
| baselines} (U-Net 0.610, Attention U-Net 0.604) and in fact exceeds SegFormer (0.555). This is an |
| important sanity check: on a small dataset, a strong classical method is not trivially beaten, and only |
| an architecture with the right inductive bias offers a decisive gain. The Swin-UNet provides that gain, |
| exceeding the Random Forest by +0.052~Dice and the plain U-Net by +0.043~Dice and +0.046~IoU. Notably, |
| SegFormer---despite the strongest \emph{validation} score---generalises worst on the test set, |
| indicating that its global efficient attention over-adapts to the training scenes; the Swin-UNet's |
| shifted-window attention is a better inductive bias for this small-data, terrain-dominated regime. |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Test-set performance of a classical baseline and four deep architectures on 101 held-out |
| patches (identical 20-band input; deep models use 8-way TTA). Best model in \textbf{bold}.} |
| \small |
| \begin{tabular}{lccccc} |
| \toprule |
| \textbf{Model} & \textbf{Dice} & \textbf{IoU} & \textbf{Precision} & \textbf{Recall} & \textbf{Thr.} \\ |
| \midrule |
| Random Forest (classical) & 0.6005 & 0.4291 & 0.5190 & 0.7125 & 0.60 \\ |
| \midrule |
| Plain U-Net & 0.6104 & 0.4392 & 0.6072 & 0.6135 & 0.30 \\ |
| Attention U-Net & 0.6035 & 0.4322 & 0.5910 & 0.6166 & 0.40 \\ |
| SegFormer (MiT-B0) & 0.5547 & 0.3838 & 0.5056 & 0.6145 & 0.35 \\ |
| \textbf{Swin-UNet} & \textbf{0.6529} & \textbf{0.4847} & \textbf{0.6256} & \textbf{0.6827} & 0.35 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection*{6.2\quad Ensemble and cross-validation} |
|
|
| A three-model Swin-UNet ensemble (different random seeds), combined with 8-way TTA, improves the test |
| Dice to 0.668 and pushes IoU past 0.50 (Table~6), with a large recall gain (+0.057) at a small cost in |
| precision. A 5-fold cross-validation---in which each fold is validated on a genuinely held-out slice |
| never seen during that fold's training---yields a Dice of $0.583\pm0.014$, confirming the stability of |
| the result (standard deviation of only 0.014 across folds). |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Swin-UNet single model vs.\ ensemble on the held-out test set, and 5-fold cross-validation.} |
| \small |
| \begin{tabular}{lcccc} |
| \toprule |
| \textbf{Configuration} & \textbf{Dice} & \textbf{IoU} & \textbf{Precision} & \textbf{Recall} \\ |
| \midrule |
| Swin-UNet (single) & 0.6529 & 0.4847 & 0.6256 & 0.6827 \\ |
| Swin-UNet (3-model ensemble + TTA)& \textbf{0.6679} & \textbf{0.5014} & 0.6087 & \textbf{0.7398} \\ |
| \midrule |
| 5-fold CV (mean $\pm$ std) & \multicolumn{4}{c}{Dice $= 0.583 \pm 0.014$} \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| We evaluated five ensembling strategies to select the final predictor (Table~7). The three-seed |
| ensemble and the larger eight-model ``mega'' ensemble tie for the best Dice (0.668) and IoU (0.501); |
| however, the three-seed ensemble attains this with far fewer models and the \emph{highest recall} |
| (0.740), which is preferable for inventory completeness, where a missed glacier is costlier than slight |
| boundary over-segmentation. The five-fold cross-validation ensemble is marginally lower, and an |
| ensemble of models trained with copy-paste augmentation did not improve over the plain seeds. We |
| therefore adopt the parsimonious three-seed ensemble as the final predictor. |
|
|
| \begin{table}[h] |
| \centering |
| \caption{Ensemble-strategy comparison on the held-out test set (all use the Swin-UNet, 20 bands, |
| 8-way TTA). The three-seed ensemble is selected: best Dice/IoU with the fewest models and highest |
| recall.} |
| \small |
| \begin{tabular}{lccccc} |
| \toprule |
| \textbf{Ensemble strategy} & \textbf{\# Models} & \textbf{Dice} & \textbf{IoU} & \textbf{Precision} & \textbf{Recall} \\ |
| \midrule |
| Single model (no ensemble) & 1 & 0.6529 & 0.4847 & 0.6256 & 0.6827 \\ |
| \textbf{3-seed ensemble (chosen)} & \textbf{3} & \textbf{0.6679} & \textbf{0.5014} & 0.6087 & \textbf{0.7398} \\ |
| 5-fold cross-validation ensemble & 5 & 0.6629 & 0.4957 & 0.6278 & 0.7021 \\ |
| 8-model ``mega'' ensemble & 8 & 0.6679 & 0.5014 & 0.6482 & 0.6889 \\ |
| 5-model copy-paste ensemble & 5 & 0.6528 & 0.4845 & 0.6328 & 0.6741 \\ |
| \bottomrule |
| \end{tabular} |
| \end{table} |
|
|
| \subsection*{6.3\quad Training dynamics and qualitative results} |
|
|
| Figure~4 shows the training and validation Dice/loss curves for the Swin-UNet under the |
| cosine-annealing schedule; the model improves smoothly throughout training without abrupt plateaus. |
| Figure~5 presents detailed qualitative results for the five best-segmented held-out test patches, |
| comparing (left to right) the RGB composite, ground-truth mask, model prediction, predicted |
| probability, and a colour-coded error map. Figure~6 provides a nine-patch gallery in which the |
| predicted glacier outline (cyan) is overlaid on the ground-truth outline (yellow). All panels are |
| georeferenced (WGS-84 longitude/latitude). The model delineates well-defined glacier bodies accurately, |
| with most residual error confined to the gradational glacier--talus boundary. |
|
|
| \begin{figure}[h] |
| \centering |
| \includegraphics[width=\textwidth]{paper_figures/training_history.png} |
| \caption{Training dynamics of the Swin-UNet with cosine-annealing learning rate over 250 epochs: |
| (a) Dice and (b) loss for the training and validation sets.} |
| \end{figure} |
|
|
| \begin{figure}[h] |
| \centering |
| \includegraphics[width=0.98\textwidth]{paper_figures/top5_detailed.png} |
| \caption{Detailed qualitative results on the five best-segmented held-out test patches. Columns: |
| RGB composite (with 500\,m scale bar and north arrow), ground truth, predicted mask, predicted |
| probability, and error map (green = true positive, red = false positive, blue = false negative). |
| Axes are WGS-84 longitude/latitude.} |
| \end{figure} |
|
|
| \begin{figure}[h] |
| \centering |
| \includegraphics[width=0.82\textwidth]{paper_figures/montage_best.png} |
| \caption{Gallery of representative test patches. The Swin-UNet prediction (cyan) is overlaid on the |
| ground-truth glacier outline (yellow) over the RGB composite; per-patch Dice is annotated. Axes are |
| WGS-84 longitude/latitude.} |
| \end{figure} |
|
|
| \section*{7.\quad Discussion} |
|
|
| Three findings stand out. First, \emph{data quality dominates}. Replacing a naive all-season composite |
| (with $\sim$20\% cloud/snow no-data) by an October--November cloud-probability composite (0\% optical |
| no-data) was the single largest source of improvement in this study, underlining that in high-mountain |
| segmentation the compositing strategy matters as much as the model. Second, \emph{rock glaciers are a |
| terrain problem}: the Cohen's~$d$ ranking and modality ablation both place terrain derivatives---surface |
| roughness, slope, topographic position---above all optical features, with terrain contributing roughly |
| twice the discriminative power of reflectance. Third, \emph{architecture matters, but with the right |
| inductive bias}: a classical Random Forest already matches the convolutional baselines (Dice~0.601), |
| so the value of deep learning here is not automatic---only the Swin-UNet's shifted-window attention |
| yields a decisive margin (+0.052~Dice over the Random Forest, +0.043 over U-Net), while a |
| global-attention transformer (SegFormer) actually underperforms the classical baseline. The Swin-UNet's |
| advantage is realised without catastrophic overfitting (5-fold CV std of 0.014). |
|
|
| The main limitation is dataset size: with one patch per glacier (671 total), the achievable ceiling is |
| bounded by label quantity, and standard augmentations (CutOut, copy-paste, MixUp) did not yield |
| consistent gains beyond the ensemble in our experiments. The moderate precision (0.63) reflects the |
| gradual glacier--talus boundary, where even expert labels are uncertain at 10\,m. Future work should |
| therefore (i)~expand the dataset with multiple offset patches per glacier and additional Himalayan |
| sub-ranges; (ii)~explore self-training on unlabelled terrain; and (iii)~incorporate multi-temporal SAR |
| interferometric coherence to exploit the diagnostic surface creep of active rock glaciers. |
|
|
| \section*{8.\quad Conclusion} |
|
|
| We presented an end-to-end deep-learning framework for rock-glacier segmentation in the Sikkim |
| Himalayas from free 10\,m multi-modal satellite data. A cloud-free October--November collection |
| pipeline, a separability-driven 20-band selection, and a Swin-UNet architecture together achieve a |
| held-out test Dice of 0.653 (single model) and 0.668 (3-model ensemble with TTA), with a 5-fold |
| cross-validated Dice of $0.583\pm0.014$. The Swin-UNet outperforms U-Net, Attention U-Net, and |
| SegFormer under identical training. Beyond the model, the study contributes a reproducible pipeline, a |
| curated Himalayan rock-glacier dataset, and clear evidence that terrain morphology is the primary |
| discriminator for automated rock-glacier mapping. The approach is directly transferable to other |
| periglacial regions of the Himalaya. |
|
|
| \section*{References} |
|
|
| \noindent |
| Bolch, T., Kulkarni, A., K\"a\"ab, A., Huggel, C., Paul, F., Cogley, J.~G., \ldots\ \& Bajracharya, S. |
| (2012). The state and fate of Himalayan glaciers. \textit{Science, 336}(6079), 310--314. |
|
|
| \vspace{4pt}\noindent |
| Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., \& Wang, M. (2022). Swin-Unet: Unet-like |
| pure transformer for medical image segmentation. \textit{ECCV 2022 Workshops}, 205--218. |
|
|
| \vspace{4pt}\noindent |
| Haeberli, W., Hallet, B., Arenson, L., Elconin, R., Humlum, O., K\"a\"ab, A., \ldots\ \& Vonder |
| M\"uhll, D. (2006). Permafrost creep and rock glacier dynamics. \textit{Permafrost and Periglacial |
| Processes, 17}(3), 189--214. |
|
|
| \vspace{4pt}\noindent |
| Kumar, A., Malviya, A., \& Singh, G. (2026). Multi-sensor-based rock glacier detection over Sikkim |
| Himalaya. \textit{iScience, 29}, 115983. \url{https://doi.org/10.1016/j.isci.2026.115983} |
|
|
| \vspace{4pt}\noindent |
| Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., \ldots\ \& Guo, B. (2021). Swin Transformer: |
| Hierarchical vision transformer using shifted windows. \textit{ICCV 2021}, 10012--10022. |
|
|
| \vspace{4pt}\noindent |
| Oktay, O., Schlemper, J., Folgoc, L.~L., Lee, M., Heinrich, M., Misawa, K., \ldots\ \& Rueckert, D. |
| (2018). Attention U-Net: Learning where to look for the pancreas. \textit{MIDL 2018}. |
|
|
| \vspace{4pt}\noindent |
| Robson, B.~A., Bolch, T., MacDonell, S., H\"olbling, D., Rastner, P., \& Schaffer, N. (2020). Automated |
| detection of rock glaciers using deep learning and object-based image analysis. \textit{Remote Sensing |
| of Environment, 250}, 112033. |
|
|
| \vspace{4pt}\noindent |
| Ronneberger, O., Fischer, P., \& Brox, T. (2015). U-Net: Convolutional networks for biomedical image |
| segmentation. \textit{MICCAI 2015}, 234--241. |
|
|
| \vspace{4pt}\noindent |
| Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.~M., \& Luo, P. (2021). SegFormer: Simple and |
| efficient design for semantic segmentation with transformers. \textit{NeurIPS 2021}, 12077--12090. |
|
|
| \end{document} |
|
|
