Seeing through the clouds: enhanced snow and cloud segmentation in sentinel-2 imagery with mDeepLabV3+

Getnet Demil, Ali Torabi Haghighi, Björn Klöve, Mourad Oussalah

Abstract: The application of satellite-based remote sensing has become increasingly prominent in snow hydrology parameter estimation, owing to its cost-efficiency, frequent revisit times, and broad spatial coverage. However, the presence of clouds introduces significant obstacles by obscuring features in optical satellite imagery. Differentiating snow from clouds remains challenging due to their similar color profiles and reflective properties. Conventional approaches, including manual thresholding, traditional machine learning techniques, and advanced deep learning models such as Convolutional Neural Networks (CNNs) and U-Net, have achieved only moderate success, despite remote sensing imagery’s rich spatial and spectral data. To address this, we propose a novel deep-learning framework for classifying background, snow, and cloud regions in multispectral satellite imagery, based on a modified DeepLabV3+ architecture. Our approach incorporates an elevated dilation rate of 24 and leverages cross-domain transfer learning with ResNet-50 and ResNet-101 as backbone networks. Additionally, we utilized the Random Forest algorithm to evaluate the significance of spectral band features and support the semantic segmentation process. We also explored the impact of different input band combinations to determine the optimal configuration. Notably, the combination of Blue (Band 2), Short-Wave Infrared (Band 11), Red (Band 4), and Water Vapor (Band 9) with the ResNet-101 backbone outperformed all other tested configurations. This study provides a detailed assessment of the segmentation capabilities of modified DeepLabV3+ (mDeepLabV3+) models, using ResNet-50 (R50) and ResNet-101 (R101) backbones, on Sentinel-2 imagery for distinguishing cloud, snow, and background classes. Performance was benchmarked against established models, including Random Forest (RF), U-Net, and UCTNet, across various band combinations. Our results reveal that the mDeepLabV3+ model with the R50 backbone achieves exceptional precision (99.9%), F1 score (98.0%), and mean Intersection over Union (mIOU) of 91.13%, while the R101 variant excels in recall (97.8%) and overall accuracy (96.78%). These outcomes highlight the superior performance and reliability of the mDeepLabV3+ models for Sentinel-2 image segmentation. This work presents a robust and domain-adapted approach for snow and cloud segmentation in Sentinel-2 imagery, demonstrating improved accuracy over existing baselines. This thorough analysis offers critical insights into the efficacy of deep learning in such tasks, emphasizing opportunities for further refinement and practical implementation.

External IDs:dblp:journals/esi/DemilHKO25