CNN and Transformer Fusion Network for Sea Ice Classification Using GaoFen-3 Polarimetric SAR Images

Jiande Zhang, Wenyi Zhang, Xiao Zhou, Qingwei Chu, Xiaoyi Yin, Guangzuo Li, Xiangyu Dai, Shuo Hu, Fukun Jin

Abstract: This article investigates the safety risks associated with sea ice along the Arctic Route by using polar sea ice images obtained by Gaofen-3 (GF3) Synthetic Aperture Radar (SAR) satellites. We collected three SAR datasets, representative of GF3 satellites' operational modes, and constructed semantic segmentation datasets through meticulous annotation using concurrent optical satellite imagery obtained by Landsat satellites to ensure precision. We propose SI-CTFNet, an innovative sea ice semantic segmentation model that integrates convolutional neural networks (CNNs) and vision transformers (ViTs) for enhanced multiscale feature extraction. SI-CTFNet features a dual-pathway architecture designed to optimize feature extraction, complemented by the BiAFusion module, which effectively merges local and global features to improve decoding accuracy. In addition, we introduce the MSDAM module to facilitate a comprehensive multiscale contextual analysis, addressing the diverse distribution of ice types in the imagery. The model incorporates advanced decoding techniques, including a progressive upsampling approach for the CNN-fusion branch and an efficient All-MLP module for the Transformer branch. Performance evaluations across three distinct datasets reveal that SI-CTFNet significantly outperforms existing methods in key metrics and maintains efficacy with supplementary Sentinel1A C-band satellite data. Furthermore, we present a streamlined variant of SI-CTFNet, which achieves a threefold increase in inference speed with minimal reduction in classification accuracy. The ultimate objective of this work is to advance a precise sea ice forecasting and navigation system for polar regions, aimed at automating sea ice classification within a smart polar shipping framework.