Cross Modal Few Shot Learning for Tree Species Classification Using Airborne Hyperspectral Images

Lei Hu, Wei He, Hongyan Zhang

Abstract: Tree species classification is essential for forest resource surveys and monitoring activities. Although airborne hyperspectral images (HSIs) can provide rich spatial and spectral information, the lack of labeled samples and the high similarity between spectra remain challenges for achieving fine-grained tree species classification mapping. In this article, a cross modal few shot learning framework is presented for multiple tree species classification (CMTSC). Notably, we innovatively introduce language prior knowledge to guide the generation of discriminative visual features, aiming to use additional modalities to improve the uni-modal classification task. Firstly, an improved three-dimensional ghost attention network (TGAN) with strong learning capability without massive parameters is constructed. Secondly, we use linguistic features from class names to optimize the decision boundary of the visual classifier by cross-modal adaptation. Thirdly, the cross domian few shot learning (FSL) strategy is employed to overcome the dilemma of sparse labeled samples and fixed application scenarios. Experiments on Gaofeng Forest Farm B (GFF-B) in Nanning City demonstrate the effectiveness of the proposed method compared to other state-of-the-art methods. The codes will be available at: https://github.com/HlEvag/CMTSC.