SSGRAM: 3-D Spectral-Spatial Feature Network Enhanced by Graph Attention Map for Hyperspectral Image Classification

Bishmoy Paul, Shaikh Anowarul Fattah, Adnan Rajib, Mohammad Saquib

Abstract: Convolutional neural networks (CNNs) and graph neural networks (GNNs) are two widely used architectures in hyperspectral image (HSI) classification. Most CNN models tend to heavily rely on neighboring pixels of the target pixel, leading to performance degradation when target pixels differ from their surrounding pixels. To address this limitation, this article introduces a novel approach to HSI classification using a combination of CNNs and GNNs to complement the drawbacks of CNN models. We propose a CNN-based 3-D spectral-spatial feature network (3D-S2FN) to extract features in spectral, spatial, and spectral-spatial spaces, utilizing pyramid squeeze attention to prioritize their importance. Additionally, we introduce a graph attention feature processor (GAFP) module that evaluates the relevance of neighboring HSI pixels to the target pixel, generating a graph attention map (GRAM). This GRAM is applied to the CNN’s intermediate features to mitigate neighborhood bias. Unlike conventional GNNs, the proposed GAFP module directly processes the data within a local window, without requiring dimensionality reduction methods that would diminish pixel-level detail. By integrating features from the GAFP and the attention-enhanced 3D-S2FN, the proposed SSGRAM method achieves superior classification performance, as demonstrated through extensive experiments on four publicly available datasets. The model code has been published in https://github.com/BishmoyPaul/SSGRAM