Exploiting Discriminative Advantage of Spectrum for Hyperspectral Image Classification: SpectralFormer Enhanced by Spectrum Motion Feature
Abstract: As for hyperspectral images (HSIs), the discrepancy of contiguous spectral information should be the main basis for the identification of ground objects. Due to the difficulty of spectral sequence coding and the spectrum similarity between categories, successful deep-learning-based classification methods always attempt to capture the spatial information to improve the accuracy by convolutional neural networks (CNNs) or other excellent spatial feature extractors. However, extracting spatial features is generally accompanied by the distortion of ground objects distribution and categories boundary. To effectively represent spectral features, the SpectralFormer based on transformer backbone can better capture the long-term dependence of the spectrum, which improves the performance of spectral feature methods significantly. However, it is still unable to compete with advanced spectral–spatial feature methods. To exploit the discriminative advantage of the spectrum fully, this letter introduces an efficient sparse-to-dense optical flow estimation method to track the spectrum variation in the HSI. Then, we take such a variation as a spectrum motion feature to enhance the original spectrum. At last, we continue to use the SpectralFormer to encode the concatenated spectrum sequence for classification. Extensive experiments show that the SpectralFormer enhanced by the spectrum motion feature (SF-SMF) significantly improves the performance of spectral feature methods, even surpassing advanced spectral–spatial feature methods. SF-SMF can avoid interference with additional spatial information to obtain exquisite whole-domain classification maps, showing its practical value. The codes will be public at https://github.com/sssssyf/SF-SMF.
Loading