SCPSN: Spectral Clustering-based Pyramid Super-resolution Network for Hyperspectral Images
Abstract: Single hyperspectral image super-resolution (HSSR) aims to reconstruct a high-resolution hyperspectral image (HRHSI) from an observed low resolution hyperspectral image (LRHSI). Most current methods combine CNN and Transformer structures to directly extract features of all channels in LRHSI for image reconstruction, but they do not consider the interference of redundant information in adjacent bands, resulting in spectral and spatial distortions in the reconstruction results, as well as an increase in model computational complexity. To address this issue, this paper proposes a spectral clustering-based pyramid super-resolution network (SCPSN) to progressively reconstruct HRHSI at different scales. In each layer of the pyramid network, a clustering super-resolution block consisting of spectral clustering block (SCB), patch non local attention block (PNAB), and dynamic fusion block (DFB) is designed to achieve the reconstruction of detail features for LRHSI. Specifically, for the high correlation between adjacent spectral bands in LRHSI, an SCB is first constructed to achieve clustering of spectral channels and filtering of hyperchannels. This can reduce the interference of redundant spectral information and the computational complexity of the model. Then, by utilizing the non-local similarity of features within the channel, a PNAB is constructed to enhance the features in the hyperchannels. Next, a DFB is designed to reconstruct the features of all channels in LRHSI by establishing correlations between enhanced hyperchannels and other channels. Finally, the reconstructed channels are upsampled and added with the upsampled LRHSI to obtain the reconstructed HRHSI. Extensive experiments validate that the performance of SCPSN is superior to that of some state-of-the-art methods in terms of visual effects and quantitative metrics. In addition, our model does not require training on large-scale datasets compared to other methods. The dataset and code will be released on GitHub.
Primary Subject Area: [Experience] Multimedia Applications
Secondary Subject Area: [Content] Media Interpretation
Relevance To Conference: Image restoration technology is an important branch of multimedia data processing technology, including super-resolution reconstruction, image rain removal, low brightness enhancement, etc. Due to limitations in the imaging environment, satellite hyperspectral sensors typically capture hyperspectral images (HSIs) with rich spectral information but relatively low spatial resolution. HSI super-resolution reconstruction aims to improve the spatial resolution of HSI images while maintaining their spectral resolution. To solve the problems of high computational complexity and insufficient spatial detail reconstruction in HSI super-resolution reconstruction tasks, this paper proposes a pyramid super-resolution network based on spectral clustering (SCPSN). Numerous experiments have shown that the proposed SCPSN has superior performance compared to some SOTA methods.
Submission Number: 5612
Loading