Context-Aware Element Filter for Hyperspectral Image Super-Resolution

Ran Ran, Liang-Jian Deng, Chen-Yu Zhao

Abstract: Hyperspectral image super-resolution (HISR) aims to fuse a low-resolution image (LR-HSI) and a high-resolution multispectral image (HR-MSI), generating a high-resolution hyperspectral image (HR-HSI). Previous attempts to apply convolutional neural networks (CNNs) with spatial-variant adaptive filters for HISR tasks. Such filters overcome the spatial invariance and content-agnostic property of standard convolution. However, the current adaptive filters only consider pixellevel specificity, ignoring that each element of the features has unique close relationships with their neighbourhoods. To address the issue, we propose a context-aware element filter (CEF) operation, which generates adaptive filters for each element with sufficient perception of the specificity of each element to improve the representation capability. CEF can generate a single-channel filter to trade off the computational resource consumption for each element and is appropriate for HISR tasks with element-level dependencies. Specifically, we design a new network structure for HISR, which utilizes CEF to replace the standard convolution in the residual block. Extensive experiments demonstrate the superiority of the proposed CEF both visually and quantitatively compared with state-of-the-art (SOTA) methods.