Continuous Spatial-Spectral Reconstruction via Implicit Neural Representation

Ruikang Xu, Mingde Yao, Chang Chen, Lizhi Wang, Zhiwei Xiong

Abstract: Existing methods for spectral image reconstruction from low spatial/spectral resolution inputs are typically in discrete manners, only producing results with fixed spatial/spectral resolutions. However, these discrete methods neglect the continuous nature of three-dimensional spectral signals, limiting their applicability and performance. To address this limitation, we propose a novel method leveraging implicit neural representation, which allows for spectral image reconstruction with arbitrary resolutions in both spatial and spectral dimensions for the first time. Specifically, we design neural spatial-spectral representation (NeSSR), which projects the deep features extracted from low-resolution inputs to the corresponding intensity values under target 3D coordinates (including 2D spatial positions and 1D spectral wavelengths). To achieve continuous reconstruction, within NeSSR we devise: a spectral profile interpolation module, which efficiently interpolates features to the desired resolution, and a coordinate-aware neural attention mapping module, which aggregates the coordinate and content information for the final reconstruction. Before NeSSR, we design the spatial-spectral encoder leveraging large-kernel 3D attention, which effectively captures the spatial-spectral correlation in the form of deep features for subsequent high-fidelity representation. Extensive experiments demonstrate the superiority of our method over existing methods across three representative spatial-spectral reconstruction tasks, showcasing its ability to reconstruct spectral images with arbitrary and even extreme spatial/spectral resolutions beyond the training scale.