LatentHSI: Restore hyperspectral images in a latent space

Jin Cao, Xiangyu Rui, Li Pang, Deyu Meng, Xiangyong Cao

Abstract: Hyperspectral image (HSI) restoration is a critical task for remote sensing. The high dimensionality of HSIs poses significant challenges for restoration techniques. Existing methods often involve transforming the image into a lower-dimensional space to address the problem. However, most of these constructed spaces either require handcrafted prior assumptions or ignore utilizing the prior knowledge of HSIs. In this work, we propose a general unsupervised HSI restoration approach in which a new lower-dimensional latent space that fully considers the prior of HSIs is learned automatically from the HSIs. Specifically, we first decompose a clean HSI into the product of a reduced representation and a coefficient matrix by exploiting the low-rank property inherent in HSIs. Then, we construct a latent space for the reduced representation using Variational Autoencoder (VAE) and perform sampling in this space via a latent diffusion model guided by a tailored guidance function. Finally, a correction step is further used to refine the restored image produced by the constructed latent space. Experimental results demonstrate that our method can achieve state-of-the-art (SOTA) performance, surpassing existing methods in both quantitative metrics and visual quality on various HSI restoration tasks, e.g., pansharpening, HSI denoising, and noisy HSI super-resolution.