Deep Image Fusion Accounting for Inter-Image Variability

Xiuheng Wang, Ricardo Augusto Borsoi, Cédric Richard, Jie Chen

Abstract: Hyperspectral and multispectral image fusion (HMIF) allows us to overcome inherent hardware limitations of hyperspectral imaging systems with respect to their lower spatial resolution. However, existing algorithms fail to consider realistic image acquisition conditions, or to leverage the powerful representation capacity of deep neural networks. This paper introduces a general imaging model which considers inter-image variabil-ity of data from heterogeneous sources, and formulates the optimization problem. Then it presents a new image fusion method that, on the one hand, solves the optimization prob-lem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, leverages unsu-pervised light-weight CNN-based denoisers to learn realistic image priors from data. Its performance is illustrated with real data that suffer from inter-image variability.