Attention guided complementary feature integration for latent image recovery from noisy/blurry pairs

Green Rosh K. S, Sachin Deepak Lomte, Nikhil Krishnan, B. H. Pawan Prasad

Abstract: Low-light imaging using a hand-held camera is a challenging task due to excessive noise in the scene. Most of the existing methods try to address this problem either by denoising an image captured using high ISO or by deblurring an image captured using long exposure time. However these methods use a single image to estimate the latent scene and hence fail to leverage the complimentary information available in the scene. In this paper, we propose a method to estimate the latent image using a pair of images captured using high ISO and high exposure time respectively, to leverage the complimentary information present in the two captures. We propose a novel deep learning based method to efficiently extract and integrate the information present in the two images. Contrary to other methods, we use separate filters to extract the complimentary information from the two images. We also progressively integrate the extracted features using a novel attention-guided mechanism. Further, we address the spatially varying nature and localization of motion blur in real life captures by using spatial attention layers. The proposed method achieves state-of-the-art performance against single as well as other noisy/blurry approaches to the problem. We also show that the network learns spatial attention maps with strong correlation to the blur in the scene, and thus the proposed method is more interpretable and easier to analyze.