Spatial-Aware GAN for Instance-Guided Cross-Spectral Face Hallucination

Wenpeng Xiao, Cheng Xu, Huaidong Zhang, Xuemiao Xu

Abstract: An efficient strategy to solve the Heterogeneous Face Recognition (HFR) is to translate the probes to the same spectrum domain of the galleries using generative models. However, without or with only globally-pooled appearance representation from a reference, the low-quality generated images restrict the recognition accuracy. The intuition of our paper is the spatially-distributed appearance contains details beneficial to higher-quality image synthesis. Particularly, we propose a semantic spatial adaptive alignment module to solve the inevitable misalignment between the content from the near-infrared (NIR) image and the appearance from the visible (VIS) reference. In this way, arbitrary VIS reference can provide appearance with sufficient details to assist the NIR-to-VIS translation. Based on this, we propose an unsupervised spatial-aware instance-guided cross-spectral facial hallucination network (SICFH) for visual-pleasing and identity-preserved VIS image translation. Qualitative and quantitative experiments on three challenging NIR-VIS datasets demonstrate the synthesized VIS images address the HFR problem effectively and achieve state-of-the-art recognition accuracy.