Reconstructing Prototype From Contaminated Face With Variations Across Heterogeneous Domains

Meng Pang, Binghui Wang, Nanrun Zhou, Yintao Zhou, Wei Huang

Abstract: This paper focuses on a new heterogeneous prototype learning (HPL) problem, which aims at reconstructing the variation-free and identity-preserved prototype in the target domain from a contaminated input image in the source domain. Most existing heterogeneous face synthesis (HFS) methods are unsuitable for HPL, as these methods focus on performing accurate image-to-image translation with facial details unaltered, but cannot effectively remove the input facial variations. In this paper, we propose an identity-aware cycle-consistent network, dubbed IAC2N, for image-to-prototype transformation across domains. To address HPL, IAC2N designs three effective losses, i.e., prototype adversarial loss, label information guided identity loss, and prototype learning cycle loss, in its objective. The first loss is used for transferring the domain style as well as removing the universal facial variations. The latter two losses are used for maintaining the identity consistency during HPL from an explicit and an implicit perspectives, respectively. Furthermore, IAC2N is a joint learning framework that is able to learn the identity feature for the contaminated image via its encoder-decoder structural generator in order to perform heterogeneous face recognition (HFR). Extensive experiments on various heterogeneous face datasets demonstrate the effectiveness of IAC2N in both tasks of HPL and HFR.