Multimodal Face Synthesis From Visual Attributes

Xing Di, Vishal M. Patel

Abstract: Synthesis of face images from visual attributes is an important problem in computer vision and biometrics due to its applications in law enforcement and entertainment. Recent advances in deep generative networks have made it possible to synthesize high-quality face images from visual attributes. However, existing methods are specifically designed for generating unimodal images (i.e., visible faces) from attributes. In this paper, we propose a novel generative adversarial network which simultaneously synthesizes identity preserving multimodal face images (i.e., visible, sketch, thermal, etc.) from visual attributes without requiring paired data in different domains for training the network. We introduce a novel generator with multimodal stretch-out modules to simultaneously synthesize multimodal face images. Additionally, multimodal stretch-in modules are introduced in the discriminator which discriminate between real and fake images. Extensive experiments and comparison with several state-of-the-art methods are performed to verify the effectiveness of the proposed attribute-based multimodal synthesis method.