A Deep Multi-task Generative Adversarial Network for Face Completion

Qiang Wang, Huijie Fan, Yandong Tang

Abstract: Face completion is a challenging task that requires a known mask as prior information to restore the missing content of a corrupted image. In contrast to well-studied face completion methods, we present a Deep Multi-task Generative Adversarial Network (DMGAN) for simultaneous missing region detection and completion in face imagery tasks. Specifically, our model first learns rich hierarchical representations, which are critical for missing region detection and completion, automatically. With these hierarchical representations, we then design two complementary sub-networks: (1) DetectionNet, which is built upon a fully convolutional neural net and detects the location and geometry information of the missing region in a coarse-to-fine manner, and (2) CompletionNet, which is designed with a skip connection architecture and predicts the missing region with multi-scale and multi-level features. Additionally, we train two context discriminators to ensure the consistency of the generated image. In contrast to existing models, our model can generate realistic face completion results without any prior information about the missing region, which allows our model to produce missing regions with arbitrary shapes and locations. Extensive quantitative and qualitative experiments on benchmark datasets demonstrate that the proposed model generates higher quality results compared to state-of-the-art methods.