Towards Controllable and Interpretable Face Completion via Structure-Aware and Frequency-Oriented Attentive GANs

Zeyuan Chen, Shaoliang Nie, Tianfu Wu, Christopher G. Healey

Keywords: Face Completion, GANs, Conditional Image Synthesis, Interpretability, Frequency-Oriented Attention

TL;DR: Structure-aware and frequency-oriented attentive GANs for high-resolution and fast face completion

Abstract: Face completion is a challenging conditional image synthesis task. This paper proposes controllable and interpretable high-resolution and fast face completion by learning generative adversarial networks (GANs) progressively from low resolution to high resolution. We present structure-aware and frequency-oriented attentive GANs. The proposed structure-aware component leverages off-the-shelf facial landmark detectors and proposes a simple yet effective method of integrating the detected landmarks in generative learning. It facilitates facial expression transfer together with facial attributes control, and helps regularize the structural consistency in progressive training. The proposed frequency-oriented attentive module (FOAM) encourages GANs to attend to only finer details in the coarse-to-fine progressive training, thus enabling progressive attention to face structures. The learned FOAMs show a strong pattern of switching its attention from low-frequency to high-frequency signals. In experiments, the proposed method is tested on the CelebA-HQ benchmark. Experiment results show that our approach outperforms state-of-the-art face completion methods. The proposed method is also fast with mean inference time of 0.54 seconds for images at 1024x1024 resolution (using a Titan Xp GPU).