Learning Unoccluded Face Texture Completion from Single Image in the Wild

Yongtang Bao, Pengfei Zhou, Peng Zhang, Yue Qi

Abstract: In recent years, single tasks such as face frontalization, image inpainting, and glasses removal have improved face de-occlusion. However, there is little work on joint learning of multiple de-occlusion tasks. To achieve multi-task learning, we propose an unoccluded face synthesis (UFS) framework for multi-tasks such as face frontalization, image inpainting, and glasses removal, which can remove glasses, face self-occlusion, and external occlude. Our UFS framework consists of an encoder, an image reconstruction module, a decoder, and an image discriminator. First, Gaussian random noise extracts high-dimensional features from images in the encoder module. Next, the image reconstruction module includes multi-scale feature fusion, residual hole block, and self-attention network. As a result, it can strengthen the learning of multi-level fine-grained features and achieve better results in face restoration and face frontalization tasks. Then, we synthesize unoccluded face textures from multi-level fine-grained elements in the decoder. Finally, the image discriminator learns the global information structure of the synthesized image, preventing problems such as distortion and blurring of the picture. Experiments show that our UFS framework can achieve better results on single tasks such as face frontalization, image inpainting, and glasses removal. It also can obtain acceptable results on multiple tasks such as face frontalization and glasses removal simultaneously.