Learning Pixel Perception for Identity and Illumination Consistency Face Frontalization in the Wild

Pengfei Zhou, Yongtang Bao, Yue Qi

Abstract:

In recent years, single tasks such as face frontalization, image inpainting, and glasses removal
have improved face de-occlusion. However, there is little work on joint learning of multiple
de-occlusion tasks. To achieve multi-task learning, we propose an unoccluded face synthesis
(UFS) framework for multi-tasks such as face frontalization, image inpainting, and glasses
removal, which can remove glasses, face self-occlusion, and external occlude. Our UFS
framework consists of an encoder, an image reconstruction module, a decoder, and an image
discriminator. First, Gaussian random noise extracts high-dimensional features from images
in the encoder module. Next, the image reconstruction module includes multi-scale feature
fusion, residual hole block, and self-attention network. As a result, it can strengthen the
learning of multi-level fine-grained features and achieve better results in face restoration
and face frontalization tasks. Then, we synthesize unoccluded face textures from multilevel fine-grained elements in the decoder. Finally, the image discriminator learns the global
information structure of the synthesized image, preventing problems such as distortion and
blurring of the picture. Experiments show that our UFS framework can achieve better results
on single tasks such as face frontalization, image inpainting, and glasses removal. It also can
obtain acceptable results on multiple tasks such as face frontalization and glasses removal
simultaneously.