FACE SUPER-RESOLUTION GUIDED BY 3D FACIAL PRIORS

xiaobin hu, wenqi ren, jiaolong yang, xiaochun cao, Xiaoming Li, John LaMaster, Bjoern Menze, wei liu

TL;DR: We propose a novel face super resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures.

Abstract: State-of-the-art face super-resolution methods employ deep convolutional neural networks to learn a mapping between low- and high-resolution facial patterns by exploring local appearance knowledge. However, most of these methods do not well exploit facial structures and identity information, and struggle to deal with facial images that exhibit large pose variation and misalignment. In this paper, we propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures. Firstly, the 3D face rendering branch is set up to obtain 3D priors of salient facial structures and identity knowledge. Secondly, the Spatial Attention Mechanism is used to better exploit this hierarchical information (i.e. intensity similarity, 3D facial structure, identity content) for the super-resolution problem. Extensive experiments demonstrate that the proposed algorithm achieves superior face super-resolution results and outperforms the state-of-the-art.

Keywords: Super-resolution, 3D Facial priors, Spatial Attention Mechanism

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2007.09454/code)