Cascade of Encoder-Decoder CNNs with Learned Coordinates Regressor for Robust Facial Landmarks Detection

Roberto Valle a, José M. Buenaposada, Luis Baumela

Abstract:

Convolutional Neural Nets (CNNs) have become the reference technology for many computer vision
problems. Although CNNs for facial landmark detection are very robust, they still lack accuracy
when processing images acquired in unrestricted conditions. In this paper we investigate the use of a
cascade of Neural Net regressors to increase the accuracy of the estimated facial landmarks. To this
end we append two encoder-decoder CNNs with the same architecture. The first net produces a set
of heatmaps with a rough estimation of landmark locations. The second, trained with synthetically
generated occlusions, refines the location of ambiguous and occluded landmarks. Finally, a densely
connected layer with shared weights among all heatmaps, accurately regresses the landmark coordi-
nates. The proposed approach achieves state-of-the-art results in 300W, COFW and WFLW that are
widely considered the most challenging public data sets.