Foreign Shadow Robust Makeup Transfer via Hierarchical Deep Aggregation and Disentangled Representation

Chang Zhan, Lin Wang, Feng Liu, Zhaoyang Sun, Jinli Zhou, Shengwu Xiong

Abstract: Facial makeup transfer aims to transfer the makeup style from a reference makeup image to a source image. In daily life, the reference makeup images from various scenes are uneasy about guaranteeing quality. In contrast, the source images photoed by ourselves are likely to control illumination and avoid shadows cast by occlusion. To this end, we propose a novel robust makeup transfer network (SRMT) for facial makeup and de-makeup under the foreign shadow. The reference image is hierarchically aggregated multi-context features for predicting makeup style which is input into a disentangled network with source image for achieving shadow robust makeup transfer. In particular, we first incorporate a shadow manipulation module(SMM) to manipulate the reference makeup, which furtherly restores the original color distribution on the face. Then we propose a makeup transfer module(MTM) based on disentangled representation, producing a high-quality source image from the predicted reference makeup. Extensive experiments show the superiority of our method in terms of visual effects. Moreover, a carefully designed makeup dataset with paired shadow-deshadow makeup images is available at https://github.com/lucuspring/Deshadow-dataset.