Unsupervised Cross-View Facial Expression Image Generation and Recognition

Ning Sun, Qingyi Lu, Wenming Zheng, Jixin Liu, Guang Han

Abstract: We propose an unsupervised cross-view facial expression adaptation network (UCFEAN) to simultaneously generate and recognize cross-view facial expressions in images in an unsupervised manner. The main idea of UCFEAN is to convert the unsupervised domain adaptation between two image spaces with different appearance into semi-supervised learning (SSL) in feature spaces with the same semantic content. The cyclic image generation of cross-view facial expressions based on the generative adversarial network (GAN) is carried out to project unlabelled target images and labelled source images to the corresponding feature spaces with the same semantic content. This helps realize the unsupervised feature learning of the target image. Labels of facial expressions represented in the projected target features can then be learned using the projected source features, because the distributions of the projected features in the two domains are close enough for knowledge transfer by using SSL. Three techniques are developed to train UCFEAN in an effective and stable manner. Extensive experiments are conducted to evaluate the UCFEAN on two multi-view facial expression image databases including RaFD and Multi-PIE. The results show that the proposed method can generate realistic target images of the facial expression and recognize cross-view facial expressions with high precision.