Facial Expression Recognition Based on Depth Fusion and Discriminative Association Learning

Xing Jin, Zhihui Lai, Wenyun Sun, Zhong Jin

Abstract: As an effective deep semi-supervised network, deep discrimination association learning has achieved impressive performance in facial expression recognition (FER). However, the instability of facial appearance (i.e. illumination change) is still a big challenge for FER. To this point, we propose a novel multi-modal deep discriminative association (MDDA) framework to better exploit facial depth information and unlabeled data. Firstly, the facial depth map is generated via 3D face reconstruction and encoded in three channels to learn more representative features. Secondly, we design a novel deep multi-loss semi-supervised network based on association learning and exploit multi-modal information through cross fusion mechanism. We evaluate the proposed method on RaFD and Oulu-CASIA datasets and achieve accuracies of 95.64% and 66.75%, respectively. Compared to the existing deep discrimination association learning approach, the accuracies are increased by 1.01% and 4.88% with the encoded facial depth map information. Moreover, extensive experiments confirm that the proposed approach has comparable performance compared to existing deep networks and is more robust to illumination changes.