Deep Convolutional Neural Network Fusing Local Feature and Two-stage Attention Weight Learning for Facial Expression Recognition

Jian Zheng, Chi Zheng, Hao Liu, Xiangchun Yu

Abstract: Facial local detail information plays an important role in facial expression recognition (FER). However, most of the existing methods only focus on the high-level semantic information of facial expressions, while ignoring the fine-grained information of local facial regions. To solve this problem, this paper proposed a Deep Convolutional Neural Network Fusing Local Feature and Two-stage Attention Weight Learning (FLF-TAWL), which can adaptively capture important facial regions to improve the effectiveness of facial expression recognition. The FLF-TAWL network is composed of a dual-branch framework, one branch extracts local features from image blocks, and the other branch extracts global features from the entire expression image. Firstly, this paper proposes a two-stage attention weight learning strategy. In the first stage, the importance weights of global and local features are roughly learned; In the second stage, the attention weight is further refined, and the local and global features are fused.. Secondly, we use a region-biased loss function to encourage the most important regions to obtain higher attention weights. Finally, extensive experiments are carried out on FERPlus, Cohn-Kanada(CK+) and JAFFE datasets to obtain accuracy rates of 90.92%,98.90% and 97.39%, respectively. The experimental results verify the effectiveness and feasibility of the FLF-TAWL model.