Face Expression Recognition via Product-Cross Dual Attention and Neutral-Aware Anchor Loss

Yongwei Nie, Rong Pan, Qing Zhang, Xuemiao Xu, Guiqing Li, Hongmin Cai

Abstract: Face expression recognition is an important task whose aim is to classify a face image to a kind of expression such as happy, sad, or surprise, etc. This task is challenging due to the ambiguities in expressions and also in the diverse poses and occlusions of the head. To handle this challenging task, recent approaches usually rely on attention mechanism to make the network focus on the most critical regions of a face, or apply a consistency loss that enforces extracting similar features from the same expressions. This paper proposes a new attention mechanism that combines the advantages of dot-product attention and feature cross-attention. The proposed new product-cross dual attention mechanism can better leverage the landmarks to extract more discriminative features from an input image. Second, although previous approaches can enforce similarity between features of the same expressions, they do not consider the arousal degree of an expression. We propose a neutral-expression-aware expression feature similarity loss based on the traditional anchor loss, which can further guide the network to learn better features from an input image. Extensive experiments demonstrate the advantages of our method over previous approaches.