Micro-Expression Recognition by Combining Progressive-Learning Intensity Magnification with Self-Attention-Convolution Classification

Wei Zhao, Ling Lei, Tong Chen, Shigang Li, Jianfeng Li

Abstract: How to detect the subtle intensity change and find the inherent relation of feature maps in a face image is the two key issues for micro-expression recognition. In this study, in order to cope with the subtle intensity change, we design an innovative Cascade Micro-Expression Magnification Network (Cascade-MEMN) to magnify the low amplitude of the intensity change for learning the intensity progressively, which helps suppress overlapping artifacts and produce more realistic magnification. In order to find the inherent relation of facial feature maps, we design a Self-attention-based Convolution Layer (SCL), which introduces self-attention to every pixel when calculating the weights in the sliding window, considering that the variation of micro-expressions is very subtle and local on face. Concretely, the SCLs are used to replace 3 Resblocks from the last stage of ResNet50 with 3 SCL blocks. Finally, a new micro-expression recognition system is realized by combining the progressive-learning-based intensity magnification network with the modified self-attention-convolution classification network. Consequently, the proposed method achieves competitive results for the composite database evaluation (CDE) protocol from MEGC 2019, as shown in the experimental result.