Entire-detail motion dual-branch network for micro-expression recognition

Bingyang Ma, Lu Wang

Abstract: Micro-expression recognition is becoming an increasingly attractive research topic due to its useful applications in a widespread area including psychology, criminology, and security. Different from macro-expressions, the facial muscle movements of micro-expressions have the characteristics of being short duration, and weak intensity, which makes micro-expression recognition extremely challenging. To deal with these problems, we propose a dual-branch classification network that integrates entire and detail motions for effective micro-expression recognition. In this network, one branch is responsible for capturing the overall motion, while the other branch focuses on capturing the detail motion. In addition, to improve the recognition accuracy, we also design a Swin-Transformer module with accumulated attention to focus more on the Region of Interest. By utilizing Grad-CAM to obtain the facial expression activation heatmaps, we find a good match between the activated regions and facial action units. Finally, we validate the effectiveness of the method on the SMIC, CASME II, SAMM, and MMEW datasets, achieving recognition performance that are more competitive than many other state-of-the-art methods. Code is available at https://github.com/likemby/EDMDBN