Facial Micro-Expression Recognition Based on Multi-Scale Temporal and Spatial Features

Hao Zhang, Bin Liu, Jianhua Tao, Zhao Lv

Abstract: Micro-expression is a kind of facial activity with weak changes and short duration that can reflect people’s true feelings. For micro-expression recognition, it is not only necessary to extract the spatial feature information of the face movement changes in the image, but also to consider the time series information of the continuous image sequence. Thus, we propose a multiple aggregation networks to verify the impact of local facial regions and temporal features on micro-expression recognition in detail. It can learn the temporal and spatial feature from the whole micro expression video frame and combined the local region where the micro-expression mainly occurs with the global region. The spatial features of micro-expressions frames are extracted by 3D CNN, and the extracted video sequences features are input into LSTM processing temporal features. Experiments from two public datasets, CASME-II and SAMM, show that our method obtains higher performance than several existing studies.