Meta-MMFNet: Meta-learning-based multi-model fusion network for micro-expression recognition

Wenjuan Gong, Yue Zhang, Wei Wang, Peng Cheng, Jordi Gonzàlez

Abstract: Despite its wide applications in criminal investigations and clinical communications with patients sufferingfrom autism, automatic micro-expression recognition remains a challenging problem because of the lackof training data and imbalanced classes problems. In this study, we proposed a meta-learning-based multi-model fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on themetric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable formodel-level fusion. The frame difference and optical flow features were fused, deep features were extractedfrom the fused feature, and finally in the meta-learning-based framework, weighted sum model fusion methodwas applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-artmethods on four datasets. The code is available at https://github.com/wenjgong/meta-fusion-based-method.