Facial dynamic modelling using long short-term memory network: Analysis and application to face authentication

Seong-Tae Kim, Dae Hoe Kim, Yong Man Ro

Abstract: According to the supplementary information hypothesis in psychology, facial motion benefits the perception of identity for human. In this study, we propose a face authentication framework which exploits facial dynamics with appearance to effectively improve the authentication performance. In our face authentication scenario, users are guided to make smile expression and the identity behind smile dynamics has been utilized. In order to model the facial dynamics, the recurrent neural network with long short-term memory cells is adopted and the facial dynamics from onset to offset duration is encoded. Comparative experiment has showed that the combination of facial dynamic features with appearance features improves the accuracy of the face authentication system compared to conventional appearance features and spatio-temporal features by effectively capturing facial dynamic and appearance features.