Examining Transfer Learning with Neural Network and Bidirectional Neural Network on Thermal Imaging for Deception Recognition

Zishan Qin, Xuanying Zhu, Tom Gedeon

Abstract: Deception is a common feature in our daily life, which can be recognised by thermal imaging. Previous research has attempted to identify deception with causality features extracted from thermal images using the extended Granger causality (eGC) method. As the eGC transformation is complicated, in this paper we explore whether a transfer learning model trained on the eGC-transformed thermal deception dataset can be applied to the original thermal data to recognise deception. We explore two feature selection methods, namely linear discriminant analysis (LDA) and t-distributed random neighborhood embedding (t-SNE), and three classifiers, including a support vector machine (SVM), a feed forward neural network (NN) and a bidirectional neural network (BDNN). We find that using features selected by LDA, a transfer learning NN is able to recognise deception with an accuracy of 91.7% and an F1 score of 0.92. We believe this study helps foster a deeper understanding of eGC and provides a foundation for building transfer learning models for deception recognition.