Combining Informative Regions and Clips for Detecting Depression from Facial Expressions

Xiaoyan Yuan, Zhenyu Liu, Qiongqiong Chen, Gang Li, Zhijie Ding, Zixuan Shangguan, Bin Hu

Abstract: Artificial intelligence methods are widely applied to depression recognition and provide an objective solution. Many effective automated methods for detecting depression use facial expressions, which are strong indicators of psychiatric disorders. However, existing approaches ignore the uneven distribution of depression information in time and space. Therefore, these approaches have limitations in their ability to form discriminative depression representations. In this paper, we propose a framework based on information regions and clips for depression detection. Specifically, we first divide the regions of interest (ROIs), which are regarded as spatially informative regions, according to pathological knowledge of depression. Following this, the local-MHHLBP-BiLSTM (LMB) module is proposed as a feature extractor to exploit short-term and long-term temporal information. Finally, an improved attention mechanism with a balancing factor is introduced into LMB to increase attention to information segments. The proposed model performs tenfold cross-validation on our 150-subject video dataset and outperforms most state-of-the-art approaches with accuracy = 0.757, precision = 0.767, recall = 0.786, and F1 score = 0.761. The obtained results demonstrate that focusing on information regions, and clips can effectively reduce the error in depression diagnosis. More importantly, we observe that the area near the eye is fairly informative and that depressed individuals blink more frequently.