Multi-Modal Depression Detection Based on High-Order Emotional Features

Yuran Ru, Ning Geng, Li Li, Hui Wang, Yongxiang Zheng, Zhenhua Tan

Abstract: The diagnosis of depression has always been a difficulty in its treatment. At present, the research on automatic depression detection mostly directly uses low-order features such as video, audio and text as input. The lack of guidance of high-order features may be a potential problem. This paper proposed a multi-modal depression detection method based on high-order emotional features. A two-stage network is designed to realize emotion recognition and depression detection at the same time, and input the emotional results as high-order semantic features into the improved TBJE-E multi-modal network. This process guided the learning of other modalities with the help of co-attention module, and finally gave the prediction results. The results of experiments on DAIC-WOZ dataset show that the addition of emotional features effectively complements the high-order semantics. Compared with the original TBJE model, the F1 performance of TBJE-E model with emotional features is relatively improved by 6.3%. The method in this paper has reached the SOTA level in the depression detection task. The experimental data also show that at present, the risk of individual internal psychological privacy being stolen by this technology without their knowledge is very low, and this technology has some application value in criminal investigation, psychological diagnosis and treatment and other professional fields.