Multi-Modal Depression Detection in Interview via Exploring Emotional Distribution Information

Zhiyuan Zhou, Yanrong Guo, Shijie Hao, Richang Hong

Abstract: In recent years, automatic depression detection (ADD) technology has been rapidly developed to boost an objective and assistive diagnosis for major depressive disorder (MDD) with the help of artificial intelligence technology and various physiological and psychological data. Despite emotion being an important reflection of mental status and frequently related to depression symptoms, few recent multi-modal ADD methods take emotional information into account. To address the above issue, we propose to explore emotional distribution information in interviews to assist multi-modal ADD model. On one hand, we use large language models (LLMs) to automatically recognize emotion of text data, and re-organize the data guided by the valence attribute of emotion, which facilitates our model being aware of difference in emotion distribution. On the other hand, we design the emotion encoding which enhances the proposed model to consider the emotional distribution information in its decision-making process. Extensive experiments are conducted by comparing with state-of-the-art ADD methods as well as the ablation study on different modules of the proposed method. More importantly, our experimental results can confirm the research findings in the psychology field, where more attention on negative emotion information is demanded in distinguishing different depressive status.

External IDs:dblp:journals/tmm/ZhouGHH25