Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic and Empathetic Response Generation

Anonymous

Abstract: The detection of depression through non-verbal cues has gained significant attention. Previous research predominantly centered on identifying depression within the confines of controlled laboratory environments, often with the supervision of psychologists or counselors. Unfortunately, datasets generated in such controlled settings may struggle to account for individual's behaviors in real-life situations. In response to this limitation, we present the Extended D-vlog dataset, encompassing a collection of $1,261$ YouTube vlogs. Additionally, the emergence of large language models (LLMs) like ChatGPT has sparked interest in their potential if they can act like a Mental Health Professionals. yet, the readiness of these LLM models to use it in real life setting is still a concern as they can give wrong responses which can be harmful for the users. We Proposed a Virtual Agent that can act as a first point of contact for mental health patient and provide a empathetic as well as the Therapeutic response alike human therapists. The system is divided into two part 1. Detection of a Disorder 2. Providing Empathetic and Therapeutic response to the user. The use of TVLT model on our Multimodal Extended D-vlog Dataset produced the most promising results, achieving a remarkable \textbf{F1-score of $67.8$\%}

Paper Type: long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models, Data resources

Languages Studied: English

Preprint Status: We are considering releasing a non-anonymous preprint in the next two months (i.e., during the reviewing process).

A1: yes

A1 Elaboration For Yes Or No: Section 9

A2: yes

A2 Elaboration For Yes Or No: Section 10

A3: yes

A3 Elaboration For Yes Or No: Section 1

B: yes

B1: yes

B1 Elaboration For Yes Or No: All the Section whenever necessary

B2: n/a

B3: n/a

B4: no

B5: n/a

B6: yes

B6 Elaboration For Yes Or No: Section 5

C: yes

C1: yes

C1 Elaboration For Yes Or No: section 5

C2: yes

C2 Elaboration For Yes Or No: section 5

C3: n/a

C4: n/a

D: yes

D1: n/a

D2: n/a

D3: n/a

D4: n/a

D5: n/a

E: no

E1: n/a