Extending Cognitive Reframing Therapy: Multimodal Support and Multi-hop Psychotherapeutic Reasoning
Abstract: Previous studies have shown that Large Language Models (LLMs) have significant potential in supporting cognitive reframing therapy.
However, these studies have primarily focused on uni-modal therapy, often overlooking the importance of the client's non-verbal cues.
Identifying non-verbal emotions plays a crucial role in effective communication and is considered a central skill in psychotherapy.
To alleviate this gap, we extend the concept of cognitive reframing conversation to multimodality.
Specifically, we present a new dataset called Multi Modal-Cognitive Support Conversation (MM-CSConv), which pairs each dialogue with an image of the client's facial expression.
Additionally, we introduce a multi-hop psychotherapeutic reasoning approach to enhance the capabilities of Vision-Language Models (VLMs) as psychotherapists.
This approach uses multi-hop reasoning over the conversations, incorporating implicit evidence crucial in psychotherapy.
Our extensive experiments with both LLMs and VLMs show that the abilities of VLMs as psychotherapists are significantly enhanced through the MM-CSConv.
Moreover, the multi-hop psychotherapeutic reasoning method allows VLMs to offer more rational and empathetic suggestions, outperforming standard prompting methods.
Paper Type: Long
Research Area: Computational Social Science and Cultural Analytics
Research Area Keywords: human behavior analysis, multi-modal dialogue systems
Contribution Types: Publicly available software and/or pre-trained models, Data resources
Languages Studied: English
Submission Number: 2757
Loading