From Traits to Empathy: Personality-Aware Multimodal Empathetic Response Generation
Abstract: Empathetic dialogue systems improve the user experience across various domains. Existing approaches mainly focus on acquiring affective and cognitive information from text, often neglecting the unique personality traits of individuals and the inherently multimodal nature of human conversation. To this end, we propose enhancing dialogue systems with the ability to generate customized empathetic responses, considering the diverse personality traits of speakers, and we advocate for the incorporation of multimodal data analysis to gain a more detailed comprehension of speakers' emotional states and context.
Specifically, we initially identify the speaker's trait across the context. The dialogue system then comprehends the speaker's emotion and situation by emotion perception through the analysis of multimodal inputs. Finally, the response generator models the correlations among the captured personality, emotion, and multimodal data, thereby generating empathetic responses. Extensive experiments are conducted utilizing the MELD dataset and the IEMOCAP dataset to investigate the influence of personality traits on empathetic response generation and validate the effectiveness of the proposed approach.
Paper Type: Long
Research Area: Human-Centered NLP
Research Area Keywords: human-AI interaction
Languages Studied: English
Submission Number: 4168
Loading