Beyond Statistical Correlation: Causal Insights into Emotion Recognition

Tao Chen, Yanrong Guo, Shijie Hao, Richang Hong

Abstract: Emotion recognition has gained significant attention recently due to its wide-ranging applications like human-computer interaction, affective computing, and social robotics. Despite the promising results, critical issues need to be addressed. One primary challenge is that existing models typically establish spurious statistical correlations between the input image and the label instead of investigating causal-and-effect relationships. Besides, some similar emotional states often appear simultaneously, rendering the model unable to establish spurious correlations between these similar labels empirically. To address these issues, we develop a Dual-Disentanglement Causal Learning (D2CL) framework consisting of two disentanglement modules: a Feature Disentanglement module and a Label Disentanglement module. The first one extracts emotion-related representation and context-specific embedding to explore the underlying causal relationships between facial features and emotions, thereby mitigating spurious correlations. The second proposes a Feature Similarity-based classifier to capture subtle distinctions between similar labels. Extensive experiments on the EMOTIC and JAFFE datasets validate the effectiveness and superiority of the proposed method.

External IDs:dblp:conf/icmcs/ChenGHH25