Causal-Aware Reconstruction Network for Robust Multimodal Emotion Recognition with Missing Modalities

ACL ARR 2026 January Submission3506 Authors

Keywords: Multimodal Emotion Recognition, Missing Modality, Causal Inference

Abstract: Multimodal Emotion Recognition (MER) often suffers from missing modalities in real-world scenarios due to sensor malfunctions, asynchronous signals, or degraded inputs. While recent studies have explored modality reconstruction to alleviate this issue, most existing methods rely heavily on dominant co-occurrence patterns in contextual information, which may induce spurious correlations and lead to biased reconstruction results under incomplete modalities. To address this limitation, we introduce a causality-aware perspective into missing-modality emotion recognition. Specifically, we propose a Causal-Aware Reconstruction Network that explicitly models causal cues from conversation history based on the Causal-Cue Encoder to guide the reconstruction process, rather than relying solely on surface-level correlations. Moreover, we design a Granger causality–inspired self-supervised constraint to effectively capture and leverage causal dependencies within multimodal contexts. Extensive experiments on two benchmark datasets demonstrate that our method outperforms existing methods under incomplete modalities.

Paper Type: Long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: Emotion Detection and Analysis, Causality, Contrastive Learning

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 3506