Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recognition in conversations

Yuntao Shou, Tao Meng, Wei Ai, Fuchen Zhang, Nan Yin, Keqin Li

Abstract: Highlights•A multimodal emotion recognition architecture through adversarial alignment and graph fusion is proposed.•A cross-modal feature alignment method with adversarial learning is designed to eliminate inter-modal heterogeneity.•A graph contrastive learning method via information bottleneck is proposed to enhance multimodal semantic association.•Our method can be applied to other multimodal tasks in a plug-and-play manner, e.g., humor detection.