Modeling the Clause-Level Structure to Multimodal Sentiment Analysis via Reinforcement Learning

Dong Zhang, Shoushan Li, Qiaoming Zhu, Guodong Zhou

Abstract: In this paper, we propose a novel approach to multimodal sentiment analysis with focus on both textual and acoustic modalities. Especially, we utilize deep reinforcement learning to explore the clause-level structure in an utterance. On the basis, we perform multimodal interactions at clause-level to model hierarchical interactive representation for multimodal senitment analysis. Detailed evaluation on two benchmark datasets demonstrates the great effectiveness of our approach over several state-of-the-art baselines.