BACN: Bi-direction Attention Capsule-based Network for Multimodal Sentiment Analysis

Anonymous

Abstract: Capsule-based network has currently identified its effectiveness in analyzing the heterogeneity issue of multimodal sentiment analysis. However, existing manners could only exploit the spatial relation between representation and output layer via down-top attention, which fails to effectively explore both inter-modality and intra-modality context. In this paper, during the preprocess period, we first present the multimodal dynamic enhanced module to facilitate the intra-modality context, which significantly boost the learning efficiency in dealing with multimodal heterogeneity issue. Furthermore, the bi-direction attention capsule-based network (BACN) is proposed to capture dynamic inter-modality context via the novel bi-direction dynamic routing mechanism. Specifically, BACN firstly highlights the static and low-level inter-modality context based on top-down attention. Then, the static multimodal context is transmitted to dynamic routing procedure, naturally allowing us to investigate dynamic and high-level inter-modality context. This indeed unleash the expressive power and provides the superior capability to bridge the modality gap among all the modalities. The experiments demonstrate that BACN can achieve state-of-the-art performance.