Identification of Multimodal Stance Towards Frames of Communication
Submission Type: Regular Long Paper
Submission Track: Sentiment Analysis, Stylistic Analysis, and Argument Mining
Submission Track 2: Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.
Keywords: stance detection, covid-19, social media, twitter, multimodal, images, multimedia
TL;DR: Constructed first multimedia text/image stance detection towards communication framings dataset and demonstrated high stance detection performance exploiting relations between text and image stance.
Abstract: Frames of communication are often evoked in multimedia documents. When an author decides to add an image to a text, one or both of the modalities may evoke a communication frame. Moreover, when evoking the frame, the author also conveys her/his stance towards the frame. Until now, determining if the author is in favor of, against or has no stance towards the frame was performed automatically only when processing texts. This is due to the absence of stance annotations on multimedia documents. In this paper we introduce MMVax-Stance, a dataset of 11,300 multimedia documents retrieved from social media, which have stance annotations towards 113 different frames of communication. This dataset allowed us to experiment with several models of multimedia stance detection, which revealed important interactions between texts and images in the inference of stance towards communication frames. When inferring the text/image relations, a set of 46,606 synthetic examples of multimodal documents with known stance was generated. This greatly impacted the quality of identifying multimedia stance, yielding an improvement of 20% in F1-score.
Submission Number: 4220
Loading