How Challenging is Multimodal Irony Detection?

Manuj Malik, David Tomás, Paolo Rosso

Abstract: The possibility that social networks offer to attach audio, video, and images to textual information has led many users to create messages with multimodal irony. Over the last years, a series of approaches have emerged trying to leverage all these formats to address the problem of multimodal irony detection. The question that the present work tries to answer is whether multimodal irony systems are being properly evaluated. More specifically, this work studies the most popular dataset used in multimodal irony detection combining text and images, identifying whether image information is really necessary to understand the ironic intention of the text. This corpus was compared to a text-only corpus, and different textual and multimodal Transformer models were evaluated on them. This study reveals that, in many situations, Transformer models were able to identify the ironic nature of the posts considering only textual information.