An Emoji-aware Multitask Framework for Multimodal Sarcasm Detection

Anonymous

Abstract: Sarcasm is a case of implicit emotion and needs additional information like context and multimodality for its better detection. But sometimes this additional information also fails to help in sarcasm detection. For example, the utterance "Oh yes, you’ve been so helpful. Thank you so much for all your help", said in a polite tone with a smiling face, can be understood easily as non-sarcastic because of its positive sentiment. But, if the above message is accompanied with a frustrated emoji 😤, the negative sentiment of emoji becomes evident and the intended sarcasm can be easily understood. Thus, in this paper, we propose the SEEmoji MUStARD, an extension of the multimodal MUStARD dataset. We annotate each utterance with relevant emoji, emoji's sentiment and emoji's emotion. We propose an emoji-aware multitask deep learning framework for multimodal sarcasm detection (i.e. primary task), and sentiment and emotion detection (i.e. secondary task) in a multimodal conversational scenario. Experimental results on the SEEmoji MUStARD show the efficacy of our proposed approach for sarcasm detection over the state-of-the-art.

Paper Type: short