ICDAR 2021 Competition on Multimodal Emotion Recognition on Comics Scenes

Nhu-Van Nguyen, Xuan-Son Vu, Christophe Rigaud, Lili Jiang, Jean-Christophe Burie

Abstract: The paper describes the “Multimodal Emotion Recognition on Comics scenes” competition presented at the ICDAR conference 2021. This competition aims to tackle the problem of emotion recognition of comic scenes (panels). Emotions are assigned manually by multiple annotators for each comic scene of a subset of a public large-scale dataset of golden age American comics. As a multi-modal analysis task, the competition proposes to extract the emotions of comic characters in comic scenes based on visual information, text in speech balloons or captions and the onomatopoeia. Participants were competing on CodaLab.org from December 16 $$^{th}$$ 2020 to March 31 $$^{th}$$ 2021. The challenge has attracted 145 registrants, 21 teams have joined the public test phase, and 7 teams have competed in the private test phase. In this paper we present the motivation, dataset preparation, task definition of the competition, the analysis of participant’s performance and submitted methods. We believe that the competition have drawn attention from the document analysis community in both fields of computer vision and natural language processing on the task of emotion recognition in documents.