MuSe '24: The 5th Multimodal Sentiment Analysis Challenge and Workshop: Social Perception & Humor

Lukas Christ, Shahin Amiriparian, Andreas König, Simone Eulitz, Erik Cambria, Björn W. Schuller

Abstract: The 5th Multimodal Sentiment Analysis Challenge (MuSe), a workshop in conjunction with ACM Multimedia '24, is focused on Multimodal Machine Learning in the domain of Affective Computing. Two different sub-challenges are proposed: Social Perception Sub-Challenge(MuSe-Perception) is a regression task on 16 different perceived characteristics including arrogance, sincerity and likability. For this, the novel LMU-ELP dataset is introduced, consisting of short audiovisual recordings of 177 subjects annotated by MTurk workers. Cross-Cultural Humour Detection Sub-Challenge(MuSe-Humor) poses the problem of cross-cultural humor recognition. An extension of the Passau Spontaneous Football Coach Humour (Passau-SFCH) dataset featuring football coach press conference recordings is utilized in this task. The training set consists of German data, while the test data comprises English recordings only. In both sub-challenges, audiovisual recordings, transcripts, pre-computed features, and baseline models are made available. In this summary, we detail the challenge's motives, organization, participation statistics, and results.