Overview of the CLEF 2024 JOKER Track - Automatic Humour Analysis

Liana Ermakova, Anne-Gwenn Bosser, Tristan Miller, Victor Manuel Palma-Preciado, Grigori Sidorov, Adam Jatowt

Abstract: The JOKER Lab series at the Conference and Labs of the Evaluation Forum (CLEF) was established in 2022 to promote collaborative, interdisciplinary research on the automated processing of wordplay and verbal humour. This paper provides an overview of the setup and results of the Lab’s 2024 edition. We describe the data and evaluation metrics used for the Lab’s three shared tasks (on humour-aware information retrieval, humour classification according to genre and technique, and translation of puns from English to French) and introduce and compare the systems that participated in each task, with particular attention to their approaches and performance.