FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task

Alan Ramponi, Sara Tonelli

Keywords: Natural language processing, fallacy detection, argumentation mining, human label variation

Abstract: FadeIT is the first shared task on fallacy detection in social media texts in Italian, an understudied language for this task. FadeIT relies on Faina, a fallacy detection dataset that includes span-level annotations with overlaps for 20 fallacy types in social media texts about migration, climate change, and public health over a 4-year time period. The shared task is articulated into two subtasks at different granularities: i) post-level fallacy detection, aiming at predicting the fallacy types expressed in each input post, and ii) span-level fallacy detection, aiming at predicting all text segments expressing any given fallacy type in each input post. Participants' systems are evaluated against two equally valid gold standards (i.e., parallel annotations in Faina) to account for natural disagreement, in line with recent work advocating the importance of considering human label variation in subjective tasks. FadeIT has attracted wide interest at Evalita 2026 with a total of 25 runs submitted by 7 participant teams. In this paper, we present the task setup, including the data used and the evaluation criteria, as well as the results obtained by all participant teams, an analysis of their approaches, and insights for future research on the topic.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 5