Enhancing Reproducibility and Replicability in Information Retrieval: A Path Towards Scientific Integrity and Effective Research

Download PDF
Open Webpage

Antonio Ferrara, Claudio Pomo, Nicola Tonellotto

Published: 01 Jan 2025, Last Modified: 10 Nov 2025ICWE 2025EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: While Information Retrieval (IR) and Recommender Systems (RecSys) have made significant strides, reproducibility and replicability remain elusive due to methodological inconsistencies and pressures for high-impact results. Despite reproducibility tracks, guidelines, and frameworks, inconsistent application and limited motivation for rigorous reproducibility hinder progress. This tutorial introduces tools and methodologies for reproducibility in IR and RecSys research. It focuses on foundational concepts like selecting appropriate baselines and designing replicable experimental pipelines, and practical case studies demonstrating the impact of minor experimental variations. This session equips researchers with actionable strategies for producing robust, transparent studies, fostering scientific integrity within the IR and RecSys communities and supporting more reliable and impactful research advancements.