Towards Reproducibility of Interactive Retrieval Experiments: Framework and Case Study

Download PDF
Open Webpage

Jana Isabelle Friese, Norbert Fuhr

Published: 2025, Last Modified: 23 Jan 2026ECIR (4) 2025EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: In interactive information retrieval (IIR), the inherent variability in user behavior and the complexities of user-system interactions pose significant challenges to reproducibility, which remain largely underexplored. To address these challenges, we propose a three-level model for evaluating the reproducibility of IIR experiments through the similarity of experimental findings, underlying measurements, and user behavior across original and reproduction studies. For each level, we introduce specific criteria, offering a structured framework for assessing reproducibility in IIR research.