Towards Reproducibility of Interactive Retrieval Experiments: Framework and Case Study

Jana Isabelle Friese, Norbert Fuhr

Abstract: In interactive information retrieval (IIR), the inherent variability in user behavior and the complexities of user-system interactions pose significant challenges to reproducibility, which remain largely underexplored. To address these challenges, we propose a three-level model for evaluating the reproducibility of IIR experiments through the similarity of experimental findings, underlying measurements, and user behavior across original and reproduction studies. For each level, we introduce specific criteria, offering a structured framework for assessing reproducibility in IIR research.

External IDs:doi:10.1007/978-3-031-88717-8_11