REVEAL 2018: offline evaluation for recommender systems

Thorsten Joachims, Adith Swaminathan, Yves Raimond, Olivier Koch, Flavian Vasile

Abstract: The inaugural REVEAL workshop1 focuses on revisiting the offline evaluation problem for recommender systems. Being able to perform offline experiments is key to rapid innovation; however practitioners often observe significant differences between offline results and the outcome of an online experiment, where users are actually exposed to the resulting recommendations. This is unfortunate because online experiments take time, can be costly, and require access to a live recommender system, when offline experiments are inherently scalable. How can we bridge that gap between offline and online experiments?