Towards Reliable Item Sampling for Recommendation Evaluation

Open Webpage

Zhi Liu, Dong Li, Ruoming Jin, Zhenming Liu, Bin Ren, Jing Gao

Published: 01 Jun 2023, Last Modified: 15 May 2025Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37(4), pp. 4409–4416EveryoneCC BY-SA 4.0
Abstract: We address the “blind-spot” issue in item-sampling metrics highlighted by Rendle & Krichene. First, we propose a new item-sampling estimator optimizing error w.r.t. ground truth and provide theoretical analysis of its advantage. Second, we develop an adaptive sampling method to mitigate blind spots and generalize EM-based algorithms for this setting. Experiments confirm improved accuracy and strong theoretical grounding for sampling evaluation.