Scalable Evaluation of Rule-Based Recommender Systems: Algorithms and Benchmarks

Len Feremans, Bart Goethals

Abstract: Recommender systems help users to identify the most relevant items from a huge collection of items. Rule-based recommenders offer efficient, interpretable, accurate, and trustworthy recommendations, addressing key challenges in recommender design. Using association rules having a single or multiple conditions, we build transparent white-box models, especially for long-tail items. Moreover, recent studies challenge the trade-off between interpretability and accuracy. However, aspects beyond accuracy and efficiency—such as popularity bias, coverage, diversity, and comprehensibility—have largely been overlooked in prior evaluations. Additionally, well-known higher-order rule-based recommender methods lack scalability. Finally, many methods have been proposed that vary in rule form, scoring measures, aggregation and inference strategies. We introduce RuleRec, a scalable toolkit offering six seminal rule-based recommenders. We extend Apriori, MSApriori and adaptive-support rule mining, thereby presenting novel algorithms based on the generalization of pairwise rule-mining using an inverted index. We find that the proposed algorithms are an order of magnitude more efficient. Finally, we empirically evaluate six rule-based recommender algorithms on six benchmark datasets, comparing their accuracy, efficiency, diversity, popularity bias, and comprehensibility. To our knowledge, this is the first work to provide an efficient open-source implementation and comparative evaluation over multiple rule-based recommenders.