Generalizable Recommender System During Temporal Popularity Distribution Shifts

Hyunsik Yoo, Ruizhong Qiu, Charlie Xu, Fei Wang, Hanghang Tong

Abstract: Many modern recommender systems represent user and item attributes as embedding vectors, relying on them for accurate recommendations. However, entangled embeddings often capture not only intrinsic property factors (e.g., user interest in item property) but also popularity factors (e.g., user conformity to item popularity) indistinguishably. These embeddings, influenced by popularity distribution, may face challenges when the popularity distribution at test time differs from historical distribution. Existing remedies in the literature involve disentangled embedding learning, which aims to separately capture intrinsic and popularity factors, demonstrating plausible generalization during popularity distribution shifts. However, we highlight that these methods often overlook a crucial aspect of popularity shifts—their temporal nature—in both training and inference phases. To address this, we propose Temporal Popularity distribution shift generalizABle recommender system (TPAB), a novel disentanglement framework incorporating temporal popularity. TPAB introduce a new (1) temporal-aware embedding design for users and items. Within this design, (2) popularity coarsening and (3) popularity bootstrapping are proposed to enhance generalization further. We also provide theoretical analysis showing that the bootstrapping loss eliminates the effect of popularity on the learned model. During inference, we infer test-time popularity and corresponding embeddings, using them alongside property embeddings for prediction. Extensive experiments on real-world datasets validate TPAB, showcasing its outstanding generalization ability during temporal popularity distribution shifts.