Adaptive multi-round retrieval with knowledge distillation for sequential recommendation

Yuhua Mo, Yang Liu, Chaowen Ye, Zhangtao Cheng, Chao Deng, Zhencheng Zhuo, Kaidi Chen, Fan Zhou

Abstract: Sequential Recommendation (SR) involves predicting the next item that a user is likely to interact with based on their historical interactions. SR models examine the sequence of a user’s actions to analyze complex behavioral patterns and capture diverse user preferences. However, existing works primarily rely on a single-round inference paradigm, which limits their ability to capture the ever-changing diversity of user preferences, and overlooks the influence of user noisy interactions. In this work, we propose MRKD, an adaptive multi-round retrieval framework for sequential recommendation via past-future knowledge distillation. MRKD comprises three key modules: user-wise translator, item-wise translator and past-future knowledge distillation. User-wise and item-wise translator extract meaningful context information from multi-round retrieval processes for refining the representations of items and users in proximity to the target item. The past-future knowledge distillation is to supervise the contextual aggregation process and prevent information loss via distilling valuable knowledge from users’ future interactions. We conduct experiments on five datasets and compare MRKD with 10 competitive baselines to evaluate its performance. Experimental results demonstrate the superiority of our MRKD, equipped with the adaptive multi-round retrieval strategy, over existing state-of-the-art models.

External IDs:dblp:journals/jiis/MoLYCDZCZ25