Reembedding and Reweighting are Needed for Tail Item Sequential Recommendation
Track: User modeling, personalization and recommendation
Keywords: sequential recommendation, tail item, language models and vision models, reweighting and reembedding
Abstract: Large vision models (LVMs) and large language models (LLMs) are becoming cutting-edge for sequential recommendation, given their success in broad applications. Despite their advantages over traditional approaches, these models suffer more significant performance degradation on tail items against conventional ID-based solutions, which are largely overlooked by recent research. In this paper, we substantiate the above challenges as (1) all-in ground-truth, i.e., the standard cross-entropy (CE) loss focuses solely on the target items while treating all non-ground-truth equally, causing insufficient optimization for tail items, and (2) knowledge transfer tax, i.e., the knowledge encapsulated in LLMs and LVMs dominates the optimization process due to insufficient training for tail items. We propose reweighting and reembedding, a simple yet efficient method to address the above challenges. Specifically, we reinitialize tail item embedding via a Gaussian distribution to alleviate knowledge transfer tax; besides, a reweighting function is incorporated in the CE loss, which adaptively adjusts item weights during training to encourage the model to pay more attention to tail items rather than exclusively optimizing for ground-truth. Overall, our method enables a more nuanced optimization and is mathematically comparable to the direct preference optimization (DPO) in LLMs. Our extensive experiments on three public datasets show our method outperforms fourteen baselines in overall performance and improves the performance on tail items by a large margin. Our code is available at https://anonymous.4open.science/r/R2Rec-0AE0.
Submission Number: 1181
Loading