On the Role of Numerical Encoding in Foundation Model of Sequential Recommendation with Sequential Indexing

Refaldi Putra, Toyotaro Suzumura

Keywords: Foundation model, Numerical encoding, Sequential recommendation

TL;DR: The prior of numerical encoding in the foundation model of recommender affects the performance.

Abstract: We study a foundation model for recommender systems named P5 on a sequential recommendation task with sequential indexing. The P5 needs to handle numerical values due to the usage of user-item's IDs in the prompt. However, it is unclear how the prior numerical encoding affects the task. We think this requires special attention since it might pose a challenge on the performance and future development of recommender systems. To do so, we conducted experiments where the prior of numerical encoding is set from an addition task. We found that by doing so, it can improve the performance compared with the vanilla P5. We also found that this performance is affected by the structure of the priors. However, the models no longer retain its addition ability. This gives insight on the role of numerical encoding in the foundation models for recommender systems.

Submission Number: 5