Pre-trained Neural Recommenders: Learning Statistical Representations for Zero-shot Recommender Systems

Junting Wang, ADIT KRISHNAN, Hari Sundaram, Yunzhe Li

Keywords: Recommender System, Zero-shot Learning, Pre-trained Models, Neural Collaborative Filtering

Abstract: Modern neural collaborative filtering (NCF) techniques are critical to the success of e-commerce, social media, and content-sharing platforms. However, despite technical advances---for every new application domain, we need to train an NCF model from scratch. In contrast, pre-trained vision and language models are routinely applied to diverse applications directly (zero-shot) or with limited fine-tuning. Inspired by the impact of pre-trained models, we explore the possibility of pre-trained recommender models that support building recommender systems in new domains, with minimal or no retraining, without the use of any auxiliary user or item information. We propose a framework that computes the user and item representations via learning the representations of the user/item activity quantiles. With extensive experiments on five diverse datasets, we show that the framework can not only generalize to unseen users and unseen items within a dataset and across different datasets (i.e., cross-domain, zero-shot) but with comparable performance to state-of-the-art neural recommenders.

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5907