Multimodal Generative Recommendation for Fusing Semantic and Collaborative Signals

ICLR 2026 Conference Submission21371 Authors

Keywords: Generative Recommendation, Generative Retrieval, Residual Quantization, Recommender Systems

Abstract: Sequential recommender systems rank relevant items by modeling a user's interaction history and computing the inner product between the resulting user representation and stored item embeddings. To avoid the significant memory overhead of storing large item sets, the generative recommendation paradigm instead models each item as a series of discrete semantic codes. Here, the next item is predicted by an autoregressive model that generates the code sequence corresponding to the predicted item. However, despite promising ranking capabilities on small datasets, these methods have yet to surpass traditional sequential recommenders on large item sets, limiting their adoption in the very scenarios they were designed to address. We identify two key limitations underlying the performance deficit of current generative recommendation approaches: 1) Existing methods mostly focus on the text modality for capturing semantics, while real-world data contains richer information spread across multiple modalities, and 2) the fixation on semantic codes neglects the synergy of collaborative and semantic signals. To address these challenges, we propose MSCGRec, a Multimodal Semantic and Collaborative Generative Recommender. MSCGRec incorporates multiple semantic modalities and introduces a novel self-supervised quantization learning approach for images based on the DINO framework. To fuse collaborative and semantic signals, MSCGRec also extracts collaborative features from sequential recommenders and treats them as a separate modality. Finally, we propose constrained sequence learning that restricts the large output space during training to the set of permissible tokens. We empirically demonstrate on three large real-world datasets that MSCGRec outperforms both sequential and generative recommendation baselines, and provide an extensive ablation study to validate the impact of each component.

Primary Area: generative models

Submission Number: 21371