From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models

Mingjia Yin, Junwei Pan, Hao Wang, Ximei Wang, Shangyu Zhang, Jie Jiang, Defu Lian, Enhong Chen

Abstract:

Click-Through Rate (CTR) prediction, a core task in recommendation systems, aims to estimate the probability of users clicking on items.
Existing models predominantly follow a discriminative paradigm, which relies heavily on explicit interactions between raw ID embeddings.
However, this paradigm inherently renders them susceptible to two critical issues: embedding dimensional collapse and information redundancy, stemming from the over-reliance on feature interactions *over raw ID embeddings*.
To address these limitations, we propose a novel *Supervised Feature Generation (SFG)* framework, *shifting the paradigm from discriminative "feature interaction" to generative "feature generation"*.
Specifically, SFG comprises two key components: an *Encoder* that constructs hidden embeddings for each feature, and a *Decoder* tasked with regenerating the feature embeddings of all features from these hidden representations.
Unlike existing generative approaches that adopt self-supervised losses, we introduce a supervised loss to utilize the supervised signal, \ie, click or not, in the CTR prediction task.
This framework exhibits strong generalizability: it can be seamlessly integrated with most existing CTR models, reformulating them under the generative paradigm.
Extensive experiments demonstrate that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains across various datasets and base models.
The code is available at https://github.com/USTC-StarTeam/GE4Rec.

Lay Summary:

CTR prediction models predict the probability of a user clicking on an item based on the direct interaction of a set of features. However, we have identified that this direct interaction results in a limitation of the information abundance of the learned features. We hypothesized that this issue could be addressed by avoiding direct interactions.

Instead of interacting those features, we propose to generate them using a new paradigm. Our paper presents results that demonstrate this simple yet effective new paradigm can significantly enhance the information abundance of the learned features, as it can bypass direct feature interactions through a feature generation process.

Our findings have revealed the inherent limitations of traditional CTR models. However, these models can be easily reformulated using our feature generation paradigm to increase the information abundance of learned features.

Link To Code: https://github.com/USTC-StarTeam/GE4Rec

Primary Area: Applications

Keywords: Generative model, Recommender system, Feature interaction

Submission Number: 1748