Enhancing Embeddings Quality with Stacked Gate for Click-Through Rate Prediction

Caihong Mu, Yunfei Fang, Jialiang Zhou, Yi Liu

Abstract: Click-Through Rate (CTR) prediction is a critical task in commercial recommender systems. Feature embedding is the cornerstone of CTR models. Most previous work on CTR prediction to improve the model performance focused on modelling feature interactions. However, the quality of embeddings and the effective information of embeddings carried are overlooked. As a result, performing feature interactions on inferior embeddings will be a bottleneck to capture informative knowledge. To address this problem, we optimize embeddings from the perspective of expanding the weight gap and masking noisy information of the embedding vector. We first propose a novel module named Simple Stacked Gate (SSG), which selects the salient information of feature embeddings and enhances the weight gap between it and other information. Based on this, we further propose an Automatic Stacked Gate (AutoSG) framework, which utilizes Neural Architecture Search (NAS) method to automatically identify and mask noisy information of the embedding vector obtained after the SSG module. Both SSG and AutoSG can be seen as plug-and-play components and added to any existing deep CTR models flexibly. Extensive experiments on four benchmark datasets demonstrate that these two methods can further improve the performance of the backbone CTR models. Even only half of the bit information retained in the optimized embeddings, better performance can still be achieved compared to the original model. The implementation of our work can be found at https://github.com/SG-CTR/Stacked-Gate.