ConvAOA: A Convolutional Attention Over Attention Model for Click-Through Rate Prediction

Bin Yang, Tianmu Sha, Ying Xing, Ziyang Wang, Zhipu Xie, Xin Wang

Abstract: Click-Through Rate (CTR) prediction is a crucial task in online advertising and recommender systems. To achieve better prediction results, it is important to accurately model feature interactions. Through the embedding layer, the relationships and interactions between the original features are reflected between the embedding vectors. Additionally, for specific features, the different dimensions within the embedding vectors also represent different perspectives and angles, which should be assigned varying importance weights. However, existing research on CTR prediction tasks using convolution operations faces significant challenges. Firstly, the comprehensive exploration of attention mechanisms within and between embedding vectors is often overlooked, resulting in a limited understanding of feature importance. Additionally, the neglect of global feature interactions hinders the practical application of CTR prediction. In this paper, we propose Convolutional Attention Over Attention (ConvAOA), a novel model that fully and efficiently leverages convolution operations and attention mechanisms for CTR prediction. ConvAOA sequentially processes features using convolutional intra-attention and convolutional inter-attention mechanisms, enabling dynamic attention to feature importance and preservation of meaningful information within and across different features. Extensive experiments are conducted on four real-world datasets, demonstrating the significant performance of ConvAOA in CTR prediction scenarios. The results show that ConvAOA outperforms current mainstream models and is an efficient and advanced solution in the field of CTR prediction.