A Dual Adaptive Factorization Network for CTR Prediction

Xianzhuang Li, Zhen Wang, Xuesong Wu, Bo Yuan, Xueqian Wang

Abstract: Factorization-based methods, which can model cross features automatically and generalize to unseen features, have been the benchmark models for the click-through rate (CTR) prediction. In general, they enumerate all cross features with a pre-determined order and then filter out useless interactions through model training. However, two major challenges remain. First, they need to make a trade-off between the computational cost and the expression ability of high-order interactions; Second, enumerating all feature interactions may introduce harmful noise. Inspired by the success of Adaptive Factorization Network (AFN), which aims to learn arbitrary-order feature interactions adaptively, we propose a novel model called Dual Adaptive Factorization Network (DAFN) with the following benefits. First of all, DAFN can learn arbitrary-order cross features at the element-wise and vector-wise levels simultaneously. More importantly, it leverages the information about the original embedding vectors thoroughly. Furthermore, it strategically integrates multiple components, including Talking-heads Attention, Logarithmic Transformation Layer and Residual Networks, into a unified end-to-end model. Extensive experiments on four real-world datasets demonstrate that our DAFN model outperforms several start-of-the-art methods.