Ad-Rec: Advanced Feature Interactions to Address Covariate-Shifts in Recommendation Networks

NeurIPS 2023 Workshop MLSys Submission38 Authors

Keywords: recommendation systems, training, covariate shifts

TL;DR: We introduce Ad-Rec, an advanced network that leverages feature interaction techniques to tackle covariate-shifts by using masked transformers to learn higher-order cross-features.

Abstract: Recommendation models enhance user experiences by utilizing input feature correlations. However, deep learning-based models encounter challenges from changing user behavior and item features, leading to data distribution shifts. Effective cross-feature learning is crucial in addressing this. We introduce Ad-Rec, an advanced network that leverages feature interaction techniques to tackle these issues. It utilizes masked transformers to learn higher-order cross-features while mitigating data distribution drift. Our approach improves model quality, accelerates convergence, and reduces training time. We demonstrate scalability of Ad-Rec and its superior model quality through extensive ablation studies.

Submission Number: 38