Rotative Factorization Machines

Zhen Tian, Yuhong Shi, Xiangkun Wu, Wayne Xin Zhao, Ji-Rong Wen

Abstract: Feature interaction learning (FIL) focuses on capturing the complex relationships among multiple features for building predictive models, which is widely used in real-world tasks. Despite the research progress, existing FIL methods suffer from two major limitations. Firstly, they mainly model the feature interactions within a bounded order (e.g., small integer order) due to the exponential growth of the interaction terms. Secondly, the interaction order of each feature is often independently learned, which lacks the flexibility to capture the feature dependencies in varying contexts.To address these issues, we present Rotative Factorization Machines (RFM), based on the key idea that represents each feature as a polar angle in the complex plane. As such, the feature interactions are converted into a series of complex rotations, where the orders are cast into the rotation coefficients, thereby allowing for the learning of arbitrarily large order. Further, we propose a novel self-attentive rotation function that models the rotation coefficients through a rotation-based attention mechanism, which can adaptively learn the interaction orders under different interaction contexts. Moreover, it incorporates a modulus amplification network to learn the modulus of the complex features, which further enhances the expressive capacity. Our proposed approach provides a general FIL framework, and many existing models can be instantiated in this framework, e.g., factorization machines. In theory, it possesses more strong capacities to model complex feature relationships, and can learn arbitrary features from varied contexts. Extensive experiments conducted on five widely used datasets have demonstrated the effectiveness of our approach.