Over 100x Speedup in Relational Deep Learning via Static GNNs and Tabular Distillation

ICLR 2025 Conference Submission12247 Authors

Keywords: Relational Databases, Relational Deep Learning, GNNs, Training Acceleration, Real-time Inference, Distillation

Abstract: Relational databases, organized into tables connected by primary-foreign key relationships, are widely used in industry. Companies leverage this data to build highly accurate, feature-engineered tabular models—often using boosted decision trees—to predict key metrics such as customer transactions and product revenues. However, these models need frequent retraining as new data is introduced, which is both expensive and time-consuming. Despite this, by being the result of extensive engineering effort, they remain difficult to outperform using generalist methods, like Temporal Graph Neural Networks (TGNNs) trained over the same relational data. Rather than attempting to replace tabular models with generalist approaches, we propose to combine the strengths of tabular models and static Graph Neural Networks (GNNs). GNNs offer better speed and scalability than TGNNs, and, as we argue, the primary strength of graph representation learning for these tasks does not lie in modeling temporal dynamics—something highly- engineered tabular models excel at—but in capturing complex relationships within the database, which are hard to featurize. Our approach integrates all predictive embeddings of all tabular models developed for various tasks into a single static GNN framework. Experimental results on the RelBench benchmark show that our approach achieves a performance improvement of up to 33% and an inference speedup of up to 1050x, making it highly suitable for real-time inference.

Primary Area: learning on graphs and other geometries & topologies

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12247