RDBench: ML Benchmark for Relational Databases
Keywords: Relational Databases, Graph Representation Learning, Machine Learning Benchmark
Abstract: Benefiting from high-quality datasets and standardized evaluation metrics, machine learning (ML) has achieved sustained progress and widespread applications. However, while applying machine learning to relational databases, the absence of a well-established benchmark remains a significant obstacle to the development of ML. To address this issue, we introduce \textit{ML Benchmark For Relational Databases} (RDBench), a benchmark that aims to promote hierarchical, robust, and reproducible ML research on relational databases. RDBench offers hierarchical datasets of varying scales, domains, and relations. It provides three types of data: tabular data, homogeneous graphs, and heterogeneous graphs. Importantly, all data formats share the same task definition, allowing for meaningful comparisons between methods across different data formats. Reported results are averaged over the same datasets and tasks (classification or regression), further enhancing the robustness of the experimental findings. In addition to dataset construction, we conduct extensive experiments to uncover performance differences between models. To better present our proposed RDBench, we offer a user-friendly API that provides standardized formats for three types of data.
Primary Area: datasets and benchmarks
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 21
Loading