Taxonomy of Benchmarks in Graph Representation LearningDownload PDF

Published: 24 Nov 2022, Last Modified: 05 May 2023LoG 2022 OralReaders: Everyone
Keywords: graph datasets, dataset taxonomy, graph representation learning, graph classification, node classification
TL;DR: We provide a systematic approach for categorization of graph learning datasets based on their empirical properties.
Abstract: Graph Neural Networks (GNNs) extend the success of neural networks to graph-structured data by accounting for their intrinsic geometry. While extensive research has been done on developing GNN models with superior performance according to a collection of graph representation learning benchmarks, it is currently not well understood what aspects of a given model are probed by them. For example, to what extent do they test the ability of a model to leverage graph structure vs. node features? Here, we develop a principled approach to taxonomize benchmarking datasets according to a $\textit{sensitivity profile}$ that is based on how much GNN performance changes due to a collection of graph perturbations. Our data-driven analysis provides a deeper understanding of which benchmarking data characteristics are leveraged by GNNs. Consequently, our taxonomy can aid in selection and development of adequate graph benchmarks, and better informed evaluation of future GNN methods. Finally, our approach is designed to be extendable to multiple graph prediction task types and future datasets.
PDF File: pdf
Type Of Submission: Full paper proceedings track submission (max 9 main pages).
Agreement: Check this if you are okay with being contacted to participate in an anonymous survey.
Type Of Submission: Full paper proceedings track submission.
Poster: png
Poster Preview: png
6 Replies