Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarks

Jiaqi Ma, Xingjian Zhang, Hezheng Fan, Jin Huang, Tianyue Li, Ting Wei Li, Yiwen Tu, Chenshu Zhu, Qiaozhu Mei

Abstract:

Establishing open and general benchmarks has been a critical driving force behind
the success of modern machine learning techniques. As machine learning is being applied to broader domains and tasks, there is a need to establish richer and
more diverse benchmarks to better reflect the reality of the application scenarios.
Graph learning is an emerging field of machine learning that urgently needs more
and better benchmarks. To accommodate the need, we introduce Graph Learning
Indexer (GLI), a benchmark curation platform for graph learning. In comparison
to existing graph learning benchmark libraries, GLI highlights two novel design
objectives. First, GLI is designed to incentivize dataset contributors. In particular, we incorporate various measures to minimize the effort of contributing and
maintaining a dataset, increase the usability of the contributed dataset, as well
as encourage attributions to different contributors of the dataset. Second, GLI is
designed to curate a knowledge base, instead of a plain collection, of benchmark
datasets. We use multiple sources of meta information to augment the benchmark
datasets with rich characteristics, so that they can be easily selected and used
in downstream research or development. The source code of GLI is available at
https://github.com/Graph-Learning-Benchmarks/gli.