Benchmarking the Lifecycle of Knowledge Graphs

Michael Röder, Mohamed Ahmed Sherif, Muhammad Saleem, Felix Conrads, Axel-Cyrille Ngonga Ngomo

Abstract: The growing interest in making use of Knowledge Graphs for developing explainable artificial intelligence, there is an increasing need for a comparable and repeatable comparison of the performance of Knowledge Graph-based systems. History in computer science has shown that a main driver to scientific advances, and in fact a core element of the scientific method as a whole, is the provision of benchmarks to make progress measurable. Benchmarks have several purposes: (1) they highlight weak and strong points of systems, (2) they stimulate technical progress and (3) they make technology viable. Benchmarks are an essential part of the scientific method as they allow to track the advancements in an area over time and make competing systems comparable. This chapter gives an overview of benchmarks used to evaluate systems that process Knowledge Graphs.