A Practical Framework for Evaluating the Quality of Knowledge Graph

Haihua Chen, Gaohui Cao, Jiangping Chen, Junhua Ding

Abstract: Knowledge graphs have become much large and complex during past several years due to its wide applications in knowledge discovery. Many knowledge graphs were built using automated construction tools and via crowdsourcing. The graph may contain significant amount of syntax and semantics errors that great impact its quality. A low quality knowledge graph produce low quality application that is built on it. Therefore, evaluating quality of knowledge graph is necessary for building high quality applications. Many frameworks were proposed for systematic evaluation of knowledge graphs, but they are either too complex to be practical or lacking of scalability to large scale knowledge graphs. In this paper, we conducted a comprehensive study of existing frameworks and proposed a practical framework for evaluating quality on “fit for purpose” of knowledge graphs. We first selected a set of quality dimensions and their corresponding metrics based on the requirements of knowledge discovery based on knowledge graphs through systematic investigation of representative published applications. Then we recommended an approach for evaluating each metric considering its feasibility and scalability. The framework can be used for checking the essential quality requirements of knowledge graphs for serving the purpose of knowledge discovery.