Continuous generation of versioned collection's members with RML and LDES

Dylan Van Assche, Sitt Min Oo, Julian Rojas, Pieter Colpaert

Keywords: RML, FnO, LDES, update, immutable, knowledge graph

TL;DR: Incrementally generating knowledge graphs to incorporate data source updates.

Abstract: When evolving datasets are used to generate a knowledge graph, it is usually challenging to keep the graph synchronized in a timely manner when changes occur in the source data. Current approaches fully regenerate a knowledge graph in such cases, which may be time consuming depending on the data type, size, and update frequency. We propose a continuous knowledge graph generation approach that can be applied on different types of data sources. We describe continuously updating knowledge graph versions represented as a Linked Data Events Stream, and use an RML processor for RDF generation. In this paper, we present our approach and demonstrate it on different types of data such as bike-sharing, public transport timetables, and weather data. By describing entities with unique, immutable, and reproducible iris, we were able to identify changes in the original data collection, reducing the number of materialized triples and generation time. Our use-cases show the importance of mechanisms to derive unique and stable IRI strategies of data source updates, to enable efficient knowledge graph generation pipelines. In the future, we will extend our approach to handle deletions in data collections, and conduct an extensive performance evaluation.