Provenance-based Explanations for Machine Learning (ML) Models

Justin Turnau, Nkechi Akwari, Seokki Lee, Dwarkesh Rajput

Abstract: Understanding machine learning (ML) models is extremely important since it improves trust and transparency and debugging the misbehavior of the models. This paper proposes a technique for computing well-known ML explanations from the literature using provenance. Observing that widely used ML models can be expressed into a set of Datalog queries, we develop a technique that rewrites the query to capture provenance and computes explanations based on that while training the model. The preliminary evaluation shows the reasonable computational cost of our algorithm.