Extracting Decision Paths via Surrogate Modeling for Explainability of Black Box Classifiers

Jaber Rad, Karthik K. Tennankore, Samina Abidi, Syed Sibte Raza Abidi

Abstract: A common challenge in using intricate machine learning (ML) classifiers in critical domains is the lack of transparency in making predictions despite exhibiting high performance. Besides, relying solely on a single ML model may introduce uncertainties due to each algorithm's distinct strengths and weaknesses in classification. In this study, we propose a novel method that collects explanations derived from multiple ML classifiers, and subsequently, through subset optimization, extracts a high-quality explanation represented as a set of rules in disjunctive normal form—referred to as decision paths. Quality check of the shortlisted explanation is done considering accuracy of the underlying ML model, fidelity, confidence, instance coverage, and interpretability. We applied our method to a large and complicated real-life dataset related to kidney transplants, addressing a binary classification problem. The experiments show that our method optimally balances the reliability and coverage of the explanation, while minimizing its complexity.