Faithfulness of Local Explanations for Tree-Based Ensemble Models

Amir Hossein Akhavan Rahnama, Pierre Geurts, Henrik Boström

Abstract: Local explanation techniques provide insights into the predicted outputs of machine learning models for individual data instances. These techniques can be model-agnostic, treating the machine learning model as a black box, or model-based, leveraging access to the model’s internal properties or logic. Evaluating these techniques is crucial for ensuring the transparency of complex machine-learning models in real-world applications. However, most evaluation studies have focused on the faithfulness of these techniques in explaining neural networks. Our study empirically evaluates the faithfulness of local explanations in explaining tree-based ensemble models. In our study, we have included local model-agnostic explanations of LIME, KernelSHAP, and LPI, along with local model-based explanations of TreeSHAP, Sabaas, and Local MDI for gradient-boosted trees and random forests models trained on 20 tabular datasets. We evaluate local explanations using two perturbation-based measures: Importance by Preservation and Importance by Deletion. We show that model-agnostic explanations of KernelSHAP and LPI consistently outperform model-based explanations from TreeSHAP, Saabas, and Local MDI when gradient-boosted tree and random forest models. Moreover, LIME explanations of gradient-boosted tree and random forest models consistently demonstrate low faithfulness across all datasets.