Explanation Shift: How Did Distribution Shift Impact the Model?

Carlos Mougan, Klaus Broelemann, David Masip, Gjergji Kasneci, Thanassis Tiropanis, Steffen Staab

Keywords: Model Monitoring, Distribution Shift, Explainable AI

TL;DR: We find that the modeling of explanation shifts can be a better indicator for analyzing interactions between models and shifting data distributions than state-of-the-art techniques based on representations of distribution shifts

Abstract: The performance of machine learning models on new data is critical for their success in real-world applications. However, the model's performance may deteriorate if the new data is sampled from a different distribution than the training data. Current methods to detect shifts in the input or output data distributions have limitations in identifying model behavior changes. In this paper, we define \emph{explanation shift} as the statistical comparison between how predictions from training data are explained and how predictions on new data are explained. We propose explanation shift as a key indicator to investigate the interaction between distribution shifts and learned models. We introduce an Explanation Shift Detector that operates on the explanation distributions, providing more sensitive and explainable changes in interactions between distribution shifts and learned models. We compare explanation shifts with other methods based on distribution shifts, showing that monitoring for explanation shifts results in more sensitive indicators for varying model behavior. We provide theoretical and experimental evidence and demonstrate the effectiveness of our approach on synthetic and real data. Additionally, we release an open-source Python package, skshift, which implements our method and provides usage tutorials for further reproducibility.

Submission Number: 3170