SUBPLEX: A Visual Analytics Approach to Understand Local Model Explanations at the Subpopulation Level

Jun Yuan, Gromit Yeuk-Yin Chan, Brian Barr, Kyle Overton, Kim Rees, Luis Gustavo Nonato, Enrico Bertini, Cláudio T. Silva

Abstract: Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts, such as transport control, financial activities, and medical diagnosis. While local explanation techniques are popular methods to interpret ML models on a single instance, they do not scale to the understanding of a model’s behavior on the whole dataset. In this article, we outline the challenges and needs of visually analyzing local explanations and propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SUBPLEX</i> , a visual analytics approach to help users understand local explanations with subpopulation visual analysis. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SUBPLEX</i> provides steerable clustering and projection visualization techniques that allow users to derive interpretable subpopulations of local explanations with users’ expertise. We evaluate our approach through two use cases and experts’ feedback.