CRAFT: explaining using Concepts from Recursive Activation FacTorization

Thomas FEL, Agustin Martin Picard, Louis Béthune, Thibaut Boissin, Julien Colin, David Vigouroux, Remi Cadene, Thomas Serre

Keywords: Explainability, Concept, Matrix Factorization, Implicit Differentiation, Attribution Methods, Sensitivity Analysis

TL;DR: Revisiting ACE to automatically discover Concepts for a classifier. Introduce Sobol total indice for Concept Importance and Concept Attribution Maps.

Abstract: Despite their considerable potential, concept-based explainability methods have received relatively little attention, and explaining what’s driving models’ decisions and where it’s located in the input is still an open problem. To tackle this, we revisit unsupervised concept extraction techniques for explaining the decisions of deep neural networks and present CRAFT – a framework to generate concept-based explanations for understanding individual predictions and the model’s high-level logic for whole classes. CRAFT takes advantage of a novel method for recursively decomposing higher-level concepts into more elementary ones, combined with a novel approach for better estimating the importance of identified concepts with Sobol indices. Furthermore, we show how implicit differentiation can be used to generate concept-wise attribution explanations for individual images. We further demonstrate through fidelity metrics that our proposed concept importance estimation technique is more faithful to the model than previous methods, and, through human psychophysic experiments, we confirm that our recursive decomposition can generate meaningful and accurate concepts. Finally, we illustrate CRAFT’s potential to enable the understanding of predictions of trained models on multiple use-cases by producing meaningful concept-based explanations.