Uncovering Unique Concept Vectors through Latent Space Decomposition

Mara Graziani, Laura O'Mahony, An-phi Nguyen, Henning Müller, Vincent Andrearczyk

Abstract: Interpreting the inner workings of deep learning models is crucial for establishing trust and ensuring model safety. Concept-based explanations have emerged as a superior approach that is more interpretable than feature attribution estimates such as pixel saliency. However, defining the concepts for the interpretability analysis biases the explanations by the user’s expectations on the concepts. To address this, we propose a novel post-hoc unsupervised method that automatically uncovers the concepts learned by deep models during training. By decomposing the latent space of a layer in singular vectors and refining them by unsupervised clustering, we uncover concept vectors aligned with directions of high variance that are relevant to the model prediction, and that point to semantically distinct concepts. Our extensive experiments reveal that the majority of our concepts are readily understandable to humans, exhibit coherency, and bear relevance to the task at hand. Moreover, we showcase the practical utility of our method in dataset exploration, where our concept vectors successfully identify outlier training samples affected by various confounding factors. This novel exploration technique has remarkable versatility to data types and model architectures and it will facilitate the identification of biases and the discovery of sources of error within training data.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

We thank the AE for the comments and support so far!

We have now uploaded the camera ready with a few minor changes that address the AE comments. In particular, we have extended the related works section to include references that help positioning the paper in the context of existing literature and recent works on the topic of concept discovery.

Bests,
The authors

Code: https://github.com/maragraziani/concept_discovery_svd

Assigned Action Editor: ~Nicolas_THOME2

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1368