PRIME: Prioritizing Interpretability in Failure Mode Extraction

Keivan Rezaei, Mehrdad Saberi, Mazda Moayeri, Soheil Feizi

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: interpretability, failure modes, bias

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a new method to detect and explain failure mode of trained models in human-understandable terms.

Abstract:

In this work, we study the challenge of providing human-understandable descriptions for failure modes in trained image classification models.
Existing works address this problem by first identifying clusters (or directions) of incorrectly classified samples in a latent space and then aiming to provide human-understandable text descriptions for them.
We observe that in some cases, describing text does not match well
with identified failure modes, partially owing to the fact that shared interpretable attributes of failure modes may not be captured using clustering in the feature space.
To improve on these shortcomings, we propose a novel approach that prioritizes interpretability in this problem: we start by obtaining human-understandable concepts (tags) of images in the dataset and
then analyze the model's behavior based on the presence or absence of combinations of these tags.
Our method also ensures that the tags describing a failure mode form a minimal set,
avoiding redundant and noisy descriptions.
Through several experiments on different datasets, we show that our method successfully identifies failure modes and generates high-quality text descriptions associated with them.
These results highlight the importance of prioritizing interpretability in understanding model failures.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Primary Area: visualization or interpretation of learned representations

Submission Number: 2058