DSEG-LIME: Improving Image Explanation by Hierarchical Data-Driven Segmentation

ICLR 2025 Conference Submission273 Authors

Keywords: XAI, LIME, Segmentation

TL;DR: The integration of semantic-segmentation foundation models for improved image classification within LIME through user-steered hierarchical feature calculation.

Abstract: Explainable Artificial Intelligence (XAI) is crucial in unraveling decision-making processes in complex machine learning models. LIME (Local Interpretable Model-agnostic Explanations) is a well-known XAI framework for image analysis. It utilizes image segmentation to create features to identify relevant areas for classification. Consequently, poor segmentation can compromise the consistency of the explanation and undermine the importance of the segments, affecting the overall interpretability. Addressing these challenges, we introduce DSEG-LIME (Data-Driven Segmentation LIME), featuring: i) a data-driven segmentation for human-recognized feature generation by foundation model integration, and ii) a user steered granularity in the hierarchical segmentation procedure through composition. We evaluate DSEG-LIME on pre-trained models using ImageNet classes, explicitly targeting scenarios without domain-specific knowledge. Our findings demonstrate that DSEG outperforms most of the XAI metrics and enhances the alignment of explanations with human-recognized concepts, significantly improving interpretability.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 273