GPT as Visual Explainer

Xingyi Yang, Xinchao Wang

Keywords: Explainable AI, Multi-modal Learning, Foundation Models

TL;DR: we present Language Model as Visual Explainer (LVX) for interpreting the vision-only models with large language model.

Abstract:

In this paper, we present Language Model as Visual Explainer (\texttt{LVX}), a systematic approach for interpreting the internal workings of vision models using a tree-structured linguistic explanation, without the need for model training. Central to our strategy is the collaboration between vision models and LLM to craft explanation. On one hand, the LLM is harnessed to delineate hierarchical visual attributes, while concurrently, a text-to-image API retrieves images that are most align with these textual concepts. By mapping the collected text and image to the vision model's embedding space, we construct a hierarchy-structured visual embedding tree. This tree is dynamically pruned and grown by querying the LLM using language templates, tailoring the explanation to the model. Such a scheme allows us to seamlessly incorporate new attributes while eliminating undesired concepts based on the model's representations. When applied to testing samples,
our method provides human-understandable explanations in the form of attribute-laden trees. Beyond explanation, we retrained the vision model by calibrating the model on the generated concept hierarchy, {allowing the model to incorporate the refined knowledge of visual attributes}. To access the effectiveness of our approach, we introduce new benchmarks and conduct rigorous evaluations. The results unequivocally demonstrate the plausibility, faithfulness, and stability of our approach compared to existing interpretability techniques.

Primary Area: visualization or interpretation of learned representations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5388