Intriguing Properties of Visual-Language Model Explanations

Chirag Agarwal

Keywords: Explainability, TrustworthyML, Zero-shot, Fine-tune, Visual-Language Model

TL;DR: An empirical study to establish the trustworthy properties of explanations generated for visual-language models used in zero-shot vs. fine-tune settings.

Abstract: The growing popularity of large-scale visual-language models (VLMs) has led to their employment in various downstream applications as they provide a rich source of image and text representations. However, these representations are highly entangled and complex to interpret by machine learning developers and practitioners. Recent works have shown visualizations of image regions that VLMs focus on but fail to describe the change in explanations generated for visual-language classifiers in zero-shot (using image and text representations) vs. fine-tuned settings (using image representations). In this work, we perform the first empirical study to establish the trustworthy properties of explanations generated for VLMs used in zero-shot vs. fine-tune settings. We show that explanations for zero-shot visual-language classifiers are more faithful than their fine-tuned counterpart. Further, we demonstrate that VLMs tend to attribute high importance to gender, despite being non-indicative of the downstream task. Our experiments on multiple real-world datasets show interesting VLM behavior in zero-shot vs. fine-tuned settings, opening up new frontiers in understanding the trustworthiness of large-scale visual-language models.