Pairwise explanations: towards a new task-agnostic paradigm in explainable artificial intelligence

ICLR 2026 Conference Submission20150 Authors

Keywords: Explainable AI, Pairwise attributions, Model agnostic methods

Abstract: In this study, we introduce Pairwise-IBISA (P-IBISA), a novel extension of the Information Bottleneck with Input Sampling for Attribution (IBISA). Unlike traditional approaches, P-IBISA generates explanations directly from encoder representations, eliminating the need for task-specific logits. This design enables interpretability across a wide range of applications, including image retrieval and vision–language grounding, and is compatible with models trained for classification as well as those pre-trained using self-supervised learning strategies. P-IBISA operates by computing a mask over the input image using a pairwise loss that aligns the embeddings of the masked image with a target embedding. This target can be derived from another image, the image itself, or a different modality—such as text in models like CLIP. We conducted a quantitative evaluation of P-IBISA on models designed for three distinct tasks: image classification, vision–language grounding, and image retrieval. Across these tasks, P-IBISA consistently demonstrated superior or competitive performance compared to state-of-the-art methods, despite being task- and model-agnostic. Qualitative analysis further reveals that P-IBISA produces sharper and semantically richer saliency maps, effectively highlighting meaningful features in both CNNs and ViTs pre-trained on unlabeled data. By decoupling explanations from final outputs, P-IBISA advances the field of xAI beyond task-specific evaluation, offering a unified framework for attribution across diverse scenarios.

Primary Area: interpretability and explainable AI

Submission Number: 20150