eXIAA: eXplainable Injections for Adversarial Attack

ICLR 2026 Conference Submission17211 Authors

Keywords: Explainable Artificial Intelligence, Adversarial attack

TL;DR: We introduce a novel black-box model-agnostic one-step adversarial attack on image explanations generated by XAI methods, by leveraging the method itself, minimally altering the original classification prediction, and making the attack hard to detect

Abstract:

Post-hoc explainability methods are a subset of Machine Learning (ML) that aim to provide a reason for why a model behaves in a certain way. In this paper, we show a new black-box model-agnostic adversarial attack for post-hoc explainable Artificial Intelligence (XAI), particularly in the image domain. The goal of the attack is to modify the original explanations while being undetected by the human eye and maintain the same predicted class. In contrast to previous methods, we do not require any access to the model or its weights, but only to the model's computed predictions and explanations. Additionally, the attack is accomplished in a single step while significantly changing the provided explanations, as demonstrated by empirical evaluation. The low requirements of our method expose a critical vulnerability in current explainability methods, raising concerns about their reliability in safety-critical applications. We systematically generate attacks based on the explanations generated by post-hoc explainability methods (saliency maps, integrated gradients, and DeepLIFT SHAP) for pretrained ResNet-18 and ViT-B16 on ImageNet.
The results show that our attacks could lead to dramatically different explanations without changing the predictive probabilities.
We validate the effectiveness of our attack, compute the induced change based on the explanation with mean absolute difference, and verify the closeness of the original image and the corrupted one with the Structural Similarity Index Measure (SSIM).

Primary Area: interpretability and explainable AI

Submission Number: 17211