Understanding Parameter Saliency via Extreme Value Theory

Shuo Wang, Issei Sato

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: parameter saliency, extreme value theory, XAI

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We attempt to bridge the knowledge gap in terms of understanding why parameter saliency ranking can find the filters inducing misidentification from a statistical viewpoint.

Abstract:

Deep neural networks are being increasingly implemented throughout society in recent years. It is useful to identify which parameters trigger misclassification in diagnosing undesirable model behaviors.
The concept of parameter saliency is proposed and used to diagnose convolutional neural networks (CNNs) by ranking
convolution filters that may have caused misclassification on the basis of parameter saliency.
It is also shown that fine-tuning the top ranking salient filters efficiently corrects misidentification on ImageNet.
However, there is still a knowledge gap in terms of understanding why parameter saliency ranking can find the filters inducing misidentification.
In this work, we attempt to bridge the gap by analyzing parameter saliency ranking from a statistical viewpoint, namely, extreme value theory.
We first show that the existing work implicitly assumes that the gradient norm computed for each filter follows a normal distribution.
Then, we clarify the relationship between parameter saliency and the score based on the peaks-over-threshold (POT) method, which is often used to model extreme values.
Finally, we reformulate parameter saliency in terms of the POT method, where this reformulation is regarded as statistical anomaly detection and does not require the implicit assumptions of the existing formulation of parameter saliency.
Our experimental results demonstrate that our reformulation can detect malicious filters as well.
Furthermore, we show that the existing parameter saliency method exhibits a bias against the depth of layers in deep neural networks.
In particular, this bias has the potential to inhibit the discovery of filters that cause misidentification in situations where domain shift occurs.
In contrast, parameter saliency based on POT shows less of this bias.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5615