Go to TMLR homepageUsing the Path of Least Resistance to Explain Deep Networks
Abstract: Integrated Gradients (IG), a widely used axiomatic path-based attribution method, assigns importance scores to input features by integrating model gradients along a straight path from a baseline to the input. While effective in some cases, we show that straight paths can lead to flawed attributions. In this paper, we identify the cause of these misattributions and propose an alternative approach that equips the input space with a model-induced Riemannian metric (derived from the explained model's Jacobian) and computes attributions by integrating gradients along geodesics under this metric. We call this method Geodesic Integrated Gradients (GIG).
To approximate geodesic paths, we introduce two techniques: a k-Nearest Neighbours-based approach for smaller models and a Stochastic Variational Inference-based method for larger ones. Additionally, we propose a new axiom, No-Cancellation Completeness (NCC), which strengthens completeness by ruling out feature-wise cancellation. We prove that, for path-based attributions under the model-induced metric, NCC holds if and only if the integration path is a geodesic.
Through experiments on both synthetic and real-world image classification data, we provide empirical evidence supporting our theoretical analysis and showing that GIG produces more faithful attributions than existing methods, including IG, on the benchmarks considered.
Submission Type: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=M6cL4nWOqK¬eId=L13y21AOZL
Changes Since Last Submission: We are submitting a revised version of our paper, which was previously considered by TMLR. We are grateful for the detailed feedback from Reviewers ayqH, NFF4, and 6v6K, and for the public comment by Zaher et al. regarding Manifold Integrated Gradients (MIG). We have substantially revised the manuscript in response to all of these.
We note that an earlier version of this work was submitted to ICML 2023 (https://openreview.net/forum?id=c8iDBe2YhX), before MIG was publicly available, and our core ideas were developed independently. We sincerely apologise for not citing MIG in our previous submission and have now included a thorough discussion of the relationship between the two approaches.
Summary of contributions and revisions:
This paper makes a primarily theoretical contribution: we identify a natural Riemannian structure on the input space induced by the explained model's Jacobian, introduce the No-Cancellation Completeness (NCC) axiom, and prove that under this metric, NCC holds if and only if the integration path is a geodesic. This provides the first formal characterisation of why geodesic paths are uniquely appropriate for path-based attribution. We support this theory with two proof-of-concept geodesic solvers and experiments on synthetic and image classification benchmarks.
The major revisions are as follows:
Relationship to MIG (Zaher et al., 2024). We now cite and discuss MIG in detail, acknowledging it as the closest related work and recognising MIG's convincing demonstration that geodesic paths yield perceptually cleaner and more robust feature visualisations. We present the two methods as complementary perspectives on how Riemannian geometry can improve path-based attributions. MIG derives its metric from a VAE decoder, so that geodesics respect the curved geometry of the data manifold; our metric comes from the explained model itself, so that geodesics avoid high-gradient regions. MIG's data-geometric perspective offers the appealing property that interpolants remain on the data manifold, producing realistic intermediate images and providing robustness to adversarial attacks. Our model-geometric perspective instead tailors the path to the specific classifier being explained. Our work additionally provides a formal characterisation (Theorem 1) connecting geodesic paths to the NCC axiom. We position the two as complementary and suggest their combination as future work.
Axiom renamed and justified. 'Strong Completeness' is now 'No-Cancellation Completeness (NCC)' to avoid implying it logically subsumes standard completeness. We added discussion of its motivation and relationship to standard completeness.
Paper restructured. A new Background section consolidates definitions and metrics. The introduction was rewritten for clarity. Forward references were reduced, ambiguous terms clarified, and metric conventions (AUC-Purity, AOC-Log-odds) explained explicitly to resolve confusion identified by Reviewer NFF4.
Computational cost and scalability. Per-method runtimes are now reported in Table 2. We discuss four concrete directions for efficiency improvements and frame the current solvers as proof-of-concept implementations whose cost will decrease as geodesic solvers improve. Importantly, our theoretical framework is solver-independent.
Hyperparameter guidance. Practical guidance for selecting beta and SVI learning rate is now provided, including initialisation heuristics and stable ranges.
Theoretical presentation strengthened. New remarks address metric degeneracy (rank-1 for scalar outputs), the relationship between the energy heuristic and true geodesics, kNN approximation quality, and regularity conditions for Theorem 1.
Limitations and scope. A new Limitations subsection and a discussion of attribution under model bias are included.
Extended related work. We now discuss SHAP/LIME, Blur IG, information geometry connections, and the Isomap analogy.
On experimental scope: Our experiments are designed to validate the theoretical claims rather than to serve as a comprehensive benchmark study. The synthetic half-moons experiment provides a controlled setting where ground-truth behaviour is known and our theoretical predictions can be directly verified. The Pascal VOC experiment demonstrates that the framework transfers to real-world deep learning, with effect sizes that are substantial (29% relative improvement in comprehensiveness, 15% in log-odds over the strongest baseline) and consistent across masking thresholds and qualitative examples. The computational cost of the current SVI solver (approximately 14 minutes per image) is the primary bottleneck for larger-scale evaluation; as we discuss, several promising directions exist for reducing this cost, and the theoretical contribution, which is the paper's central focus, is independent of the solver. We look forward to the consideration of our revised manuscript.
Sincerely, Authors
Assigned Action Editor: ~Pierre-Alexandre_Mattei3
Submission Number: 7664
Loading