Influence Estimation in Statistical Models Using the Fisher Information Matrix

TMLR Paper6174 Authors

Abstract: Quantifying how infinitesimal perturbations of training data affect a model is key to diagnosing and improving learning systems. We cast this task as a weighted empirical risk minimization problem and derive an influence estimator that refines classical influence estimation approaches. The formulation is broadly applicable, accommodates non-differentiable regularizers, and admits an efficient algorithm with favorable computational complexity. Simulations on realistic setups show that our estimator remains informative and reliable, while offering clear runtime advantages over existing techniques, and that it further works in settings with non-differentiable regularizers as encountered in many modern learning systems.

Submission Type: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=CkbO0xBEsl&nesting=2&sort=date-desc

Changes Since Last Submission: We set the font size and margins in accordance with the TMLR author guide: https://jmlr.org/tmlr/author-guide.html

Assigned Action Editor: ~Mohammad_Emtiyaz_Khan1

Submission Number: 6174