Linearized Laplace Inference in Neural Additive Models

Kouroche Bouchiat, Alexander Immer, Hugo Yèche, Vincent Fortuin

Keywords: Linearized, Laplace, Neural Additive Model, Uncertainty, Marginal Likelihood, Feature Selection, Automatic Relevance Determination

TL;DR: We improve the interpretability of Neural Additive Models (NAMs) with uncertainty estimation and automatic feature selection by virtue of Laplace approximations.

Abstract: Deep neural networks are highly effective but suffer from a lack of interpretability due to their black-box nature. Neural additive models (NAMs) solve this by separating into additive sub-networks, revealing the interactions between features and predictions. In this paper, we approach the NAM from a Bayesian perspective in order to quantify the uncertainty in the recovered interactions. Linearized Laplace approximation enables inference of these interactions directly in function space and yields a tractable estimate of the marginal likelihood, which can be used to perform implicit feature selection through an empirical Bayes procedure. Empirically, we show that Laplace-approximated NAMs (LA-NAM) are both more robust to noise and easier to interpret than their non-Bayesian counterpart for tabular regression and classification tasks.