Detecting Extrapolation with Local Ensembles

David Madras, James Atwood, Alexander D'Amour

Keywords: extrapolation, reliability, influence functions, laplace approximation, ensembles, Rashomon set

TL;DR: We present local ensembles, a method for detecting extrapolation in trained models, which approximates the variance of an ensemble using local-second order information.

Abstract: We present local ensembles, a method for detecting extrapolation at test time in a pre-trained model. We focus on underdetermination as a key component of extrapolation: we aim to detect when many possible predictions are consistent with the training data and model class. Our method uses local second-order information to approximate the variance of predictions across an ensemble of models from the same class. We compute this approximation by estimating the norm of the component of a test point's gradient that aligns with the low-curvature directions of the Hessian, and provide a tractable method for estimating this quantity. Experimentally, we show that our method is capable of detecting when a pre-trained model is extrapolating on test data, with applications to out-of-distribution detection, detecting spurious correlates, and active learning.

Code: https://github.com/dmadras/local-ensembles

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1910.09573/code)