Keywords: bayesian deep learning, variational methods, bayesian last layers
TL;DR: We provide a simple, cheap, and deterministic Bayesian training procedure for neural network last layers
Abstract: We introduce a deterministic variational formulation for training Bayesian last layer neural networks. This yields a sampling-free, single-forward pass objective that effectively improves network uncertainty representation. Our variational Bayesian last layer can be trained and evaluated inexpensively, with only quadratic complexity in last layer width, and is thus (nearly) computationally free to add to existing architectures.