Regression Prior Networks

Andrey Malinin, Sergey Chervontsev, Ivan Provilkov, Mark Gales

Keywords: uncertainty, prior networks, regression, ensemble distribution distillation, depth estimation.

Abstract: Prior Networks are a recently developed class of models which yield interpretable measures of uncertainty and have been shown to outperform state-of-the-art ensemble approaches on a range of tasks. They can also be used to distill an ensemble of models via \emph{Ensemble Distribution Distillation} (EnD$^2$), such that its accuracy, calibration and uncertainty estimates are retained within a single model. However, Prior Networks have so far been developed only for classification tasks. This work extends Prior Networks and EnD$^2$ to regression tasks by considering the Normal-Wishart distribution. The properties of Regression Prior Networks are demonstrated on synthetic data, selected UCI datasets and a monocular depth estimation task, where they yield performance competitive with ensemble approaches.

One-sentence Summary: Development of Prior Networks and Ensemble Distribution Distillation for Regression Tasks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2006.11590/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=zttNivMlGv