Hierarchical Probabilistic Neural Network: Efficient and Accurate Uncertainty Quantification
Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Uncertainty Quantification, Evidential Deep Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Bayesian neural networks (BNNs) are known for accurately estimating the posterior distribution of model parameters, showcasing their effectiveness in uncertainty quantification (UQ). However, the computational demands of Bayesian inference can be challenging. Evidential deep learning methods address this by treating target distribution parameters as random variables with a learnable conjugate distribution, thus allowing for efficient UQ. In our paper, we present the Hierarchical Probabilistic Neural Network (HPNN), offering new insights into existing evidential deep learning methods. Firstly, it distills BNN knowledge into a single deterministic network, endowing it with a Bayesian perspective and theoretical guarantees. Secondly, we introduce a self-regularized training strategy using Laplacian approximation (LA) for self-distillation, bypassing the heavy computational load with BNNs. Thirdly, we propose to utilize flexible normalizing flows to alleviate the conjugate prior assumption in a post-processing manner, where a few training iterations can enhance model performance. Lastly, we present the Hierarchical Bayesian Neural Network, which treats the NN parameters in HPNN as random variables, for further improving UQ accuracy. The experiment results demonstrate the effectiveness of our proposed methods in both UQ accuracy and robustness.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6353
Loading