Implicit Neural Representation Inference for Low-Dimensional Bayesian Deep Learning

Panagiotis Dimitrakopoulos, Giorgos Sfikas, Christophoros Nikou

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Bayesian Deep Learning, Implicit neural representations, Probabilistic machine learning, Hypernetworks

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Bayesian inference is the standard for providing full predictive distributions with well calibrated uncertainty estimates.
However, scaling to a modern, overparameterized deep learning setting
typically comes at the cost of severe and restrictive approximations, sacrificing model predictive strength.
With our approach, we factor model parameters as a function of deterministic and probabilistic components;
the model is solved by combining maximum a posteriori estimation of the former,
with inference over a low-dimensional, Implicit Neural Representation of the latter.
This results in a solution that combines both predictive accuracy and good calibration,
as it entails inducing stochasticity over the full set of model weights while being comparatively cheap to compute.
Experimentally, our approach compares favorably to the state of the art,
including much more expensive methods as well as less expressive posterior approximations over full network parameters.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 2362