Incorporating Unlabelled Data into Bayesian Neural Networks

Mrinank Sharma, Tom Rainforth, Yee Whye Teh, Vincent Fortuin

Authors that are also TMLR Expert Reviewers: ~Vincent_Fortuin1

Abstract: Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions. To overcome this limitation, we introduce Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn models with suitable prior predictive distributions. This is achieved by leveraging contrastive pretraining techniques and optimising a variational lower bound. We then show that the prior predictive distributions of self-supervised BNNs capture problem semantics better than conventional BNN priors. In turn, our approach offers improved predictive performance over conventional BNNs, especially in low-budget regimes.

Certifications: Expert Certification

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Manuel_Haussmann1

Submission Number: 2167