A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning

Carl Allen, Ivana Balazevic, Timothy Hospedales

Keywords: semi-supervised learning, probabilistic model, neuro-symbolic learning

Abstract: Strong progress has been achieved in semi-supervised learning (SSL) by combining several methods, some of which relate to properties of the data distribution p(x), others to the model outputs p(y|x), e.g. minimising the entropy of unlabelled predictions. Focusing on the latter, we fill a gap in the standard text by introducing a probabilistic model for discriminative semi-supervised learning, mirroring the classical generative model. Several SSL methods are theoretically explained by our model as inducing (approximate) strong priors over parameters of p(y|x). Applying this same probabilistic model to tasks in which labels represent binary attributes, we theoretically justify a family of neuro-symbolic SSL approaches, taking a step towards bridging the divide between statistical learning and logical reasoning.

One-sentence Summary: A probabilistic model for discriminative and neuro-symbolic semi-supervised learning.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=U9PIbxt_Zp