Neural Bayesian Network Understudy

Paloma Rabaey, Cedric De Boom, Thomas Demeester

Keywords: causal machine learning, bayesian networks, neural networks

TL;DR: We show how to train a neural network to act as an understudy to a Bayesian network, approximating both its probabilistic and causal properties.

Abstract: Bayesian Networks may be appealing for clinical decision-making due to their inclusion of causal knowledge, but their practical adoption remains limited as a result of their inability to deal with unstructured data. While neural networks do not have this limitation, they are not interpretable and are inherently unable to deal with causal structure in the input space. Our goal is to build neural networks that combine the advantages of both approaches. Motivated by the perspective to inject causal knowledge while training such neural networks, this work presents initial steps in that direction. We demonstrate how a neural network can be trained to output conditional probabilities, providing approximately the same functionality as a Bayesian Network. Additionally, we propose two training strategies that allow encoding the independence relations inferred from a given causal structure into the neural network. We present initial results in a proof-of-concept setting, showing that the neural model acts as an understudy to its Bayesian Network counterpart, approximating its probabilistic and causal properties.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2211.08243/code)