Probabilistic Graphical Models and Tensor Networks: A Hybrid Framework

Jacob Miller, Geoffrey Roeder, Tai-Danae Bradley

Abstract:

We investigate a correspondence between two formalisms for discrete probabilistic
modeling: probabilistic graphical models (PGMs) and tensor networks (TNs), a
powerful modeling framework for simulating complex quantum systems. The
graphical calculus of PGMs and TNs exhibits many similarities, with discrete
undirected graphical models (UGMs) being a special case of TNs. However,
more general probabilistic TN models such as Born machines (BMs) employ
complex-valued hidden states to produce novel forms of correlation among the
probabilities. While representing a new modeling resource for capturing structure in
discrete probability distributions, this behavior also renders the direct application of
standard PGM tools impossible. We aim to bridge this gap by introducing a hybrid
PGM-TN formalism that integrates quantum-like correlations into PGM models
in a principled manner, using the physically-motivated concept of decoherence.
We first prove that applying decoherence to the entirety of a BM model converts it
into a discrete UGM, and conversely, that any subgraph of a discrete UGM can be
represented as a decohered BM. This method allows a broad family of probabilistic
TN models to be encoded as partially decohered BMs, a fact we leverage to
combine the representational strengths of both model families. We experimentally
verify the performance of such hybrid models in a sequential modeling task, and
identify promising uses of our method within the context of existing applications
of graphical models.