Efficient Learning of Probabilistic Tensor Networks

Marawan Gamal, Guillaume Rabusseau

Keywords: tensor networks, matrix product states, generative modelling

TL;DR: We propose a numerically stable method for learning probabilistic tensor networks

Abstract: Tensor networks (TNs) enable compact representations of large tensors through shared parameters. Their use in probabilistic modeling is particularly appealing, as probabilistic tensor networks (PTNs) allow for tractable computation of marginals. However, existing approaches for learning parameters of PTNs are either computationally demanding and not fully compatible with automatic differentiation frameworks, or numerically unstable. In this work, we demonstrate a conceptually simple approach for learning PTNs efficiently, that is numerically stable. We stabilize the computation of the negative log-likelihood computation by iteratively rescaling intermediate computations using logarithmic scale factors. We show our method provides significant improvements in time and space complexity, achieving 10× reduction in latency for generative modeling on the MNIST dataset. Furthermore, our approach enables learning of distributions with 10× more variables than previous approaches when applied to a variety of density estimation benchmarks. Our code is publicly available at https://github.com/ptensnet/ptn.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 22718