Deep Model Compression and Inference Speedup of Sum-Product Networks on Tensor Trains

Ching-Yun Ko, Cong Chen, Zhuolun He, Yuke Zhang, Kim Batselier, Ngai Wong

Abstract: Sum-product networks (SPNs) constitute an emerging class of neural networks with clear probabilistic semantics and superior inference speed over other graphical models. This brief reveals an important connection between SPNs and tensor trains (TTs), leading to a new canonical form which we call tensor SPNs (tSPNs). Specifically, we demonstrate the intimate relationship between a valid SPN and a TT. For the first time, through mapping an SPN onto a tSPN and employing specially customized optimization techniques, we demonstrate improvements up to a factor of 100 on both model compression and inference speedup for various data sets with negligible loss in accuracy.