Coordinate Descent Method for Log-linear Model on Posets

Shota Hayashi, Mahito Sugiyama, Shin Matsushima

Abstract: In this study, we address a learning problem of probabilistic models that represent high-order interactions among discrete attributes. To include the second-order interaction of discrete attributes, probabilistic models such as Ising models and Boltzmann machines are widely used. Both are regarded as special cases of the log-linear model on partially ordered sets (posets), which can represent not only second-order but also higher-order interactions between discrete attributes. Such a model is also known by its preferable information-geometric structure. We propose a coordinate descent method for efficient learning of the log-linear model on posets and present an information-geometric understanding of its functionality. The proposed method has no hyperparameter, whereas the standard gradient descent method requires the stepsize to be set appropriately. We theoretically and empirically show that our proposed method is faster than the gradient descent method in learning distributions by the log-linear model on posets.