Beyond maximum likelihood: Boosting the Chow-Liu algorithm for large alphabets

Jiantao Jiao, Yanjun Han, Tsachy Weissman

Abstract: We show that in high dimensional distributions, i.e., the regime where the alphabet size of each node is comparable to the number of observations, the Chow-Liu algorithm on learning graphical models is highly sub-optimal. We propose a new approach, where the key ingredient is to replace the empirical mutual information in the Chow-Liu algorithm with a minimax rate-optimal estimator proposed recently by Jiao, Venkat, Han, and Weissman. We demonstrate the improved performance of the new approach in two problems: learning tree graphical models and Bayesian network classification.