Efficient profile maximum likelihood for universal symmetric property estimation

Moses Charikar, Kirankumar Shiragur, Aaron Sidford

Abstract: Estimating symmetric properties of a distribution, e.g. support size, coverage, entropy, distance to uniformity, are among the most fundamental problems in algorithmic statistics. While these properties have been studied extensively and separate optimal estimators have been produced, in striking recent work Acharya et al. provided a single estimator that is competitive for each. They showed that the value of the property on the distribution that approximately maximizes profile likelihood (PML), i.e. the probability of observed frequency of frequencies, is sample competitive with respect to a broad class of estimators. Unfortunately, prior to this work, there was no known polynomial time algorithm to compute such an approximation or use PML to obtain a universal plug-in estimator. In this paper we provide an algorithm that, given n samples from a distribution, computes an approximate PML distribution up to a multiplicative error of exp(n2/3 poly log(n)) in nearly linear time. Generalizing work of Acharya et al. we show that our algorithm yields a universal plug-in estimator that is competitive with a broad range of estimators up to accuracy є = Ω(n−0.166). Further, we provide efficient polynomial-time algorithms for computing a d-dimensional generalization of PML (for constant d) that allows for universal plug-in estimation of symmetric relationships between distributions.