On Semi-Supervised Estimation of Discrete Distributions Under f-Divergences

H. S. Melihcan Erol, Lizhong Zheng

Abstract: We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed variable. We adopt the minimax framework with $l_{p}^{p}$ loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk with the optimal first-order constant for $p\geq 2$ in the regime $m=o(n)$, questions remained for $p\leq 2$ and various $f$ -divergences. In our study, we affirm that these composite estimators are indeed minimax optimal for $l_{p}^{p}$ loss functions, specifically for the range $1\leq p\leq 2$, including the critical $l_{1}$ loss. Additionally, we ascertain their optimality for a suite of $f$ -divergences, such as KL, $\chi^{2}$, Squared Hellinger, and Le Cam divergences.

External IDs:dblp:conf/isit/ErolZ24