Alpha-NML Universal Predictors

Marco Bondaschi, Michael Gastpar

Abstract: Inspired by Sibson's $\alpha$-mutual information, we introduce a new class of universal predictors that depend on a real parameter $\alpha \geq 1$. This class interpolates two well-known predictors, the mixture estimator, that includes the Laplace and the Krichevsky-Trofimov predictors, and the Normalized Maximum Likelihood (NML) estimator. We point out some advantages of this class of predictors and study its performance from two complementary viewpoints: (1) we analyze it in terms of worst-case regret, as an approximation of the optimal NML, for the class of discrete memoryless sources; (2) we discuss its optimality when the maximal Renyi divergence is considered as a regret measure, which can be interpreted operationally as a middle way between the standard average and worst-case regret measures. Finally, we study how our class of predictors relates to other generalizations of NML, such as Luckiness NML and Conditional NML.