Entropy property testing with finitely many errors

Changlong Wu, Narayana Santhanam

Abstract: Let P be a class of distributions over natural numbers, and A be a subset of ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> . We study the problem of deciding, using i.i.d. samples X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ,... from an unknown p ∈ P, whether the entropy H(p) is in A or not. The decision is updated based on every new observation X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> -we are interested in decision rules that make only finitely many errors no matter what the underlying source is. We give necessary and sufficient conditions on the class P and A that can be decided with only finitely many errors. We show for example that such rules exist for testing the rationality of entropy within a given interval, for testing if the entropy falls in an interval of form (a,b], but no such decision rule exists to determine if the entropy is finite or if the entropy falls in an interval of form [a,b]. In the process, we also highlight the conceptual foundation this framework shares with regularization.