Online Prediction with Limited Selectivity

Licheng Liu, Mingda Qiao

Keywords: Online prediction, selective prediction

TL;DR: We introduce a model of prediction with limited selectivity, and prove instance-dependent bounds on the optimal error rate.

Abstract: Selective prediction [Dru13, QV19] models the scenario where a forecaster freely decides on the prediction window that their forecast spans. Many data statistics can be predicted to a non-trivial error rate *without any* distributional assumptions or expert advice, yet these results rely on that the forecaster may predict at any time. We introduce a model of Prediction with Limited Selectivity (PLS) where the forecaster can start the prediction only on a subset of the time horizon. We study the optimal prediction error both on an instance-by-instance basis and via an average-case analysis. We introduce a complexity measure that gives instance-dependent bounds on the optimal error. For a randomly-generated PLS instance, these bounds match with high probability.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 5261