On Sharpness Diagrams

Alexander I. Jordan

Keywords: autocalibration, entropy, proper scoring rules, sharpness principle

TL;DR: Only autocalibrated forecasts can be meaningfully compared in terms of sharpness, measured using an expected entropy.

Abstract: The sharpness principle states that the goal in probabilistic forecasting is to "maximize the sharpness of the predictive distributions subject to calibration". For that statement to be consistent with the principle of the best-informed forecast, the goal should be refined to "minimize the expected entropy subject to autocalibration". Not all popular sharpness measures follow this principle -- the average length of the predictive interquartile range being a prime example. Valid alternatives include the expected interval score entropy and the expected tail probability with respect to the modal interval.

Submission Number: 17