Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity

Published: 02 Feb 2023, Last Modified: 28 Feb 2023Accepted by TMLREveryoneRevisionsBibTeX
Abstract: Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to construct risk measures which exhibit a desired tail sensitivity and may replace the expectation operator in ERM. Our method relies on the specification of a reference distribution with a desired tail behaviour, which is in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is compatible with this upper probability, displays a tail sensitivity which is finely tuned to the reference distribution. As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems. For instance, we show how ambiguity sets based on the Kullback-Leibler divergence are intricately tied to the class of subexponential random variables. We elaborate the connection of divergence risk measures and rearrangement invariant Banach norms.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: 1. Only substantial change: we added a diagram to the appendix which illustrates key elements of the paper; we refer to it in the first sentence of the introduction. 2. We made multiple minor edits such as restructuring a sentence or replacing "\{" by "\left\{" in some cases.
Assigned Action Editor: ~Daniel_M_Roy1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 339