Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity

Christian Fröhlich, Robert Williamson

Abstract:

Expected risk minimization (ERM) is at the core of many machine learning systems. This means
that the risk inherent in a loss distribution is summarized using a single number - its average.
In this paper, we propose a general approach to construct risk measures which exhibit a
desired tail sensitivity and may replace the expectation operator in ERM. Our method
relies on the specification of a reference distribution with a desired tail behaviour, which is
in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is
compatible with this upper probability, displays a tail sensitivity which is finely tuned to the
reference distribution. As a concrete example, we focus on divergence risk measures based
on f-divergence ambiguity sets, which are a widespread tool used to foster distributional
robustness of machine learning systems. For instance, we show how ambiguity sets based on
the Kullback-Leibler divergence are intricately tied to the class of subexponential random
variables. We elaborate the connection of divergence risk measures and rearrangement
invariant Banach norms.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

1. Only substantial change: we added a diagram to the appendix which illustrates key elements of the paper; we refer to it in the first sentence of the introduction.
2. We made multiple minor edits such as restructuring a sentence or replacing "\{" by "\left\{" in some cases.

Assigned Action Editor: ~Daniel_M_Roy1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 339