Efficient Risk-sensitive Planning via Entropic Risk Measures

Alexandre Marthe, Samuel Bounan, Aurélien Garivier, Claire Vernade

Keywords: planning, risk-sensitive, entropic risk measure, distributional, value at risk

Abstract:

Risk-sensitive planning aims to identify policies maximizing some tail-focused metrics in Markov Decision Processes (MDPs), such as (Conditional) Values at Risk.
In general, such optimization problems can be very costly as it is known that only Entropic Risk Measures (EntRM) can be efficiently optimized through dynamic programming.
We show that EntRM can serve as an approximation of other metrics of interest and we propose a dual optimization problem that requires to compute the set of optimal policies for EntRM across all parameter values.
We prove that this optimality front can be computed effectively thanks to a novel structural analysis of the smoothness properties of entropic risks.
Empirical results demonstrate that our approach achieves strong performance in risk-sensitive decision-making scenarios.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Alexandre_Marthe1

Track: Regular Track: unpublished work

Submission Number: 24