Constrained Multi-Objective Reinforcement Learning with Max-Min Criterion

Giseung Park, Hyunyoung Nam, Woohyeon Byeon, Amir Leshem, Youngchul Sung

Keywords: Reinforcement learning, Multi-objective reinforcement learning

TL;DR: We propose a unified framework for constrained MORL that integrates the max-min criterion with constraint satisfaction, supported by a theoretical foundation.

Abstract: Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies over multiple and often conflicting objectives. Although max-min scalarization has emerged as a powerful approach to promote fairness in MORL, it has limited applicability, especially when incorporating constraints. In this paper, we propose a unified framework for constrained MORL that combines the max-min criterion with constraint satisfaction and generalizes prior formulations such as unconstrained max-min MORL and constrained weighted-sum MORL. We establish a theoretical foundation for our framework and validate our algorithm through a formal convergence analysis and experiments in tabular environments. We extend our framework to practical applications, including simulated edge computing resource allocation and locomotion control. Across these domains, the method demonstrates strong handling of fairness and constraint satisfaction in multi-objective decision-making.

Primary Area: reinforcement learning

Submission Number: 19386