Constrained Multi-Objective Reinforcement Learning with Max-Min Criterion

ICLR 2026 Conference Submission19386 Authors

Keywords: Reinforcement learning, Multi-objective reinforcement learning

TL;DR: We propose a unified framework for constrained MORL that integrates the max-min criterion with constraint satisfaction, supported by a theoretical foundation.

Abstract: Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies over multiple and often conflicting objectives. Although max-min scalarization has emerged as a powerful approach to promote fairness in MORL, it has limited applicability, especially when handling heterogeneous objectives or incorporating constraints. In this paper, we propose a unified framework for constrained MORL that combines the max-min criterion with constraint satisfaction and generalizes prior formulations such as unconstrained max-min MORL and constrained weighted-sum MORL. We establish a theoretical foundation for our framework and validate our algorithm through a formal convergence analysis and experiments in tabular environments. We further extend it to practical applications, including simulated edge computing resource allocation and locomotion control, demonstrating our framework’s capability to address fairness and constraint handling in multi-objective decision-making.

Primary Area: reinforcement learning

Submission Number: 19386