AGE-MORL: Agent-Guided Evolutionary Control for Multi-Objective Reinforcement Learning

Xuanang Gui, Qianlong Wang, Huan Zhao, Yuheng Cheng, Ping Chen, Junhua Zhao

Keywords: Multi-objective optimization, Deep reinforcement learning, Agent-guided evolution, Geometric analysis, Population diversity

Abstract: Multi-objective reinforcement learning (MORL) provides a powerful paradigm for solving problems with conflicting objectives. However, their performance can be highly dependent on the selection of search operators, which often follows predefined and non-adaptive strategies. In this paper, we propose a novel hierarchical framework, termed Agent-Guided Evolutionary Control framework for Multi-Objective Reinforcement Learning (AGE-MORL), which leverages Deep Reinforcement Learning (DRL) to adaptively select operators for multi-objective optimization algorithms at a high level. The operator selection problem is formulated as a Markov Decision Process (MDP), where a reinforcement learning agent learns a dynamic strategy-selection policy based on the evolving state of the optimization process. To enhance search effectiveness, we design a set of intelligent search operators based on geometric analysis of the Pareto front, including a sector-based exploration mechanism to explore sparse regions. Further, to enhance population diversity and escape local optima, we integrate a probabilistic acceptance model that combines a Simulated Annealing (SA) criterion with a Blink mechanism. Experiments on a diverse set of multi-objective optimization problems show that the proposed method significantly outperforms existing state-of-the-art methods, in terms of both solution quality and stability.

Primary Area: optimization

Submission Number: 9088