A Latent Causal Diffusion Model for Multiobjective Reinforcement Learning

Xueming Yan, Bo Yin, Wei Chen, Yaochu Jin

Keywords: Multiobjective Reinforcement Learning, Diffusion Model, Casual Learning

TL;DR: This paper introduces the Latent Causal Diffusion Model (LacaDM) for Multiobjective Reinforcement Learning (MORL), addressing challenges in dynamic environments by learning latent causal relationships between states and policies.

Abstract: Multiobjective reinforcement learning (MORL) poses significant challenges due to the inherent conflicts between objectives and the difficulty of adapting to dynamic environments. Traditional methods often struggle to generalize effectively, particularly in large and complex state-action spaces. To address these limitations, we introduce the Latent Causal Diffusion Model (LacaDM), a novel approach designed to enhance the adaptability of MORL in discrete and continuous environments. Unlike existing methods that primarily address conflicts between objectives, LacaDM learns latent temporal causal relationships between environmental states and policies, enabling efficient knowledge transfer across diverse MORL scenarios. By embedding these causal structures within a diffusion model-based framework, LacaDM achieves a balance between conflicting objectives while maintaining strong generalization capabilities in previously unseen environments. Empirical evaluations on various tasks from the MOGymnasium framework demonstrate that LacaDM consistently outperforms the state-of-art baselines in terms of hypervolume, sparsity, and expected utility maximization, showcasing its effectiveness in complex multiobjective tasks.

Primary Area: reinforcement learning

Submission Number: 10710