Multi-Objective Reinforcement Learning for Forward-Backward Markov Decision Processes

Mohsen Amidzadeh, Mario Di Francesco

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Forward-Backward Markov Decision Process, Multi-Objective Reinforcement Learning Algorithm

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

This work introduces the notion of Forward-Backward Markov Decision Process (FB-MDP)
for multi-task control problems. In this context, we devise a novel approach called Forward-Backward Multi-Objective Reinforcement Learning (FB-MORL).
Specifically, we analytically characterize its convergence towards a Pareto-optimal solution and also empirically evaluate its effectiveness.
For the latter, we consider a use case in wireless caching and perform several experiments to characterize performance in that context. Finally, an ablation study demonstrates that FB-MDP is instrumental to optimize rewards for systems with forward-backward dynamics.
The outcomes of this work pave the way for further understanding of multi-objective RL algorithms for FB-MDPs.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5148