Reinforcement Learning for Large Group Systems using Hierarchical Kernel Representations

Wei Zhang, Jr-Shin Li

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Reinforcement learning, Group Systems, Control Theory

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We provide a nested reinforcement learning framework for learning optimal policies of parameterized group systems with convergence guarantee.

Abstract:

Policy learning for targeted coordination of massive-scale populations of, in the limit a continuum spectrum of, intelligent agents has been a missing component in reinforcement learning research. The purpose of this work is to fill in this literature gap by addressing the major challenge: the curse of dimensionality caused by the huge population size. To this end, we formulate such an intelligent agent population as a parameterized deterministic dynamical system, referred to as a group system, and then introduce the novel moment representation to the system. Under this representation, we propose a nested reinforcement learning algorithm to learn the optimal policy for the system hierarchically. As a significant advantage, each hierarchy preserves the optimality of all its lower-level children, which then leads to the fast convergence of the nested algorithm.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8686