Efficient Reinforcement Learning in Resource Allocation Problems Through Permutation Invariant Multi-task Learning

Desmond Cai, Shiau Hong Lim, Laura Wynter

Abstract: One of the main challenges in real-world reinforcement learning is to learn successfully from limited training samples. We show that in certain settings, the available data can be dramatically increased through a form of multi-task learning, by exploiting an invariance property in the tasks. We provide a theoretical performance bound for the gain in sample efficiency under this setting. This motivates a new approach to multi-task learning, which involves the design of an appropriate neural network architecture and a prioritized task-sampling strategy. We demonstrate empirically the effectiveness of the proposed approach on two real-world sequential resource allocation tasks where this invariance property occurs: financial portfolio optimization and meta federated learning.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We identify a permutation invariance property of reinforcement learning problems involving sequential resource allocation, provide a theoretical performance bound and use it to define a method to increase sample efficiency for this class of problems.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=GB1lIuJIkz