Federated reinforcement learning for generalizable motion planning

Zhenyuan Yuan, Siyuan Xu, Minghui Zhu

Abstract: This paper considers the problem of learning a control policy that generalize well to novel environments given a set of sample environments. We develop a federated learning framework that enables collaborative learning of multiple learners and a centralized server without sharing their raw data. In each iteration, each learner uploads its local control policy and the corresponding estimated normalized arrival time to the server, which then computes the global optimum among the learners and broadcasts the optimal policy to the learners. Each learner then selects between its local control policy and that from the server for next iteration. By leveraging generalization error, our analysis shows that the proposed framework is able to provide generalization guarantees on arrival time and safety as well as consensus at global optimal value in the limiting case. Monte Carlo simulation is conducted for evaluation.