Distributed Model-Free Policy Iteration for Networks of Homogeneous Systems

Shahriar Talebi, Siavash Alemzadeh, Mehran Mesbahi

Abstract: Control of large-scale networked systems often necessitates modeling complex interactions amongst agents. However, as the size of the network increases, modeling these interactions often becomes exponentially expensive in applications. In this paper, we propose a distributed model-free policy iteration algorithm to design a feedback mechanism for large networks of homogeneous systems. We assume that the networked system is built upon an underlying information-exchange graph allowing the distributed controller to synthesize a feedback signal using information from adjacent agents. This model-free approach provides a stabilizing distributed feedback controller through a learning phase. In particular, a data-driven control method is utilized to circumvent model uncertainties by directly synthesizing a controller based on data that is obtained from a relatively small subgraph of the original network. Additionally, a stability margin is learned from data which is then utilized to design a suboptimal distributed controller for the entire network even during the learning phase. We showcase the performance of our methodology by examining distributed control scenarios involving modeling errors.