Generalising Battery Control in Net-Zero Buildings via Personalised Federated RL

Nicolas Mauricio Cuadrado, Samuel Horváth, Martin Takáč

Keywords: microgrids, renewables, reinforcement learning, federated learning

TL;DR: Federated, privacy-preserving reinforcement learning can train battery controllers that generalize across many net-zero buildings.

Abstract: This work studies the challenge of optimal energy management in building-based microgrids through a collaborative and privacy-preserving framework. We evaluated two common RL algorithms (PPO and TRPO) in different collaborative setups to manage distributed energy resources (DERs) efficiently. Using a customized version of the CityLearn environment and synthetically generated data, we simulate and design net-zero energy scenarios for microgrids composed of multiple buildings. Our approach emphasizes reducing energy costs and carbon emissions while ensuring privacy. Experimental results demonstrate that Federated TRPO is comparable with state-of-the-art federated RL methodologies without hyperparameter tuning. The proposed framework highlights the feasibility of collaborative learning for achieving optimal control policies in energy systems, advancing the goals of sustainable and efficient smart grids. Our code is accessible \href{https://github.com/Optimization-and-Machine-Learning-Lab/energy_fed_trpo.git}{\textit{this repo}}.

Submission Number: 29