Application of Multi-Agent Reinforcement Learning for Battery Management in Renewable Mini-Grids

Tomisin I. Dada, Pierre Thodoroff, Neil D Lawrence

Keywords: Battery Management, Operation Research, Reinforcement Learning, Multi-Agent

TL;DR: We re-frame the battery management problem in an Operations Research (OR) context as a multi-agent newsvendor problem and benchmark seven Multi-Agent Reinforcement Learning algorithms with five popular handcrafted heuristic strategies.

Abstract:

Electricity is an integral part of modern society, yet globally millions of people are without access. This lack of access, coupled with increasing concern over climate change represents a serious global challenge. Distributed energy storage will likely play a large part in the future of the grid, however, battery management remains an open problem. In this work, we re-frame the battery management problem in an Operations Research (OR) context as a multi-agent newsvendor problem. We benchmark seven Multi-Agent Reinforcement Learning (MARL) algorithms and compare their performance with five popular handcrafted heuristic strategies.
We considered MARL algorithms due to their capacity to learn novel policies from data that may outperform handcrafted rule-based policies, especially as problem complexity increases. We find that all seven methods learn policies that achieve comparable results to each other and outperform a simple keep-fully-charged heuristic consistently. However, they do not consistently outperform all the heuristics considered in all the scenarios considered.