A Memory-Based Graph Reinforcement Learning Method for Critical Load Restoration With Uncertainties of Distributed Energy Resource

Bangji Fan, Xinghua Liu, Gaoxi Xiao, Yan Xu, Xiang Yang, Peng Wang

Abstract: The integration of distributed energy resources into distribution networks, marked by its inherent uncertainties, presents a substantial challenge for devising load restoration strategies. To tackle this challenge, we develop a memory-based graph reinforcement learning approach, designed to train the agent to acquire a critical load restoration strategy in a distribution network under uncertainties. Specifically, the restoration problem under uncertainties is formulated as a novel partially observable Markov decision process, where a multimodal feature-based observation space is proposed. This space includes graph-structured data of the environment and memory information of the agent. The graph-structured data contain potential features of the current observation, thus enhancing the observable domain, while the memory information incorporates temporal correlations between sample sequences to address the partial observability of the environment. Based on the proposed Markov process, we put forth a maximum entropy-based recurrent graph soft actor-critic algorithm to train the agent in partially observable environments through a recursive structure, where entropy regularization is utilized to facilitate a more extensive exploration of possibilities in a state space with high uncertainties. The performance of the proposed approach is validated by a comparative study versus existing results on the IEEE 123-bus system containing wind power and photovoltaic sources.