Constraint-Projection-Based Actor-Critic Algorithm for Dynamic Distributed Economic Dispatch With Nonconvex Nonsmooth Cost

Jun Li, Huaqing Li, Dawen Xia, Tingwen Huang, Lifeng Zheng, Liang Ran, Lianghao Ji

Abstract: Out of consideration for the dynamic demand of energy dispatching with local affine constraints and globally shared nonconvex nonsmooth hybrid cost arising from highly prevalent intermittent renewable energy sources, load effects, and multiple fuel resources, the constraint-projection-based actor–critic algorithm is investigated to solve the dynamic distributed economic dispatch problem in microgrids. To characterize the generation power against the fuel cost, we denote various fuel choices by segmented nonconvex costs. To help address the lack of prior knowledge of the system model, we propose a novel distributed actor–critic algorithm to find the feasible power. The algorithm combines the actor policy network with the multiplier splitting approach and projected gradient method, maps the action's policy distribution to a feasible power interval, and finally realizes real-time dispatching in continuous action space by approximating the evaluation function from the critic network. The convergence of the algorithm is theoretically analyzed. The effectiveness of the approach is verified by several simulations.