Actor-Critic Methods using Physics-Informed Neural Networks: Control of a 1D PDE Model for Fluid-Cooled Battery Packs

Amartya Mukherjee, Jun Liu

Keywords: Actor Critic Method, Physics-Informed Neural Network, Fluid Cooled Battery Packs, Hamilton Jacobi Bellman Equation

TL;DR: We introduce an actor-critic method using PINNs for controlling the temperature of a battery pack using a cooling fluid.

Abstract: This paper proposes an actor-critic algorithm for controlling the temperature of a battery pack using a cooling fluid. This is modeled by a coupled 1D partial differential equation (PDE) with a controlled advection term that determines the speed of the cooling fluid. The Hamilton-Jacobi-Bellman (HJB) equation is a PDE that evaluates the optimality of the value function and determines an optimal controller. We propose an algorithm that treats the value network as a Physics-Informed Neural Network (PINN) to solve the continuous-time HJB equation rather than a discrete-time Bellman optimality equation, and we derive a control function from the HJB equation. Our experiments show that a hybrid-policy method that updates the value network using the HJB equation and updates the policy network identically to PPO achieves the best results in the control of this PDE system.

Submission Number: 5