CHAROT: Robustly controlling chaotic PDEs with partial observations

Max Weissenbacher, Anastasia Borovykh, Georgios Rigas

Keywords: Reinforcement learning, chaos, PDEs, memory, attention, Transformers, KS

TL;DR: A Transformer-based memory augmentation improves controller performance for chaotic PDEs with partial observations.

Abstract: Control of chaotic partial differential equations is challenging but valuable, with far-reaching applications in energy systems, economics, fluid dynamics and many other domains. Realistic engineering applications often only admit partial observations of the state and the controller must learn to steer the system towards a desired state using incomplete information. We introduce CHAROT, an attention-based memory architecture designed to augment actor-critic reinforcement learning algorithms and improve their performance in controlling chaotic PDEs using only partial observations. We present numerical experiments for control of the Kuramoto-Sivashinsky equation in chaotic ($0.005 \leq \nu \leq 0.05$) and partially observable regimes. In the most chaotic regime considered, our method outperforms a memoryless controller by $150 $% and an LSTM-augmented controller by $206$%.

Submission Number: 58