PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs

Mauricio Soroco, Jialin Song, Mengzhou Xia, Kye Emond, Weiran Sun, Wuyang Chen

TL;DR: We build an LLM termed "PDE-Controller" that can achieve reasoning and planning on PDE (partial differential equation) control problems.

Abstract: We present PDE-Controller, a framework that enables large language models (LLMs) to control systems governed by partial differential equations (PDEs). Traditional LLMs have excelled in commonsense reasoning but fall short in rigorous logical reasoning. While recent AI-for-math has made strides in pure mathematics, areas of applied mathematics, particularly PDEs, remain underexplored despite their significant real-world applications. Our approach enables LLMs to transform informal natural language instructions into formal specifications, and then execute reasoning and planning steps to improve the utility of PDE control. We build a holistic solution comprising datasets (both human-written cases and 2 million synthetic samples), math-reasoning models, and novel evaluation metrics, all of which require significant effort. Our PDE-Controller significantly outperforms the latest open-source and GPT models in reasoning, autoformalization, and program synthesis, achieving up to a 62% improvement in utility gain for PDE control. By bridging the gap between language generation and PDE systems, we demonstrate the potential of LLMs in addressing complex scientific and engineering challenges. We promise to release all data, model checkpoints, and code upon acceptance.

Lay Summary: We explore the abilities of large language models (LLMs) to perform scientific reasoning, beyond commonsense reasoning, in order to solve partial differential equations (PDEs). Scientific and engineering problems often rely on PDEs to model real-world systems, but AI struggles to handle them efficiently. Our framework, PDE-Controller, bridges this gap by enabling LLMs to better understand and control these equations, leading to smarter, more effective solutions in fields like aerospace engineering, physics, and material science. By improving LLMs' capabilities over the latest open-source and GPT models in reasoning, autoformalization, and program synthesis, this framework promises significant advances in automated scientific and engineering research.

Link To Code: https://github.com/delta-lab-ai/pde-controller/tree/master

Primary Area: Deep Learning->Large Language Models

Keywords: AI-for-Math, Large Language Model, Partial Differential Equation

Submission Number: 14088