Dynamic Bayesian Optimization Framework for Instruction Tuning in Partial Differential Equation Discovery

ACL ARR 2026 January Submission1666 Authors

Keywords: Large Language Models, Bayesian Optimization, Symbolic Regression, Prompt Optimization, AI for Science

Abstract: Large Language Models (LLMs) show promise for equation discovery, yet their outputs are highly sensitive to prompt phrasing—a phenomenon we term instruction brittleness. Static prompts cannot adapt to the evolving state of a multi-step generation process, causing models to plateau at suboptimal solutions. To address this, we propose NeuroSym-BO, which reframes prompt engineering as a sequential decision problem. Our method maintains a discrete library of reasoning strategies and uses Bayesian Optimization to select the optimal instruction at each step based on numerical feedback. Experiments on PDE discovery benchmarks show that adaptive instruction selection significantly outperforms fixed prompts, achieving higher recovery rates with more parsimonious solutions.

Paper Type: Short

Research Area: Mathematical, Symbolic, Neurosymbolic, and Logical Reasoning

Research Area Keywords: NLP Applications, Language Modeling

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 1666