Go to ICML 2025 Conference homepageAutoformulation of Mathematical Optimization Models Using LLMs
Mathematical optimization is fundamental to decision-making across diverse domains, from operations research to healthcare. Yet, translating real-world problems into optimization models remains a difficult task, often demanding specialized expertise. This paper approaches the problem of $\textit{autoformulation}$: the automated creation of solver-ready optimization models from natural language problem descriptions. We identify three core challenges of autoformulation: $\textit{(1)}$ the vast, problem-dependent hypothesis space, $\textit{(2)}$ efficient and diverse exploration of this space under uncertainty, and $\textit{(3)}$ evaluation of formulation correctness against problem description. To address these challenges, we present a novel method leveraging $\textit{Large Language Models}$ (LLMs) with $\textit{Monte-Carlo Tree Search}$, exploiting the hierarchical nature of optimization modeling to generate and systematically explore possible formulations. To enhance search efficiency, we introduce symbolic pruning to eliminate trivially equivalent search paths (branches), and employ LLM-based evaluation of partial formulations to guide search. Empirical analysis on linear and mixed-integer programming benchmarks demonstrates our method's effectiveness, with significant performance gains from both LLM-based value estimation and symbolic pruning techniques.
(1) Building mathematical optimization models from real-world problem descriptions is essential in fields like logistics, finance, and healthcare, but it remains a labor-intensive task requiring expert knowledge. (2) We introduce an autoformulator that uses large language models (LLMs) to automatically convert natural language problem descriptions into solver-ready optimization models. Our method uses Monte Carlo Tree Search to systematically explore modeling choices, guided by LLM-generated hypotheses and correctness evaluations, while symbolic tools prune redundant formulations to keep the search efficient. (3) This approach not only significantly outperforms prior baselines on real-world benchmarks, but also brings us closer to making powerful optimization tools accessible to non-experts, streamlining decision-making across industries and expanding the applications of AI-assisted modeling.