Evo-Step: Evolutionary Generation and Stepwise Validation for Optimizing LLMs in OR
Keywords: Large language model; Operations Research; Automated Modeling
Abstract: Large Language Models (LLMs) have revolutionized various domains but face significant challenges in tackling optimization modeling tasks for Operations Research (OR) problems, particularly when dealing with complex problem. In this work, we propose Evo-Step-Instruct, a framework that augments existing datasets and generates high-quality fine-tuning data tailored to OR modeling tasks. Evo-Step-Instruct employs iterative problem generation to progressively increase problem complexity and stepwise validation to rigorously validate data, preventing error propagation and ensuring the quality of the generated dataset. Leveraging this framework, we fine-tune open-source LLMs, including LLaMA-3-8B and Mistral-7B, to develop Evo-Step—a model that achieves state-of-the-art performance on benchmarks such as NL4OPT, MAMO, and IndustryOR. Extensive experiments demonstrate the superior performance of Evo-Step, especially in addressing complex OR tasks, with a notable 17.01\% improvement in micro average accuracy on difficult problems. These findings highlight the effectiveness of combining structured validation with gradual problem refinement to advance the automation of decision-making processes using LLMs. The code and dataset are available at [https://anonymous.4open.science/r/Evo-Step-F5AB](https://anonymous.4open.science/r/Evo-Step-F5AB).
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13668
Loading