Leveraging Large Language Models for Solving Rare MIP Challenges

NAACL 2025 Workshop AISD Submission1 Authors

Keywords: Mixed integer programming, Chain-of-Thought, dynamic temperature, ride pooling, bipartite matching

Abstract: Mixed Integer Programming (MIP) has been extensively applied to areas requiring mathematical solvers to address complex instances within tight time constraints. However, as the problem scale increases, the complexity of model formulation and finding feasible solutions escalates significantly. Beneficial from outstanding text generation capacity of Large Language Models (LLMs), building and solving industrial-level instances becomes insensitive to problem scale. While LLMs, like GPT-4, can handle some traditional medium-scale MIP problems, they struggle with uncommon or highly specialized MIP scenarios. Fine-tuning LLMs can yield some feasible solutions for medium-scale MIP instances, but these models typically fail to explore diverse solutions when constrained by a low and constant temperature. In this paper, we propose and evaluate a recursively dynamic temperature method integrated with a chain-of-thought approach to exploit a large feasible region. Our findings show that starting with a high temperature and gradually lowering it leads to better feasible solutions compared to other dynamic temperature strategies. Additionally, by comparing results generated by the LLM with those from Gurobi, we demonstrate that the LLM can produce solutions that complement traditional solvers by accelerating the pruning process and improving overall efficiency.

Submission Number: 1