Large Language Model-guided Multi-modal Motion Planning via Mixed Integer Program

ICLR 2026 Conference Submission17962 Authors

Keywords: Large Language Model, Robotics, Mix Integer Programming

Abstract: Multi-Modal Motion Planning (M$^3$P) is a rather challenging form of motion planning where the planner searches through the continuous space of motions as well as discrete space of modes. However, brute-force global search is typically sample-inefficient and computationally expensive. Recent research has explored the use of Mixed-Integer Programming (MIP) to reformulate global search problems in robotic applications. MIP leverages the branch-and-bound algorithm to efficiently prune infeasible or sub-optimal solutions. Despite its strengths, MIP is limited to problems with disjoint convex feasible domains---a constraint that is often too restrictive for general motion planning. To address this, prior work has proposed techniques to approximate non-convex motion planning problems as disjoint convex MIPs. Unfortunately, these methods are typically hand-crafted and domain-specific, limiting their generalizability. In this work, we explore the use of Large Language Models (LLMs) to automatically translate non-convex optimization problems into approximate MIP formulations. To this end, we construct a dataset comprising various M$^3$P problems paired with their known MIP approximations. We evaluate LLM performance on this reformulation task using both In-Context Learning (ICL) and Supervised Fine-Tuning (SFT). Our results demonstrate that LLMs are capable of capturing common patterns in MIP reformulations and can even generalize to complex, unseen translation tasks beyond those encountered during fine-tuning.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 17962