Large Language Models powered Neural Solvers for Generalized Vehicle Routing Problems

Cong Dao Tran, Quan Nguyen-Tri, Huynh Thi Thanh Binh, Hoang Thanh-Tung

Keywords: Large Language Models, Neural Combinatorial Optimization, Vehicle Routing Problems

Abstract:

Neural Combinatorial Optimization (NCO) has shown promise in solving combinatorial optimization problems end-to-end with minimal expert-driven algorithm design. However, existing constructive NCO methods for Vehicle Routing Problems (VRPs) often rely on attention-based node selection mechanisms that struggle with large-scale instances.
To address this, we propose a directed fine-tuning approach for NCO based on LLM-driven automatic heuristic design. We first introduce an evolution-driven process that extracts implicit structural features from input instances, forming LLM-guided attention bias. This bias is then integrated into the neural model’s attention scores, enhancing solution flexibility and scalability. Instead of retraining from scratch, we fine-tune the model on a small, diverse dataset to transfer learned heuristics effectively to larger problem instances.
Experimental results show that our approach achieves state-of-the-art performance on TSP and CVRP, significantly improving generalization to both synthetic and real-world datasets (TSPLIB and CVRPLIB) with thousands of nodes.

Submission Number: 32