ACCORD: Autoregressive Constraint-satisfying Generation for COmbinatorial Optimization with Routing and Dynamic attention

ICLR 2026 Conference Submission21260 Authors

Keywords: Large Language Models, Combinatorial Optimization, Autoregressive Generation, Constraint Satisfaction, NP-hard problems, Fine-tuning, LoRA, Attention-based Routing, TSP, VRP, Knapsack, FlowShop, JSSP, BinPacking

TL;DR: ACCORD is an end-to-end framework that uses a fine-tuned 8B-parameter Llama model with an autoregressive representation and dynamic attention to solve various NP-hard combinatorial optimization problems with high feasibility and low optimality gaps.

Abstract: Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, yet their direct application to NP-hard combinatorial problems (CPs) remains underexplored. In this work, we systematically investigate the reasoning abilities of LLMs on a variety of NP-hard combinatorial optimization tasks and introduce \textbf{ACCORD}: \textbf{A}utoregressive \textbf{C}onstraint-satisfying generation for \textbf{CO}mbinatorial optimization with \textbf{R}outing and \textbf{D}ynamic attention. ACCORD features a novel dataset representation and model architecture that leverage the autoregressive nature of LLMs to dynamically enforce feasibility constraints, coupled with attention-based routing to activate problem-specific LoRA modules. We also present the ACCORD-90k supervised dataset, covering six NP-hard combinatorial problems: TSP, VRP, Knapsack, FlowShop, JSSP, and BinPacking. Extensive experiments demonstrate that our ACCORD model, built on an 8B-parameter Llama backbone, consistently outperforms standard prompting and input-output methods, even when compared to much larger LLMs, such as gpt-4. Ablation studies further show that our output structure enhances solution feasibility. To the best of our knowledge, this is the first large-scale, end-to-end framework for exploring the applications of LLMs to a broad spectrum of combinatorial optimization problems.

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Submission Number: 21260