Hierarchical Deep Reinforcement Learning for Vehicle Routing Problem

Qianyu Liu

Keywords: Vehicle Routing Problem, Reinforcement Learning, Divide and Conquer, Markov Decision Process, Hierarchical Reinforcement Learning

Abstract: The Vehicle Routing Problem (VRP) is a fundamental combinatorial optimization problem with broad applications in logistics, supply chain management, and transportation. Traditional approaches, including exact algorithms and metaheuristics, often struggle with scalability and computational efficiency when applied to large-scale VRP instances. In this work, we propose a novel Hierarchical Deep Reinforcement Learning (HDRL) framework designed to tackle large-scale VRP efficiently. Our method employs a two-level hierarchical model, where a high-level meta-controller partitions the problem space into manageable subproblems, and a lower-level controller generates high-quality routes for these subproblems using deep reinforcement learning. This hierarchical decomposition significantly reduces the complexity of solving large-scale instances while maintaining competitive solution quality. We evaluate our approach on benchmark VRP datasets with up to 10,000 customers and compare its performance against state-of-the-art solvers, including LKH-3 and OR-Tools. Experimental results demonstrate that HDRL achieves near-optimal solutions with significantly reduced computational time, making it a promising approach for solving real-world VRP instances at scale.

Submission Number: 27