Ensemble-based Deep Reinforcement Learning for Vehicle Routing Problems under Distribution Shift

Yuan Jiang, Zhiguang Cao, Yaoxin Wu, Wen Song, Jie Zhang

Keywords: Vehicle Routing Problem, Distribution shift, Deep Reinforcement Learning, Ensemble Learning

TL;DR: An ensemble learning approach to tackle out-of-distribution issue in routing problems, which trains a set of diverse policies with ensemble-based deep reinforcement learning.

Abstract: While performing favourably on the independent and identically distributed (i.i.d.) instances, most of the existing neural methods for vehicle routing problems (VRPs) struggle to generalize in the presence of a distribution shift. To tackle this issue, we propose an ensemble-based deep reinforcement learning method for VRPs, which learns a group of diverse sub-policies to cope with various instance distributions. In particular, to prevent convergence of the parameters to the same one, we enforce diversity across sub-policies by leveraging Bootstrap with random initialization. Moreover, we also explicitly pursue inequality between sub-policies by exploiting regularization terms during training to further enhance diversity. Experimental results show that our method is able to outperform the state-of-the-art neural baselines on randomly generated instances of various distributions, and also generalizes favourably on the benchmark instances from TSPLib and CVRPLib, which confirmed the effectiveness of the whole method and the respective designs.

Submission Number: 14082