GCRL: Efficient Delivery Area Assignment for Last-mile Logistics with Group-based Cooperative Reinforcement Learning

Hai Wang, Shuai Wang, Yu Yang, Desheng Zhang

Abstract: Last-mile logistics is the final step of the delivery process from a transit station to customers. In last-mile logistics systems, a city is divided into many delivery areas for couriers to finish the parcel transition tasks. In recent years, last-mile logistics faces huge challenges in system efficiency and customer experience due to highly dynamic logistics service demand across different delivery areas. How to design a proper mechanism to improve the system efficiency and customer experience has become an important task. In this paper, we formulate the delivery area assignment problem and propose a Group-based Cooperative Reinforcement Learning (GCRL) framework to optimize the last-mile logistics system. Firstly, we design a multi-level attention mechanism to construct an optimal courier team that provides cooperative pick-up and delivery services. Secondly, A graph generator and graph-based strategy are proposed to represent the decision dependency and coordinate the dependent behaviors among couriers, respectively. Finally, we design a simultaneous training mechanism to maximize the discounted return and guide the delivery area for each courier. Being formulated in a multi-agent way, GCRL focuses on the cooperation among couriers while considering the system context and couriers’ preferences. Experiments on real-world data show that GCRL achieves an average of 12% improvements compared with state-of-the-art models.