Distributed Multiagent Deep Reinforcement Learning for Multiline Dynamic Bus Timetable Optimization

Haoyang Yan, Zhiyong Cui, Xinqiang Chen, Xiaolei Ma

Abstract: As a primary countermeasure to mitigate traffic congestion and air pollution, promoting public transit has become a global census. Designing a robust and reliable bus timetable is a pivotal step to increase ridership and reduce operating cost for transit authorities. However, most previous studies on bus timetabling rely on historical passenger count and travel time data to generate static schedules, which often yield biased results in these uncertain scenarios, such as demand surge or adverse weather. In addition, acquiring real-time passenger origin/destination from a limited number of running buses is not feasible. This article considers the multiline dynamic bus timetable optimization problem as a Markov decision process model to address the aforementioned issues, and proposes a multiagent deep reinforcement learning framework to ensure effective learning from the imperfect-information game, where the passenger demand and traffic condition are not always known in advance. Moreover, a distributed reinforcement learning algorithm is applied to overcome the limitation of high computational cost and low efficiency. A case study of multiple bus lines in Beijing, China, confirms the effectiveness and efficiency of the proposed model. The results demonstrate that our method outperforms heuristic and state-of-the-art reinforcement learning algorithms by reducing 20.30% of operating and passenger costs compared with actual timetables.