Multi-Agent Soft Actor-Critic with Coordinated Loss for Autonomous Mobility-on-Demand Fleet Control

Zeno Woywood, Jasper I. Wiltfang, Julius Luy, Tobias Enders, Maximilian Schiffer

Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.

Tracks: Main Track

Keywords: hybrid learning and optimization, multi-agent learning, deep reinforcement learning, coordinated loss, autonomous mobility on demand

TL;DR: We propose a novel coordinated critic loss function to enable accurate state value estimations when combining multi-agent SAC with a coordination layer to learn dispatching and rebalancing policies in autonomous Mobility-on-Demand systems.

Abstract: We study a sequential decision-making problem for a profit-maximizing operator of an autonomous mobility-on-demand system. Optimizing a central operator's vehicle-to-request dispatching policy requires efficient and effective fleet control strategies. To this end, we employ a multi-agent Soft Actor-Critic algorithm combined with weighted bipartite matching. We propose a novel vehicle-based algorithm architecture and adapt the critic's loss function to appropriately consider coordinated actions. Furthermore, we extend our algorithm to incorporate rebalancing capabilities. Through numerical experiments, we show that our approach outperforms state-of-the-art benchmarks by up to 12.9\% for dispatching and up to 38.9\% with integrated rebalancing.

Submission Number: 77