Novel RL Approach for Efficient Elevator Group Control Systems

Nathan Vaartjes, Vincent Francois-Lavet

Keywords: Elevator Control, Reinforcement Learning, Applied Reinforcement Learning, Partially Observable Markov Decision Process, Dueling Double Deep Q-learning

TL;DR: Novel Reinforcement Learning approach for optimizing elevator dispatching, with contributions including a customized action space and the concept of infra-steps.

Abstract:

The management of elevator traffic in large buildings is crucial for ensuring low passenger travel times and energy consumption. We optimize the Elevator Group Control System (EGCS) using a novel Reinforcement Learning (RL) approach. Existing methods, including heuristic-based and pattern detection algorithms, often fall short in handling the complex and stochastic nature of elevator systems. This research proposes an end-to-end RL-based approach. A custom elevator simulation environment representing the 6-elevator, 15-floor system at Vrije Universiteit Amsterdam (VU) is developed as a Markov Decision Process (MDP).
Key innovations include a novel action space encoding to handle the combinatorial complexity of elevator dispatching, the introduction of $\textit{infra-steps}$ to model continuous passenger arrivals, and a tailored reward signal to improve learning efficiency. Additionally, we explore various ways of adapting the discounting factor to the $\textit{infra-step}$ formulation. We investigate RL architectures based on Dueling Double Deep Q-learning, showing that the proposed RL-based EGCS adapts to fluctuating traffic patterns, learns from a highly stochastic environment, and thereby outperforms a traditional rule-based algorithm.

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 14124