A Hierarchical Reinforcement Learning Based Optimization FrameWork for Large Scale Storage Location Assignment Problem
Keywords: Hierarchical Reinforcement Learning, Storage Location Assignment Problem, Large Scale
Abstract: The Storage Location Assignment Problem(SLAP) is one of the essential problems within the domain of logistics. The objective is to dynamically allocate optimal storage locations to incoming items, aiming to maximize warehouse space utilization and operational efficiency. Prior research primarily focused on offline scenarios with predetermined goods arrival times. A smaller portion explored real-time allocation using heuristic algorithms based on manual rules and search methods. However, these methods suffer from inadequate solution quality and efficiency, particularly for large-scale problems. To overcome this limitation, we draw inspiration from the partitioned, multi-layered, and modularized layout commonly adopted in most large-scale storage spaces. Building upon this inspiration, we propose a novel hierarchical optimization framework to solve large-scale SLAPs better via reinforcement learning. Specifically, we designed a two-level model: (1) a higher-level model learns to determine which block to choose, and (2) a lower-level model learns to select the final storage location under the constraints of the selected blocks in the upper level. We have designed a policy network based on attention mechanisms for SLAP to achieve better performance. To verify the effectiveness of the proposed framework, we collected a large amount of real historical data from the terminal operating system of Ningbo-Zhoushan Port and built a realistic container terminal simulator. Besides, we conducted extensive offline simulations and online testing using the simulator based on real data and validated the superior performance of our framework compared to existing benchmark methods.
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1167
Loading