Guiding RL Agents with Classical Algorithms and Heuristics to Solve Combinatorial Optimization Problems on Graphs
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Reinforcement Learning, Combinatorial Optimization, Graphs, Deep Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We identify a method to combine classical algorithms with deep RL agents for combinatorial optimization on graphs, and propose a set of environments optimized for training deep RL agents on such problems.
Abstract: Graph Combinatorial Optimization (CO) problems are prevalent in various domains, such as supply chain management and operations. These problems are often challenging to solve at scale due to their complexity and large solution spaces. Deep Reinforcement Learning (DRL) methods have recently been explored as solutions to these problems but generally lag behind traditional heuristics. Existing DRL methods are not easily comparable due to differences in training and problem settings. Moreover, existing environments and libraries for DRL on CO lack comprehensive graph environments and predominantly focus on model design and training, overlooking the best practices of environment design. To address these challenges, we identify and emphasize the concept of \emph{parenting}, a method that combines traditional algorithms with RL agents to guide action selection. This results in hybrid solutions that surpass the performance of individual approaches. We also introduce \textit{GraphEnvs}, a package of 15 OpenAI-Gym environments that integrate nuanced aspects of environment design, significantly influencing task difficulty and model performance. Our contributions provide a unified and standardized framework that addresses the fragmentation in the research space, enabling more accurate comparisons and evaluations of DRL methods for graph CO problems.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6557
Loading