ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague, Colin Bellinger, Mark Crowley, Isaac Tamblyn

Keywords: reinforcement learning, automated chemistry, chemical synthesis, gym environment

TL;DR: This paper introduces a simulated chemistry environment for reinforcement learning, namely ChemGymRL.

Abstract: This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, **ChemGymRL**, implementing the standard Gymnasium API. ChemGymRL supports a series of interconnected virtual chemical *benches* where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.

Submission Track: Original Research

Submission Number: 198