Policy Determination in Reinforcement Learning via Quantum Mechanics Analogies

TMLR Paper3247 Authors

Abstract: This study explores the integration of quantum mechanics with reinforcement learning (RL) to determine policies. Leveraging the ground state eigenfunction of the Schrödinger equation, a direct analogy between RL policies and quantum particle probabilities is established. The approach demonstrates close alignment with conventional RL methods, offering potential for efficient policy determination in complex environments. The study extends beyond gridworld scenarios to solve maze navigation tasks, showcasing the versatility of the approach. Future research could explore extensions of these concepts to multi-agent RL scenarios.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Taking into account the comments of the reviewers I have made the following changes:

1. Added appendix section on the effect of using the larger eigenvalues for calculating the probability density distribution.
2. Added appendix section on benchmarking the speed of calculating the value function comparing this manuscript’s method versus traditional value iteration methods.
3. Expanded the introduction section with 3 more references:
* J. Rahme and R.P. Adams, A Theoretical Connection Between Statistical Physics and Reinforcement Learning, 2021
* N. Meyer et al., A survey on quantum reinforcement learning, 2022
* Z. Cheng et al., Offline Quantum Reinforcement Learning in a Conservative Manner, 2023
4. Changed title to “Policy Determination in Reinforcement Learning via Quantum Mechanics Analogies” to avoid confounding with quantum computing concepts
5. Pg.1: “Classical RL methods” -> “Classical reinforcement learning (RL) methods”
6. Pg.1: “However, the opposite case, the application of quantum physics to RL is not as thoroughly investigated.” -> “It seems promising to use the tools and ideas of quantum physics for application to RL.”
7. Pg. 3-4: “a RL” -> ”an RL”
8. Pg. 4: “for which the ground state eigenfunction Ψ0(v) is solved for numerically” -> ”for which the ground state eigenfunction Ψ0(v) is solved numerically”
9. Fixed other typographical errors in the text and references

Assigned Action Editor: ~Andrew_Kyle_Lampinen1

Submission Number: 3247