Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach

Yang Xu, Vaneet Aggarwal

Keywords: Quantum Machine Learning, Reinforcement Learning

Abstract:

We address the problem of quantum reinforcement learning (QRL) under model-free settings with quantum oracle access to the Markov Decision Process (MDP). This paper introduces a Quantum Natural Policy Gradient (QNPG) algorithm, which replaces the random sampling used in classical Natural Policy Gradient (NPG) estimators with a deterministic gradient estimation approach, enabling seamless integration into quantum systems. While this modification introduces a bounded bias in the estimator, the bias decays exponentially with increasing truncation levels. This paper demonstrates that the proposed QNPG algorithm achieves a sample complexity of $\widetilde{O}(\epsilon^{-1.5})$ for queries to the quantum oracle, significantly improving the classical lower bound of $\widetilde{O}(\epsilon^{-2})$ for queries to the MDP.

This is to be considered as an extended abstract.

Submission Number: 18