Reinforcement Learning with learned gadgets to tackle hard quantum problems on real hardware

Akash Kundu, Leopoldo Sarra

TL;DR: GRL combines reinforcement learning and program synthesis to learn efficient quantum circuit components from easy problems, enabling scalable solutions for harder quantum tasks with improved accuracy and hardware compatibility.

Abstract: Designing quantum circuits for specific computational tasks remains a fundamental challenge in quantum computing, because of the exponential growth of the state space with the number of qubits. We propose *gadget reinforcement learning* (GRL), a framework that integrates reinforcement learning (RL) with program synthesis by automatically synthesizing composite gates, or ``gadgets", and incorporating them into the RL agent's action space. This enables a more efficient exploration of the design space for parameterized quantum circuits (PQCs) that solve complex quantum tasks, such as approximating ground states of quantum Hamiltonians—an NP-hard problem. We test GRL using the transverse field Ising model (TFIM), a standard testbed for quantum algorithms, under fixed computational budgets typical of research settings (e.g., 2--3 days of GPU runtime). Our experimental results demonstrate the advantages of GRL over baseline RL methods, including: (1) *Improved accuracy*: GRL achieves ground-state energy estimation up to machine accuracy; (2) *Hardware compatibility*: GRL generates compact PQCs that are more suitable for implementation on real quantum hardware, minimizing noise and gate errors; (3) *Scalability*: GRL exhibits robust performance as the size and complexity of the problem increases, even with constrained computational resources. By integrating program synthesis into the RL framework, GRL facilitates the automatic discovery of reusable circuit components, specifically tuned for a given hardware. This bridges the gap between algorithmic design and practical quantum implementation. This makes GRL a versatile and resource-efficient framework for optimizing quantum circuits, with potential applications in hardware-specific optimizations, variational quantum algorithms, and other challenging quantum tasks.

Primary Area: Applications->Chemistry, Physics, and Earth Sciences

Keywords: Quantum computing, Reinforcement learning, Program synthesis, Gadgets, Learning

Submission Number: 4736