Reinforcement Learning for Quantum Control under Physical Constraints

Jan Ole Ernst, Aniket Chatterjee, Tim Franzmeyer, Axel Kuhn

TL;DR: We present an RL approach to improve quantum control of realistic quantum systems for a variety of applications by integrating physics informed constraints into the problem formulation.

Abstract:

Quantum control is concerned with the realisation of desired dynamics in quantum systems, serving as a linchpin for advancing quantum technologies and fundamental research.
Analytic approaches and standard optimisation algorithms do not yield satisfactory solutions for more complex quantum systems, and especially not for real world quantum systems which are open and noisy.
We devise a physics-constrained Reinforcement Learning (RL) algorithm that restricts the space of possible solutions.
We incorporate priors about the desired time scales of the quantum state dynamics - as well as realistic control signal limitations - as constraints to the RL algorithm.
These constraints improve solution quality and enhance computational scaleability.
We evaluate our method on three broadly relevant quantum systems and incorporate real-world complications, arising from dissipation and control signal perturbations.
We achieve both higher fidelities - which exceed 0.999 across all systems - and better robustness to time-dependent perturbations and experimental imperfections than previous methods.
Lastly, we demonstrate that incorporating multi-step feedback can yield solutions robust even to strong perturbations. Our implementation can be found at: https://github.com/jan-o-e/RL4qcWpc.

Lay Summary:

Controlling quantum systems—the tiny building blocks of matter like electrons and photons—is essential for developing advanced technologies such as quantum computers and sensors. It’s also key to running advanced experiments in physics and chemistry. These systems are usually controlled by applying laser pulses or microwave signals, but this is challenging in real-world settings where such systems are noisy and interact with their environment.
We present a new approach using Reinforcement Learning (RL), a type of machine learning that learns how to control a system by trial and error. The algorithm tries different control signals on a simulated quantum system, then checks how close the result is to the desired outcome. What makes our method unique is that it’s guided by physical priors: it learns to prefer control signals that are realistic for experiments and that change the quantum state on the right time scales. By focusing only on physically meaningful options, the algorithm becomes faster and more effective.
We tested this method on three relevant quantum systems that are being actively studied in laboratories today. It consistently achieved very high control accuracy and outperformed previous methods, especially in situations where the systems were affected by noise or other real-world imperfections.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/jan-o-e/RL4qcWpc

Primary Area: Applications->Chemistry, Physics, and Earth Sciences

Keywords: reinforcement learning, quantum computing, quantum control, quantum dynamics, control theory

Submission Number: 7352