Sample-efficient model-based reinforcement learning for quantum control

Khalid, Irtaza, Weidner, Carrie A., Jonckheere, Edmond A., Schirmer, Sophie G. and Langbein, Frank C. 2023. Sample-efficient model-based reinforcement learning for quantum control. Physical Review Research 5 (4) , 043002. 10.1103/PhysRevResearch.5.043002

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with reduced sample complexity over model-free RL. Sample complexity is defined as the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an autodifferentiable ODE, parametrized by a learnable Hamiltonian ansatz, to represent the model approximating the environment, whose time-dependent part, including the control, is fully known. Control alongside Hamiltonian learning of continuous time-independent parameters is addressed through interactions with the system. We demonstrate an order of magnitude advantage in sample complexity of our method over standard model-free RL in preparing some standard unitary gates with closed and open system dynamics, in realistic computational experiments incorporating single-shot measurements, arbitrary Hilbert space truncations, and uncertainty in Hamiltonian parameters. Also, the learned Hamiltonian can be leveraged by existing control methods like GRAPE (gradient ascent pulse engineering) for further gradient-based optimization with the controllers found by RL as initializations. Our algorithm, which we apply to nitrogen vacancy (NV) centers and transmons, is well suited for controlling partially characterized one- and two-qubit systems.

Item Type:	Article
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	American Physical Society
ISSN:	2643-1564
Date of First Compliant Deposit:	22 January 2024
Date of Acceptance:	12 September 2023
Last Modified:	24 Jan 2024 14:30
URI:	https://orca.cardiff.ac.uk/id/eprint/165718

Actions (repository staff only)

Edit Item