Reinforcement Learning Achieves Robust Perfect Entangling Gates for Noisy Intermediate-scale Quantum Computers

The quest for powerful quantum computers hinges on creating stable and reliable connections between quantum bits, or qubits, and a critical component of this is the perfect two-qubit gate. Leander Grech from University of Malta, Matthias G. Krauss from Freie Universität Berlin, and Mirko Consiglio from University of Malta, alongside their colleagues, demonstrate a significant advance in achieving these essential gates using a technique called reinforcement learning. Their work showcases how intelligent algorithms can discover pulse shapes that drive qubits with exceptional precision, even when faced with the inherent noise of current quantum hardware. This innovative approach not only identifies effective control strategies, but also promises to dramatically reduce the complex calibration procedures currently required for building practical quantum computers, paving the way for broader applicability across diverse computing platforms.

edu. mt. Noisy intermediate-scale quantum computers promise solutions to complex computational challenges through the massive parallelism offered by qubits. Achieving this potential relies on perfect entangling (PE) two-qubit gates, a critical building block for universal quantum computation. Researchers are now harnessing the power of quantum optimal control, shaping electromagnetic pulses to precisely manipulate quantum states.

Reinforcement Learning for Robust Quantum Control

This work details a comprehensive investigation into using reinforcement learning (RL) to design control pulses for quantum gates, offering a detailed account of the methods and results. The research focuses on justifying design choices, demonstrating robustness, and ensuring transparency in the approach. The team employed the TRPO (Trust Region Policy Optimization) algorithm, chosen for its ability to ensure monotonic policy improvement and stability during the learning process. The ZCQPEE (Z-Control Quantum Pulse Episodic Environment) served as the simulation environment, allowing the agent to interact with a virtual quantum system.

Key parameters, including pulse segment duration and the reward function, were carefully defined to guide the learning process. The reward function balances gate concurrence, unitarity, and pulse smoothness, using a total variation penalty to encourage efficient control pulses. The research also compares the RL approach to traditional optimal control theory (OCT), highlighting the trade-offs between computational cost and adaptability. While RL requires more upfront computation, it offers advantages for adaptive control scenarios and reduces the need for manual recalibration. Results from training with domain randomization demonstrate the robustness of the approach to variations in system parameters. This careful engineering, combined with a focus on robustness and transparency, provides a valuable resource for anyone interested in applying reinforcement learning to quantum control.

Reinforcement Learning Optimizes Three-Qutrit Entangling Gate

Scientists have achieved significant progress in designing high-fidelity quantum gates using reinforcement learning techniques, paving the way towards practical quantum computation. The research centers on a three-qutrit system, comprising two fixed-frequency qubits coupled by a tunable central bus, enabling the implementation of a perfect entangling (PE) gate through precise modulation of the bus frequency. Researchers developed a specialized reinforcement learning environment, ZCQPEE, to formulate the control problem as a Markov Decision Process, allowing an agent to learn optimal pulse shapes for driving the quantum system. The team simulated the quantum system using a Hamiltonian incorporating both static system properties and a time-dependent control term, accurately modeling the interactions between the qubits and the tunable coupler.

System parameters, including qubit frequencies and anharmonicities, were defined within the rotating wave approximation. The environment returns observations describing the quantum state’s evolution under applied control pulses, enabling the agent to learn over pulse segments rather than individual time steps. Experiments revealed that the reinforcement learning approach effectively identifies near-optimal pulse shapes for achieving PE gates. This work demonstrates the potential of reinforcement learning to reduce calibration overhead compared to traditional optimal control methods, offering a promising route toward scalable and robust quantum computation.

Reinforcement Learning Optimises Quantum Gate Fidelity

This research demonstrates the successful application of reinforcement learning to optimise pulse shapes for high-fidelity two-qubit gates, a crucial step towards realising the potential of quantum computation. By training agents within robust simulation environments, scientists have identified control strategies that yield near-optimal gate performance, even when accounting for realistic noise. The approach significantly reduces the calibration effort typically required in quantum control, offering a pathway to more efficient and scalable quantum systems. Importantly, the developed method is hardware agnostic, suggesting broad applicability across diverse computing platforms.

The team acknowledges that approximations, specifically truncating the Hilbert space, were employed to improve computational efficiency, but further simulations indicate that this simplification does not significantly alter the overall conclusions. Future work will focus on validating these findings on actual quantum hardware and exploring the potential for adapting the reinforcement learning framework to more complex quantum circuits and control scenarios. This work represents a significant advance in quantum control, paving the way for more reliable and powerful quantum computers.