AI Learns to Build Six-Qubit Circuits Using Reinforcement Learning

A new method simplifies Clifford quantum circuits, a key step towards utilising all-to-all qubit connectivity in quantum computing. Richie Yeung and colleagues at University of Oxford, in collaboration with College of Computing and Data Science and Nanyang Technological University, have developed a reinforcement learning approach where an agent learns to simplify Clifford circuits represented as symplectic matrices. Their new, size-agnostic neural network architecture, equivariant to qubit relabelings, enables the application of a single policy to circuits of varying qubit counts. The agent achieves near-optimal results on six-qubit circuits and outperforms existing synthesis methods, such as Qiskit’s Aaronson-Gottesman and greedy algorithms, when applied to Clifford circuits with up to thirty qubits and over a thousand gates.

Rapid Clifford circuit synthesis via reinforcement learning enables optimisation at scale

A breakthrough in quantum circuit synthesis is now possible. An agent finds optimal circuits in 99.2% of instances on six-qubit circuits within seconds, a substantial improvement over prior methods requiring 217 hours to reach 97.9% optimality. This performance crosses a critical threshold previously unattainable, as exact solutions for Clifford circuits are computationally expensive and limited to smaller systems. Heuristic approaches often lack the precision needed for complex computations.

The new reinforcement learning agent, utilising a novel neural network, extends the feasible scale of quantum circuit optimisation, enabling synthesis for circuits with up to thirty qubits and over a thousand gates. The agent’s neural network architecture is equivariant to qubit relabelings, meaning it can adapt to different qubit arrangements without retraining. It is also size-agnostic, allowing it to function across varying qubit counts without circuit modification.

This advance addresses a key challenge in quantum computing: efficiently translating abstract algorithms into executable gate sequences on physical hardware. Benchmarking against established methods like Qiskit’s synthesizers revealed lower average two-qubit gate counts. However, these results are currently confined to Clifford circuits and do not yet demonstrate comparable optimisation for the more complex, non-Clifford circuits required for many practical quantum algorithms. The authors acknowledge the limitation of relying on comparisons to existing methods rather than a definitive proof of optimality for circuits exceeding six qubits.

Symplectic matrix reduction via reinforcement learning yields efficient Clifford circuit synthesis

A reinforcement learning agent capable of synthesising Clifford quantum circuits for up to thirty qubits has been developed. It achieves lower two-qubit gate counts than existing methods like Qiskit’s Aaronson-Gottesman and greedy Clifford synthesizers. The agent learns to reduce a circuit’s symplectic matrix representation, a binary matrix describing the circuit, to the identity using a novel neural network architecture. This architecture is size-agnostic, simplifying scaling to larger circuits as it can be applied to different qubit counts without modification. The development does not address potential caveats regarding generalisation to different hardware architectures or error models. Future work will focus on extending this approach to non-Clifford circuits, essential for implementing a wider range of quantum algorithms, and the team suggests further exploration of the neural network’s size-agnostic nature for even more complex quantum computations.

Reinforcement learning optimises Clifford circuit synthesis for scalable quantum computation

Researchers at [Institution Name] have developed a reinforcement learning agent capable of synthesising Clifford quantum circuits for systems with fully connected qubits. The agent learns to apply a sequence of elementary Clifford gates, reducing a circuit’s representation, known as a symplectic matrix, to a simplified identity form. On six-qubit circuits, where optimal solutions are known, the agent identifies circuits within one two-qubit gate of the best possible result in milliseconds, achieving optimality in 99.2% of cases within seconds.

The agent consistently achieves lower two-qubit gate counts than established software like Qiskit’s synthesizers when demonstrating success on circuits up to thirty qubits, indicating improved efficiency. Demonstrating true optimality is currently limited to six-qubit circuits, with extrapolation used to assess performance on larger systems. Existing Clifford synthesis methods typically involve a trade-off between runtime and solution quality; polynomial-time algorithms like Aaronson-Gottesman scale efficiently but often use excessive entangling gates.

Stronger optimisation and exact-synthesis methods, however, demand significant computational resources. This development introduces a new approach to synthesising Clifford quantum circuits, utilising reinforcement learning and a novel neural network architecture. The agent learns to simplify circuits, represented as ‘symplectic matrices’, by discovering sequences of elementary gates, bypassing the computational limitations of existing methods for larger systems. In particular, the network’s design allows it to generalise across circuits of different sizes without needing retraining, a significant advantage for scaling quantum computations.

The research successfully demonstrated a reinforcement learning agent that optimises the synthesis of Clifford quantum circuits. This is important because efficient circuit synthesis is crucial for running complex quantum algorithms on existing and near-term quantum hardware. The agent achieved results on six-qubit circuits within one two-qubit gate of the optimal solution and outperformed existing methods on circuits with up to thirty qubits. Researchers continue to explore the agent’s ability to handle even more complex quantum computations, potentially enabling more scalable quantum computation.