Quarl Optimizes Quantum Circuits, Outperforms Existing Methods with Reinforcement Learning

Quarl, a learning-based quantum circuit optimizer, has been developed by researchers from Carnegie Mellon University, Columbia University, Microsoft, Tsinghua University, and VMware Research. The optimizer uses reinforcement learning to address the challenges of quantum circuit optimization, such as the large and varying action space and the nonuniform state representation. Quarl significantly outperforms existing circuit optimizers on almost all benchmark circuits and can learn to perform complex nonlocal circuit optimization tasks. The success of Quarl opens up new avenues for research in quantum circuit optimization and could accelerate the development of practical quantum computing applications.

What is Quarl and Why is it Important?

Quarl is a learning-based quantum circuit optimizer developed by a team of researchers from Carnegie Mellon University, Columbia University, Microsoft, Tsinghua University, and VMware Research. Quantum computing is a novel paradigm that offers significant acceleration over classical counterparts in a wide range of applications such as quantum simulation, integer factorization, and machine learning. However, programming quantum computers is a challenging task due to the scarcity of qubits and the diverse forms of noise that affect the performance of near-term intermediate-scale quantum (NISQ) devices.

Quantum programs are commonly represented as quantum circuits. To enhance the success rate of executing a circuit, a common form of optimization is applying circuit transformations, which replace a subcircuit matching a specific pattern with a functionally equivalent subcircuit that has better performance. Quarl addresses the challenges of quantum circuit optimization by applying reinforcement learning (RL) to the process.

The RL approach to quantum circuit optimization raises two main challenges: the large and varying action space and the nonuniform state representation. Quarl addresses these issues with a novel neural architecture and RL training procedure. The neural architecture decomposes the action space into two parts and leverages graph neural networks in its state representation. This approach is guided by the intuition that optimization decisions can be mostly guided by local reasoning while allowing global circuit-wide reasoning.

How Does Quarl Compare to Other Quantum Circuit Optimizers?

Quarl significantly outperforms existing circuit optimizers on almost all benchmark circuits. Surprisingly, Quarl can learn to perform rotation merging, a complex nonlocal circuit optimization implemented as a separate pass in existing optimizers.

Prior research has proposed two approaches for performing circuit transformations on an input circuit. The first approach is the use of rule-based strategies, which are employed by many quantum compilers such as Qiskit, tket, and Quilc. These strategies involve the greedy application of a set of circuit transformations that are manually designed by quantum computing experts to improve the performance of quantum circuits.

The second approach, as introduced in recent works, is a search-based approach that explores a search space of circuits that are functionally equivalent to the input circuit. For instance, Quartz automatically generates and verifies circuit transformations for a given gate set which preserves equivalence but may not necessarily improve performance. To optimize an input circuit, Quartz employs a cost-based backtracking search algorithm to apply these transformations and discover an optimized circuit.

What are the Challenges in Transformation-Based Quantum Circuit Optimization?

Transformation-based quantum circuit optimization presents several challenges. The set of circuits that can be reached from an input circuit by iteratively applying verified equivalence-preserving transformations comprises the search space in quantum circuit optimization. However, finding the optimal circuit is challenging due to the size of the space, which makes exhaustive exploration infeasible, and the inability of the cost function derived from a selected performance metric to provide enough guidance for a greedy approach.

This scenario is referred to as a planar optimization landscape since the path from one circuit to another with lower cost often contains many steps in which the cost remains unchanged. Plateaus of this sort are also present in classic program optimization.

To illustrate the challenge of discovering an optimal circuit in the search space, the researchers analyzed the search space of a relatively small circuit, barencotof3, which is implemented with 58 gates in the Nam gate set. They considered the 6206 transformations discovered by Quartz for the Nam gate set and exhaustively found all circuits reachable within seven transformations. Of these roughly 162 million circuits, they randomly sampled 200 circuits and analyzed the optimization landscape around them.

How Does Quarl Address These Challenges?

Quarl addresses the challenges of quantum circuit optimization by applying reinforcement learning (RL) to the process. The RL approach to quantum circuit optimization raises two main challenges: the large and varying action space and the nonuniform state representation.

Quarl addresses these issues with a novel neural architecture and RL training procedure. The neural architecture decomposes the action space into two parts and leverages graph neural networks in its state representation. This approach is guided by the intuition that optimization decisions can be mostly guided by local reasoning while allowing global circuit-wide reasoning.

What are the Implications of Quarl’s Success?

The success of Quarl has significant implications for the field of quantum computing. By significantly outperforming existing circuit optimizers on almost all benchmark circuits, Quarl demonstrates the potential of applying reinforcement learning to quantum circuit optimization.

Moreover, the ability of Quarl to learn to perform rotation merging, a complex nonlocal circuit optimization implemented as a separate pass in existing optimizers, suggests that reinforcement learning can be used to automate complex optimization tasks that were previously performed manually by quantum computing experts.

What are the Future Directions for Quarl and Quantum Circuit Optimization?

The success of Quarl opens up new avenues for research in quantum circuit optimization. Future work could explore the application of reinforcement learning to other aspects of quantum computing, such as quantum error correction and quantum algorithm design.

Moreover, the novel neural architecture and RL training procedure used by Quarl could be adapted for other optimization tasks in quantum computing. For instance, the decomposition of the action space into two parts and the use of graph neural networks in state representation could be applied to the optimization of quantum algorithms or the design of quantum error correction codes.

Finally, the success of Quarl highlights the need for further research into the challenges of quantum circuit optimization, such as the large and varying action space and the nonuniform state representation. By addressing these challenges, future work could further improve the performance of quantum circuits and accelerate the development of practical quantum computing applications.