Hybrid Reinforcement Learning Enables Quantum Architecture Search and Optimizes Molecular Ground-State Energy

The quest to design effective quantum circuits presents a significant hurdle in the development of practical quantum algorithms, as traditional methods often struggle to find optimal configurations. Jiayang Niu, Yan Wang, and Jie Li, along with colleagues from RMIT University and beyond, now present a new approach that tackles this challenge by combining the strengths of both discrete and continuous optimisation techniques. Their work introduces a framework called HyRLQAS, which uses reinforcement learning to simultaneously design the structure of a quantum circuit and fine-tune its parameters, dynamically improving the circuit as it learns. This unified strategy, tested within a molecular energy minimisation task, consistently produces circuits with lower errors and fewer components than existing methods, and importantly, establishes better starting points for further optimisation, paving the way for automated design of more efficient and powerful quantum hardware.

Reinforcement Learning Optimizes Quantum Circuit Design

This research details a novel approach to automated quantum circuit design using a hybrid action space within a reinforcement learning framework. The team addresses the challenge of automatically creating quantum circuits tailored to specific tasks, a crucial step towards building larger and more powerful quantum computers. Traditional methods often struggle with the vast number of possible circuits and can become trapped in suboptimal solutions. To overcome these limitations, the authors propose a hybrid action space that allows the agent to simultaneously decide both what gate to place and how to configure it, fostering a more integrated and efficient search process.

The core innovation lies in a hybrid action space consisting of discrete and continuous actions. The agent selects the type of quantum gate and its placement on the qubits using discrete actions, while simultaneously setting and refining the parameters for both selected and previously placed gates using continuous actions. This allows for iterative improvement and adaptation of the circuit throughout the learning process. The agent learns through trial and error using a reinforcement learning algorithm, employing a neural network to map the current circuit state to appropriate actions, and a masking mechanism prevents redundant or invalid operations, improving efficiency.

The research demonstrates that this integrated approach offers several benefits. By simultaneously optimizing gate placement and parameters, the method achieves a more efficient search of the circuit design space, leading to more accurate and compact circuits. The iterative refinement of circuit parameters allows for adaptive improvement, and the prevention of redundant operations further enhances efficiency and stability. These advantages suggest that this hybrid action space has the potential to scale to more complex circuits and advance the field of quantum computing.

Hybrid Reinforcement Learning for Quantum Circuit Design

This study pioneers a novel framework, HyRLQAS (Hybrid-Action Reinforcement Learning for Quantum Architecture Search), to address limitations in automated quantum circuit design. Recognizing that manually designed circuits often lack adaptability and scalability, the team developed a reinforcement learning approach where an agent learns to construct circuits optimized for minimizing molecular ground-state energy within a Variational Quantum Eigensolver (VQE) environment. Unlike previous methods that treat gate placement and parameter optimization as separate steps, HyRLQAS integrates both within a unified hybrid action space, allowing for simultaneous learning of circuit topology and initial parameter values. The team implemented a system where the agent’s actions directly influence both the structure of the quantum circuit and the initial conditions for parameter optimization, fostering a dynamic interplay between topology and parameterization.

Experiments involved training the agent to construct circuits capable of minimizing the energy of molecular Hamiltonians, and the agent’s performance was evaluated by measuring the resulting energy errors and the compactness of the generated circuits. Results demonstrate that HyRLQAS consistently achieves lower energy errors and more compact circuit structures compared to baseline methods that optimize either gate placement or parameter initialization in isolation. Furthermore, the hybrid action space yields superior parameter initializations, producing post-optimization energy distributions with consistently lower minima, indicating a more efficient exploration of the energy landscape. This suggests that HyRLQAS offers a principled pathway toward automated and hardware-efficient quantum circuit design, addressing key challenges in the field of near-term quantum computation.

Hybrid Reinforcement Learning Optimizes Quantum Circuits

This work presents HyRLQAS, a novel hybrid-action reinforcement learning framework designed to simultaneously optimize both the structure and parameters of quantum circuits. By integrating discrete gate placement with continuous parameter initialization within a unified learning process, the team demonstrates improved performance compared to methods that treat these aspects separately. Experiments demonstrate that HyRLQAS consistently achieves lower energy errors and more compact circuit structures compared to methods focusing solely on discrete gate placement or continuous parameter adjustment. The team developed an agent that learns to build circuits optimized for minimizing molecular ground-state energy within a variational quantum eigensolver environment.

Results show that HyRLQAS delivers superior parameter initializations, leading to post-optimization energy distributions with consistently lower minima. This improvement stems from the agent’s ability to reuse optimization knowledge, learning parameter initialization distributions that enhance the efficiency of subsequent optimization steps. Specifically, the research highlights that learning both gate placement and parameter initialization is crucial for success. Initial attempts to have the agent directly learn optimal parameter values without an external optimizer failed to converge, confirming the need for learned parameters as effective starting points for optimization. The team’s experiments demonstrate that HyRLQAS consistently outperforms baseline methods in terms of both energy error and circuit size, showcasing the benefits of this integrated approach to quantum circuit design. The framework addresses the challenge of barren plateaus, a common issue in quantum optimization, by providing more favorable starting points for parameter optimization.

Hybrid Reinforcement Learning Optimizes Quantum Circuits

This work presents HyRLQAS, a novel hybrid-action reinforcement learning framework designed to simultaneously optimize both the structure and parameters of quantum circuits. By integrating discrete gate placement with continuous parameter initialization within a unified learning process, the team demonstrates improved performance compared to methods that treat these aspects separately. Experimental results, obtained within a variational quantum eigensolver environment, show that HyRLQAS consistently achieves lower energy errors and more compact circuit designs. Further analysis reveals that the hybrid approach not only provides better initial parameter settings for rotation gates but also facilitates the discovery of more effective gate placement strategies.

Notably, a warm-up initialization technique accelerated convergence, reducing the number of optimization iterations required on certain molecular systems. While the current framework relies on classical optimizers for final parameter tuning and has not been tested on real quantum hardware, these findings highlight the potential of hybrid-action reinforcement learning to bridge the gap between circuit topology design and parameter optimization. Future research will focus on developing fully end-to-end optimization schemes and extending the evaluation to noisy, hardware-constrained quantum environments.