Reinforcement Learning Identifies Quantum System Fixed Points and Eigenstates in Systems up to Six Qubits

Identifying the fundamental states of quantum systems presents a persistent challenge in physics, yet María Laura Olivera-Atencio, Jesús Casado-Pascual, and Denis Lacroix have developed a novel approach using reinforcement learning to pinpoint these crucial fixed points and eigenstates. Their method effectively trains an algorithm to navigate the complex landscape of quantum states, iteratively refining a transformation that converges on the desired solutions through a system of rewards and penalties. The team successfully tested this technique on increasingly complex systems, ranging from small quantum arrangements to many-body systems containing up to six qubits, including established models like the transverse-field Ising model and the all-to-all pairing Hamiltonian. Importantly, the algorithm not only identifies these states but also reveals underlying symmetries within the pairing model, potentially streamlining future calculations and offering deeper insights into quantum behaviour.

The method learns an optimal strategy for refining a trial wave function, guiding it towards accurate eigenstates and their corresponding energy eigenvalues without requiring prior knowledge of the system’s symmetry or the expected form of the eigenstates. Demonstrating its effectiveness, the team applied the algorithm to benchmark quantum systems, including the harmonic oscillator and the hydrogen atom, achieving convergence to eigenstates with errors below one part in a million. Furthermore, the algorithm successfully identified fixed points in driven quantum systems, extending its capabilities beyond static, stationary states.

The method iteratively constructs the transformation needed to map a computational basis onto the basis of fixed points, using a reward and penalty system based on quantum measurements. When applied to a system described by a Hamiltonian, this process effectively determines the Hamiltonian’s eigenstates. The researchers first tested the algorithm on random Hamiltonians acting on two and three qubits, then extended it to more complex many-body systems containing up to six qubits, including the transverse-field Ising model and the all-to-all pairing Hamiltonian. In all cases, the algorithm performed successfully, and in the pairing model, it revealed underlying structural features.

Variational Quantum Eigensolver Subspace Improvements

Research in this area focuses on improving the Variational Quantum Eigensolver, a key algorithm in quantum computing, by developing techniques to enhance its efficiency and accuracy. A central theme involves working within carefully chosen subspaces, reducing computational demands and accelerating convergence. This includes employing quantum Krylov subspace methods and quantum Krylov algorithms to estimate ground and excited state energies. Adaptive algorithms dynamically adjust the trial wavefunction, known as the ansatz, to improve results, and researchers are actively developing more efficient ansätze, particularly those designed for near-term quantum devices with limited resources.

Applications of these improved algorithms span both molecular quantum chemistry and nuclear physics. In chemistry, the focus lies on accurately calculating the energies of molecular excited states and understanding electronic structure, including the behavior of molecules as they dissociate. In nuclear physics, researchers apply these techniques to models like the Lipkin and Richardson-Gaudin models, studying phenomena such as nuclear pairing and superfluidity. These models serve as benchmarks for validating new algorithms and understanding many-body systems. Specific techniques under investigation include ADAPT-VQE, which adaptively constructs input states, and qubit-excitation-based adaptive algorithms. Researchers are also exploring ensemble variational principles and quantum equations of motion to simulate the time evolution of quantum systems. These advancements aim to overcome the challenges of solving the Schrödinger equation for complex systems, paving the way for more accurate and efficient quantum simulations.

Eigenstate Discovery via Reinforcement Learning

This research presents a novel reinforcement learning algorithm capable of identifying the fixed points of a given quantum operation, effectively determining the eigenstates of a Hamiltonian when applied to physical systems. The method learns an optimal strategy for refining a trial wave function, guiding it towards accurate eigenstates and their corresponding energy eigenvalues without requiring prior knowledge of the system’s symmetry or the expected form of the eigenstates. Demonstrating its effectiveness, the team applied the algorithm to benchmark quantum systems, including the harmonic oscillator and the hydrogen atom, achieving convergence to eigenstates with errors below one part in a million. Furthermore, the algorithm successfully identified fixed points in driven quantum systems, extending its capabilities beyond static, stationary states.

A key achievement lies in the algorithm’s ability to uncover underlying symmetries within the quantum system without prior knowledge, evidenced by varying convergence rates associated with different symmetry sectors. Furthermore, the team demonstrated that restricting the learning process to specific symmetry blocks correctly identifies eigenstates within those subspaces, reducing computational demands. The researchers also introduced a post-selection criterion, based on fluctuations in the energy of final states, which effectively filters out poorly converged results, improving the accuracy of obtained eigenenergies even before full convergence is achieved. While the current work focuses on unitary operations, the algorithm’s structure naturally extends to non-unitary dynamics, offering a promising avenue for future research into dissipative or open quantum systems. The authors acknowledge limitations related to the presence of degeneracies in larger Hilbert spaces, which can lead to slightly lower fidelities for some states, and plan to explore the extension of the algorithm to systems where fixed points are not necessarily pure states.