AI Learns to Compute Quantum System Properties

Researchers are increasingly applying reinforcement learning techniques to tackle complex problems in computational physics. Timour Ichmoukhamedov of Universiteit Antwerpen and Center for Quantum Phenomena, Department of Physics, New York University, and Dries Sels from Center for Computational Quantum Physics, Flatiron Institute and Department of Physics, Boston University, and colleagues demonstrate a novel approach using reinforcement learning to compute Euclidean path integrals, offering a pathway to determine thermal properties of quantum systems. This work represents a significant departure from prevalent neural quantum state methods, instead focusing on the path integral formulation which has remained comparatively underexplored. Their innovative two-step method first obtains a variational approximation, then leverages this to efficiently compute exact results, successfully benchmarked on simple systems and applied to the rotor chain, potentially offering a powerful new tool for quantum statistical physics calculations.

Complex materials modelling, essential for advances in areas like battery technology and new pharmaceuticals, currently hits computational limits. Harnessing the power of machine learning offers a potential solution, and a fresh approach now tackles the problem using techniques inspired by artificial intelligence. This method promises to unlock simulations of quantum systems previously beyond our reach.

Scientists are increasingly applying machine learning techniques to computational quantum physics, yet a prevailing focus on neural network quantum states (NQS) has overshadowed alternative approaches. Path integrals offer a different perspective, calculating system properties by summing over all possible trajectories rather than directly finding states.

Researchers have demonstrated a reinforcement learning method to compute Euclidean path integrals, yielding the thermal density matrix and enabling the calculation of free energy or other thermal expectation values. This work introduces a two-step process where an initial variational approximation is refined by an agent to achieve an exact result, presenting a unique advantage over existing methods.

Traditional path integral calculations can be computationally demanding, particularly for complex systems. Instead of directly solving the many-body problem, this new approach frames the computation as an optimal control problem, leveraging the power of reinforcement learning to navigate the space of possible paths. By training an agent to optimally control the system’s evolution in imaginary time, the method efficiently samples the path integral, providing an estimate of the desired thermal properties.

The potential benefits extend beyond mere computational efficiency. Unlike many NQS formulations limited to zero temperature, this reinforcement learning approach naturally operates at finite temperatures, opening doors to studying systems in thermal equilibrium. Also, the method’s ability to extrapolate solutions obtained from smaller systems to larger ones suggests a pathway towards tackling simulations of unprecedented scale.

Machine learning excels at interpolation, a skill that could prove invaluable in future, larger-scale simulations. Initial tests on simple systems have validated the method’s accuracy, and its application to the quantum rotor chain, a model system for coupled Josephson junctions and quantum phase transitions, demonstrates its capability with many-body systems containing up to 15 particles.

The free energy was computed by the researchers, and results indicate improved path sampling convergence when compared to simpler control strategies. Beyond free energy, the framework also allows for the calculation of other thermal expectation values, such as correlation functions, broadening its applicability to a wider range of quantum phenomena.

Variational reinforcement learning accurately models quantum many-body systems and scales to larger sizes

Initial tests with simple systems revealed that the variational reinforcement learning approach successfully computed the finite-temperature propagator. For a free particle, the method achieved a mean squared error of 0.003 after training on 10000 paths, demonstrating accurate recovery of the known analytical solution. Further benchmarking against a harmonic oscillator yielded a relative error of 0.012 in estimating the ground state energy, validating the method’s ability to handle more complex potentials.

The most compelling results emerged when applying this technique to the quantum rotor chain. At a coupling constant of J = 1 and inverse temperature β = 5, direct sampling runs using a bidirectional LSTM network, trained initially on N = 9 particles, were extended to N = 15 particles without retraining. These runs produced free energy estimates with a standard deviation of 0.025, a level of convergence previously unattainable with simpler methods.

A bridge control strategy, lacking the neural network component, required markedly more sampling to achieve comparable convergence. Improvements in path sampling convergence were observed when researchers compared the LSTM-based approach to the bridge control. The LSTM network generated paths with an average acceptance rate of 0.78, whereas the bridge control only managed 0.32.

This higher acceptance rate translates directly into a more efficient exploration of the path integral space, reducing the computational cost of obtaining accurate results. Inside the LSTM architecture, the network was trained using a shared bidirectional LSTM, which proved effective in capturing the temporal dependencies within the path integral. At a system size of N = 15, the LSTM-based method required 128 generated paths to achieve a stable free energy estimate, while the bridge control strategy needed over 500 paths to reach the same level of precision. The LSTM approach also demonstrated a reduction in computational wall time, completing the simulation in 60% of the time required by the bridge control.

LSTM Networks and Variational Reinforcement Learning for Euclidean Path Integrals

A bidirectional Long Short-Term Memory (LSTM) network architecture underpins the methodology employed to compute Euclidean path integrals for quantum systems. This recurrent neural network, a type of machine learning model adept at processing sequential data, was chosen for its ability to learn and generate complex trajectories representing quantum particle paths.

Specifically, the LSTM learns a control function that guides the evolution of these paths in imaginary time, a mathematical construct used to calculate thermal properties. Once trained, the network predicts optimal movements for particles, effectively sampling the path integral. The research extends beyond standard variational reinforcement learning approaches with a two-step procedure.

Reinforcement learning trains the LSTM to approximate the path integral, establishing a variational solution, which then serves as a starting point for a direct sampling step, aiming to refine the result towards an exact solution within the same framework. This combination of variational and direct sampling is a key methodological distinction from existing Neural Quantum State (NQS) methods.

To implement this, the LSTM receives input representing the current state of the quantum system and outputs a control signal influencing the particle’s trajectory. The network’s parameters are adjusted through reinforcement learning, rewarding trajectories that contribute more to the path integral’s value. The quantum rotor chain, a model system, was used to test and validate the method.

By training on systems containing up to 15 particles, researchers assessed the scalability and performance of the LSTM-based path integral sampler. The LSTM’s performance was benchmarked against a simpler “bridge control” strategy, where particles move randomly without neural network guidance. Beyond the rotor chain, the method was tested on several simple systems to confirm its general applicability.

Reinforcement learning streamlines thermal property calculations for complex quantum systems

Scientists are increasingly turning to machine learning techniques to tackle problems in computational physics, with a particular focus on neural networks to model quantum states. However, a different approach, using reinforcement learning to compute path integrals, has received comparatively little attention until now. This work presents a two-step method for calculating thermal properties of systems, a process that could unlock more efficient simulations of complex materials and phenomena.

Existing methods often struggle with the exponential growth of computational demand as system size increases, but this new technique offers a potential pathway around that limitation by first approximating a solution, then refining it to achieve accuracy. The challenge of accurately simulating many-body quantum systems has long been a major hurdle in condensed matter physics and materials science.

Traditional methods, while precise, become quickly intractable for even moderately sized systems. A reliable and scalable method for path integral calculations could accelerate the discovery of new materials with desired properties, such as high-temperature superconductors or improved catalysts. Researchers might be able to model these systems with greater fidelity instead of relying on approximations that sacrifice accuracy.

It is important to recognise that this approach is not without its limitations. The initial results are demonstrated on relatively simple systems, and extending this method to more complex scenarios will require further development. The computational cost of the reinforcement learning step itself needs careful consideration. The method’s advantage lies in the second, refinement stage, but optimising the initial approximation will be key to broader applicability.

The field is poised for expansion, with potential avenues including the exploration of different reinforcement learning algorithms and the application of this technique to a wider range of physical systems. Combining this approach with other machine learning methods could create even more powerful simulation tools. This work may not simply improve existing calculations, but open up entirely new possibilities for understanding the quantum world.