Greplová and Colleagues Introduces Proximal Wavefunction Optimisation for Neural Quantum States

Scientists at Université de Montréal, in collaboration with Leiden University and Delft University of Technology, have presented a new approach to optimising neural quantum states, a flexible framework for approximating complex quantum systems. Minimising variational energy aligns with advantage policy-gradient problems, resulting in Proximal Wavefunction Optimisation (PWO), a trust-region algorithm designed for efficient and stable training. PWO avoids computationally expensive matrix inversions and sample reuse, achieving improved convergence and stability compared to existing methods across various spin systems. The team successfully fine-tuned a substantial 1.5 billion-parameter model, extending the scale of neural quantum state optimisation by over three orders of magnitude and enabling increasingly complex quantum simulations.

Proximal Wavefunction Optimisation scales neural quantum state parameterisation to 1.5 billion

Researchers have achieved neural quantum state optimisation with a staggering 1.5 billion parameters, surpassing previous scales by over three orders of magnitude. This breakthrough relied on Proximal Wavefunction Optimisation, or PWO, a novel algorithm linking energy minimisation with reinforcement learning to stabilise and accelerate the training of autoregressive neural quantum states. Neural quantum states (NQS) represent a promising avenue for tackling the complexities of quantum many-body problems, offering a flexible parameterisation of the wavefunction which describes the quantum state of a system. Traditional methods for solving these problems, such as exact diagonalisation, become computationally intractable as the system size increases, motivating the development of approximate methods like NQS. Autoregressive models, central to this work, function similarly to language models, predicting the next element in a sequence to efficiently sample the system’s probability distribution, offering a distinct advantage in generating independent samples from the Born distribution, a crucial requirement for accurate quantum simulations. This avoids the autocorrelation and mixing issues inherent in Markov chain Monte Carlo methods commonly used in quantum computation.

PWO avoids computationally expensive matrix inversions and sample reuse, delivering improved convergence and stability across diverse spin systems, including challenging frustrated models. The Université de Montréal team, alongside their colleagues, achieved substantial gains in stability and convergence when optimising neural quantum states using the Proximal Wavefunction Optimisation (PWO) algorithm on one- and two-dimensional spin systems. PWO outperformed established methods such as Adam, minSR, and SPRING on challenging frustrated models, notably the Heisenberg J1, J2 chain, where six of ten minSR runs failed due to numerical instability. A RWKV-7 model containing 1.5 billion parameters was successfully fine-tuned, representing an improvement of over three orders of magnitude in scale compared to previous neural quantum state optimisation efforts; this was demonstrated by achieving a relative error of 10−7 on the Ising model in approximately five minutes. The Ising model, a fundamental model in statistical mechanics, serves as a benchmark for evaluating the performance of these optimisation algorithms. The ability to achieve such high accuracy in a short timeframe highlights the efficiency of PWO.

Trust-region reinforcement learning for scalable neural quantum state optimisation

Proximal Wavefunction Optimisation addresses the challenge of training complex neural quantum states by adopting principles from reinforcement learning; the process of minimising a system’s energy is viewed as akin to refining a strategy for the best outcome. In reinforcement learning terms, the neural quantum state represents a ‘policy’ and the energy represents the ‘reward’. This connection allowed the application of trust-region optimisation, a technique for finding the best solution by taking small, cautious steps, ensuring stable progress during training. Trust-region methods constrain the parameter updates within a defined region, preventing large, destabilising changes to the wavefunction. PWO avoids computationally intensive matrix inversions, streamlining calculations and enabling the reuse of samples to accelerate learning; this is particularly important when dealing with autoregressive models, which predict the next element in a sequence, building upon previous elements. The computational cost of matrix inversions scales poorly with system size, making them a bottleneck in many quantum simulation algorithms. By avoiding these inversions, PWO significantly improves scalability.

The team addressed challenges in optimising autoregressive models, which offer exact sampling but have previously suffered from unstable training; Adam is scalable but lacks geometric awareness, while stochastic reconfiguration is computationally expensive. Adam, a popular optimisation algorithm, relies on gradient information but doesn’t explicitly consider the geometry of the function space, potentially leading to inefficient updates. Stochastic reconfiguration, while capable of finding good solutions, involves significant computational overhead. Consequently, Proximal Wavefunction Optimisation, or PWO, was developed as a trust-region algorithm clipping changes in probability ratios, avoiding computationally intensive matrix inversions and enabling sample reuse for faster learning. Clipping probability ratios ensures that the updates remain within the trust region, preventing drastic changes to the wavefunction and promoting stability. This careful control over the optimisation process is crucial for achieving accurate and reliable results.

Training neural networks unlocks high-precision quantum system modelling

A new level of precision in simulating quantum systems has been unlocked, employing neural quantum states and a technique called Proximal Wavefunction Optimisation to train models with an unprecedented 1.5 billion parameters. This advancement promises to accelerate materials discovery and fundamental physics research by allowing researchers to model increasingly complex interactions between quantum particles. Accurate modelling of quantum systems is essential for understanding the behaviour of materials at the atomic level, designing new materials with desired properties, and exploring fundamental questions in physics, such as high-temperature superconductivity and quantum magnetism. The team’s current work focuses heavily on autoregressive models, a specific type of neural network architecture, however.

The development of Proximal Wavefunction Optimisation, or PWO, establishes a new approach to training neural quantum states by drawing parallels between energy minimisation and reinforcement learning. This connection facilitated the creation of a trust-region algorithm that enhances both the stability and speed of optimising these complex models, vital for simulating quantum systems. Successfully fine-tuning a model containing 1.5 billion parameters demonstrates a significant leap in scale, exceeding previous efforts by over three orders of magnitude and enabling the investigation of more intricate quantum phenomena. The ability to handle such large models opens up new possibilities for studying systems that were previously inaccessible to neural quantum state methods, potentially leading to breakthroughs in our understanding of the quantum world. Further research will likely focus on extending PWO to even larger systems and exploring its application to a wider range of quantum problems.

The researchers successfully trained a neural network with 1.5 billion parameters to model quantum systems using a new optimisation technique, Proximal Wavefunction Optimisation. This method improves the stability and speed of training these complex models, allowing for more accurate simulations of quantum interactions. The increased scale of modelling, exceeding previous work by three orders of magnitude, enables investigation of more intricate quantum phenomena and supports advances in materials discovery and fundamental physics research. The authors intend to extend this work to larger systems and a broader range of quantum problems.