Mixed Precision Advances Variational Monte Carlo with 64-Bit Error Bounds

Scientists are increasingly exploring how to harness the power of modern hardware to tackle complex scientific problems, and a new study investigates the potential of mixed-precision arithmetic within neural network-based Variational Monte Carlo (VMC) methods. Massimo Solinas from the University of Regensburg, Agnes Valenti and Nawaf Bou-Rabee from the Flatiron Institute, alongside Roeland Wiersema, demonstrate that significant computational gains are possible by utilising lower precision formats , traditionally avoided due to concerns about accuracy , without compromising results. Their research establishes analytical bounds on errors introduced by reduced precision in Metropolis-Hastings Monte Carlo methods and validates these bounds specifically within VMC, revealing that sampling can often be performed in half precision. This work is significant because it provides a theoretical framework for applying mixed-precision techniques to machine learning algorithms reliant on Markov Chain Monte Carlo, ultimately paving the way for more scalable and energy-efficient simulations of many-body systems.

This breakthrough reveals that substantial portions of the VMC algorithm, specifically the sampling of the quantum state, can be executed in half precision without compromising accuracy, paving the way for more scalable and energy-efficient simulations. The study establishes a theoretical framework applicable to machine-learning approaches reliant on MCMC sampling, offering a means to assess the viability of mixed-precision arithmetic in diverse contexts.

Researchers began by formulating general analytical bounds quantifying the error introduced by reduced precision during Metropolis-Hastings MCMC sampling, a cornerstone of many computational methods. Experiments show that the sampling process, a computationally intensive step in VMC, can be performed using half precision without introducing unacceptable inaccuracies, offering a substantial performance boost. This approach allows for a reduction in memory footprint and improved energy efficiency, particularly when leveraging hardware accelerators like Graphics Processing Units (GPUs).
The work opens new avenues for accelerating simulations of quantum many-body systems, which are vital across numerous scientific disciplines. The ability to efficiently simulate these systems is crucial for advancing our understanding of quantum phases in physics, elucidating molecular interactions in quantum chemistry, and designing novel materials with tailored properties in materials science. Neural Quantum States (NQS) offer a promising approach to overcome the exponential scaling of computational demands in these simulations by compressing the quantum wavefunction into a neural network and optimizing it using VMC. The research demonstrates that the MCMC sampling component of this process can be significantly expedited through the use of lower-precision formats, without sacrificing the accuracy of the resulting ground-state approximation.

Furthermore, the team’s findings extend beyond NQS, providing a rigorous framework for quantifying the accuracy of low-precision sampling applicable to Bayesian learning and energy-based models. Empirical evaluations, utilising both feed-forward networks and residual convolutional neural networks, reveal that the proposed mixed-precision VMC framework can accelerate sampling by up to 3.5x without diminishing the training performance of the neural networks. Specifically, they found that the sampling stage of the algorithm can often be performed using half-precision without compromising accuracy, offering substantial gains in computational speed and energy efficiency. This work significantly advances the application of machine learning techniques to scientific computing by providing a theoretical framework for assessing the viability of mixed-precision arithmetic in MCMC-reliant algorithms.
In the context of VMC, the researchers successfully implemented mixed-precision strategies, enabling more scalable and energy-efficient simulations of many-body systems. However, the authors acknowledge that their theoretical analysis relies on a specific noise model and proposal distribution, suggesting that extending the approach to more complex scenarios requires further investigation. Future research could focus on accelerating other computationally intensive parts of VMC, such as local energy calculations and preconditioning steps, potentially yielding even greater performance improvements, particularly with advancements in mixed-precision linear algebra. Ultimately, this research highlights mixed precision as a promising, yet relatively unexplored, tool for enhancing MCMC-driven scientific computing, especially as reliance on GPU hardware increases.