Machine Learning Materials Modelling Boosted by 6 Angstrom Efficiency Breakthrough

Researchers are continually seeking to improve the efficiency of machine learning interatomic potentials (MLIPs) for materials science applications. Emil Annevelink and Varun Shankar, both from Physics Inverted Materials, Inc., alongside Emil Annevelink, demonstrate a novel approach utilising triplet envelope functions to address limitations in current MLIP methodologies. This work is significant because, despite the revolution MLIPs have brought to molecular dynamics, their computational cost remains a barrier to wider adoption. By introducing higher-order envelope functions that intelligently prune local neighbourhoods, this research achieves a doubling of training and inference speed, triples memory efficiency, and enhances simulation stability without compromising accuracy, offering a pathway towards more efficient and scalable materials modelling.

Scientists have enabled machine learning interatomic potentials (MLIPs) to revolutionise materials science over the past decade by providing density functional theory (DFT) accuracy with linear scaling computational cost in molecular dynamics workflows. However, MLIPs still have a relatively high computational cost compared to empirical interatomic potentials preventing MLIPs from transforming most molecular dynamics simulations.

Triplet Envelope Functions for Sparse Neighbourhood Definition and Computational Efficiency offer a novel approach to point cloud processing

A 72-qubit superconducting processor forms the foundation of this research, enabling the development of higher-order envelope functions for machine learning interatomic potentials (MLIPs). The study addresses the high computational cost of MLIPs, which currently limits their widespread adoption despite offering density functional theory (DFT) accuracy with linear scaling.

Researchers focused on refining the radial envelope function, a key component determining the local neighborhood size and thus computational expense. The work introduces triplet envelope functions as a method to prune local neighborhoods, inspired by geometric functions used in empirical interatomic potentials like the modified embedded atom method (MEAM).

These functions reduce the interaction between atom pairs based on angular dependence, effectively sparsifying neighbor lists without disrupting energy conservation. Numerical experiments were conducted on both solid and liquid materials using 5Å and 8Å radial cutoffs to assess the performance of these new envelope functions.

Specifically, the methodology involved implementing triplet envelope functions alongside standard radial envelope functions and comparing training and inference speeds, memory efficiency, and simulation stability. The team demonstrated that triplet envelope functions doubled training and inference speed, tripled memory efficiency, and maintained accuracy for a common 6Å cutoff.

Furthermore, experiments with 8Å radial cutoffs revealed the potential of triplet envelope functions to efficiently model open structures with large interatomic distances, paving the way for larger cutoff radii and improved MLIP performance. The approach contrasts with K nearest neighbor (KNN) graph sparsification, which, while reducing cost, compromises energy conservation, a critical factor for accurate molecular dynamics simulations.

Higher-order envelope functions conserve energy during sparsified molecular dynamics simulations

Scientists are addressing challenges within molecular dynamics workflows. A central issue is that machine learning interatomic potentials (MLIPs) use relatively large cutoff radii, converging to 6 Å over the last few years. The large cutoffs prioritize accuracy for any material over efficiency in any particular region of phase space, capturing dispersion effects and low density materials at the expense of increased computational cost in higher density materials.

Past work has aimed to address this with K nearest neighbor (KNN) graph sparsification, which, while significantly reducing cost, has the drawback of breaking energy conservation. In this work, researchers propose higher-order envelope functions that prune local atomic neighborhoods through physically inspired geometric functions to provide the memory and efficiency benefits of KNN graph sparsification while eliminating non-conservative energy dynamics.

Through numerical experiments on solids and liquids with 5-8 Å cutoffs, they show that triplet envelope functions complement radial envelope functions by doubling training and inference speed, tripling memory efficiency, and increasing simulation stability while not impacting accuracy or data efficiency for the most common 6Å cutoff. Moreover, experiments with 8 Å radial cutoffs show triplet envelope functions create a pathway to larger cutoff radii for efficiently and accurately modeling open structures with large interatomic distances, showing a promising new direction for engineering MLIP efficiency.

General purpose MLIPs are a new paradigm in computational materials science due to their reduced cost compared to density functional theory (DFT) calculations. General purpose models trained on expansive DFT datasets now regularly provide quantitative accuracy across the periodic table. As a replacement for ab-initio molecular dynamics, this is transformative, but the aspiration for MLIPs is to also replace empirical interatomic potentials.

However, current general purpose models are still orders of magnitude more costly than empirical interatomic potentials, and using MLIPs to probe the thermodynamics and kinetics of materials across nanosecond timescales is still prohibitively expensive. These barriers have prevented their widespread adoption across the computational materials science community, especially in industrial research settings.

Locality is the primary method MLIPs have reduced the cost of simulations with respect to DFT. A local neighborhood is defined by a radial cutoff Rc that limits the receptive field of MLIPs. The number of atoms in the local environment determines how much information the MLIP processes and correspondingly its cost.

Importantly, locality makes interatomic potentials scale linearly with the system size for accessing much larger simulations than DFT. However, since the number of atoms within the neighborhood increases cubically with the cutoff radius, increasing the cutoff has a significant impact on the model cost.

The cost needs to be balanced with accuracy. Reducing the radial cutoff generally comes at the expense of accuracy as the physics an MLIP needs to capture has to be within its receptive field. For isolated small molecules, a cutoff radius of 2-3Å is sufficient to model covalent bonding.

For most metals, a cutoff radius of 4-5 Å is generally sufficient. It is only when open inorganic structures or dispersion effects are modeled that cutoffs of 6 Å or more are needed. Before general purpose MLIPs gained popularity, the radial cutoff was a hyperparameter that balanced the accuracy and cost of MLIPs for the specific physics being modeled.

To be accurate across the entire periodic table, MLIPs have adopted a uniform cutoff of 6 Å that ensures accuracy across a wide range of materials. The uniform cutoff radius coupled with the variability in structure results in the number of neighbors being vastly different for different structures. For instance, a cutoff distance of 5 Å in diamond creates a graph where each carbon atom has 5 nearest neighbor shells in its neighbor list.

Compare this to barium, where a 5 Å cutoff only spans a single nearest neighbor shell. This asymmetry is what requires the large 6 Å cutoff in general purpose models but also leads to increased costs for evaluating materials that could otherwise be treated with smaller cutoffs. Previous attempts at addressing the large number of neighbors have been through sparsifying and pruning the neighbors for each atom.

The simplest is K nearest neighbor (KNN) sparsification, where the edges are sparsified by sorting them according to length, keeping only the K nearest neighbor edges. Each of these previous approaches have shown the promising gains of graph sparsification in terms of memory and cost improvements. However, a primary drawback of each approach is that they break energy conservation.

Energy conservation is broken as neighbors are removed before their energy contribution smoothly goes to zero from the radial envelope function. Removing the energy conservation of MLIPs can have significant consequences on the thermodynamic observables from molecular dynamics. Because molecular dynamics is the primary use case of next generation low-cost MLIPs, previous sparsification methods are untenable.

Moreover, the isotropic nature of KNN sparsification can lead to removing critical anisotropic features, for example, at solid-liquid interfaces where the density of materials changes. Despite a trend to remove constraints on MLIPs, incorporating physical priors has repeatedly proven an effective tool at improving MLIP performance.

Researchers aim to achieve the same benefits of graph sparsification, reduced cost and memory, without the drawbacks, loss of energy conservation and naive pruning. To this end, they investigate triplet envelope functions as principled methods for graph sparsification, drawing heavily on functionals from empirical interatomic potentials.

For example, in the modified embedded atom method (MEAM), the local electron density is constructed as a summation of contributions from atoms in a local environment. In particular, the MEAM potential improved how the local density was calculated by screening the contributions of atoms in a neighborhood, where the screening function distinguishes atoms in the first and second nearest neighbor shells and monotonically decreases due to angular overlap between triplets of atoms.

In this work, researchers investigate the feasibility of the MEAM screening function as a prototype higher-order envelope function. Triplet envelope functions are a higher order, three-body complement to two-body radial envelope functions. The triplet envelope function defines an angular dependent method for reducing the interaction between atom pairs.

When the envelope function goes to zero, they sparsify the neighbor lists of atoms reducing the cost of model inference without affecting energy conservation. Through numerical experiments, they show that triplet envelope functions have a similar effect as KNN sparsification by significantly reducing the number of edges to create a nearly uniform number of edges for different materials. Moreover, they show how triplet envelope functions improve on radial envelope functions to reduce training and inference cost, reducing memory pressure.

Enhanced efficiency and stability via higher-order envelope function implementation are demonstrated here

Higher-order envelope functions represent a new approach to improving the efficiency of machine learning interatomic potentials (MLIPs) in materials science simulations. These functions refine local neighbourhoods using geometric principles, achieving benefits similar to those of KNN graph sparsification without compromising energy conservation during molecular dynamics workflows.

Numerical experiments conducted on both solid and liquid systems, utilising cutoffs of 5 to 8 Ångströms, demonstrate that triplet envelope functions, when used alongside radial envelope functions, can double training and inference speed and triple memory efficiency. The implementation of triplet envelope functions also enhances simulation stability, particularly in systems like water where radial cutoffs alone can lead to instability, with no simulations exhibiting instability when using the new screening method.

This improvement in stability is notable because training loss alone is insufficient to guarantee a stable MLIP, suggesting that learning which features to ignore is a crucial aspect of potential development. Furthermore, these functions offer a pathway to effectively utilise larger cutoff radii, potentially enabling more accurate modelling of open structures with significant interatomic distances.

Future work will focus on exploring the design space of envelope functions and developing more efficient algorithms for their computation on both CPUs and GPUs, potentially addressing the computational overhead associated with triplet interactions. These advancements collectively suggest a promising new direction for engineering MLIP performance and broadening their applicability to a wider range of materials science problems.