Mlipaudit Benchmarks Machine Learned Interatomic Potentials for Accurate, Cost-Effective Molecular Simulations

The increasing demand for accurate and efficient atomistic simulations drives the development of machine-learned interatomic potentials, or MLIPs, which promise to model complex molecular systems at a significantly reduced computational cost. However, a lack of standardised evaluation tools hinders consistent comparison and application of these models. To address this challenge, Leon Wehrhan, Lucien Walewski, Marie Bluntzer, and colleagues at InstaDeep introduce MLIPAudit, a comprehensive and open benchmarking suite. This new tool assesses the accuracy of MLIPs across diverse application tasks, including organic compounds, liquids, proteins and peptides, and features a continuously updated leaderboard for direct performance comparison. By establishing a unified and transparent framework for validation, MLIPAudit fosters reproducibility and accelerates progress in the development of reliable MLIPs for complex molecular systems.

At present, the computational cost of traditional electronic structure methods limits their widespread application. Although collections of models and categorisation efforts have emerged recently, consistently discovering, comparing, and applying machine learning interatomic potentials (MLIPs) across diverse scenarios remains difficult. The field currently lacks a standardised and comprehensive framework for evaluating MLIP performance. To address this, researchers introduce MLIPAudit, an open, curated, and modular benchmarking suite designed to assess the accuracy of MLIP models across a variety of application tasks. MLIPAudit offers a diverse collection of benchmark systems, including small organic compounds, molecular liquids, proteins, and flexible peptides, along with pre-computed results for a range of pre-trained and publicly available MLIPs.

Machine Learning Potentials and Validation Datasets

The field is witnessing a growing trend towards machine learning potentials (MLPs) as alternatives to traditional force fields. Numerous datasets and software tools support the training, validation, and application of these MLPs. Spice provides a dataset of drug-like molecules and peptides, while Transition1x focuses on reactive MLPs. Wiggle150 offers highly strained conformers for challenging model development. Chgnet is a pre-trained universal neural network potential, and extensive datasets of reactants, products, and transition states, derived from quantum chemistry calculations, are also available.

A large dataset of 134,000 molecules further supports quantum chemistry structure analysis. Software frameworks like MLIPAudit serve as benchmarking tools, and ORCA includes the Nudged Elastic Band (NEB) method for identifying transition states. Specific MLP approaches, such as Ani-1, an extensible neural network potential, and TorsionNet, which predicts torsional energy profiles, are also gaining prominence. Alongside MLPs, traditional force fields continue to be refined. The Open Force Field (OFF) project, encompassing versions 1.

0 and 2. 0, represents a collaborative effort to develop a general-purpose force field. The Amber Force Field, alongside the General Amber Force Field (GAFF), provides established methods for molecular dynamics simulations. Various water models, including SPC/E, TIP3P, TIP4P, TIP5P, and the Jorgensen model, are essential for accurately simulating aqueous systems. These resources are complemented by quantum chemistry and electronic structure methods, such as Density Functional Theory (DFT) and the NEB method, which provide the data needed to train and validate these models.

Molecular dynamics simulations and analysis techniques, including the calculation of radial distribution functions (RDFs) for liquids like water, carbon tetrachloride, methanol, and acetonitrile, are crucial for characterising molecular behaviour. Datasets focused on specific molecular systems, such as extensive water datasets including RDFs and diffraction measurements, alongside data for carbon tetrachloride, methanol, and acetonitrile, provide valuable benchmarks. Tautobase, an open tautomer database, and Spice, the dataset of drug-like molecules and peptides, offer resources for specific applications. The overarching themes highlight a shift towards MLPs, the importance of rigorous validation, the central role of water modelling, and a focus on achieving DFT-level accuracy at a reduced computational cost, all driven by open science and collaborative development.

MLIPAudit Benchmarks Interatomic Potential Performance

Scientists have developed MLIPAudit, a comprehensive benchmarking suite designed to rigorously assess the performance of machine-learned interatomic potentials (MLIPs). This new tool addresses a critical need within the field, as existing methods often focus solely on energy and force errors, overlooking crucial aspects like model stability and transferability. The work introduces a standardised framework for evaluating MLIPs across diverse applications, moving beyond simple error metrics to reflect real-world simulation demands. MLIPAudit incorporates a diverse collection of benchmark systems, encompassing small organic compounds, molecular liquids, and biomolecules.

The suite provides reference datasets and tools for systematic validation and comparison of different MLIP models, fostering reproducibility and transparency in the field. The benchmarking suite evaluates models not only on energy and force accuracy, but also on their performance in predicting properties relevant to downstream applications. Tests include assessments of model stability, transferability, and robustness, providing a holistic evaluation of model capabilities. Researchers demonstrate the utility of MLIPAudit by applying it to a series of internal and publicly available models, including UMA-Small, MACE-OFF, and MACE-MP.

The resulting data provides a clear comparison of model performance across various benchmarks, enabling informed selection of the most appropriate model for specific simulations. MLIPAudit’s modular design allows for easy expansion and contribution from the wider scientific community. The suite is freely available on GitHub and PyPI under the Apache License 2. 0, and a continuously updated leaderboard, accessible on HuggingFace, tracks performance across benchmarks. This open-source approach promotes collaboration and accelerates progress in the development and deployment of accurate and efficient MLIPs for complex molecular systems.

Standardised Benchmarking of Interatomic Potentials

MLIPAudit represents a significant advance in the field of machine-learned interatomic potentials, offering a comprehensive and open benchmarking suite for evaluating model performance. Researchers developed a curated repository of benchmarks encompassing small molecules, molecular liquids, and biomolecules, addressing the existing need for standardised and reproducible evaluation protocols. By shifting the focus from model-centric testing to systematic validation and comparison, MLIPAudit facilitates a more rigorous assessment of accuracy, transferability, and robustness in these increasingly important predictive models. The suite provides a diverse collection of benchmark systems and reference datasets, enabling direct comparison of different MLIP models on a common set of tasks.

This standardised approach allows researchers to move beyond simple error metrics, such as energy and force errors, and assess performance in a manner that reflects real-world simulation demands. The team acknowledges a current limitation in the scope of benchmarks, specifically noting that the current suite focuses on a limited range of systems and properties. Future development will expand the benchmark suite to include a wider variety of materials and application scenarios, further enhancing its utility for the broader scientific community. The openly available library and leaderboard, accessible via GitHub, PyPI, and HuggingFace, promote transparency and collaboration in the ongoing development and deployment of machine-learned interatomic potentials.