Scientists Compress Complex Chemical Data by 99 Percent with New Method

Scientists at the King’s College London, led by Kemal Atalar, have developed a new protocol to compress two-body reduced density matrices (2RDMs), addressing substantial memory demands inherent in computational chemistry calculations. The research team achieves approximately 99% data reduction for complex systems such as octane, with scaling that is effective for system size and reduces memory costs from quartic to quadratic. This compression preserves essential physical properties vital for understanding electronic states and opens avenues for simulating larger, more complex molecular systems, as demonstrated through nonadiabatic dynamics simulations of photoexcited H$_{28}$ chains.

Quadratic scaling of two-body reduced density matrices unlocks complex molecular simulations

Approximately 99% compression of the coupled-cluster 2RDM for octane has been achieved, marking a significant advance in managing the computational cost of electronic structure calculations. Traditionally, the storage requirements for two-body reduced density matrices, essential for accurately modelling electron interactions and correlations within molecules, scaled to the power of four with system size, a prohibitive factor for larger systems. This quartic scaling arises from the need to represent all possible pairs of electron interactions, leading to a rapid increase in memory usage as the number of electrons and orbitals grows. The new method reduces this scaling to quadratic, effectively circumventing the memory bottleneck and unlocking simulations previously limited by computational resources. Two-body RDMs are crucial because they contain all the information needed to calculate static and dynamic properties of the electronic structure, without requiring the full many-body wavefunction.

Statistically resolved observables from nonadiabatic molecular dynamics simulations of photoexcited H$_{28}$ chains demonstrate the technique’s practical application to complex systems. Nonadiabatic dynamics are particularly demanding, requiring accurate representation of excited state potential energy surfaces and efficient propagation of the molecular wavefunction. A linear scaling of effective rank with system size was observed during compression of these matrices, a finding relevant for modelling electron interactions in materials, particularly those exhibiting strong electron correlation. This linear scaling suggests that the method remains effective even as the system size increases, offering a promising pathway for tackling larger and more challenging materials. Analysis of alkane chains revealed that only 490 low-rank vectors were needed to achieve chemical accuracy for a molecule with 202 orbitals, demonstrating the substantial reduction in computational effort.

A full-rank representation would require 40,804 vectors, highlighting the efficiency gains as systems increase in size and demonstrating approximately 99% compression for octane. The technique maintains accuracy by preserving essential two-body correlations, scaling quadratically with system size for a fixed relative energy error, and operates independently of basis set locality. This independence is crucial, as it allows the method to be applied with various basis sets without requiring modifications. Achieving substantially sub-milli-Hartree accuracy, however, requires larger ranks, and the prefactor governing this scaling remains a practical consideration for wider application. The prefactor reflects the overhead associated with the compression algorithm itself and its impact on computational time. Future research will focus on refining this prefactor and exploring the limits of accuracy achievable with this compression method, potentially through the development of more sophisticated compression algorithms or adaptive rank selection strategies.

Reduced density matrix compression enables simulations of larger molecular systems

A pathway to more efficient molecular simulations has been unlocked by addressing the computational burden of representing electron interactions. The strong performance of this technique across diverse chemical systems, particularly those with strongly correlated electrons, where traditional methods often struggle, is yet to be fully explored. Strongly correlated systems, such as transition metal complexes or systems undergoing bond breaking, require more accurate descriptions of electron correlation, increasing the computational cost. Compressing two-body reduced density matrices, which represent electron behaviour, delivers a substantial reduction in the computational resources needed for molecular simulations. The 2RDM provides a compact representation of the electronic state, capturing the essential information about electron correlations without the need to explicitly calculate the full many-body wavefunction.

Consequently, larger and more intricate molecular systems become accessible, potentially accelerating discoveries in materials science and drug design. In materials science, this could lead to the development of new materials with tailored properties, while in drug design, it could enable the accurate prediction of drug-target interactions and the design of more effective therapies. This new method for compressing the data needed to simulate molecular behaviour has been developed, building upon existing techniques for tensor decomposition and low-rank approximation. By representing these complex matrices in a lower-rank format, memory requirements have been reduced from a computationally expensive scaling to a more practical quadratic level. This advance enables statistically resolved simulations of molecular dynamics, previously limited by system size; simulations of photoexcited hydrogen chains demonstrated this capability, providing a benchmark for future applications and investigations into more complex molecular systems. The ability to perform statistically resolved simulations is crucial for obtaining reliable results, as it allows for the averaging over multiple trajectories to reduce statistical noise and improve the accuracy of the predictions. The method’s ability to handle nonadiabatic dynamics is particularly significant, as these simulations are essential for understanding photochemical processes and energy transfer in molecular systems.

The researchers successfully compressed two-body reduced density matrices, a key component in modelling electron behaviour, while maintaining chemical accuracy. This compression reduces the memory cost of simulations from a quartic to a quadratic scaling, meaning significantly larger molecular systems can now be studied. For example, the method achieved approximately 99% compression for the coupled-cluster 2RDM of octane. The authors demonstrated this capability through simulations of photoexcited H$_{28}$ chains, and developed metrics to systematically control the decomposition process.