Search

The use of machine learning to estimate the energy of a group of atoms, and the forces that drive them to more stable configurations, have revolutionized the fields of computational chemistry and materials discovery. In this domain, rigorous enforcement of symmetry and conservation laws has traditionally been considered essential. For this reason, interatomic forces are usually computed as the derivatives of the potential energy, ensuring energy conservation. Several recent works have questioned this physically-constrained approach, suggesting that using the forces as explicit learning targets yields a better trade-off between accuracy and computational efficiency - and that energy conservation can be learned during training. The present work investigates the applicability of such non-conservative models in microscopic simulations. We identify and demonstrate several fundamental issues, from ill-defined convergence of geometry optimization to instability in various types of molecular dynamics. Contrary to the case of rotational symmetry, lack of energy conservation is hard to learn, control, and correct. The best approach to exploit the acceleration afforded by direct force evaluation might be to use it in tandem with a conservative model, reducing - rather than eliminating - the additional cost of backpropagation, but avoiding most of the pathological behavior associated with non-conservative forces.

Most atomistic machine learning (ML) models rely on a locality ansatz, and decompose the energy into a sum of short-ranged, atom-centered contributions. This leads to clear limitations when trying to describe problems that are dominated by long-range physical effects - most notably electrostatics. Many approaches have been proposed to overcome these limitations, but efforts to make them efficient and widely available are hampered by the need to incorporate an ad hoc implementation of methods to treat long-range interactions. We develop a framework aiming to bring some of the established algorithms to evaluate non-bonded interactions - including Ewald summation, classical particle-mesh Ewald (PME), and particle-particle/particle-mesh (P3M) Ewald - into atomistic ML. We provide a reference implementation for PyTorch as well as an experimental one for JAX. Beyond Coulomb and more general long-range potentials, we introduce purified descriptors which disregard the immediate neighborhood of each atom, and are more suitable for general long-ranged ML applications. Our implementations are fast, feature-rich, and modular: They provide an accurate evaluation of physical long-range forces that can be used in the construction of (semi)empirical baseline potentials; they exploit the availability of automatic differentiation to seamlessly combine long-range models with conventional, local ML schemes; and they are sufficiently flexible to implement more complex architectures that use physical interactions as building blocks. We benchmark and demonstrate our torch-pme and jax-pme libraries to perform molecular dynamics simulations, to train range-separated ML potentials, and to evaluate long-range equivariant descriptors of atomic structures.

In computational physics, chemistry, and biology, the implementation of new techniques in a shared and open source software lowers barriers to entry and promotes rapid scientific progress. However, effectively training new software users presents several challenges. Common methods like direct knowledge transfer and in-person workshops are limited in reach and comprehensiveness. Furthermore, while the COVID-19 pandemic highlighted the benefits of online training, traditional online tutorials can quickly become outdated and may not cover all the software's functionalities. To address these issues, here we introduce ``PLUMED Tutorials'', a collaborative model for developing, sharing, and updating online tutorials. This initiative utilizes repository management and continuous integration to ensure compatibility with software updates. Moreover, the tutorials are interconnected to form a structured learning path and are enriched with automatic annotations to provide broader context. This paper illustrates the development, features, and advantages of PLUMED Tutorials, aiming to foster an open community for creating and sharing educational resources., Comment: 26 pages, 5 figures

The widespread application of machine learning (ML) to the chemical sciences is making it very important to understand how the ML models learn to correlate chemical structures with their properties, and what can be done to improve the training efficiency whilst guaranteeing interpretability and transferability. In this work, we demonstrate the wide utility of prediction rigidities, a family of metrics derived from the loss function, in understanding the robustness of ML model predictions. We show that the prediction rigidities allow the assessment of the model not only at the global level, but also on the local or the component-wise level at which the intermediate (e.g. atomic, body-ordered, or range-separated) predictions are made. We leverage these metrics to understand the learning behavior of different ML models, and to guide efficient dataset construction for model training. We finally implement the formalism for a ML model targeting a coarse-grained system to demonstrate the applicability of the prediction rigidities to an even broader class of atomistic modeling problems., Comment: 15 pages, 9 figures, submitted to Faraday Discussions: "Data-driven discovery in the chemical sciences", 2024

The electronic density of states (DOS) provides information regarding the distribution of electronic energy levels in a material, and can be used to approximate its optical and electronic properties and therefore guide computational material design. Given its usefulness and relative simplicity, it has been one of the first electronic properties used as target for machine-learning approaches going beyond interatomic potentials. A subtle but important point, well-appreciated in the condensed matter community but usually overlooked in the construction of data-driven models, is that for bulk configurations the absolute energy reference of single-particle energy levels is ill-defined. Only energy differences matter, and quantities derived from the DOS are typically independent on the absolute alignment. We introduce an adaptive scheme that optimizes the energy reference of each structure as part of the training process, and show that it consistently improves the quality of ML models compared to traditional choices of energy reference, for different classes of materials and different model architectures. On a practical level, we trace the improved performance to the ability of this self-aligning scheme to match the most prominent features in the DOS. More broadly, we believe that this work highlights the importance of incorporating insights into the nature of the physical target into the definition of the architecture and of the appropriate figures of merit for machine-learning models, that translate in better transferability and overall performance.

Symmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both established and state-of-the-art approaches, with almost no exceptions, are built to be exactly equivariant to translations, permutations, and rotations of the atoms. Incorporating symmetries -- rotations in particular -- constrains the model design space and implies more complicated architectures that are often also computationally demanding. There are indications that non-symmetric models can easily learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We put a model that obeys rotational invariance only approximately to the test, in realistic scenarios involving simulations of gas-phase, liquid, and solid water. We focus specifically on physical observables that are likely to be affected -- directly or indirectly -- by symmetry breaking, finding negligible consequences when the model is used in an interpolative, bulk, regime. Even for extrapolative gas-phase predictions, the model remains very stable, even though symmetry artifacts are noticeable. We also discuss strategies that can be used to systematically reduce the magnitude of symmetry breaking when it occurs, and assess their impact on the convergence of observables.

Atomic-scale simulations have progressed tremendously over the past decade, largely due to the availability of machine-learning interatomic potentials. These potentials combine the accuracy of electronic structure calculations with the ability to reach extensive length and time scales. The i-PI package facilitates integrating the latest developments in this field with advanced modeling techniques, thanks to a modular software architecture based on inter-process communication through a socket interface. The choice of Python for implementation facilitates rapid prototyping but can add computational overhead. In this new release, we carefully benchmarked and optimized i-PI for several common simulation scenarios, making such overhead negligible when i-PI is used to model systems up to tens of thousands of atoms using widely adopted machine learning interatomic potentials, such as Behler-Parinello, DeePMD and MACE neural networks. We also present the implementation of several new features, including an efficient algorithm to model bosonic and fermionic exchange, a framework for uncertainty quantification to be used in conjunction with machine-learning potentials, a communication infrastructure that allows deeper integration with electronic-driven simulations, and an approach to simulate coupled photon-nuclear dynamics in optical or plasmonic cavities.

Statistical learning algorithms provide a generally-applicable framework to sidestep time-consuming experiments, or accurate physics-based modeling, but they introduce a further source of error on top of the intrinsic limitations of the experimental or theoretical setup. Uncertainty estimation is essential to quantify this error, and make application of data-centric approaches more trustworthy. To ensure that uncertainty quantification is used widely, one should aim for algorithms that are reasonably accurate, but also easy to implement and apply. In particular, including uncertainty quantification on top of an existing architecture should be straightforward, and add minimal computational overhead. Furthermore, it should be easy to manipulate or combine multiple machine-learning predictions, propagating uncertainty over further modeling steps. We compare several well-established uncertainty quantification frameworks against these requirements, and propose a practical approach, which we dub direct propagation of shallow ensembles, that provides a good compromise between ease of use and accuracy. We present benchmarks for generic datasets, and an in-depth study of applications to the field of atomistic machine learning for chemistry and materials. These examples underscore the importance of using a formulation that allows propagating errors without making strong assumptions on the correlations between different predictions of the model.

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or be combined explicitly with physically-grounded operations. We present an example of an integrated modeling approach, in which a symmetry-adapted ML model of an effective Hamiltonian is trained to reproduce electronic excitations from a quantum-mechanical calculation. The resulting model can make predictions for molecules that are much larger and more complex than those that it is trained on, and allows for dramatic computational savings by indirectly targeting the outputs of well-converged calculations while using a parameterization corresponding to a minimal atom-centered basis. These results emphasize the merits of intertwining data-driven techniques with physical approximations, improving the transferability and interpretability of ML models without affecting their accuracy and computational efficiency, and providing a blueprint for developing ML-augmented electronic-structure methods.

One essential ingredient in many machine learning (ML) based methods for atomistic modeling of materials and molecules is the use of locality. While allowing better system-size scaling, this systematically neglects long-range (LR) effects, such as electrostatics or dispersion interaction. We present an extension of the long distance equivariant (LODE) framework that can handle diverse LR interactions in a consistent way, and seamlessly integrates with preexisting methods by building new sets of atom centered features. We provide a direct physical interpretation of these using the multipole expansion, which allows for simpler and more efficient implementations. The framework is applied to simple toy systems as proof of concept, and a heterogeneous set of molecular dimers to push the method to its limits. By generalizing LODE to arbitrary asymptotic behaviors, we provide a coherent approach to treat arbitrary two- and many-body non-bonded interactions in the data-driven modeling of matter.

Machine learning (ML) models for molecules and materials commonly rely on a decomposition of the global target quantity into local, atom-centered contributions. This approach is convenient from a computational perspective, enabling large-scale ML-driven simulations with a linear-scaling cost, and also allow for the identification and post-hoc interpretation of contributions from individual chemical environments and motifs to complicated macroscopic properties. However, even though there exist practical justifications for these decompositions, only the global quantity is rigorously defined, and thus it is unclear to what extent the atomistic terms predicted by the model can be trusted. Here, we introduce a quantitative metric, which we call the local prediction rigidity (LPR), that allows one to assess how robust the locally decomposed predictions of ML models are. We investigate the dependence of LPR on the aspects of model training, particularly the composition of training dataset, for a range of different problems from simple toy models to real chemical systems. We present strategies to systematically enhance the LPR, which can be used to improve the robustness, interpretability, and transferability of atomistic ML models.

This work examines the conformational ensemble involved in $\beta$-hairpin folding by means of advanced molecular dynamics simulations and dimensionality reduction. A fully atomistic description of the protein and the surrounding solvent molecules is used and this complex energy landscape is sampled by means of parallel tempering metadynamics simulations. The ensemble of configurations explored is analysed using the recently proposed sketch-map algorithm. Further simulations allow us to probe how mutations affect the structures adopted by this protein. We find that many of the configurations adopted by a mutant are the same as those adopted by the wild type protein. Furthermore, certain mutations destabilize secondary structure containing configurations by preventing the formation of hydrogen bonds or by promoting the formation of new intramolecular contacts. Our analysis demonstrates that machine-learning techniques can be used to study the energy landscapes of complex molecules and that the visualizations that are generated in this way provide a natural basis for examining how the stabilities of particular configurations of the molecule are affected by factors such as temperature or structural mutations., Comment: Pre-print of doi:10.1021/ct500950z, reproduced with permission from the ACS

Point clouds are versatile representations of 3D objects and have found widespread application in science and engineering. Many successful deep-learning models have been proposed that use them as input. The domain of chemical and materials modeling is especially challenging because exact compliance with physical constraints is highly desirable for a model to be usable in practice. These constraints include smoothness and invariance with respect to translations, rotations, and permutations of identical atoms. If these requirements are not rigorously fulfilled, atomistic simulations might lead to absurd outcomes even if the model has excellent accuracy. Consequently, dedicated architectures, which achieve invariance by restricting their design space, have been developed. General-purpose point-cloud models are more varied but often disregard rotational symmetry. We propose a general symmetrization method that adds rotational equivariance to any given model while preserving all the other requirements. Our approach simplifies the development of better atomic-scale machine-learning schemes by relaxing the constraints on the design space and making it possible to incorporate ideas that proved effective in other domains. We demonstrate this idea by introducing the Point Edge Transformer (PET) architecture, which is not intrinsically equivariant but achieves state-of-the-art performance on several benchmark datasets of molecules and solids. A-posteriori application of our general protocol makes PET exactly equivariant, with minimal changes to its accuracy., Comment: Enhancing figures; minor polishing

Machine-learning models based on a point-cloud representation of a physical object are ubiquitous in scientific applications and particularly well-suited to the atomic-scale description of molecules and materials. Among the many different approaches that have been pursued, the description of local atomic environments in terms of their neighbor densities has been used widely and very succesfully. We propose a novel density-based method which involves computing ``Wigner kernels''. These are fully equivariant and body-ordered kernels that can be computed iteratively with a cost that is independent of the radial-chemical basis and grows only linearly with the maximum body-order considered. This is in marked contrast to feature-space models, which comprise an exponentially-growing number of terms with increasing order of correlations. We present several examples of the accuracy of models based on Wigner kernels in chemical applications, for both scalar and tensorial targets, reaching state-of-the-art accuracy on the popular QM9 benchmark dataset, and we discuss the broader relevance of these ideas to equivariant geometric machine-learning.

In this paper, we address the challenge of obtaining a comprehensive and symmetric representation of point particle groups, such as atoms in a molecule, which is crucial in physics and theoretical chemistry. The problem has become even more important with the widespread adoption of machine-learning techniques in science, as it underpins the capacity of models to accurately reproduce physical relationships while being consistent with fundamental symmetries and conservation laws. However, some of the descriptors that are commonly used to represent point clouds -- most notably those based on discretized correlations of the neighbor density, that underpin most of the existing ML models of matter at the atomic scale -- are unable to distinguish between special arrangements of particles in three dimensions. This makes it impossible to machine learn their properties. Atom-density correlations are provably complete in the limit in which they simultaneously describe the mutual relationship between all atoms, which is impractical. We present a novel approach to construct descriptors of \emph{finite} correlations based on the relative arrangement of particle triplets, which can be employed to create symmetry-adapted models with universal approximation capabilities, which have the resolution of the neighbor discretization as the sole convergence parameter. Our strategy is demonstrated on a class of atomic arrangements that are specifically built to defy a broad class of conventional symmetric descriptors, showcasing its potential for addressing their limitations.

Spherical harmonics provide a smooth, orthogonal, and symmetry-adapted basis to expand functions on a sphere, and they are used routinely in physical and theoretical chemistry as well as in different fields of science and technology, from geology and atmospheric sciences to signal processing and computer graphics. More recently, they have become a key component of rotationally equivariant models in geometric machine learning, including applications to atomic-scale modeling of molecules and materials. We present an elegant and efficient algorithm for the evaluation of the real-valued spherical harmonics. Our construction features many of the desirable properties of existing schemes and allows to compute Cartesian derivatives in a numerically stable and computationally efficient manner. To facilitate usage, we implement this algorithm in sphericart, a fast C++ library which also provides C bindings, a Python API, and a PyTorch implementation that includes a GPU kernel.

Alloys composed of several elements in roughly equimolar composition, often referred to as high-entropy alloys, have long been of interest for their thermodynamics and peculiar mechanical properties, and more recently for their potential application in catalysis. They are a considerable challenge to traditional atomistic modeling, and also to data-driven potentials that for the most part have memory footprint, computational effort and data requirements which scale poorly with the number of elements included. We apply a recently proposed scheme to compress chemical information in a lower-dimensional space, which reduces dramatically the cost of the model with negligible loss of accuracy, to build a potential that can describe 25 d-block transition metals. The model shows semi-quantitative accuracy for prototypical alloys, and is remarkably stable when extrapolating to structures outside its training set. We use this framework to study element segregation in a computational experiment that simulates an equimolar alloy of all 25 elements, mimicking the seminal experiments by Cantor et al., and use our observations on the short-range order relations between the elements to define a data-driven set of Hume-Rothery rules that can serve as guidance for alloy design. We conclude with a study of three prototypical alloys, CoCrFeMnNi, CoCrFeMoNi and IrPdPtRhRu, determining their stability and the short-range order behavior of their constituents.

Due to the subtle balance of intermolecular interactions that govern structure-property relations, predicting the stability of crystal structures formed from molecular building blocks is a highly non-trivial scientific problem. A particularly active and fruitful approach involves classifying the different combinations of interacting chemical moieties, as understanding the relative energetics of different interactions enables the design of molecular crystals and fine-tuning their stabilities. While this is usually performed based on the empirical observation of the most commonly encountered motifs in known crystal structures, we propose to apply a combination of supervised and unsupervised machine-learning techniques to automate the construction of an extensive library of molecular building blocks. We introduce a structural descriptor tailored to the prediction of the binding (lattice) energy and apply it to a curated dataset of organic crystals and exploit its atom-centered nature to obtain a data-driven assessment of the contribution of different chemical groups to the lattice energy of the crystal. We then interpret this library using a low-dimensional representation of the structure-energy landscape and discuss selected examples of the insights into crystal engineering that can be extracted from this analysis, providing a complete database to guide the design of molecular materials.

Machine learning frameworks based on correlations of interatomic positions begin with a discretized description of the density of other atoms in the neighbourhood of each atom in the system. Symmetry considerations support the use of spherical harmonics to expand the angular dependence of this density, but there is as yet no clear rationale to choose one radial basis over another. Here we investigate the basis that results from the solution of the Laplacian eigenvalue problem within a sphere around the atom of interest. We show that this generates the smoothest possible basis of a given size within the sphere, and that a tensor product of Laplacian eigenstates also provides the smoothest possible basis for expanding any higher-order correlation of the atomic density within the appropriate hypersphere. We consider several unsupervised metrics of the quality of a basis for a given dataset, and show that the Laplacian eigenstate basis has a performance that is much better than some widely used basis sets and is competitive with data-driven bases that numerically optimize each metric. In supervised machine learning tests, we find that the optimal function smoothness of the Laplacian eigenstates leads to comparable or better performance than can be obtained from a data-driven basis of a similar size that has been optimized to describe the atom-density correlation for the specific dataset. We conclude that the smoothness of the basis functions is a key and hitherto largely overlooked aspect of successful atomic density representations.

The electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. A natural choice to construct a model that yields transferable and linear-scaling predictions is to represent the scalar field using a multi-centered atomic basis analogous to that routinely used in density fitting approximations. However, the non-orthogonality of the basis poses challenges for the learning exercise, as it requires accounting for all the atomic density components at once. We devise a gradient-based approach to directly minimize the loss function of the regression problem in an optimized and highly sparse feature space. In so doing, we overcome the limitations associated with adopting an atom-centered model to learn the electron density over arbitrarily complex datasets, obtaining extremely accurate predictions. The enhanced framework is tested on 32-molecule periodic cells of liquid water, presenting enough complexity to require an optimal balance between accuracy and computational efficiency. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom with respect to the reference density functional calculations. Finally, we test our method on the highly heterogeneous QM9 benchmark dataset, showing that a small fraction of the training data is enough to derive ground-state total energies within chemical accuracy.

The "quasi-constant" SOAP and ACSF fingerprint manifolds recently discovered by Parsaeifard and Goedecker are a direct consequence of the presence of degenerate pairs of configurations, a known shortcoming of all low-body-order atom-density correlation representations of molecular structures. Contrary to the configurations that are rigorously singular -- that we demonstrate can only occur in finite, discrete sets -- the continuous "quasi-constant" manifolds exhibit low, but non-zero, sensitivity to atomic displacements. Thus, it is possible to build interpolative machine-learning models of high-order interactions along the manifold, even though the numerical instabilities associated with proximity to the exact singularities affect the accuracy and transferability of such models, to an extent that depends on numerical details of the implementation., Comment: Comment on arXiv:2102.06915

Data-driven schemes that associate molecular and crystal structures with their microscopic properties share the need for a concise, effective description of the arrangement of their atomic constituents. Many types of models rely on descriptions of atom-centered environments, that are associated with an atomic property or with an atomic contribution to an extensive macroscopic quantity. Frameworks in this class can be understood in terms of atom-centered density correlations (ACDC), that are used as a basis for a body-ordered, symmetry-adapted expansion of the targets. Several other schemes, that gather information on the relationship between neighboring atoms using "message-passing" ideas, cannot be directly mapped to correlations centered around a single atom. We generalize the ACDC framework to include multi-centered information, generating representations that provide a complete linear basis to regress symmetric functions of atomic coordinates, and provides a coherent foundation to systematize our understanding of both atom-centered and message-passing, invariant and equivariant machine-learning schemes.

Graph neural networks (GNN) are very popular methods in machine learning and have been applied very successfully to the prediction of the properties of molecules and materials. First-order GNNs are well known to be incomplete, i.e., there exist graphs that are distinct but appear identical when seen through the lens of the GNN. More complicated schemes have thus been designed to increase their resolving power. Applications to molecules (and more generally, point clouds), however, add a geometric dimension to the problem. The most straightforward and prevalent approach to construct graph representation for molecules regards atoms as vertices in a graph and draws a bond between each pair of atoms within a chosen cutoff. Bonds can be decorated with the distance between atoms, and the resulting "distance graph NNs" (dGNN) have empirically demonstrated excellent resolving power and are widely used in chemical ML, with all known indistinguishable configurations being resolved in the fully-connected limit, which is equivalent to infinite or sufficiently large cutoff. Here we present a counterexample that proves that dGNNs are not complete even for the restricted case of fully-connected graphs induced by 3D atom clouds. We construct pairs of distinct point clouds whose associated graphs are, for any cutoff radius, equivalent based on a first-order Weisfeiler-Lehman test. This class of degenerate structures includes chemically-plausible configurations, both for isolated structures and for infinite structures that are periodic in 1, 2, and 3 dimensions. The existence of indistinguishable configurations sets an ultimate limit to the expressive power of some of the well-established GNN architectures for atomistic machine learning. Models that explicitly use angular or directional information in the description of atomic environments can resolve this class of degeneracies.

Zeolites are nanoporous alumino-silicate frameworks widely used as catalysts and adsorbents. Even though millions of distinct siliceous networks can be generated by computer-aided searches, no new hypothetical framework has yet been synthesized. The needle-in-a-haystack problem of finding promising candidates among large databases of predicted structures has intrigued materials scientists for decades; most work to date on the zeolite problem has been limited to intuitive structural descriptors. Here, we tackle this problem through a rigorous data science scheme-the "zeolite sorting hat"-that exploits interatomic correlations to produce a 95% real versus theoretical zeolites classification accuracy. The hypothetical frameworks that are grouped together with known zeolites are promising candidates for synthesis, that can be further ranked by estimating their thermodynamic stability. A critical analysis of the classifier reveals the decisive structural features. Further partitioning into compositional classes provides guidance in the design of synthetic strategies.

Symmetry considerations are at the core of the major frameworks used to provide an effective mathematical representation of atomic configurations that is then used in machine-learning models to predict the properties associated with each structure. In most cases, the models rely on a description of atom-centered environments, and are suitable to learn atomic properties, or global observables that can be decomposed into atomic contributions. Many quantities that are relevant for quantum mechanical calculations, however -- most notably the single-particle Hamiltonian matrix when written in an atomic-orbital basis -- are not associated with a single center, but with two (or more) atoms in the structure. We discuss a family of structural descriptors that generalize the very successful atom-centered density correlation features to the N-centers case, and show in particular how this construction can be applied to efficiently learn the matrix elements of the (effective) single-particle Hamiltonian written in an atom-centered orbital basis. These N-centers features are fully equivariant -- not only in terms of translations and rotations, but also in terms of permutations of the indices associated with the atoms -- and are suitable to construct symmetry-adapted machine-learning models of new classes of properties of molecules and materials.

The increasingly common applications of machine-learning schemes to atomic-scale simulations have triggered efforts to better understand the mathematical properties of the mapping between the Cartesian coordinates of the atoms and the variety of representations that can be used to convert them into a finite set of symmetric descriptors or features. Here, we analyze the sensitivity of the mapping to atomic displacements, showing that the combination of symmetry and smoothness leads to mappings that have singular points at which the Jacobian has one or more null singular values (besides those corresponding to infinitesimal translations and rotations). This is in fact desirable, because it enforces physical symmetry constraints on the values predicted by regression models constructed using such representations. However, besides these symmetry-induced singularities, there are also spurious singular points, that we find to be linked to the incompleteness of the mapping, i.e. the fact that, for certain classes of representations, structurally distinct configurations are not guaranteed to be mapped onto different feature vectors. Additional singularities can be introduced by a too aggressive truncation of the infinite basis set that is used to discretize the representations.

The resolving power of solid-state nuclear magnetic resonance (NMR) crystallography depends heavily on the accuracy of computational predictions of NMR chemical shieldings of candidate structures, which are usually taken to be local minima in the potential energy. To test the limits of this approximation, we systematically study the importance of finite-temperature and quantum nuclear fluctuations for $^1$H, $^{13}$C, and $^{15}$N shieldings in polymorphs of three paradigmatic molecular crystals -- benzene, glycine, and succinic acid. The effect of quantum fluctuations is comparable to the typical errors of shielding predictions for static nuclei with respect to experiments, and their inclusion to improve the agreement with measurements, translating to more reliable assignment of the NMR spectra to the correct candidate structure. The use of integrated machine-learning models, trained on first-principles energies and shieldings, renders rigorous sampling of nuclear fluctuations affordable, setting a new standard for the calculations underlying NMR structure determinations.

We introduce a local machine-learning method for predicting the electron densities of periodic systems. The framework is based on a numerical, atom-centred auxiliary basis, which enables an accurate expansion of the all-electron density in a form suitable for learning isolated and periodic systems alike. We show that using this formulation the electron densities of metals, semiconductors and molecular crystals can all be accurately predicted using symmetry-adapted Gaussian process regression models, properly adjusted for the non-orthogonal nature of the basis. These predicted densities enable the efficient calculation of electronic properties which present errors on the order of tens of meV/atom when compared to ab initio density-functional calculations. We demonstrate the key power of this approach by using a model trained on ice unit cells containing only 4 water molecules to predict the electron densities of cells containing up to 512 molecules, and see no increase in the magnitude of the errors of derived electronic properties when increasing the system size. Indeed, we find that these extrapolated derived energies are more accurate than those predicted using a direct machine-learning model. Finally, on heterogeneous datasets SALTED can predict electron densities with errors below 4%., Comment: 13 pages, 7 figures; accepted version, SI added

The input of almost every machine learning algorithm targeting the properties of matter at the atomic scale involves a transformation of the list of Cartesian atomic coordinates into a more symmetric representation. Many of the most popular representations can be seen as an expansion of the symmetrized correlations of the atom density, and differ mainly by the choice of basis. Considerable effort has been dedicated to the optimization of the basis set, typically driven by heuristic considerations on the behavior of the regression target. Here we take a different, unsupervised viewpoint, aiming to determine the basis that encodes in the most compact way possible the structural information that is relevant for the dataset at hand. For each training dataset and number of basis functions, one can determine a unique basis that is optimal in this sense, and can be computed at no additional cost with respect to the primitive basis by approximating it with splines. We demonstrate that this construction yields representations that are accurate and computationally efficient, particularly when constructing representations that correspond to high-body order correlations. We present examples that involve both molecular and condensed-phase machine-learning models.

Physically-motivated and mathematically robust atom-centred representations of molecular structures are key to the success of modern atomistic machine learning (ML) methods. They lie at the foundation of a wide range of methods to predict the properties of both materials and molecules as well as to explore and visualize the chemical compound and configuration space. Recently, it has become clear that many of the most effective representations share a fundamental formal connection: that they can all be expressed as a discretization of N-body correlation functions of the local atom density, suggesting the opportunity of standardizing and, more importantly, optimizing the calculation of such representations. We present an implementation, named librascal, whose modular design lends itself both to developing refinements to the density-based formalism and to rapid prototyping for new developments of rotationally equivariant atomistic representations. As an example, we discuss SOAP features, perhaps the most widely used member of this family of representations, to show how the expansion of the local density can be optimized for any choice of radial basis set. We discuss the representation in the context of a kernel ridge regression model, commonly used with SOAP features, and analyze how the computational effort scales for each of the individual steps of the calculation. By applying data reduction techniques in feature space, we show how to further reduce the total computational cost by at up to a factor of 4 or 5 without affecting the model's symmetry properties and without significantly impacting its accuracy.

The first step in the construction of a regression model or a data-driven analysis, aiming to predict or elucidate the relationship between the atomic scale structure of matter and its properties, involves transforming the Cartesian coordinates of the atoms into a suitable representation. The development of atomic-scale representations has played, and continues to play, a central role in the success of machine-learning methods for chemistry and materials science. This review summarizes the current understanding of the nature and characteristics of the most commonly used structural and chemical descriptions of atomistic structures, highlighting the deep underlying connections between different frameworks, and the ideas that lead to computationally efficient and universally applicable models. It emphasizes the link between properties, structures, their physical chemistry and their mathematical description, provides examples of recent applications to a diverse set of chemical and materials science problems, and outlines the open questions and the most promising research directions in the field.

Selecting the most relevant features and samples out of a large set of candidates is a task that occurs very often in the context of automated data analysis, where it can be used to improve the computational performance, and also often the transferability, of a model. Here we focus on two popular sub-selection schemes which have been applied to this end: CUR decomposition, that is based on a low-rank approximation of the feature matrix and Farthest Point Sampling, that relies on the iterative identification of the most diverse samples and discriminating features. We modify these unsupervised approaches, incorporating a supervised component following the same spirit as the Principal Covariates Regression (PCovR) method. We show that incorporating target information provides selections that perform better in supervised tasks, which we demonstrate with ridge regression, kernel ridge regression, and sparse kernel regression. We also show that incorporating aspects of simple supervised learning models can improve the accuracy of more complex models, such as feed-forward neural networks. We present adjustments to minimize the impact that any subselection may incur when performing unsupervised tasks. We demonstrate the significant improvements associated with the use of PCov-CUR and PCov-FPS selections for applications to chemistry and materials science, typically reducing by a factor of two the number of features and samples which are required to achieve a given level of regression accuracy.

Statistical learning algorithms are finding more and more applications in science and technology. Atomic-scale modeling is no exception, with machine learning becoming commonplace as a tool to predict energy, forces and properties of molecules and condensed-phase systems. This short review summarizes recent progress in the field, focusing in particular on the problem of representing an atomic configuration in a mathematically robust and computationally efficient way. We also discuss some of the regression algorithms that have been used to construct surrogate models of atomic-scale properties. We then show examples of how the optimization of the machine-learning models can both incorporate and reveal insights onto the physical phenomena that underlie structure-property relations.

Machine learning models have emerged as a very effective strategy to sidestep time-consuming electronic-structure calculations, enabling accurate simulations of greater size, time scale and complexity. Given the interpolative nature of these models, the reliability of predictions depends on the position in phase space, and it is crucial to obtain an estimate of the error that derives from the finite number of reference structures included during the training of the model. When using a machine-learning potential to sample a finite-temperature ensemble, the uncertainty on individual configurations translates into an error on thermodynamic averages, and provides an indication for the loss of accuracy when the simulation enters a previously unexplored region. Here we discuss how uncertainty quantification can be used, together with a baseline energy model, or a more robust although less accurate interatomic potential, to obtain more resilient simulations and to support active-learning strategies. Furthermore, we introduce an on-the-fly reweighing scheme that makes it possible to estimate the uncertainty in the thermodynamic averages extracted from long trajectories. We present examples covering different types of structural and thermodynamic properties, and systems as diverse as water and liquid gallium., Comment: 17 pages, 9 figures

Mapping an atomistic configuration to an $N$-point correlation of a field associated with the atomic positions (e.g. an atomic density) has emerged as an elegant and effective solution to represent structures as the input of machine-learning algorithms. While it has become clear that low-order density correlations do not provide a complete representation of an atomic environment, the exponential increase in the number of possible $N$-body invariants makes it difficult to design a concise and effective representation. We discuss how to exploit recursion relations between equivariant features of different orders (generalizations of $N$-body invariants that provide a complete representation of the symmetries of improper rotations) to compute high-order terms efficiently. In combination with the automatic selection of the most expressive combination of features at each order, this approach provides a conceptual and practical framework to generate systematically-improvable, symmetry adapted representations for atomistic machine learning.

We present a generally-applicable computational framework for the efficient and accurate characterization of molecular structural patterns and acid properties in explicit solvent using H$_2$O$_2$ and CH$_3$SO$_3$H in phenol as an example. In order to address the challenges posed by the complexity of the problem, we resort to a set of data-driven methods and enhanced sampling algorithms. The synergistic application of these techniques makes the first-principle estimation of the chemical properties feasible without renouncing to the use of explicit solvation, involving extensive statistical sampling. Ensembles of neural network potentials are trained on a set of configurations carefully selected out of preliminary simulations performed at a low-cost density-functional tight-binding (DFTB) level. The energy and forces of these configurations are then recomputed at the hybrid density functional theory (DFT) level and used to train the neural networks. The stability of the NN model is enhanced by using DFTB energetics as a baseline, but the efficiency of the direct NN (i.e., baseline-free) is exploited via a multiple-time step integrator. The neural network potentials are combined with enhanced sampling techniques, such as replica exchange and metadynamics, and used to characterize the relevant protonated species and dominant non-covalent interactions in the mixture, also considering nuclear quantum effects.

The molecular dipole moment ($\boldsymbol{\mu}$) is a central quantity in chemistry. It is essential in predicting infrared and sum-frequency generation spectra, as well as induction and long-range electrostatic interactions. Furthermore, it can be extracted directly from high-level quantum mechanical calculations, making it an ideal target for machine learning (ML). In this work, we choose to represent this quantity with a physically inspired ML model that captures two distinct physical effects: local atomic polarization is captured within the symmetry-adapted Gaussian process regression (SA-GPR) framework, which assigns a (vector) dipole moment to each atom, while movement of charge across the entire molecule is captured by assigning a partial (scalar) charge to each atom. The resulting "MuML" models are fitted together to reproduce molecular $\boldsymbol{\mu}$ computed using high-level coupled-cluster theory (CCSD) and density functional theory (DFT) on the QM7b dataset. The combined model shows excellent transferability when applied to a showcase dataset of larger and more complex molecules, approaching the accuracy of DFT at a small fraction of the computational cost. We also demonstrate that the uncertainty in the predictions can be estimated reliably using a calibrated committee model. The ultimate performance of the models depends, however, on the details of the system at hand, with the scalar model being clearly superior when describing large molecules whose dipole is almost entirely generated by charge separation. These observations point to the importance of simultaneously accounting for the local and non-local effects that contribute to $\boldsymbol{\mu}$; further, they define a challenging task to benchmark future models, particularly those aimed at the description of condensed phases., Comment: 16 pages, 7 figures, plus supplementary information; added Materials Cloud link and corrected values in Table I. To be published in the Journal of Chemical Physics

Data analyses based on linear methods constitute the simplest, most robust, and transparent approaches to the automatic processing of large amounts of data for building supervised or unsupervised machine learning models. Principal covariates regression (PCovR) is an underappreciated method that interpolates between principal component analysis and linear regression, and can be used to conveniently reveal structure-property relations in terms of simple-to-interpret, low-dimensional maps. Here we provide a pedagogic overview of these data analysis schemes, including the use of the kernel trick to introduce an element of non-linearity, while maintaining most of the convenience and the simplicity of linear approaches. We then introduce a kernelized version of PCovR and a sparsified extension, and demonstrate the performance of this approach in revealing and predicting structure-property relations in chemistry and materials science, showing a variety of examples including elemental carbon, porous silicate frameworks, organic molecules, amino acid conformers, and molecular materials.

Many-body descriptors are widely used to represent atomic environments in the construction of machine learned interatomic potentials and more broadly for fitting, classification and embedding tasks on atomic structures. It was generally believed that 3-body descriptors uniquely specify the environment of an atom, up to a rotation and permutation of like atoms. We produce several counterexamples to this belief, with the consequence that any classifier, regression or embedding model for atom-centred properties that uses 3 (or 4)-body features will incorrectly give identical results for different configurations. Writing global properties (such as total energies) as a sum of many atom-centred contributions mitigates, but does not eliminate, the impact of this fundamental deficiency -- explaining the success of current "machine-learning" force fields. We anticipate the issues that will arise as the desired accuracy increases, and suggest potential solutions.

At room temperature, the quantum contribution to the kinetic energy of a water molecule exceeds the classical contribution by an order of magnitude. The quantum kinetic energy (QKE) of a water molecule is modulated by its local chemical environment and leads to uneven partitioning of isotopes between different phases in thermal equilibrium, which would not occur if the nuclei behaved classically. In this work, we use ab initio path integral simulations to show that QKEs of the water molecules and the equilibrium isotope fractionation ratios of the oxygen and hydrogen isotopes are sensitive probes of the hydrogen bonding structures in aqueous ionic solutions. In particular, we demonstrate how the QKE of water molecules in path integral simulations can be decomposed into translational, rotational and vibrational degrees of freedom, and use them to determine the impact of solvation on different molecular motions. By analyzing the QKEs and isotope fractionation ratios, we show how the addition of the Na$^+$, Cl$^-$ and HPO$_4^{2-}$ ions perturbs the competition between quantum effects in liquid water and impacts their local solvation structures.

The properties of molecules and materials containing light nuclei are affected by their quantum mechanical nature. Modelling these quantum nuclear effects accurately requires computationally demanding path integral techniques. Considerable success has been achieved in reducing the cost of such simulations by using generalized Langevin dynamics to induce frequency-dependent fluctuations. Path integral generalized Langevin equation methods, however, have this far been limited to the study of static, thermodynamic properties due to the large perturbation to the system's dynamics induced by the aggressive thermostatting. Here we introduce a post-processing scheme, based on analytical estimates of the dynamical perturbation induced by the generalized Langevin dynamics, that makes it possible to recover meaningful time correlation properties from a thermostatted trajectory. We show that this approach yields spectroscopic observables for model and realistic systems which have an accuracy comparable to much more demanding approximate quantum dynamics techniques based on full path integral simulations.

The most successful and popular machine learning models of atomic-scale properties derive their transferability from a locality ansatz. The properties of a large molecule or a bulk material are written as a sum over contributions that depend on the configurations within finite atom-centered environments. The obvious downside of this approach is that it cannot capture non-local, non-additive effects such as those arising due to long-range electrostatics or quantum interference. We propose a solution to this problem by introducing non-local representations of the system that are remapped as feature vectors that are defined locally and are equivariant in O(3). We consider in particular one form that has the same asymptotic behavior as the electrostatic potential. We demonstrate that this framework can capture non-local, long-range physics by building a model for the electrostatic energy of randomly distributed point-charges, for the unrelaxed binding curves of charged organic molecular dimers, and for the electronic dielectric response of liquid water. By combining a representation of the system that is sensitive to long-range correlations with the transferability of an atom-centered additive model, this method outperforms current state-of-the-art machine-learning schemes, and provides a conceptual framework to incorporate non-local physics into atomistic machine learning.

Nuclear Magnetic Resonance (NMR) spectroscopy is particularly well-suited to determine the structure of molecules and materials in powdered form. Structure determination usually proceeds by finding the best match between experimentally observed NMR chemical shifts and those of candidate structures. Chemical shifts for the candidate configurations have traditionally been computed by electronic-structure methods, and more recently predicted by machine learning. However, the reliability of the determination depends on the errors in the predicted shifts. Here we propose a Bayesian framework for determining the confidence in the identification of the experimental crystal structure, based on knowledge of the typical error in the electronic structure methods. We also extend the recently-developed ShiftML machine-learning model, including the evaluation of the uncertainty of its predictions. We demonstrate the approach on the determination of the structures of six organic molecular crystals. We critically assess the reliability of the structure determinations, facilitated by the introduction of a visualization of the of similarity between candidate configurations in terms of their chemical shifts and their structures. We also show that the commonly used values for the errors in calculated $^{13}$C shifts are underestimated, and that more accurate, self-consistently determined uncertainties make it possible to use $^{13}$C shifts to improve the accuracy of structure determinations.

Infrared pump-probe spectroscopy provides detailed information on the dynamics of hydrogen-bonded liquids. Due to dissipation of the absorbed pump pulse energy, also thermal equilibration dynamics contributes to the observed signal. Disentangling this contribution from the molecular response remains an open challenge. Performing non-equilibrium molecular dynamics simulat ions of liquid deuterated methanol, we show that faster molecular vibrational relaxation and slower heat diffusion are decoupled and occur on different length scale. Transient structures of the hydrogen bonding network influence thermal relaxation by affecting thermal diffusivity over the length-scale of several nanometers.

This chapter discusses the importance of incorporating three-dimensional symmetries in the context of statistical learning models geared towards the interpolation of the tensorial properties of atomic-scale structures. We focus on Gaussian process regression, and in particular on the construction of structural representations, and the associated kernel functions, that are endowed with the geometric covariance properties compatible with those of the learning targets. We summarize the general formulation of such a symmetry-adapted Gaussian process regression model, and how it can be implemented based on a scheme that generalizes the popular smooth overlap of atomic positions representation. We give examples of the performance of this framework when learning the polarizability and the ground-state electron density of a molecule.

Automated analyses of the outcome of a simulation have been an important part of atomistic modeling since the early days, addressing the need of linking the behavior of individual atoms and the collective properties that are usually the final quantity of interest. Methods such as clustering and dimensionality reduction have been used to provide a simplified, coarse-grained representation of the structure and dynamics of complex systems, from proteins to nanoparticles. In recent years, the rise of machine learning has led to an even more widespread use of these algorithms in atomistic modeling, and to consider different classification and inference techniques as part of a coherent toolbox of data-driven approaches. This perspective briefly reviews some of the unsupervised machine-learning methods -- that are geared towards classification and coarse-graining of molecular simulations -- seen in relation to the fundamental mathematical concepts that underlie all machine-learning techniques. It discusses the importance of using concise yet complete representations of atomic structures as the starting point of the analyses, and highlights the risk of introducing preconceived biases when using machine learning to rationalize and understand structure-property relations. Supervised machine-learning techniques, that explicitly attempt to predict the properties of a material given its structure, are less susceptible to such biases. Current developments in the field suggest that using these two classes of approaches side-by-side and in a fully integrated mode, while keeping in mind the relations between the data analysis framework and the fundamental physical principles, will be key for realizing the full potential of machine learning to help understanding the behavior of complex molecules and materials.

Metal-organic frameworks show both fundamental interest and great promise for applications in adsorption-based technologies, such as the separation and storage of gases. The flexibility and complexity of the molecular scaffold poses a considerable challenge to atomistic modeling, especially when also considering the presence of guest molecules. We investigate the role played by quantum and anharmonic fluctuations in the archetypical case of MOF-5, comparing the material at various levels of methane loading. Accurate path integral simulations of such effects are made affordable by the introduction of an accelerated simulation scheme and the use of an optimized force field based on first-principles reference calculations. We find that the level of statistical treatment that is required for predictive modeling depends significantly on the property of interest. The thermal properties of the lattice are generally well described by a quantum harmonic treatment, with the adsorbate behaving in a classical but strongly anharmonic manner. The heat capacity of the loaded framework - which plays an important role in the characterization of the framework and in determining its stability to thermal fluctuations during adsorption/desorption cycles - requires, however, a full quantum and anharmonic treatment, either by path integral methods or by a simple but approximate scheme. We also present molecular-level insight into the nanoscopic interactions contributing to the material's properties and suggest design principles to optimize them

Thermodynamic properties of liquid water as well as hexagonal (Ih) and cubic (Ic) ice are predicted based on density functional theory at the hybrid-functional level, rigorously taking into account quantum nuclear motion, anharmonic fluctuations and proton disorder. This is made possible by combining advanced free energy methods and state-of-the-art machine learning techniques. The ab initio description leads to structural properties in excellent agreement with experiments, and reliable estimates of the melting points of light and heavy water. We observe that nuclear quantum effects contribute a crucial 0.2 meV/H$_2$O to the stability of ice Ih, making it more stable than ice Ic. Our computational approach is general and transferable, providing a comprehensive framework for quantitative predictions of ab initio thermodynamic properties using machine learning potentials as an intermediate step.

We present a scheme to obtain an inexpensive and reliable estimate of the uncertainty associated with the predictions of a machine-learning model of atomic and molecular properties. The scheme is based on resampling, with multiple models being generated based on sub-sampling of the same training data. The accuracy of the uncertainty prediction can be benchmarked by maximum likelihood estimation, which can also be used to correct for correlations between resampled models, and to improve the performance of the uncertainty estimation by a cross-validation procedure. In the case of sparse Gaussian Process Regression models, this resampled estimator can be evaluated at negligible cost. We demonstrate the reliability of these estimates for the prediction of molecular energetics, and for the estimation of nuclear chemical shieldings in molecular crystals. Extension to estimate the uncertainty in energy differences, forces, or other correlated predictions is straightforward. This method can be easily applied to other machine learning schemes, and will be beneficial to make data-driven predictions more reliable, and to facilitate training-set optimization and active-learning strategies.

The electronic charge density plays a central role in determining the behavior of matter at the atomic scale, but its computational evaluation requires demanding electronic-structure calculations. We introduce an atom-centered, symmetry-adapted framework to machine-learn the valence charge density based on a small number of reference calculations. The model is highly transferable, meaning it can be trained on electronic-structure data of small molecules and used to predict the charge density of larger compounds with low, linear-scaling cost. Applications are shown for various hydrocarbon molecules of increasing complexity and flexibility, and demonstrate the accuracy of the model when predicting the density on octane and octatetraene after training exclusively on butane and butadiene. This transferable, data-driven model can be used to interpret experiments, initialize electronic structure calculations, and compute electrostatic interactions in molecules and condensed-phase systems.