Search

Achieving high performance and performance portability for large-scale scientific applications is a major challenge on heterogeneous computing systems such as many-core CPUs and accelerators like GPUs. In this work, we implement a widely used block eigensolver, Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG), using two popular directive based programming models (OpenMP and OpenACC) for GPU-accelerated systems. Our work differs from existing work in that it adopts a holistic approach that optimizes the full solver performance rather than narrowing the problem into small kernels (e.g., SpMM, SpMV). Our LOPBCG GPU implementation achieves a 2.8$${\times }$$–4.3$${\times }$$ speedup over an optimized CPU implementation when tested with four different input matrices. The evaluated configuration compared one Skylake CPU to one Skylake CPU and one NVIDIA V100 GPU. Our OpenMP and OpenACC LOBPCG GPU implementations gave nearly identical performance. We also consider how to create an efficient LOBPCG solver that can solve problems larger than GPU memory capacity. To this end, we create microbenchmarks representing the two dominant kernels (inner product and SpMM kernel) in LOBPCG and then evaluate performance when using two different programming approaches: tiling the kernels, and using Unified Memory with the original kernels. Our tiled SpMM implementation achieves a 2.9$${\times }$$ and 48.2$${\times }$$ speedup over the Unified Memory implementation on supercomputers with PCIe Gen3 and NVLink 2.0 CPU to GPU interconnects, respectively.

We describe a number of recently developed techniques for improving the performance of large-scale nuclear configuration interaction calculations on high performance parallel computers. We show the benefit of using a preconditioned block iterative method to replace the Lanczos algorithm that has traditionally been used to perform this type of computation. The rapid convergence of the block iterative method is achieved by a proper choice of starting guesses of the eigenvectors and the construction of an effective preconditioner. These acceleration techniques take advantage of special structure of the nuclear configuration interaction problem which we discuss in detail. The use of a block method also allows us to improve the concurrency of the computation, and take advantage of the memory hierarchy of modern microprocessors to increase the arithmetic intensity of the computation relative to data movement. We also discuss the implementation details that are critical to achieving high performance on massively parallel multi-core supercomputers, and demonstrate that the new block iterative solver is two to three times faster than the Lanczos based algorithm for problems of moderate sizes on a Cray XC30 system.

As on-node parallelism increases and the performance gap between the processor and the memory system widens, achieving high performance in large-scale scientific applications requires an architecture-aware design of algorithms and solvers. We focus on the eigenvalue problem arising in nuclear Configuration Interaction (CI) calculations, where a few extreme eigenpairs of a sparse symmetric matrix are needed. We consider a block iterative eigensolver whose main computational kernels are the multiplication of a sparse matrix with multiple vectors (SpMM), and tall-skinny matrix operations. We present techniques to significantly improve the SpMM and the transpose operation SpMMT by using the compressed sparse blocks (CSB) format. We achieve 3-4× speedup on the requisite operations over good implementations with the commonly used compressed sparse row (CSR) format. We develop a performance model that allows us to correctly estimate the performance of our SpMM kernel implementations, and we identify cache bandwidth as a potential performance bottleneck beyond DRAM. We also analyze and optimize the performance of LOBPCG kernels (inner product and linear combinations on multiple vectors) and show up to 15× speedup over using high performance BLAS libraries for these operations. The resulting high performance LOBPCG solver achieves 1.4× to 1.8× speedup over the existing Lanczos solver on a series of CI computations on high-end multicore architectures (Intel Xeons). We also analyze the performance of our techniques on an Intel Xeon Phi Knights Corner (KNC) processor.

Obtaining highly accurate predictions on the properties of light atomic nuclei using the configuration interaction (CI) approach requires computing a few extremal Eigen pairs of the many-body nuclear Hamiltonian matrix. In the Many-body Fermion Dynamics for nuclei (MFDn) code, a block Eigen solver is used for this purpose. Due to the large size of the sparse matrices involved, a significant fraction of the time spent on the Eigen value computations is associated with the multiplication of a sparse matrix (and the transpose of that matrix) with multiple vectors (SpMM and SpMM-T). Existing implementations of SpMM and SpMM-T significantly underperform expectations. Thus, in this paper, we present and analyze optimized implementations of SpMM and SpMM-T. We base our implementation on the compressed sparse blocks (CSB) matrix format and target systems with multi-core architectures. We develop a performance model that allows us to understand and estimate the performance characteristics of our SpMM kernel implementations, and demonstrate the efficiency of our implementation on a series of real-world matrices extracted from MFDn. In particular, we obtain 3-4 speedup on the requisite operations over good implementations based on the commonly used compressed sparse row (CSR) matrix format. The improvements in the SpMM kernel suggest we may attain roughly a 40% speed up in the overall execution time of the block Eigen solver used in MFDn. © 2014 IEEE.

We report the development and testing of new integrated cyberinfrastructure for performing free energy simulations with generalized hybrid quantum mechanical/molecular mechanical (QM/MM) and machine learning potentials (MLPs) in Amber. The Sander molecular dynamics program has been extended to leverage fast, density-functional tight-binding models implemented in the DFTB+ and xTB packages, and an interface to the DeePMD-kit software enables the use of MLPs. The software is integrated through application program interfaces that circumvent the need to perform "system calls" and enable the incorporation of long-range Ewald electrostatics into the external software's self-consistent field procedure. The infrastructure provides access to QM/MM models that may serve as the foundation for QM/MM-ΔMLP potentials, which supplement the semiempirical QM/MM model with a MLP correction trained to reproduce ab initio QM/MM energies and forces. Efficient optimization of minimum free energy pathways is enabled through a new surface-accelerated finite-temperature string method implemented in the FE-ToolKit package. Furthermore, we interfaced Sander with the i-PI software by implementing the socket communication protocol used in the i-PI client-server model. The new interface with i-PI allows for the treatment of nuclear quantum effects with semiempirical QM/MM-ΔMLP models. The modular interoperable software is demonstrated on proton transfer reactions in guanine-thymine mispairs in a B-form deoxyribonucleic acid helix. The current work represents a considerable advance in the development of modular software for performing free energy simulations of chemical reactions that are important in a wide range of applications., (© 2024 Author(s). Published under an exclusive license by AIP Publishing.)

AmberTools is a free and open-source collection of programs used to set up, run, and analyze molecular simulations. The newer features contained within AmberTools23 are briefly described in this Application note.

We have ported and optimized the graphics processing unit (GPU)-accelerated QUICK and AMBER-based ab initio quantum mechanics/molecular mechanics (QM/MM) implementation on AMD GPUs. This encompasses the entire Fock matrix build and force calculation in QUICK including one-electron integrals, two-electron repulsion integrals, exchange-correlation quadrature, and linear algebra operations. General performance improvements to the QUICK GPU code are also presented. Benchmarks carried out on NVIDIA V100 and AMD MI100 cards display similar performance on both hardware for standalone HF/DFT calculations with QUICK and QM/MM molecular dynamics simulations with QUICK/AMBER. Furthermore, with respect to the QUICK/AMBER release version 21, significant speedups are observed for QM/MM molecular dynamics simulations. This significantly increases the range of scientific problems that can be addressed with open-source QM/MM software on state-of-the-art computer hardware.

The reactive force field (ReaxFF) model bridges the gap between traditional classical models and quantum mechanical (QM) models by incorporating dynamic bonding and polarizability. To achieve realistic simulations using ReaxFF, model parameters must be optimized against high fidelity training data which typically come from QM calculations. Existing parameter optimization methods for ReaxFF consist of black box techniques using genetic algorithms or Monte Carlo methods. Due to the stochastic behavior of these methods, the optimization process oftentimes requires millions of error evaluations for complex parameter fitting tasks, thereby significantly hampering the rapid development of high quality parameter sets. Rapid optimization of the parameters is essential for developing and refining Reax force fields because producing a force field which exhibits empirical accuracy in terms of dynamics typically requires multiple refinements to the training data as well as to the parameters under optimization. In this work, we present JAX-ReaxFF, a novel software tool that leverages modern machine learning infrastructure to enable fast optimization of ReaxFF parameters. By calculating gradients of the loss function using the JAX library, JAX-ReaxFF utilizes highly effective local optimization methods that are initiated from multiple guesses in the high dimensional optimization space to obtain high quality results. Leveraging the architectural portability of the JAX framework, JAX-ReaxFF can execute efficiently on multicore CPUs, graphics processing units (GPUs), or even tensor processing units (TPUs). As a result of using the gradient information and modern hardware accelerators, we are able to decrease ReaxFF parameter optimization time from days to mere minutes. Furthermore, the JAX-ReaxFF framework can also serve as a sandbox environment for domain scientists to explore customizing the ReaxFF functional form for more accurate modeling.

A novel locally polarizable multisite model based on the original cation dummy atom (CDA) model is described for molecular dynamics simulations of ions in condensed phases. Polarization effects are introduced by the electronegativity equalization model (EEM) method where charges on the metal ion and its dummy atoms can fluctuate to respond to the environment. This model includes explicit polarization and ion-induced interactions and can be coupled with nonpolarizable or polarizable water models, making it more transferable to simpler force fields. This approach allows us to enhance the original fixed charge CDA model where the charge distribution cannot adapt to the local solvent structure. To illustrate the new CDA pol model, we examined properties of the Zn 2+ , Al 3+ , and Zr 4+ ions in aqueous solution. The polarizable model and Lennard-Jones parameters were refined for octahedrally coordinated Zn 2+ , Al 3+ , and Zr 4+ CDAs to reproduce thermodynamic and geometrical properties. Using this locally polarizable model, we were able to obtain the experimental hydration free energy, ion-oxygen distance, and coordination number coupled with the standard 12-6 Lennard-Jones model. This model can be used in myriad additional applications where local polarization and charge transfer is important.

The growing interest in the effects of external electric fields on reactive processes requires predictive methods that can reach longer length and time scales than quantum mechanical simulations. Recently, many studies have included electric fields in ReaxFF, a widely used reactive molecular dynamics method. In the case of modeling an external electric field, the charge distribution method used in ReaxFF is critical. The most common charge distribution method used in previous studies of electric fields is the charge equilibration (QEq) method, which assumes that the system is a contiguous conductor and that charge transfer can occur across any distance. In contrast, many systems of interest are insulators or semiconductors, and long-distance charge transfer should not occur in response to a small difference in potential. This study focuses on the limitations of the QEq method in the context of water in an external electric field. We demonstrate that QEq can predict unphysical charge distributions and exhibits properties that do not converge as a function of system size. Furthermore, we show that electric fields within the recently developed atom-condensed Kohn-Sham density functional theory (DFT) approximated to the second-order (ACKS2) approach address the major limitations of electric fields in QEq. With ACKS2, we observe more physical charge distributions and properties that converge as a function of system size. We do not suggest that ACKS2 is perfect in all circumstances but rather show specific cases where it addresses the major shortcomings of QEq in the context of an external electric field.

We report a new multi-GPU capable ab initio Hartree-Fock/density functional theory implementation integrated into the open source QUantum Interaction Computational Kernel (QUICK) program. Details on the load balancing algorithms for electron repulsion integrals and exchange correlation quadrature across multiple GPUs are described. Benchmarking studies carried out on up to four GPU nodes, each containing four NVIDIA V100-SXM2 type GPUs demonstrate that our implementation is capable of achieving excellent load balancing and high parallel efficiency. For representative medium to large size protein/organic molecular systems, the observed parallel efficiencies remained above 82% for the Kohn-Sham matrix formation and above 90% for nuclear gradient calculations. The accelerations on NVIDIA A100, P100, and K80 platforms also have realized parallel efficiencies higher than 68% in all tested cases, paving the way for large-scale ab initio electronic structure calculations with QUICK.

Reactive force fields provide an affordable model for simulating chemical reactions at a fraction of the cost of quantum mechanical approaches. However, classically accounting for chemical reactivity often comes at the expense of accuracy and transferability, while computational cost is still large relative to nonreactive force fields. In this Perspective, we summarize recent efforts for improving the performance of reactive force fields in these three areas with a focus on the ReaxFF theoretical model. To improve accuracy, we describe recent reformulations of charge equilibration schemes to overcome unphysical long-range charge transfer, new ReaxFF models that account for explicit electrons, and corrections for energy conservation issues of the ReaxFF model. To enhance transferability we also highlight new advances to include explicit treatment of electrons in the ReaxFF and hybrid nonreactive/reactive simulations that make it possible to model charge transfer, redox chemistry, and large systems such as reverse micelles within the framework of a reactive force field. To address the computational cost, we review recent work in extended Lagrangian schemes and matrix preconditioners for accelerating the charge equilibration method component of ReaxFF and improvements in its software performance in LAMMPS.

Combined quantum mechanical/molecular mechanical (QM/MM) models using semiempirical and ab initio methods have been extensively reported on over the past few decades. These methods have been shown to be capable of providing unique insights into a range of problems, but they are still limited to relatively short time scales, especially QM/MM models using ab initio methods. An intermediate approach between a QM based model and classical mechanics could help fill this time-scale gap and facilitate the study of a range of interesting problems. Reactive force fields represent the intermediate approach explored in this paper. A widely used reactive model is ReaxFF, which has largely been applied to materials science problems and is generally used as a stand-alone (i.e., the full system is modeled using ReaxFF). We report a hybrid ReaxFF/AMBER molecular dynamics (MD) tool, which introduces ReaxFF capabilities to capture bond breaking and formation within the AMBER MD software package. This tool enables us to study local reactive events in large systems at a fraction of the computational costs of QM/MM models. We describe the implementation of ReaxFF/AMBER, validate this implementation using a benzene molecule solvated in water, and compare its performance against a range of similar approaches. To illustrate the predictive capabilities of ReaxFF/AMBER, we carried out a Claisen rearrangement study in aqueous solution. In a first for ReaxFF, we were able to use AMBER's potential of mean force (PMF) capabilities to perform a PMF study on this organic reaction. The ability to capture local reaction events in large systems using combined ReaxFF/AMBER opens up a range of problems that can be tackled using this model to address both chemical and biological processes.

Studies using molecular dynamics (MD) have long struggled to simulate the failure modes of materials, predicting unrealistically high ductility and failing to capture brittle fracture. The primary cause of this shortcoming is an inadequate description of bond breaking. While reactive force fields such as ReaxFF show improvements compared to traditional force fields, the charge models used yield unphysical partial charges, especially during dissociation of ionic bonds. This flaw may be remedied by using the atom-condensed Kohn-Sham density functional theory (DFT) approximated to a second order (ACKS2) charge model for determining partial charges. In this work, we present a new ACKS2-enabled Reax force field for fracture simulations of lithium oxide systems, which was obtained by training against an extensive set of DFT, multireference configuration interaction (MRCI), and MRCI+Q reference data using genetic optimization techniques. This new force field significantly improves the bond breaking behavior, but still cannot fully capture the brittle fracture in MD simulations, suggesting more research is needed to improve simulation of brittle fracture.

Reactive force fields make low-cost simulations of chemical reactions possible. However, optimizing them for a given chemical system is difficult and time-consuming. We present a high-performance implementation of global force-field parameter optimization, which delivers parameter sets of the same quality with much less effort and in far less time than before, and also offers excellent parallel scaling. We demonstrate these features with example applications targeting the ReaxFF force field. © 2015 Wiley Periodicals, Inc., (© 2015 Wiley Periodicals, Inc.)

We report our study of a silica-water interface using reactive molecular dynamics. This first-of-its-kind simulation achieves length and time scales required to investigate the detailed chemistry of the system. Our molecular dynamics approach is based on the ReaxFF force field of van Duin et al. [J. Phys. Chem. A 107, 3803 (2003)]. The specific ReaxFF implementation (SERIALREAX) and force fields are first validated on structural properties of pure silica and water systems. Chemical reactions between reactive water and dangling bonds on a freshly cut silica surface are analyzed by studying changing chemical composition at the interface. In our simulations, reactions involving silanol groups reach chemical equilibrium in approximately 250 ps. It is observed that water molecules penetrate a silica film through a proton-transfer process we call "hydrogen hopping," which is similar to the Grotthuss mechanism. In this process, hydrogen atoms pass through the film by associating and dissociating with oxygen atoms within bulk silica, as opposed to diffusion of intact water molecules. The effective diffusion constant for this process, taken to be that of hydrogen atoms within silica, is calculated to be 1.68 x 10(-6) cm(2)/s. Polarization of water molecules in proximity of the silica surface is also observed. The subsequent alignment of dipoles leads to an electric potential difference of approximately 10.5 V between the silica slab and water.

Questions of understanding and quantifying the representation and amount of information in organisms have become a central part of biological research, as they potentially hold the key to fundamental advances. In this paper, we demonstrate the use of information-theoretic tools for the task of identifying segments of biomolecules (DNA or RNA) that are statistically correlated. We develop a precise and reliable methodology, based on the notion of mutual information, for finding and extracting statistical as well as structural dependencies. A simple threshold function is defined, and its use in quantifying the level of significance of dependencies between biological segments is explored. These tools are used in two specific applications. First, they are used for the identification of correlations between different parts of the maize zmSRp32 gene. There, we find significant dependencies between the 5' untranslated region in zmSRp32 and its alternatively spliced exons. This observation may indicate the presence of as-yet unknown alternative splicing mechanisms or structural scaffolds. Second, using data from the FBI's combined DNA index system (CODIS), we demonstrate that our approach is particularly well suited for the problem of discovering short tandem repeats-an application of importance in genetic profiling.