Search

Variational ab-initio methods in quantum chemistry stand out among other methods in providing direct access to the wave function. This allows in principle straightforward extraction of any other observable of interest, besides the energy, but in practice this extraction is often technically difficult and computationally impractical. Here, we consider the electron density as a central observable in quantum chemistry and introduce a novel method to obtain accurate densities from real-space many-electron wave functions by representing the density with a neural network that captures known asymptotic properties and is trained from the wave function by score matching and noise-contrastive estimation. We use variational quantum Monte Carlo with deep-learning ans\"atze (deep QMC) to obtain highly accurate wave functions free of basis set errors, and from them, using our novel method, correspondingly accurate electron densities, which we demonstrate by calculating dipole moments, nuclear forces, contact densities, and other density-based properties., Comment: 12 pages, 9 figures in the main text

Seismology has witnessed significant advancements in recent years with the application of deep learning methods to address a broad range of problems. These techniques have demonstrated their remarkable ability to effectively extract statistical properties from extensive datasets, surpassing the capabilities of traditional approaches to an extent. In this study, we present SAIPy, an open source Python package specifically developed for fast data processing by implementing deep learning. SAIPy offers solutions for multiple seismological tasks, including earthquake detection, magnitude estimation, seismic phase picking, and polarity identification. We introduce upgraded versions of previously published models such as CREIMERT capable of identifying earthquakes with an accuracy above 99.8 percent and a root mean squared error of 0.38 unit in magnitude estimation. These upgraded models outperform state of the art approaches like the Vision Transformer network. SAIPy provides an API that simplifies the integration of these advanced models, including CREIMERT, DynaPickerv2, and PolarCAP, along with benchmark datasets. The package has the potential to be used for real time earthquake monitoring to enable timely actions to mitigate the impact of seismic events. Ongoing development efforts aim to enhance the performance of SAIPy and incorporate additional features that enhance exploration efforts, and it also would be interesting to approach the retraining of the whole package as a multi-task learning problem.

Reliable earthquake forecasting methods have long been sought after, and so the rise of modern data science techniques raises a new question: does deep learning have the potential to learn this pattern? In this study, we leverage the large amount of earthquakes reported via good seismic station coverage in the subduction zone of Japan. We pose earthquake forecasting as a classification problem and train a Deep Learning Network to decide, whether a timeseries of length greater than 2 years will end in an earthquake on the following day with magnitude greater than 5 or not. Our method is based on spatiotemporal b value data, on which we train an autoencoder to learn the normal seismic behaviour. We then take the pixel by pixel reconstruction error as input for a Convolutional Dilated Network classifier, whose model output could serve for earthquake forecasting. We develop a special progressive training method for this model to mimic real life use. The trained network is then evaluated over the actual dataseries of Japan from 2002 to 2020 to simulate a real life application scenario. The overall accuracy of the model is 72.3 percent. The accuracy of this classification is significantly above the baseline and can likely be improved with more data in the future

High rate Global Navigation Satellite System (HR GNSS) data can be highly useful for earthquake analysis as it provides continuous high-rate measurements of ground motion. This data can be used to estimate the magnitude, to assess the potential of an earthquake for generating tsunamis, and to analyze diverse parameters related to the seismic source. Particularly, in this work, we present the first results of a deep learning model based on a convolutional neural network for earthquake magnitude estimation, using HR GNSS displacement time series. The influence of different dataset configurations, such as station numbers, epicentral distances, signal duration, and earthquake size, were analyzed to figure out how the model can be adapted to various scenarios. We explored the potential of the model for global application and compared its performance using both synthetic and real data from different seismogenic regions. The performance of our model at this stage was satisfactory in estimating earthquake magnitude from synthetic data. Comparable results were observed in tests using synthetic data from a different tectonic region than the training data. Furthermore, the model was tested using real data from different regions and magnitudes, resulting in a good accuracy, provided that the data from a particular group of stations had similar epicentral distance constraints to those used during the model training. The robustness of the DL model can be improved to work independently from the window size of the time series and the number of stations, enabling faster estimation by the model using only near field data. Overall, this study provides insights for the development of future DL approaches for earthquake magnitude estimation with HR-GNSS data, emphasizing the importance of proper handling and careful data selection for further model improvements.

Normalizing flows (NF) are a class of powerful generative models that have gained popularity in recent years due to their ability to model complex distributions with high flexibility and expressiveness. In this work, we introduce a new type of normalizing flow that is tailored for modeling positions and orientations of multiple objects in three-dimensional space, such as molecules in a crystal. Our approach is based on two key ideas: first, we define smooth and expressive flows on the group of unit quaternions, which allows us to capture the continuous rotational motion of rigid bodies; second, we use the double cover property of unit quaternions to define a proper density on the rotation group. This ensures that our model can be trained using standard likelihood-based methods or variational inference with respect to a thermodynamic target density. We evaluate the method by training Boltzmann generators for two molecular examples, namely the multi-modal density of a tetrahedral system in an external field and the ice XI phase in the TIP4P water model. Our flows can be combined with flows operating on the internal degrees of freedom of molecules and constitute an important step towards the modeling of distributions of many interacting molecules., Comment: International Conference on Machine Learning, 2023

Seismic phase picking and magnitude estimation are essential components of real time earthquake monitoring and earthquake early warning systems. Reliable phase picking enables the timely detection of seismic wave arrivals, facilitating rapid earthquake characterization and early warning alerts. Accurate magnitude estimation provides crucial information about the size of an earthquake and potential impact. Together, these steps contribute to effective earthquake monitoring, enhancing our ability to implement appropriate response measures in seismically active regions and mitigate risks. In this study, we explore the potential of deep learning in real time earthquake monitoring. To that aim, we begin by introducing DynaPicker which leverages dynamic convolutional neural networks to detect seismic body wave phases. Subsequently, DynaPicker is employed for seismic phase picking on continuous seismic recordings. To showcase the efficacy of Dynapicker, several open source seismic datasets including window format data and continuous seismic data are used for seismic phase identification, and arrival time picking. Additionally,the robustness of DynaPicker in classifying seismic phases was tested on the low magnitude seismic data polluted by noise. Finally, the phase arrival time information is integrated into a previously published deep learning model for magnitude estimation. This workflow is then applied and tested on the continuous recording of the aftershock sequences following the Turkey earthquake to detect the earthquakes, seismic phase picking and estimate the magnitude of the corresponding event. The results obtained in this case study exhibit a high level of reliability in detecting the earthquakes and estimating the magnitude of aftershocks following the Turkey earthquake.

Coarse-grained (CG) molecular simulations have become a standard tool to study molecular processes on time- and length-scales inaccessible to all-atom simulations. Parameterizing CG force fields to match all-atom simulations has mainly relied on force-matching or relative entropy minimization, which require many samples from costly simulations with all-atom or CG resolutions, respectively. Here we present flow-matching, a new training method for CG force fields that combines the advantages of both methods by leveraging normalizing flows, a generative deep learning method. Flow-matching first trains a normalizing flow to represent the CG probability density, which is equivalent to minimizing the relative entropy without requiring iterative CG simulations. Subsequently, the flow generates samples and forces according to the learned distribution in order to train the desired CG free energy model via force matching. Even without requiring forces from the all-atom simulations, flow-matching outperforms classical force-matching by an order of magnitude in terms of data efficiency, and produces CG models that can capture the folding and unfolding transitions of small proteins.

Many active volcanoes in the world exhibit Strombolian activity, which is typically characterized by relatively frequent mild events and also by rare and much more destructive major explosions and paroxysms. Detailed analyses of past major and minor events can help to understand the eruptive behavior of the volcano and the underlying physical and chemical processes. Catalogs of volcanic eruptions may be established using continuous seismic recordings at stations in the proximity of volcanoes. However, in many cases, the analysis of the recordings relies heavily on the manual picking of events by human experts. Recently developed Machine Learning-based approaches require large training data sets which may not be available a priori. Here, we propose an alternative automated approach: the Adaptive-Window Volcanic Event Selection Analysis Module (AWESAM). This process of creating event catalogs consists of three main steps: (i) identification of potential volcanic events based on squared ground-velocity amplitudes, an adaptive MaxFilter, and a prominence threshold. (ii) catalog consolidation by comparing and verification of the initial detections based on recordings from two different seismic stations. (iii) identification and exclusion of signals from regional tectonic earthquakes. The software package is applied to publicly accessible continuous seismic recordings from two almost equidistant stations at Stromboli volcano in Italy. We tested AWESAM by comparison with a hand-picked catalog and found that around 95 percent of the eruptions with a signal-to-noise ratio above three are detected. In a first application, we derive a new amplitude-frequency relationship from over 290.000 volcanic events at Stromboli during 2019-2020. The module allows for a straightforward generalization and application to other volcanoes worldwide.

Normalizing flows are a promising tool for modeling probability distributions in physical systems. While state-of-the-art flows accurately approximate distributions and energies, applications in physics additionally require smooth energies to compute forces and higher-order derivatives. Furthermore, such densities are often defined on non-trivial topologies. A recent example are Boltzmann Generators for generating 3D-structures of peptides and small proteins. These generative models leverage the space of internal coordinates (dihedrals, angles, and bonds), which is a product of hypertori and compact intervals. In this work, we introduce a class of smooth mixture transformations working on both compact intervals and hypertori. Mixture transformations employ root-finding methods to invert them in practice, which has so far prevented bi-directional flow training. To this end, we show that parameter gradients and forces of such inverses can be computed from forward evaluations via the inverse function theorem. We demonstrate two advantages of such smooth flows: they allow training by force matching to simulation data and can be used as potentials in molecular dynamics simulations., Comment: Neural Information Proceessing Systems (NeurIPS) 2021

Spatial and temporal data for earthquakes with magnitude M greater than or equal to 6.5 can provide crucial information about the seismic history and potential for large earthquakes in a region. We analyzed approximately 313,500 events that occurred in the Sunda-arc region during the last 56 years, from 1964 to 2020, reported by the International Seismological Center. We report a persistent increase in the annual number of the events with mb greater than or equal to 6.5. We tested this increase against the null hypothesis and discarded the possibility of the increase being due to random groupings. The trend given by Auto-Regressive Integrated Moving Average suggests continuing increase of such large-magnitude events in the region during the next decade. At the same time, the computed Gutenberg Richter b value shows anomalies that can be related to the occurrence of the mega 2004 Sumatra earthquake, and to possible state of high tectonic stress in the eastern parts of the region.

Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.

Many types of neural network layers rely on matrix properties such as invertibility or orthogonality. Retaining such properties during optimization with gradient-based stochastic optimizers is a challenging task, which is usually addressed by either reparameterization of the affected parameters or by directly optimizing on the manifold. This work presents a novel approach for training invertible linear layers. In lieu of directly optimizing the network parameters, we train rank-one perturbations and add them to the actual weight matrices infrequently. This P$^{4}$Inv update allows keeping track of inverses and determinants without ever explicitly computing them. We show how such invertible blocks improve the mixing and thus the mode separation of the resulting normalizing flows. Furthermore, we outline how the P$^4$ concept can be utilized to retain properties other than invertibility., Comment: 17 pages, 10 figures

Normalizing flows are exact-likelihood generative neural networks which approximately transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry. To scale and generalize these results, it is essential that the natural symmetries in the probability density -- in physics defined by the invariances of the target potential -- are built into the flow. We provide a theoretical sufficient criterion showing that the distribution generated by \textit{equivariant} normalizing flows is invariant with respect to these symmetries by design. Furthermore, we propose building blocks for flows which preserve symmetries which are usually found in physical/chemical many-body particle systems. Using benchmark systems motivated from molecular physics, we demonstrate that those symmetry preserving flows can provide better generalization capabilities and sampling efficiency.

The sampling of probability distributions specified up to a normalization constant is an important problem in both machine learning and statistical mechanics. While classical stochastic sampling methods such as Markov Chain Monte Carlo (MCMC) or Langevin Dynamics (LD) can suffer from slow mixing times there is a growing interest in using normalizing flows in order to learn the transformation of a simple prior distribution to the given target distribution. Here we propose a generalized and combined approach to sample target densities: Stochastic Normalizing Flows (SNF) -- an arbitrary sequence of deterministic invertible functions and stochastic sampling blocks. We show that stochasticity overcomes expressivity limitations of normalizing flows resulting from the invertibility constraint, whereas trainable transformations between sampling steps improve efficiency of pure MCMC/LD along the flow. By invoking ideas from non-equilibrium statistical mechanics we derive an efficient training procedure by which both the sampler's and the flow's parameters can be optimized end-to-end, and by which we can compute exact importance weights without having to marginalize out the randomness of the stochastic blocks. We illustrate the representational power, sampling efficiency and asymptotic correctness of SNFs on several benchmarks including applications to sampling molecular systems in equilibrium.

Developing a differentially private deep learning algorithm is challenging, due to the difficulty in analyzing the sensitivity of objective functions that are typically used to train deep neural networks. Many existing methods resort to the stochastic gradient descent algorithm and apply a pre-defined sensitivity to the gradients for privatizing weights. However, their slow convergence typically yields a high cumulative privacy loss. Here, we take a different route by employing the method of auxiliary coordinates, which allows us to independently update the weights per layer by optimizing a per-layer objective function. This objective function can be well approximated by a low-order Taylor's expansion, in which sensitivity analysis becomes tractable. We perturb the coefficients of the expansion for privacy, which we optimize using more advanced optimization routines than SGD for faster convergence. We empirically show that our algorithm provides a decent trained model quality under a modest privacy budget.

Flows are exact-likelihood generative neural networks that transform samples from a simple prior distribution to the samples of the probability distribution of interest. Boltzmann Generators (BG) combine flows and statistical mechanics to sample equilibrium states of strongly interacting many-body systems such as proteins with 1000 atoms. In order to scale and generalize these results, it is essential that the natural symmetries of the probability density - in physics defined by the invariances of the energy function - are built into the flow. Here we develop theoretical tools for constructing such equivariant flows and demonstrate that a BG that is equivariant with respect to rotations and particle permutations can generalize to sampling nontrivially new configurations where a nonequivariant BG cannot.

Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular Dynamics. Combining deep learning and statistical mechanics, we here develop Boltzmann Generators, that are shown to generate unbiased one-shot equilibrium samples of representative condensed matter systems and proteins. Boltzmann Generators use neural networks to learn a coordinate transformation of the complex configurational equilibrium distribution to a distribution that can be easily sampled. Accurate computation of free energy differences and discovery of new configurations are demonstrated, providing a statistical mechanics tool that can avoid rare events during sampling without prior knowledge of reaction coordinates.

Wave-particle duality is an inherent peculiarity of the quantum world. The double-slit experiment has been frequently used for understanding different aspects of this fundamental concept. The occurrence of interference rests on the lack of which-way information and on the absence of decoherence mechanisms, which could scramble the wave fronts. In this letter, we report on the observation of two-center interference in the molecular frame photoelectron momentum distribution upon ionization of the neon dimer by a strong laser field. Postselection of ions, which were measured in coincidence with electrons, allowed choosing the symmetry of the continuum electronic wave function, leading to observation of both, gerade and ungerade, types of interference.

Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression., Comment: Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018

The success of convolutional networks in learning problems involving planar signals such as images is due to their ability to exploit the translation symmetry of the data distribution through weight sharing. Many areas of science and egineering deal with signals with other symmetries, such as rotation invariant data on the sphere. Examples include climate and weather science, astrophysics, and chemistry. In this paper we present spherical convolutional networks. These networks use convolutions on the sphere and rotation group, which results in rotational weight sharing and rotation equivariance. Using a synthetic spherical MNIST dataset, we show that spherical convolutional networks are very effective at dealing with rotationally invariant classification problems.