Search

Automatic differentiation (AD) emerged as an integral part of machine learning, accelerating model development by enabling gradient-based optimization without explicit analytical derivatives. Recently, the benefits of AD and computing arbitrary-order derivatives with respect to any variable were also recognized in the field of quantum chemistry. In this work, we present dxtb—an open-source, fully differentiable framework for semiempirical extended tight-binding (xTB) methods. Developed entirely in Python and leveraging PyTorch for array operations, dxtb facilitates extensibility and rapid prototyping while maintaining computational efficiency. Through comprehensive code vectorization and optimization, we essentially reach the speed of compiled xTB programs for high-throughput calculations of small molecules. The excellent performance also scales to large systems, and batch operability yields additional benefits for execution on parallel hardware. In particular, energy evaluations are on par with existing programs, whereas the speed of automatically differentiated nuclear derivatives is only 2 to 5 times slower compared to their analytical counterparts. We showcase the utility of AD in dxtb by calculating various molecular and spectroscopic properties, highlighting its capacity to enhance and simplify such evaluations. Furthermore, the framework streamlines optimization tasks and offers seamless integration of semiempirical quantum chemistry in machine learning, paving the way for physics-inspired end-to-end differentiable models. Ultimately, dxtb aims to further advance the capabilities of semiempirical methods, providing an extensible foundation for future developments and hybrid machine learning applications. The framework is accessible at https://github.com/grimme-lab/dxtb. [ABSTRACT FROM AUTHOR]

In this work, we introduce a differentiable implementation of the local natural orbital coupled cluster (LNO-CC) method within the automatic differentiation framework of the PySCFAD package. The implementation is comprehensively tuned for enhanced performance, which enables the calculation of first-order static response properties on medium-sized molecular systems using coupled cluster theory with single, double, and perturbative triple excitations [CCSD(T)]. We evaluate the accuracy of our method by benchmarking it against the canonical CCSD(T) reference for nuclear gradients, dipole moments, and geometry optimizations. In addition, we demonstrate the possibility of property calculations for chemically interesting systems through the computation of bond orders and Mössbauer spectroscopy parameters for a [NiFe]-hydrogenase active site model, along with the simulation of infrared spectra via ab initio LNO-CC molecular dynamics for a protonated water hexamer. [ABSTRACT FROM AUTHOR]

We present a method for calculating first-order response properties in phaseless auxiliary field quantum Monte Carlo by applying automatic differentiation (AD). Biases and statistical efficiency of the resulting estimators are discussed. Our approach demonstrates that AD enables the calculation of reduced density matrices with the same computational cost scaling per sample as energy calculations, accompanied by a cost prefactor of less than four in our numerical calculations. We investigate the role of self-consistency and trial orbital choice in property calculations. We find that orbitals obtained using density functional theory perform well for the dipole moments of selected molecules compared to those optimized self-consistently. [ABSTRACT FROM AUTHOR]

Machine-learning potentials provide computationally efficient and accurate approximations of the Born–Oppenheimer potential energy surface. This potential determines many materials properties and simulation techniques usually require its gradients, in particular forces and stress for molecular dynamics, and heat flux for thermal transport properties. Recently developed potentials feature high body order and can include equivariant semi-local interactions through message-passing mechanisms. Due to their complex functional forms, they rely on automatic differentiation (AD), overcoming the need for manual implementations or finite-difference schemes to evaluate gradients. This study discusses how to use AD to efficiently obtain forces, stress, and heat flux for such potentials, and provides a model-independent implementation. The method is tested on the Lennard-Jones potential, and then applied to predict cohesive properties and thermal conductivity of tin selenide using an equivariant message-passing neural network potential. [ABSTRACT FROM AUTHOR]

As a widely used equation in electrostatics, the Poisson equation has significant research value in numerical solution. The basic principle of existing methods is to divide the solution domain into various grids and solve the numerical solutions at each grid node. Therefore, the accuracy of the solution is strongly correlated with the grid density divided. Based on this, this paper proposes a grid-free numerical calculation method that requires far fewer model parameters than traditional methods, and can ignore the order of the equation to solve high-dimensional Poisson equations. Given a Poisson equation, which has a certain type of boundary condition. A certain number of coordinate points are selected on the solution space and its boundary to construct a dataset. Using automatic differentiation technique to fit the differential operator in the equation, a loss function is constructed by incorporating the given boundary conditions or initial conditions, and the final numerical solution is obtained through iterative optimization algorithms. In the numerical experiment section, the algorithm proposed in this paper was used to solve the two-dimensional and three-dimensional Poisson equations with given exact solutions. The relative errors between the numerical solution and the true solution were 8. 5 3 e − 4 and 6. 4 0 e − 2 , which are within the acceptable range. This proves that the proposed algorithm is feasible for solving the two-dimensional and three-dimensional Poisson equations with precise solutions. Secondly, the proposed algorithm is used to solve the four-dimensional Poisson equation with first-type boundary conditions, and the relative error range of the solution was within [0,0.56], which successfully extends the algorithm to solve high-dimensional Poisson equations and verifies its feasibility and efficiency in solving high-dimensional Poisson equations regardless of the dimension restriction. [ABSTRACT FROM AUTHOR]

Recently, one artificial intelligence technique, known as artificial neural network (ANN), has brought advanced development to the arena of mathematical research. It competes effectively with other traditional methods in providing accurate solutions for fractional differential equations (FDEs). This work aims to implement a feedforward ANN with two hidden layers to solve nonlinear systems based on the fractional Riccati differential equation (FRDE). The network parameters are trained using the Adam optimization method with the aid of automatic differentiation. A vectorization algorithm is designated for the selected step to make the computation process more efficient. Two different initial value problems in integer-order derivatives and fractional-order derivatives are discussed. Numerical results demonstrate that the proposed method not only closely matches the exact solutions and reference solutions but also is more accurate than other existing methods. [ABSTRACT FROM AUTHOR]

A workshop at MIT discussed the importance of data assimilation and Earth system reanalysis in improving climate models. The workshop emphasized the need to bring models and observations into consistency to reduce uncertainties in climate projections. Recommendations were made for actions and funding in areas such as exploiting Earth observations, developing infrastructure for data assimilation and machine learning, and performing Earth system reanalysis. The document highlights the significance of using observations to improve models and calls for collaboration between different scientific communities to achieve these goals. The recommendations aim to enhance the quality and usability of climate information. [Extracted from the article]

An isogeometric topology optimization (ITO) model for multi-material structures under thermal-mechanical loadings using neural networks is proposed. In the proposed model, a non-uniform rational B-spline (NURBS) function is employed for geometric description and analytical calculation, which realizes the unification of the geometry and computational models. Neural networks replace the optimization algorithms of traditional topology optimization to update the relative densities of multi-material structures. The weights and biases of neural networks are taken as design variables and updated by automatic differentiation without derivation of the sensitivity formula. In addition, the grid elements can be refined directly by increasing the number of refinement nodes, resulting in high-resolution optimal topology without extra computational costs. To obtain comprehensive performance from ITO for multi-material structures, a weighting coefficient is introduced to regulate the proportion between thermal compliance and compliance in the loss function. Some numerical examples are given and the validity is verified by performance analysis. The optimal topological structures obtained based on the proposed model exhibit both excellent heat dissipation and stiffness performance under thermal-mechanical loadings. [ABSTRACT FROM AUTHOR]

Topology optimization has emerged as a versatile design tool embraced across diverse domains. This popularity has led to great efforts in the development of education-centric topology optimization codes with various focuses, such as targeting beginners seeking user-friendliness and catering to experienced users emphasizing computational efficiency. In this study, we introduce FEniTop, a novel 2D and 3D topology optimization software developed in Python and built upon the open-source FEniCSx library, designed to harmonize usability with computational efficiency and post-processing for fabrication. FEniTop employs a modular architecture, offering a unified input script for defining topology optimization problems and six replaceable modules to streamline subsequent optimization tasks. By enabling users to express problems in the weak form, FEniTop eliminates the need for matrix manipulations, thereby simplifying the modeling process. The software also integrates automatic differentiation to mitigate the intricacies associated with chain rules in finite element analysis and sensitivity analysis. Furthermore, FEniTop provides access to a comprehensive array of readily available solvers and preconditioners, bolstering flexibility in problem-solving. FEniTop is designed for scalability, furnishing robust support for parallel computing that seamlessly adapts to diverse computing platforms, spanning from laptops to distributed computing clusters. It also facilitates effortless transitions for various spatial dimensions, mesh geometries, element types, and orders, and quadrature degrees. Apart from the computational benefits, FEniTop facilitates the automated exportation of optimized designs, compatible with open-source ParaView software for post-processing. This functionality allows for visualizing optimized designs across diverse mesh geometries and element shapes, automatically smoothing 3D designs, and converting smoothed designs into STereoLithography (STL) files for 3D printing. To illustrate the capabilities of FEniTop, we present five representative examples showcasing topology optimization across 2D and 3D geometries, structured and unstructured meshes, solver switching, and complex boundary conditions. We also assess the parallel computational efficiency of FEniTop by examining its performance across diverse computing platforms, process counts, problem sizes, and solver configurations. Finally, we demonstrate a physical 3D-printed model utilizing the STL file derived from the design optimized by FEniTop. These examples showcase not only FEniTop’s rich functionality but also its parallel computing performance. The open-source FEniTop is given in "Appendix B" and will be available on GitHub. [ABSTRACT FROM AUTHOR]

Copyright of Automatisierungstechnik is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We present ReactionMechanismSimulator.jl (RMS), a modern differentiable software for the simulation and analysis of chemical kinetic mechanisms, including multiphase systems. RMS has already been applied to problems in combustion, pyrolysis, polymers, pharmaceuticals, catalysis, and electrocatalysis. RMS is written in Julia, making it easy to develop and allowing it to take advantage of Julia's extensive numerical computing ecosystem. In addition to its extensive library of optimized analytic Jacobians, RMS can generate and use Jacobians computed using automatic differentiation and symbolically generated analytic Jacobians. RMS is demonstrated to be faster than Cantera and Chemkin in several benchmarks. RMS also implements an extensive set of features for analyzing chemical mechanisms, including a library of easy‐to‐call plotting functions, molecular structure resolved flux diagram generation, crash analysis, traditional sensitivity analysis, transitory sensitivity analysis, and an automatic mechanism analysis toolkit. RMS implements efficient adjoint and parallel forward sensitivity analyses. We also demonstrate the ease of adding new features to RMS. [ABSTRACT FROM AUTHOR]

Direct design of complex functional materials would revolutionize technologies ranging from printable organs to novel clean energy devices. However, even incremental steps toward designing functional materials have proven challenging. If the material is constructed from highly complex components, the design space of materials properties rapidly becomes too computationally expensive to search. On the other hand, very simple components such as uniform spherical particles are not powerful enough to capture rich functional behavior. Here, we introduce a differentiable materials design model with components that are simple enough to design yet powerful enough to capture complex materials properties: rigid bodies composed of spherical particles with directional interactions (patchy particles). We showcase the method with selfassembly designs ranging from open lattices to self-limiting clusters, all of which are notoriously challenging design goals to achieve using purely isotropic particles. By directly optimizing over the location and interaction of the patches on patchy particles using gradient descent, we dramatically reduce the computation time for finding the optimal building blocks. [ABSTRACT FROM AUTHOR]

Investigating subsurface shear wave velocity (v s) structures using surface wave dispersion data involves minimizing a misfit function that is commonly solved through gradient-based optimization. Sensitivity kernels for model updates are commonly estimated using numerical differentiation, variational methods or implicit functions which however, may involve numerical instability and computational challenges when dealing with complex velocity models and large data sets. In this study, we propose a novel surface wave inversion framework in which error-free gradients are calculated by automatic differentiation (AD) and forward modelling is implemented by convenient computational graphs in the state-of-the-art deep learning framework. The AD-based inversion approach is first validated using two synthetic data sets. Then, the subsurface structures at three distinct locations, namely the Great Plains and the Long Beach in the US and Tong Zhou in China, are also derived using this method with seismic ambient noise data, which show nice consistency with those obtained using traditional methods. With the significantly improved computational efficiency, a great number of initial models can be inverted simultaneously to mitigate the impact of local minima and to estimate the uncertainty in the invert models. We have developed a new surface wave inversion package named ADsurf based on automatic differentiation and computational graphs in the deep learning framework, and its computational efficiency is also compared with the traditional finite-difference-based gradient estimation approach. While a great number of intriguing studies on the geophysical inverse problems have been conducted recently using deep learning for end-to-end mapping, the use of AD provided in the in the deep learning frameworks to assist and expedite the gradient computations are still underexploited in geophysics. Thus, it is expected that various geophysical inverse problems in many different areas beyond the surface wave inversion can also be tackled with this new paradigm in the future. [ABSTRACT FROM AUTHOR]

Neural networks have gained popularity for modeling complex non-linear relationships. Their computational efficiency has led to their growing adoption in optimization methods, including topology optimization. Recently, there have been several contributions toward improving derivatives of neural network outputs, which can improve their use in gradient-based optimization. However, a comparative study has yet to be conducted on the different derivative methods for the sensitivity of the input features on the neural network outputs. This paper aims to evaluate four derivative methods: analytical neural network's Jacobian, central finite difference method, complex step method, and automatic differentiation. These methods are implemented into density-based and homogenization-based topology optimization using multilayer perceptrons (MLPs). For density-based topology optimization, the MLP approximates Young's modulus for the solid isotropic material with penalization (SIMP) model. For homogenization-based topology optimization, the MLP approximates the homogenized stiffness tensor of a representative volume element, e.g., square cell microstructure with a rectangular hole. The comparative study is performed by solving two-dimensional topology optimization problems using the sensitivity coefficients from each derivative method. Evaluation includes initial sensitivity coefficients, convergence plots, and the final topologies, compliance, and design variables. The findings demonstrate that neural network-based sensitivity coefficients are sufficiently accurate for density-based and homogenization-based topology optimization. The neural network's Jacobian, complex step method, and automatic differentiation produced identical sensitivity coefficients to working precision. The study's open-source code is provided through a python repository. [ABSTRACT FROM AUTHOR]

We present a high-level, differentiable, and composable abstraction for the point evaluation of the solution fields of partial differential equation models. The new functionality, embedded in the Firedrake automated finite element system, enables modellers to easily assimilate point data into their models at the point locations, rather than resorting to extrapolation to a computational mesh. We demonstrate the expressiveness and ease with which more mathematically defensible data assimilation can be performed with examples in the fields of groundwater hydrology and glaciology. In various geoscience disciplines, modellers seek to estimate fields that are challenging to directly observe using measurements of other related fields. These measurements are often sparse, and it is common practice to first extrapolate these measurements to the grid or mesh used for computations. When this estimation procedure is viewed as a deterministic inverse problem, the extrapolation step is undesirable because the choice of extrapolation method introduces an arbitrary algorithmic degree of freedom that can alter the outcomes. When the estimation procedure is instead viewed through the lens of statistical inference, the extrapolation step is undesirable for the additional reason that it obscures the number of statistically independent measurements that are assimilated and thus makes it impossible to apply statistical goodness-of-fit tests or model selection criteria. The introduction of point evaluation into Firedrake, together with its integration into the automatic differentiation features of the system, greatly facilitates the direct assimilation of point data and thus improved methodology for solving both deterministic and statistical inverse problems. [ABSTRACT FROM AUTHOR]

Nonlinear optimization (NOPT) is a meaningful tool for solving complex tasks in fields like engineering, economics, and operations research, among others. However, NOPT has problems when it comes to dealing with data variability and noisy input measurements that lead to incorrect solutions. Furthermore, nonlinear constraints may result in outcomes that are either infeasible or suboptimal, such as nonconvex optimization. This paper introduces a novel regularized physics-informed neural network (RPINN) framework as a new NOPT tool for both supervised and unsupervised data-driven scenarios. Our RPINN is threefold: By using custom activation functions and regularization penalties in an artificial neural network (ANN), RPINN can handle data variability and noisy inputs. Furthermore, it employs physics principles to construct the network architecture, computing the optimization variables based on network weights and learned features. In addition, it uses automatic differentiation training to make the system scalable and cut down on computation time through batch-based back-propagation. The test results for both supervised and unsupervised NOPT tasks show that our RPINN can provide solutions that are competitive compared to state-of-the-art solvers. In turn, the robustness of RPINN against noisy input measurements makes it particularly valuable in environments with fluctuating information. Specifically, we test a uniform mixture model and a gas-powered system as NOPT scenarios. Overall, with RPINN, its ANN-based foundation offers significant flexibility and scalability. [ABSTRACT FROM AUTHOR]

Recent advances in differentiable modeling, a genre of physics-informed machine learning that trains neural networks (NNs) together with process-based equations, have shown promise in enhancing hydrological models' accuracy, interpretability, and knowledge-discovery potential. Current differentiable models are efficient for NN-based parameter regionalization, but the simple explicit numerical schemes paired with sequential calculations (operator splitting) can incur numerical errors whose impacts on models' representation power and learned parameters are not clear. Implicit schemes, however, cannot rely on automatic differentiation to calculate gradients due to potential issues of gradient vanishing and memory demand. Here we propose a "discretize-then-optimize" adjoint method to enable differentiable implicit numerical schemes for the first time for large-scale hydrological modeling. The adjoint model demonstrates comprehensively improved performance, with Kling–Gupta efficiency coefficients, peak-flow and low-flow metrics, and evapotranspiration that moderately surpass the already-competitive explicit model. Therefore, the previous sequential-calculation approach had a detrimental impact on the model's ability to represent hydrological dynamics. Furthermore, with a structural update that describes capillary rise, the adjoint model can better describe baseflow in arid regions and also produce low flows that outperform even pure machine learning methods such as long short-term memory networks. The adjoint model rectified some parameter distortions but did not alter spatial parameter distributions, demonstrating the robustness of regionalized parameterization. Despite higher computational expenses and modest improvements, the adjoint model's success removes the barrier for complex implicit schemes to enrich differentiable modeling in hydrology. [ABSTRACT FROM AUTHOR]

Four-dimensional (4D) constellations with up to 131 072 points (17 bit/4D-sym) are designed for the first time using geometric shaping. The constellations are optimized in terms of mutual information (MI) and generalized MI (GMI) for the additive white Gaussian noise (AWGN) channel, targeting a forward error correction (FEC) rate of 0.8 at finite signal-to-noise ratios. The presented 15–17 bit constellations are currently the highest-performing constellations in the literature, having a gap to the AWGN capacity as low as 0.17 dB (MI) and 0.45 dB (GMI) at 17 bit/4D-sym. For lower cardinalities, our constellations match or closely approach the performance of previously published optimized constellations. We also show that (GMI-)optimized constellations with a symmetry constraint, optimized for a FEC rate of 0.8, perform nearly identical to their unconstrained counterparts for cardinalities above 8 bit/4D-sym. A symmetry constraint for MI-optimized constellations is shown to have a negative impact in general. The proposed procedure relies on a Monte-Carlo-based approach for evaluating performance and is extendable to other (nonlinear) channels. Stochastic gradient descent is used for the optimization algorithm for which the gradients are computed using automatic differentiation. This article is part of the theme issue 'Celebrating the 15th anniversary of the Royal Society Newton International Fellowship'. [ABSTRACT FROM AUTHOR]

Semiempirical quantum chemistry has recently seen a renaissance with applications in high-throughput virtual screening and machine learning. The simplest semiempirical model still in widespread use in chemistry is Hückel's π-electron molecular orbital theory. In this work, we implemented a Hückel program using differentiable programming with the JAX framework based on limited modifications of a pre-existing NumPy version. The auto-differentiable Hückel code enabled efficient gradient-based optimization of model parameters tuned for excitation energies and molecular polarizabilities, respectively, based on as few as 100 data points from density functional theory simulations. In particular, the facile computation of the polarizability, a second-order derivative, via auto-differentiation shows the potential of differentiable programming to bypass the need for numeric differentiation or derivation of analytical expressions. Finally, we employ gradient-based optimization of atom identity for inverse design of organic electronic materials with targeted orbital energy gaps and polarizabilities. Optimized structures are obtained after as little as 15 iterations using standard gradient-based optimization algorithms. [ABSTRACT FROM AUTHOR]

We present a simple, differentiable method for predicting emission line strengths from rest-frame optical continua using an empirically determined mapping. Extensive work has been done to develop mock galaxy catalogues that include robust predictions for galaxy photometry, but reliably predicting the strengths of emission lines has remained challenging. Our new mapping is a simple neural network implemented using the JAX Python automatic differentiation library. It is trained on Dark Energy Spectroscopic Instrument Early Release data to predict the equivalent widths (EWs) of the eight brightest optical emission lines (including H α, H β, [O ii ], and [O iii ]) from a galaxy's rest-frame optical continuum. The predicted EW distributions are consistent with the observed ones when noise is accounted for, and we find Spearman's rank correlation coefficient ρ s > 0.87 between predictions and observations for most lines. Using a non-linear dimensionality reduction technique, we show that this is true for galaxies across the full range of observed spectral energy distributions. In addition, we find that adding measurement uncertainties to the predicted line strengths is essential for reproducing the distribution of observed line-ratios in the BPT diagram. Our trained network can easily be incorporated into a differentiable stellar population synthesis pipeline without hindering differentiability or scalability with GPUs. A synthetic catalogue generated with such a pipeline can be used to characterize and account for biases in the spectroscopic training sets used for training and calibration of photo- z 's, improving the modelling of systematic incompleteness for the Rubin Observatory LSST and other surveys. [ABSTRACT FROM AUTHOR]

A dynamical systems approach to turbulence envisions the flow as a trajectory through a high-dimensional state space [Hopf, Commun. Appl. Maths 1, 303 (1948)]. The chaotic dynamics are shaped by the unstable simple invariant solutions populating the inertial manifold. The hope has been to turn this picture into a predictive framework where the statistics of the flow follow from a weighted sum of the statistics of each simple invariant solution. Two outstanding obstacles have prevented this goal from being achieved: 1) paucity of known solutions and 2) the lack of a rational theory for predicting the required weights. Here, we describe a method to substantially solve these problems, and thereby provide compelling evidence that the probability density functions (PDFs) of a fully developed turbulent flow can be reconstructed with a set of unstable periodic orbits. Our method for finding solutions uses automatic differentiation, with high-quality guesses constructed by minimizing a trajectorydependent loss function. We use this approach to find hundreds of solutions in turbulent, two-dimensional Kolmogorov flow. Robust statistical predictions are then computed by learning weights after converting a turbulent trajectory into a Markov chain for which the states are individual solutions, and the nearest solution to a given snapshot is determined using a deep convolutional autoencoder. In this study, the PDFs of a spatiotemporally chaotic system have been successfully reproduced with a set of simple invariant states, and we provide a fascinating connection between self-sustaining dynamical processes and the more well-known statistical properties of turbulence. [ABSTRACT FROM AUTHOR]

A long‐standing challenge in volume visualization is the effective communication of relevant spatial structures that might be hidden due to occlusions. Given a scalar field that indicates the importance of every point in the domain, previous work synthesized volume visualizations by weighted averaging of samples along view rays or by optimizing a spatially‐varying extinction field through an energy minimization. This energy minimization, however, did not directly measure the contribution of an individual sample to the final pixel color. In this paper, we measure the visibility of relevant structures directly by incorporating the transmittance into a non‐linear energy minimization. For the first time, we not only perform a transmittance‐based extinction optimization, we concurrently optimize the camera position to find ideal viewpoints. We derive the partial derivatives for the gradient‐based optimization symbolically, which makes the application of automatic differentiation methods unnecessary. The transmittance‐based formulation gives a direct visibility measure that is communicated to the user in order to make aware of potentially overlooked relevant structures. Our approach is compatible with any measure of importance and its versatility is demonstrated in multiple data sets. [ABSTRACT FROM AUTHOR]

Continuum-kinematics-inspired peridynamics (CPD) has been recently proposed as a novel reformulation of peridynamics that is characterized by one-, two- and three-neighbor interactions. CPD is geometrically exact and thermodynamically consistent and does not suffer from zero-energy modes, displacement oscillations or material interpenetration. In this manuscript, for the first time, we develop a computational framework furnished with automatic differentiation for the implementation of CPD. Thereby, otherwise tedious analytical differentiation is automatized by employing hyper-dual numbers (HDN). This differentiation method does not suffer from round-off errors, subtractive cancellation errors or truncation errors and is thereby highly stable with superb accuracy being insensitive to perturbation values. The computational framework provided here is compact and model-independent, thus once the framework is implemented, any other material model can be incorporated via modifying the potential energy solely. Finally, to illustrate the versatility of our proposed framework, various potential energies are considered and the corresponding material response is examined for different scenarios. [ABSTRACT FROM AUTHOR]

The paper is devoted to a problem of acquiring elastic properties of a composite material from the vibration testing data with a simplified experimental acquisition scheme. The specimen is considered to abide by the linear elasticity laws and subject to viscoelastic damping. The boundary value problem for transverse movement of such a specimen in the frequency domain is formulated and solved with finite-element method. The correction method is suggested for the finite element matrices to account for the mass of the accelerometer. The problem of acquiring the elastic parameters is then formulated as a nonlinear least-square optimization problem. The usage of the automatic differentiation technique for stable and efficient computation of the gradient and hessian allows to use well-studied first and second order local optimization methods. We also explore the possibility of generating initial guesses for local minimization by heuristic global methods. The results of the numerical experiments on simulated data are analyzed in order to provide insights for the following real life experiments. [ABSTRACT FROM AUTHOR]

Defect reconstruction is essential in non-destructive testing and structural health monitoring with guided ultrasonic waves. This paper presents an algorithm for reconstructing notches in steel plates, which can be seen as artificial defects representing cracks by comparing measured results with those from a simulation model. The model contains a parameterized notch, and its geometrical parameters are to be reconstructed. While the algorithm is formulated and presented in a general notation, a special case of guided wave propagation is used to investigate one of the simplest possible simulation models that discretizes only the cross section of the steel plate. An efficient simulation model of the plate cross section is obtained by the semi-analytical scaled boundary finite element method. The reconstruction algorithm applied is gradient-based, and algorithmic differentiation calculates the gradient. The dedicated experimental setup excites nearly plane wave fronts propagating orthogonal to the notch. A scanning laser Doppler vibrometer records the velocity field at certain points on the plate surface as input to the reconstruction algorithm. Using two plates with notches of different depths, it is demonstrated that accurate geometry reconstruction is possible. [ABSTRACT FROM AUTHOR]

Training in Feed Forward Deep Neural Networks is a memory-intensive operation which is usually performed on GPUs with limited memory capacities. This may force data scientists to limit the depth of the models or the resolution of the input data if data does not fit in the GPU memory. The re-materialization technique, whose idea comes from the checkpointing strategies developed in the Automatic Differentiation literature, allows data scientists to limit the memory requirements related to the storage of intermediate data (activations), at the cost of an increase in the computational cost. This paper introduces a new strategy of re-materialization of activations that significantly reduces memory usage. It consists in selecting which activations are saved and which activations are deleted during the forward phase and then recomputing the deleted activations when they are needed during the backward phase. We propose an original computation model that combines two types of activation savings: either only storing the layer inputs or recording the complete history of operations that produced the outputs. This paper focuses on the fully heterogeneous case, where the computation time and the memory requirement of each layer is different. We prove that finding the optimal solution is NP-hard and that classical techniques from Automatic Differentiation literature do not apply. Moreover, the classical assumption of memory persistence of materialized activations, used to simplify the search of optimal solutions, does not hold anymore. Thus, we propose a weak memory persistence property and provide a dynamic program to compute the optimal sequence of computations. This algorithm is made available through the Rotor software, a PyTorch plug-in dealing with any network consisting of a sequence of layers, each of them having an arbitrarily complex structure. Through extensive experiments, we show that our implementation consistently outperforms existing re-materialization approaches for a large class of networks, image sizes, and batch sizes. [ABSTRACT FROM AUTHOR]

Time evolution of the populations of spin states coupled with bosons, which can be a model of photosynthetic excitation energy transfer of dye molecules surrounded by proteins, is simulated using the projected-variational quantum dynamics algorithm. By a transformation of the Hamiltonian describing the spin-boson model into the one-dimensional nearest-neighbor form, it is shown that the spin-boson model can be simulated using the sequential ansatz even if a quantum computer has limited connectivity. The optimization of the parametrized quantum circuits is performed by the gradient descent method on a classical computer using the automatic differentiation, and the population of the spins is simulated on a noisy intermediate-scale quantum computer. The error originating from the quantum computing is mitigated by the Clifford data regression, in which the noise channel is estimated using the data obtained from all the time steps. [ABSTRACT FROM AUTHOR]

Macromolecular complexes are often composed of diverse subunits. The self-assembly of these subunits is inherently nonequilibrium and must avoid kinetic traps to achieve high yield over feasible timescales. We show how the kinetics of self-assembly benefits from diversity in subunits because it generates an expansive parameter space that naturally improves the "expressivity" of self-assembly, much like a deeper neural network. By using automatic differentiation algorithms commonly used in deep learning, we searched the parameter spaces of mass-action kinetic models to identify classes of kinetic protocols that mimic biological solutions for productive self-assembly. Our results reveal how high-yield complexes that easily become kinetically trapped in incomplete intermediates can instead be steered by internal design of rate-constants or external and active control of subunits to efficiently assemble. Internal design of a hierarchy of subunit binding rates generates self-assembly that can robustly avoid kinetic traps for all concentrations and energetics, but it places strict constraints on selection of relative rates. External control via subunit titration is more versatile, avoiding kinetic traps for any system without requiring molecular engineering of binding rates, albeit less efficiently and robustly. We derive theoretical expressions for the timescales of kinetic traps, and we demonstrate our optimization method applies not just for design but inference, extracting intersubunit binding rates from observations of yield-vs.- time for a heterotetramer. Overall, we identify optimal kinetic protocols for self-assembly as a powerful mechanism to achieve efficient and high-yield assembly in synthetic systems whether robustness or ease of "designability" is preferred. [ABSTRACT FROM AUTHOR]

In the present article, a stability- and consistency-preserving integration scheme for polynomial Galerkin approaches of arbitrary order is presented. The basis is formed by Taylor series expansions of the stresses with respect to the strains, which in turn are expanded towards the spatial directions. With a split of the material and geometric nonlinearities and the assumption of a material behavior linearly variable within an element, the strain energy in elements of arbitrary shape and polynomial order can be evaluated exactly. Therefore, geometric moments have to be calculated in preprocessing, requiring only evaluations of derivatives at a single integration point during the analysis. The moments can be effectively integrated analytically over the boundary of the elements. As one of the manifold applications, the use in the context of second order virtual elements is elaborated for which the assembly time can be significantly reduced. The combination with the automatic differentiation and expression optimization software AceGen provides performant element routines. In the numerical examples, the integration scheme shows promising accuracy and makes the application in more complex material models up to computational homogenization attractive. [ABSTRACT FROM AUTHOR]

A modification of Newton's method for solving systems of n > 0 nonlinear equations is presented. The new matrix-free method is exact as opposed to a range of inexact Newton methods in the sense that both the Jacobians and the solutions to the linear Newton systems are computed without truncation. It relies on a given decomposition of a structurally dense invertible Jacobian of the residual into a product of q > 0 structurally sparse invertible elemental Jacobians according to the chain rule of differentiation. Inspired by the adjoint mode of algorithmic differentiation, explicit accumulation of the Jacobian of the residual is avoided. Prospective, generally applicable implementations of the new method can be based on similar ideas. Sparsity is exploited for the direct solution of the linear Newton systems. Optimal exploitation of sparsity yields various well-known computationally intractable combinatorial optimization problems in sparse linear algebra such as BANDWIDTH or DIRECTED ELIMINATION ORDERING. The method is motivated in the context of a decomposition into elemental Jacobians with bandwidth 2µ + 1 for 0 ≤ µ << n. In the likely scenario of q > n, the computational cost of the standard Newton algorithm is dominated by the cost of accumulating the Jacobian of the residual. It can be estimated as O(qµn²), thus exceeding the cost of O(n³) for the direct solution of the linear Newton system. The new method reduces this cost to O(qnµ²), yielding a potential improvement by a factor of O(n/µ). Supporting run time measurements are presented for the tridiagonal case showing a reduction of the computational cost by O(n). Generalization yields the combinatorial Matrix-Free Exact Newton Step problem. We prove NP-completeness, and we present algorithmic components for building methods for the approximate solution. Potential applications of the matrix-free exact Newton method in machine learning of surrogates for computationally expensive nonlinear residuals are touched on briefly as part of various conclusions to be drawn. [ABSTRACT FROM AUTHOR]

Inverse design methods use optimization and learning algorithms to pair desired functionalities with the corresponding high‐performing systems. Such methods have significant potential for discovering novel photonics solutions, with inverse design techniques already mediating significant milestones in nanophotonics, quantum optics, and lens systems. However, while computational tools for identifying optimal system parameters (i.e., component settings) have reached significant maturity, the identification of suitable system topologies (i.e., component choice and arrangement) has remained challenging, especially for the design of complex photonic schemes. Here, a framework for the inverse design of practical photonic systems is presented, capable of efficiently and automatically searching for high‐performance topologies and their associated operational parameters. It is demonstrated that the approach can aid in the discovery of practical photonic systems, that are both physically feasible and non‐trivial, by leveraging system‐level automatic differentiation and discrete topological changes. The versatility of the platform is supported with example designs for waveform generation, noise suppression, and sensing, among others. [ABSTRACT FROM AUTHOR]

Clustered competing risks data are a complex failure time data scheme. Its main characteristics are the cluster structure, which implies a latent within-cluster dependence between its elements, and its multiple variables competing to be the one responsible for the occurrence of an event, the failure. To handle this kind of data, we propose a full likelihood approach, based on generalized linear mixed models instead the usual complex frailty model. We model the competing causes in the probability scale, in terms of the cumulative incidence function (CIF). A multinomial distribution is assumed for the competing causes and censorship, conditioned on the latent effects that are accommodated by a multivariate Gaussian distribution. The CIF is specified as the product of an instantaneous risk level function with a failure time trajectory level function. The estimation procedure is performed through the R package Template Model Builder, an C++ based framework with efficient Laplace approximation and automatic differentiation routines. A large simulation study was performed, based on different latent structure formulations. The model fitting was challenging and our results indicated that a latent structure where both risk and failure time trajectory levels are correlated is required to reach reasonable estimation. [ABSTRACT FROM AUTHOR]

The adjoint method provides an efficient way to compute sensitivities for system models with a large number of inputs. However, implementing the adjoint method requires significant effort that limits its use. The effort is exacerbated in large-scale multidisciplinary design optimization. We propose the adoption of a three-stage compiler as the method for constructing computational models for large-scale multidisciplinary design optimization to enable accurate and efficient adjoint sensitivity analysis. We develop a new modeling language called the Computational System Design Language that provides an appropriate input to the compiler front end that works well with multidisciplinary models. This paper describes the three-stage compiler methodology and the Computational System Design Language. The proposed solution uses a graph representation of the numerical model to automatically generate a computational model that computes adjoint-based sensitivities for use within an optimization framework. For two engineering models, this approach reduces the amount of user code by a factor of approximately two compared to their original implementations, without a measurable increase in computation time. This paper also includes a best-case complexity analysis that is built into the compiler implementation to allow users to estimate the memory required to evaluate a computational model and its derivatives, which is independent of the compiler back end that ultimately generates the computational model. Future compiler implementations are expected to approach the theoretical best-case memory cost and improve run time performance for both model evaluation and derivative computation. [ABSTRACT FROM AUTHOR]

A new method is developed for accurately approximating the solution to state‐variable inequality path constrained optimal control problems using a multiple‐domain adaptive Legendre–Gauss–Radau collocation method. The method consists of the following parts. First, a structure detection method is developed to estimate switch times in the activation and deactivation of state‐variable inequality path constraints. Second, using the detected structure, the domain is partitioned into multiple‐domains where each domain corresponds to either a constrained or an unconstrained segment. Furthermore, additional decision variables are introduced in the multiple‐domain formulation, where these additional decision variables represent the switch times of the detected active state‐variable inequality path constraints. Within a constrained domain, the path constraint is differentiated with respect to the independent variable until the control appears explicitly, and this derivative is set to zero along the constrained arc while all preceding derivatives are set to zero at the start of the constrained arc. The time derivatives of the active state‐variable inequality path constraints are computed using automatic differentiation and the properties of the chain rule. The method is demonstrated on two problems, the first being a benchmark optimal control problem which has a known analytical solution and the second being a challenging problem from the field of aerospace engineering in which there is no known analytical solution. When compared against previously developed adaptive Legendre–Gauss–Radau methods, the results show that the method developed in this paper is capable of computing accurate solutions to problems whose solution contain active state‐variable inequality path constraints. [ABSTRACT FROM AUTHOR]

The nonlinear sine-Gordon equation is a prevalent feature in numerous scientific and engineering problems. In this paper, we propose a machine learning-based approach, physics-informed neural networks (PINNs), to investigate and explore the solution of the generalized non-linear sine-Gordon equation, encompassing Dirichlet and Neumann boundary conditions. To incorporate physical information for the sine-Gordon equation, a multiobjective loss function has been defined consisting of the residual of governing partial differential equation (PDE), initial conditions, and various boundary conditions. Using multiple densely connected independent artificial neural networks (ANNs), called feedforward deep neural networks designed to handle partial differential equations, PINNs have been trained through automatic differentiation to minimize a loss function that incorporates the given PDE that governs the physical laws of phenomena. To illustrate the effectiveness, validity, and practical implications of our proposed approach, two computational examples from the nonlinear sine-Gordon are presented. We have developed a PINN algorithm and implemented it using Python software. Various experiments were conducted to determine an optimal neural architecture. The network training was employed by using the current state-of-the-art optimization methods in machine learning known as Adam and L-BFGS-B minimization techniques. Additionally, the solutions from the proposed method are compared with the established analytical solutions found in the literature. The findings show that the proposed method is a computational machine learning approach that is accurate and efficient for solving nonlinear sine-Gordon equations with a variety of boundary conditions as well as any complex nonlinear physical problems across multiple disciplines. [ABSTRACT FROM AUTHOR]

Efficiently creating a concise but comprehensive data set for training machine-learned interatomic potentials (MLIPs) is an under-explored problem. Active learning, which uses biased or unbiased molecular dynamics (MD) to generate candidate pools, aims to address this objective. Existing biased and unbiased MD-simulation methods, however, are prone to miss either rare events or extrapolative regions—areas of the configurational space where unreliable predictions are made. This work demonstrates that MD, when biased by the MLIP's energy uncertainty, simultaneously captures extrapolative regions and rare events, which is crucial for developing uniformly accurate MLIPs. Furthermore, exploiting automatic differentiation, we enhance bias-forces-driven MD with the concept of bias stress. We employ calibrated gradient-based uncertainties to yield MLIPs with similar or, sometimes, better accuracy than ensemble-based methods at a lower computational cost. Finally, we apply uncertainty-biased MD to alanine dipeptide and MIL-53(Al), generating MLIPs that represent both configurational spaces more accurately than models trained with conventional MD. [ABSTRACT FROM AUTHOR]

In global optimization problems, diversification approaches are often necessary to overcome the convergence toward local optima. One approach is the multi-start method, where a set of different starting configurations are taken into account to designate the best local minimum returned by the multiple optimization procedures as the (possible) global optimum. Therefore, parallelization is crucial for multi-start. In this work, we present a new multi-start approach for gradient-based optimization methods that exploits the reverse Automatic Differentiation to perform efficiently. In particular, for each step, this Automatic Differentiation-based method is able to compute the N gradients of N optimization procedures extremely quickly, exploiting the implicit parallelization guaranteed by the computational graph representation of the multi-start problem. The practical advantages of the proposed method are illustrated by analyzing the time complexity from a theoretical point of view and showing numerical examples where the speed-up is between × 40 and × 100 , with respect to classic parallelization methods. Moreover, we show that our AD-based multi-start approach can be implemented by using tailored shallow Neural Networks, taking advantage of the built-in optimization procedures of the Deep Learning frameworks. [ABSTRACT FROM AUTHOR]

We study the excitation transfer across a fully connected quantum network whose sites energies can be artificially designed. Starting from a simplified model of a broadly-studied physical system, we systematically optimize its local energies to achieve high excitation transfer for various environmental conditions, using an adaptive Gradient Descent technique and Automatic Differentiation. We show that almost perfect transfer can be achieved with and without local dephasing, provided that the dephasing rates are not too large. We investigate our solutions in terms of resilience against variations in either the network connection strengths, or size, as well as coherence losses. We highlight the different features of a dephasing-free and dephasing-driven transfer. Our work gives further insight into the interplay between coherence and dephasing effects in excitation-transfer phenomena across fully connected quantum networks. In turn, this will help designing optimal transfer in artificial open networks through the simple manipulation of local energies. [ABSTRACT FROM AUTHOR]

Data-driven approximations of the Koopman operator are promising for predicting the time evolution of systems characterized by complex dynamics. Among these methods, the approach known as extended dynamic mode decomposition with dictionary learning (EDMD-DL) has garnered significant attention. Here, we present a modification of EDMD-DL that concurrently determines both the dictionary of observables and the corresponding approximation of the Koopman operator. This innovation leverages automatic differentiation to facilitate gradient descent computations through the pseudoinverse. We also address the performance of several alternative methodologies. We assess a "pure" Koopman approach, which involves the direct time-integration of a linear, high-dimensional system governing the dynamics within the space of observables. Additionally, we explore a modified approach where the system alternates between spaces of states and observables at each time step—this approach no longer satisfies the linearity of the true Koopman operator representation. For further comparisons, we also apply a state-space approach (neural ordinary differential equations). We consider systems encompassing two- and three-dimensional ordinary differential equation systems featuring steady, oscillatory, and chaotic attractors, as well as partial differential equations exhibiting increasingly complex and intricate behaviors. Our framework significantly outperforms EDMD-DL. Furthermore, the state-space approach offers superior performance compared to the "pure" Koopman approach where the entire time evolution occurs in the space of observables. When the temporal evolution of the Koopman approach alternates between states and observables at each time step, however, its predictions become comparable to those of the state-space approach. [ABSTRACT FROM AUTHOR]

The study of nonlinear problems related to heat transfer in a substance is of great practical important. Earlier, this paper's authors proposed an effective algorithm for determining the volumetric heat capacity and thermal conductivity of a substance based on experimental observations of the dynamics of the temperature field in the object. In this paper, the problem of simultaneous identification of temperature-dependent volumetric heat capacity and thermal conductivity of the substance under study from the heat flux at the boundary of the domain is investigated. The consideration is based on the first (Dirichlet) boundary value problem for a one-dimensional unsteady heat equation. The coefficient inverse problem under consideration is reduced to a variational problem, which is solved by gradient methods based on the application of fast automatic differentiation. The uniqueness of the solution of the inverse problem is investigated. [ABSTRACT FROM AUTHOR]

Differentiable physics is an approach that effectively combines physical models with deep learning, providing valuable information about physical systems during the training process of neural networks. This integration enhances the generalization ability and ensures better consistency with physical principles. In this work, we propose a framework for estimating the temperature of a permanent magnet synchronous motor by combining neural networks with the differentiable physical thermal model, as well as utilizing the simulation results. In detail, we first implement a differentiable thermal model based on a lumped parameter thermal network within an automatic differentiation framework. Subsequently, we add a neural network to predict thermal resistances, capacitances, and losses in real time and utilize the thermal parameters' optimized empirical values as the initial output values of the network to improve the accuracy and robustness of the final temperature estimation. We validate the conceivable advantages of the proposed method through extensive experiments based on both synthetic data and real-world data and then provide some further potential applications. [ABSTRACT FROM AUTHOR]

In this work, we propose a new deep learning-based scheme for solving high-dimensional nonlinear backward stochastic differential equations (BSDEs). The idea is to reformulate the problem as a global optimization problem where local loss functions are included. Essentially, we approximate the unknown solution of a BSDE using a deep neural network and the gradient of the approximated solution with automatic differentiation. The approximations are performed by globally minimizing the quadratic local loss function defined at each time step, which always includes the terminal condition. This kind of loss functions are obtained by iterating the Euler-Maruyama discretization of the integrals with the terminal condition. Our formulation can prompt the stochastic gradient descent algorithm not only to take the accuracy at each time layer into account, but also converge to a good local minima, namely an estimated solution near to the exact solution. In order to demonstrate performances of our algorithm, several high-dimensional nonlinear BSDEs including pricing problems in finance are provided. [ABSTRACT FROM AUTHOR]

In this work, we illustrate the implementation of physics informed neural networks (PINNs) for solving forward and inverse problems in structural vibration. Physics informed deep learning has lately proven to be a powerful tool for the solution and data-driven discovery of physical systems governed by differential equations. In spite of the popularity of PINNs, their application in structural vibrations is limited. This motivates the extension of the application of PINNs in yet another new domain and leverages from the available knowledge in the form of governing physical laws. On investigating the performance of conventional PINNs in vibrations, it is mostly found that it suffers from a very recently pointed out similar scaling or regularization issue, leading to inaccurate predictions. It is thereby demonstrated that a simple strategy of modifying the loss function helps to combat the situation and enhance the approximation accuracy significantly without adding any extra computational cost. In addition to the above two contributing factors of this work, the implementation of the conventional and modified PINNs is performed in the MATLAB environment owing to its recently developed rich deep learning library. Since all the developments of PINNs till date is Python based, this is expected to diversify the field and reach out to greater scientific audience who are more proficient in MATLAB but are interested to explore the prospect of deep learning in computational science and engineering. As a bonus, complete executable codes of all four representative (both forward and inverse) problems in structural vibrations have been provided along with their line-by-line lucid explanation and well-interpreted results for better understanding.

We introduce an extension to the PySCF package, which makes it automatically differentiable. The implementation strategy is discussed, and example applications are presented to demonstrate the automatic differentiation framework for quantum chemistry methodology development. These include orbital optimization, properties, excited-state energies, and derivative couplings, at the mean-field level and beyond, in both molecules and solids. We also discuss some current limitations and directions for future work. [ABSTRACT FROM AUTHOR]