Search

We characterize the connectivity structure of feed-forward, deep neural networks (DNNs) using network motif theory. To address whether a particular motif distribution is characteristic of the training task, or function of the DNN, we compare the connectivity structure of 350 DNNs trained to simulate a bio-mechanical flight control system with different randomly initialized parameters. We develop and implement algorithms for counting second- and third-order motifs and calculate their significance using their Z-score. The DNNs are trained to solve the inverse problem of the flight dynamics model in Bustamante, et al. (2022) (i.e., predict the controls necessary for controlled flight from the initial and final state-space inputs) and are sparsified through an iterative pruning and retraining algorithm Zahn, et al. (2022). We show that, despite random initialization of network parameters, enforced sparsity causes DNNs to converge to similar connectivity patterns as characterized by their motif distributions. The results suggest how neural network function can be encoded in motif distributions, suggesting a variety of experiments for informing function and control.

Particle tracking is crucial for almost all physics analysis programs at the Large Hadron Collider. Deep learning models are pervasively used in particle tracking related tasks. However, the current practice is to design and train one deep learning model for one task with supervised learning techniques. The trained models work well for tasks they are trained on but show no or little generalization capabilities. We propose to unify these models with a language model. In this paper, we present a tokenized detector representation that allows us to train a BERT model for particle tracking. The trained BERT model, namely TrackingBERT, offers latent detector module embedding that can be used for other tasks. This work represents the first step towards developing a foundational model for particle detector understanding., Comment: 7 pages, 3 figures, A Proceeding of the Connecting the Dots Workshop (CTD 2023)

Systems in nature are stochastic as well as nonlinear. In traditional applications, engineered filters aim to minimize the stochastic effects caused by process and measurement noise. Conversely, a previous study showed that the process noise can reveal the observability of a system that was initially categorized as unobservable when deterministic tools were used. In this paper, we develop a stochastic framework to explore observability analysis and sensor placement. This framework allows for direct studies of the effects of stochasticity on optimal sensor placement and selection to improve filter error covariance. Numerical results are presented for sensor selection that optimizes stochastic empirical observability in a bioinspired setting., Comment: 12 pages, 5 figures, 2 tables, to be published in the proceedings of the 2024 American Control Conference

We investigate a new production mechanism of axion-like particle (ALP) from $B$ meson annihilation decays and its observation potential at the Belle and Belle II experiments. This mechanism allows for the production of ALP from $B$ meson decays in association with a large variety of mesons. In this article, we first estimate the branching ratios of such processes with a perturbative QCD method. Focussing on the most promising $B \to h a^{\prime}$ ($h=K^\pm, \pi^\pm, D^0$ and $D_s$) channels, we perform sensitivity studies for $a^{\prime}$ decaying invisibly or into diphoton with Belle and Belle II experiments.

We introduce a novel variant of GNN for particle tracking called Hierarchical Graph Neural Network (HGNN). The architecture creates a set of higher-level representations which correspond to tracks and assigns spacepoints to these tracks, allowing disconnected spacepoints to be assigned to the same track, as well as multiple tracks to share the same spacepoint. We propose a novel learnable pooling algorithm called GMPool to generate these higher-level representations called "super-nodes", as well as a new loss function designed for tracking problems and HGNN specifically. On a standard tracking problem, we show that, compared with previous ML-based tracking algorithms, the HGNN has better tracking efficiency performance, better robustness against inefficient input graphs, and better convergence compared with traditional GNNs., Comment: 7 pages, 5 figures, submitted to the 21st International Workshop on Advanced Computing and Analysis Techniques in Physics Research

Insect flight is a strongly nonlinear and actuated dynamical system. As such, strategies for understanding its control have typically relied on either model-based methods or linearizations thereof. Here we develop a framework that combines model predictive control on an established flight dynamics model and deep neural networks (DNN) to create an efficient method for solving the inverse problem of flight control. We turn to natural systems for inspiration since they inherently demonstrate network pruning with the consequence of yielding more efficient networks for a specific set of tasks. This bio-inspired approach allows us to leverage network pruning to optimally sparsify a DNN architecture in order to perform flight tasks with as few neural connections as possible, however, there are limits to sparsification. Specifically, as the number of connections falls below a critical threshold, flight performance drops considerably. We develop sparsification paradigms and explore their limits for control tasks. Monte Carlo simulations also quantify the statistical distribution of network weights during pruning given initial random weights of the DNNs. We demonstrate that on average, the network can be pruned to retain approximately 7% of the original network weights, with statistical distributions quantified at each layer of the network. Overall, this work shows that sparsely connected DNNs are capable of predicting the forces required to follow flight trajectories. Additionally, sparsification has sharp performance limits.

Model order reduction has been extensively studied over the last two decades. Projection-based methods such as the Proper Orthogonal Decomposition and the Reduced Basis Method enjoy the important advantages of Galerkin methods in the derivation of the reduced problem, but are limited to linear data compression for which the reduced solution is sought as a linear combination of spatial modes. Nonlinear data compression must be used when the solution manifold is not embedded in a low-dimensional subspace. Early methods involve piecewise linear data compression, by constructing a dictionary of reduced-order models tailored to a partition of the solution manifold. In this work, we introduce the concept of optimal partition of the solution manifold in terms of normalized Kolmogorov widths, and prove that the optimal partitions can be found by means of a representative-based clustering algorithm using the sine dissimilarity measure on the solution manifold.

We consider the dictionary-based ROM-net (Reduced Order Model) framework [T. Daniel, F. Casenave, N. Akkari, D. Ryckelynck, Model order reduction assisted by deep neural networks (ROM-net), Advanced modeling and Simulation in Engineering Sciences 7 (16), 2020] and summarize the underlying methodologies and their recent improvements. The main contribution of this work is the application of the complete workflow to a real-life industrial model of an elastoviscoplastic high-pressure turbine blade subjected to thermal, centrifugal and pressure loadings, for the quantification of the uncertainty on dual quantities (such as the accumulated plastic strain and the stress tensor), generated by the uncertainty on the temperature loading field. The dictionary-based ROM-net computes predictions of dual quantities of interest for 1008 Monte Carlo draws of the temperature loading field in 2 hours and 48 minutes, which corresponds to a speedup greater than 600 with respect to a reference parallel solver using domain decomposition, with a relative error in the order of 2%. Another contribution of this work consists in the derivation of a meta-model to reconstruct the dual quantities of interest over the complete mesh from their values on the reduced integration points.

A highly organized and densely packed lattice of molecular machinery within the sarcomeres of muscle cells powers contraction. Although many of the proteins that drive contraction have been studied extensively, the mechanical impact of fluid shearing within the lattice of molecular machinery has received minimal attention. It was recently proposed that fluid flow augments substrate transport in the sarcomere, however, this analysis used analytical models of fluid flow in the molecular machinery that could not capture its full complexity. By building a finite element model of the sarcomere, we estimate the explicit flow field, and contrast it with analytical models. Our results demonstrate that viscous drag forces on sliding filaments are surprisingly small in contrast to the forces generated by single myosin molecular motors. This model also indicates that the energetic cost of fluid flow through viscous shearing with lattice proteins is likely minimal. The model also highlights a steep velocity gradient between sliding filaments and demonstrates that the maximal radial fluid velocity occurs near the tips of the filaments. To our knowledge, this is the first computational analysis of fluid flow within the highly structured sarcomere.

Nonlinear model order reduction has opened the door to parameter optimization and uncertainty quantification in complex physics problems governed by nonlinear equations. In particular, the computational cost of solving these equations can be reduced by means of local reduced-order bases. This article examines the benefits of a physics-informed cluster analysis for the construction of cluster-specific reduced-order bases. We illustrate that the choice of the dissimilarity measure for clustering is fundamental and highly affects the performances of the local reduced-order bases. It is shown that clustering with an angle-based dissimilarity on simulation data efficiently decreases the intra-cluster Kolmogorov $N$-width. Additionally, an a priori efficiency criterion is introduced to assess the relevance of a ROM-net, a methodology for the reduction of nonlinear physics problems introduced in our previous work in [T. Daniel, F. Casenave, N. Akkari, D. Ryckelynck, Model order reduction assisted by deep neural networks (ROM-net), Advanced Modeling and Simulation in Engineering Sciences 7 (16), 2020]. This criterion also provides engineers with a very practical method for ROM-nets' hyperparameters calibration under constrained computational costs for the training phase. On five different physics problems, our physics-informed clustering strategy significantly outperforms classic strategies for the construction of local reduced-order bases in terms of projection errors.

Classification algorithms have recently found applications in computational physics for the selection of numerical methods or models adapted to the environment and the state of the physical system. For such classification tasks, labeled training data come from numerical simulations and generally correspond to physical fields discretized on a mesh. Three challenging difficulties arise: the lack of training data, their high dimensionality, and the non-applicability of common data augmentation techniques to physics data. This article introduces two algorithms to address these issues, one for dimensionality reduction via feature selection, and one for data augmentation. These algorithms are combined with a wide variety of classifiers for their evaluation. When combined with a stacking ensemble made of six multilayer perceptrons and a ridge logistic regression, they enable reaching an accuracy of 90% on our classification problem for nonlinear structural mechanics.

We present a framework for calibration of parameters in elastoplastic constitutive models that is based on the use of automatic differentiation (AD). The model calibration problem is posed as a partial differential equation-constrained optimization problem where a finite element (FE) model of the coupled equilibrium equation and constitutive model evolution equations serves as the constraint. The objective function quantifies the mismatch between the displacement predicted by the FE model and full-field digital image correlation data, and the optimization problem is solved using gradient-based optimization algorithms. Forward and adjoint sensitivities are used to compute the gradient at considerably less cost than its calculation from finite difference approximations. Through the use of AD, we need only to write the constraints in terms of AD objects, where all of the derivatives required for the forward and inverse problems are obtained by appropriately seeding and evaluating these quantities. We present three numerical examples that verify the correctness of the gradient, demonstrate the AD approach's parallel computation capabilities via application to a large-scale FE model, and highlight the formulation's ease of extensibility to other classes of constitutive models.

Time delays due to signal latency, computational complexity, and sensor-denied environments, pose a critical challenge in both engineered and biological control systems. In this work, we investigate biologically inspired strategies to develop precisely timed feedforward control laws for engineered systems with large time delays. We demonstrate this approach on the nonlinear pendulum with partially denied observations, so that it is only possible to measure the state of the system near the upright position. Given a large disturbance that overwhelms the local feedback controller, it is necessary to add or remove energy from the pendulum so that it returns to the upright position after one full revolution. The partial observation near the upright position introduces a significant delay between observations and the region where actuation is most effective. Thus, we develop a learning algorithm that integrates sensor information into a precisely timed feedforward control signal to overcome this delay with minimal computation, training data, and set of control decisions. This simple controller can serve as a model for many biological systems, and can be implemented in engineered systems with time delays., Comment: 6 pages, 5 figures

Sparse sensor placement is a central challenge in the efficient characterization of complex systems when the cost of acquiring and processing data is high. Leading sparse sensing methods typically exploit either spatial or temporal correlations, but rarely both. This work introduces a new sparse sensor optimization that is designed to leverage the rich spatiotemporal coherence exhibited by many systems. Our approach is inspired by the remarkable performance of flying insects, which use a few embedded strain-sensitive neurons to achieve rapid and robust flight control despite large gust disturbances. Specifically, we draw on nature to identify targeted neural-inspired sensors on a flapping wing to detect body rotation. This task is particularly challenging as the rotational twisting mode is three orders-of-magnitude smaller than the flapping modes. We show that nonlinear filtering in time, built to mimic strain-sensitive neurons, is essential to detect rotation, whereas instantaneous measurements fail. Optimized sparse sensor placement results in efficient classification with approximately ten sensors, achieving the same accuracy and noise robustness as full measurements consisting of hundreds of sensors. Sparse sensing with neural inspired encoding establishes a new paradigm in hyper-efficient, embodied sensing of spatiotemporal data and sheds light on principles of biological sensing for agile flight control., Comment: 21 pages, 19 figures

Control theory arose from a need to control synthetic systems. From regulating steam engines to tuning radios to devices capable of autonomous movement, it provided a formal mathematical basis for understanding the role of feedback in the stability (or change) of dynamical systems. It provides a framework for understanding any system with feedback regulation, including biological ones such as regulatory gene networks, cellular metabolic systems, sensorimotor dynamics of moving animals, and even ecological or evolutionary dynamics of organisms and populations. Here we focus on four case studies of the sensorimotor dynamics of animals, each of which involves the application of principles from control theory to probe stability and feedback in an organism's response to perturbations. We use examples from aquatic (electric fish station keeping and jamming avoidance), terrestrial (cockroach wall following) and aerial environments (flight control in moths) to highlight how one can use control theory to understand how feedback mechanisms interact with the physical dynamics of animals to determine their stability and response to sensory inputs and perturbations. Each case study is cast as a control problem with sensory input, neural processing, and motor dynamics, the output of which feeds back to the sensory inputs. Collectively, the interaction of these systems in a closed loop determines the behavior of the entire system., Comment: Submitted to Integr Comp Biol

The primary challenge of GOLF-NG (Global Oscillations at Low Frequency New Generation) is the detection of the low-frequency solar gravity and acoustic modes, as well as the possibility to measure the high-frequency chromospheric modes. On June 8th 2008, the first sunlight observations with the multichannel resonant GOLF-NG prototype spectrometer were obtained at the Observatorio del Teide (Tenerife). The instrument performs integrated (Sun-as-a-star), Doppler velocity measurements, simultaneously at eight different heights in the D1 sodium line profile, corresponding to photospheric and chromospheric layers of the solar atmosphere. In order to study its performances, to validate the conceived strategy, and to estimate the necessary improvements, this prototype has been running on a daily basis over the whole summer of 2008 at the Observatorio del Teide. We present here the results of the first GOLF-NG observations, clearly showing the characteristics of the 5-minute oscillatory signal at different heights in the solar atmosphere. We compare these signals with simultaneous observations from GOLF/SOHO and from the Mark-I instrument -- a node of the BiSON network, operating at the same site., Comment: 4 pages, 7 figures, Conference proceeding GONG 2008 / SOHO XXI Meeting

This article quickly summarizes the performances and results of the GOLF/SoHO resonant spectrometer, thus justifying to go a step further. We then recall the characteristics of the multichannel resonant GOLF-NG spectrometer and present the first successful performances of the laboratory tests on the prototype and also the limitations of this first technological instrument. Scientific questions and an observation strategy are discussed., Comment: 8 pages, 8 figures, published in Astronomical Notes

Statistical power in music education was examined by taking an in-depth look at quantitative articles published in the "Journal of Research in Music Education" between 1987 and 1991, inclusive. Of the 109 articles of the period, 78 were quantitative, with both parametric and nonparametric procedures considered. Sample sizes were those reported by the authors. Effect sizes were estimated according to the guidelines developed by J. Cohen (1988), and his power analysis tables were used. The overall median power for the articles was 0.13 for detecting small effects, 0.64 for detecting medium effects, and 0.97 for detecting large effects. Implications of these findings, limitations of the study, and suggestions for future music-education research are discussed. In general, more attention should be placed on a priori power analyses of research designs. Adequate sample sizes should be chosen and a greater understanding and application of the concept of effect size is needed in music education research. Six tables are included. (Contains 18 references.) (SLD)

This paper studies statistical procedures used in the "Journal of Research in Music Education" (JRME) over a 5-year period (1987 through 1991). The results are compared with those of L. D. Goodwin and W. L. Goodwin (1985) concerning the "American Educational Research Journal" for the same period. In all, 78 quantitative and 31 non-quantitative JRME articles were used. A 10-item summation chart was developed (power, alpha, effect size, sample size, basic procedures, method of computation, basic assumptions, references, repeated univariate, and other). Results for individual articles were grouped into issue totals, volume totals, and overall item totals. The results agree with those of Goodwin and Goodwin; basic, intermediate, advanced, and other techniques are used at about the same rate in both educational journals. Analysis of variance is currently the dominant statistical procedure in educational research. The chi square procedure was the next dominant statistical procedure used. Neither qualitative research nor non-parametric procedures are widely used. Power and effect size are mostly ignored in the behavioral sciences. Music education researchers must deal with power and effect size. Problems caused by researcher unfamiliarity with advanced computer packages and an over-abundance of stepwise regression are considered. Three tables are included. (RLC)