Search

Foundation models have demonstrated remarkable generalization, data efficiency, and robustness properties across various domains. In this paper, we explore the feasibility of foundation models for applications in the control domain. The success of these models is enabled by large-scale pretaining on Internet-scale datasets. These are available in fields like natural language processing and computer vision, but do not exist for dynamical systems. We address this challenge by pretraining a transformer-based foundation model exclusively on synthetic data and propose to sample dynamics functions from a reproducing kernel Hilbert space. Our pretrained model generalizes for prediction tasks across different dynamical systems, which we validate in simulation and hardware experiments, including cart-pole and Furuta pendulum setups. Additionally, the model can be fine-tuned effectively to new systems to increase performance even further. Our results demonstrate the feasibility of foundation models for dynamical systems that outperform specialist models in terms of generalization, data efficiency, and robustness., Comment: 10 pages

We study homogeneous instantons on the seven dimensional Stiefel manifold V in the context of G2 and Sasakian geometry. According to the reductive decomposition of V we provide an explicit description of all invariant G2 and Sasakian structures. In particular, we characterise the invariant G2- structures inducing a Sasakian metric, among which the well known nearly parallel G2-structure (Sasaki- Einstein) is included. As a consequence, we classify the invariant connections on homogeneous principal bundles over V with gauge group U(1) and SO(3), satisfying either the G2 or the Sasakian instanton condition. Finally, we analyse the Yang Mills condition for those invariant connections.

Long-baseline neutrino oscillation experiments require external constraints on $\sin^2\theta_{12}$ and $\Delta m_{21}^2$ to make precision measurements of the leptonic mixing matrix. These constraints come from measurements of the Mikheyev-Smirnov-Wolfenstein (MSW) mixing in solar neutrinos. Here we develop an MSW large mixing angle approximation in the presence of heavy neutral leptons which adds a single new parameter ($\alpha_{11}$) representing the magnitude of the mixing between the $\nu_e$ state and the heavy sector. We use data from the Borexino, SNO and KamLAND collaborations to find a solar constraint appropriate for heavy neutral lepton searches in long-baseline oscillation experiments. Solar data limits the magnitude of the non-unitary parameter to $(1-\alpha_{11}) < 0.046$ at the $99\%$ credible interval and yields a strongly correlated constraint on the solar mass splitting and the magnitude of $\nu_e$ non-unitary mixing., Comment: 8 pages, 5 figures

Convolutional Neural Networks (CNN) have become a common choice for industrial quality control, as well as other critical applications in the Industry 4.0. When these CNNs behave in ways unexpected to human users or developers, severe consequences can arise, such as economic losses or an increased risk to human life. Concept extraction techniques can be applied to increase the reliability and transparency of CNNs through generating global explanations for trained neural network models. The decisive features of image datasets in quality control often depend on the feature's scale; for example, the size of a hole or an edge. However, existing concept extraction methods do not correctly represent scale, which leads to problems interpreting these models as we show herein. To address this issue, we introduce the Scale-Preserving Automatic Concept Extraction (SPACE) algorithm, as a state-of-the-art alternative concept extraction technique for CNNs, focused on industrial applications. SPACE is specifically designed to overcome the aforementioned problems by avoiding scale changes throughout the concept extraction process. SPACE proposes an approach based on square slices of input images, which are selected and then tiled before being clustered into concepts. Our method provides explanations of the models' decision-making process in the form of human-understandable concepts. We evaluate SPACE on three image classification datasets in the context of industrial quality control. Through experimental results, we illustrate how SPACE outperforms other methods and provides actionable insights on the decision mechanisms of CNNs. Finally, code for the implementation of SPACE is provided., Comment: 22 pages, 7 figures

The industry 4.0 is leveraging digital technologies and machine learning techniques to connect and optimize manufacturing processes. Central to this idea is the ability to transform raw data into human understandable knowledge for reliable data-driven decision-making. Convolutional Neural Networks (CNNs) have been instrumental in processing image data, yet, their ``black box'' nature complicates the understanding of their prediction process. In this context, recent advances in the field of eXplainable Artificial Intelligence (XAI) have proposed the extraction and localization of concepts, or which visual cues intervene on the prediction process of CNNs. This paper tackles the application of concept extraction (CE) methods to industry 4.0 scenarios. To this end, we modify a recently developed technique, ``Extracting Concepts with Local Aggregated Descriptors'' (ECLAD), improving its scalability. Specifically, we propose a novel procedure for calculating concept importance, utilizing a wrapper function designed for CNNs. This process is aimed at decreasing the number of times each image needs to be evaluated. Subsequently, we demonstrate the potential of CE methods, by applying them in three industrial use cases. We selected three representative use cases in the context of quality control for material design (tailored textiles), manufacturing (carbon fiber reinforcement), and maintenance (photovoltaic module inspection). In these examples, CE was able to successfully extract and locate concepts directly related to each task. This is, the visual cues related to each concept, coincided with what human experts would use to perform the task themselves, even when the visual cues were entangled between multiple classes. Through empirical results, we show that CE can be applied for understanding CNNs in an industrial context, giving useful insights that can relate to domain knowledge.

In this work, we approach the Laplacian coflow of a coclosed $G_2$-structure $\varphi$ using the formulae for the irreducible $G_2$-decomposition of the Hodge Laplacian and the Lie derivative of the Hodge dual $4$-form of $\varphi$. In terms of this decomposition, we characterise the conditions for a vector field as an infinitesimal symmetry of a coclosed $G_2$-structure, as well as the soliton condition for the Laplacian coflow. More specifically, we provide an easier proof for the absence of compact shrinking solitons of the Laplacian coflow. Moreover, we revisit the Laplacian coflow of coclosed $G_2$-structures on almost Abelian Lie groups addressed by Fino-Bagaglini (2018). However, our approach is based on the bracket flow point of view. Notably, by showing that the norm of the Lie bracket is strictly decreasing, we prove that we have long-time existence for any coclosed Laplacian coflow solution., Comment: 22 pages, 1 figure. Comments are welcome

We study the G$_2$-instanton condition for a family of metric connections arisen from the characteristic connection, on $7$-dimensional $2$-step nilpotent Lie groups with left-invariant coclosed G$_2$-structures. According to the dimension of the commutator subgroup, we establish necessary and sufficient conditions for the connection to be an instanton, in terms of the torsion of the G$_2$-structure, the torsion of the connection and the Lie group structure.Moreover, we show that in our setup, G$_2$-instantons define a naturally reductive structure on the simply connected $2$-step nilpotent Lie group with left-invariant Riemannian metric. Taking quotient by lattices, one obtains G$_2$-instantons on compact nilmanifolds., Comment: 37 pages. Comments are welcome. v2: Minor changes, references added

We consider left-invariant $G_2$-structures on $7$-dimensional almost Abelian Lie groups. Also, we characterise the associated torsion forms and the full torsion tensor according to the Lie bracket $A$ of the corresponding Lie algebra. In those terms, we establish the algebraic condition on $A$ for each of the possible $16$-torsion classes of a $G_2$-structure. In particular, we show that four of those torsion classes are not admissible, since $\tau_3=0$ implies $\tau_0=0$. Finally, we use the above results to provide the algebraic criteria on $A$, satisfying the harmonic condition $div T=0$, and this allows to identify which torsion classes are harmonic., Comment: 15 pages, v3: Additional reference [17] to the bibliography, the same version as v2

We develop an abstract theory of flows of geometric $H$-structures, i.e., flows of tensor fields defining $H$-reductions of the frame bundle, for a closed and connected subgroup $H\subset SO(n)$, on any connected and oriented $n$-manifold with sufficient topology to admit such structures. The first part of the article sets up a unifying theoretical framework for deformations of $H$-structures, by way of the natural infinitesimal action of $\mathrm{GL}(n,\mathbb{R})$ on tensors combined with various bundle decompositions induced by $H$-structures. We compute evolution equations for the intrinsic torsion under general flows of $H$-structures and, as applications, we obtain general Bianchi-type identities for $H$-structures, and, for closed manifolds, a general first variation formula for the $L^2$-Dirichlet energy functional $\mathcal{E}$ on the space of $H$-structures. We then specialise the theory to the negative gradient flow of $\mathcal{E}$ over isometric $H$-structures, i.e., their harmonic flow. The core result is an almost monotonocity formula along the flow for a scale-invariant localised energy, similar to the classical formulae by Chen-Struwe for the harmonic map heat flow. This yields an $\varepsilon$-regularity theorem and an energy gap result for harmonic structures, as well as long-time existence for the flow under small initial energy, relative to the $L^\infty$-norm of initial torsion, in the spirit of Chen-Ding. Moreover, below a certain energy level, the absence of a torsion-free isometric $H$-structure in the initial homotopy class imposes the formation of finite-time singularities. These seemingly contrasting statements are illustrated by examples on flat $n$-tori, so long as $[\mathbb{S}^n,SO(n)/H]$ contains more than one element and the universal cover of $SO(n)/H$ is a sphere; e.g. when $n=7$ and $H=\rm G_2$, or $n=8$ and $H=\rm Spin(7)$., Comment: v4: 61 pages, no figures. Final version, to appear in Journal f\"ur die reine und angewandte Mathematik. Completely revised article. Main corrections and clarifications: Lemma 2.4, proofs of Theorems 2.5, 2.7 and 2.10; Example 2.17 and Remark 2.18; proof of Lemma 2.23; Proposition 2.24; and proof of Theorem 2.29

Convolutional neural networks (CNNs) are increasingly being used in critical systems, where robustness and alignment are crucial. In this context, the field of explainable artificial intelligence has proposed the generation of high-level explanations of the prediction process of CNNs through concept extraction. While these methods can detect whether or not a concept is present in an image, they are unable to determine its location. What is more, a fair comparison of such approaches is difficult due to a lack of proper validation procedures. To address these issues, we propose a novel method for automatic concept extraction and localization based on representations obtained through pixel-wise aggregations of CNN activation maps. Further, we introduce a process for the validation of concept-extraction techniques based on synthetic datasets with pixel-wise annotations of their main components, reducing the need for human intervention. Extensive experimentation on both synthetic and real-world datasets demonstrates that our method outperforms state-of-the-art alternatives., Comment: 34 pages, under review

We describe the $10$-dimensional space of $Sp(2)$-invariant $G_2$-structures on the homogeneous $7$-sphere $S^7=Sp(2)/Sp(1)$ as $\mathbb{R}^+\times Gl^+(3,\mathbb{R})$. In those terms, we formulate a general Ansatz for $G_2$-structures, which realises representatives in each of the $7$ possible isometric classes of homogeneous $G_2$-structures. Moreover, the well-known nearly parallel round and squashed metrics occur naturally as opposite poles in an $S^3$-family, the equator of which is a new $S^2$-family of coclosed $G_2$-structures satisfying the harmonicity condition $div T=0$. We show general existence of harmonic representatives of $G_2$-structures in each isometric class through explicit solutions of the associated flow and describe the qualitative behaviour of the flow. We study the stability of the Dirichlet gradient flow near these critical points, showing explicit examples of degenerate and nondegenerate local maxima and minima, at various regimes of the general Ansatz. Finally, for metrics outside of the Ansatz, we identify families of harmonic $G_2$-structures, prove long-time existence of the flow and study the stability properties of some well-chosen examples., Comment: 34 pages, several modifications according to the referee's report, the major changes in: Theorem 2, Proposition 4.1 and the proofs of Lemma 1.1 and Lemma 3.4. We add a brief explanation about G-equivariant equivalent geometric structures in Section 1.2 and an Appendix with the Maple code of some standard computations on tensors

Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t microscopy image data of fluorescently labeled cellular nuclei. The method combines spatiotemporal statistical shape models of different cell cycle stages with a conditional GAN to generate time series of cell populations and provides instance-level control of cell cycle stage and the fluorescence intensity of generated cells. We show the effect of the GAN conditioning and create a set of synthetic images that can be readily used for training and benchmarking of cell segmentation and tracking approaches., Comment: 5 pages, 3 figures

The need for more transparency of the decision-making processes in artificial neural networks steadily increases driven by their applications in safety critical and ethically challenging domains such as autonomous driving or medical diagnostics. We address today's lack of transparency of neural networks and shed light on the roles of single neurons and groups of neurons within the network fulfilling a learned task. Inspired by research in the field of neuroscience, we characterize the learned representations by activation patterns and network ablations, revealing functional neuron populations that a) act jointly in response to specific stimuli or b) have similar impact on the network's performance after being ablated. We find that neither a neuron's magnitude or selectivity of activation, nor its impact on network performance are sufficient stand-alone indicators for its importance for the overall task. We argue that such indicators are essential for future advances in transfer learning and modern neuroscience., Comment: 17 pages, 14 figures, adjusted "under review" statement

We apply the general Ansatz in geometric flows on homogeneous spaces proposed by Jorge Lauret for the Laplacian co-flow of invariant $G_2$-structures on a Lie group, finding an explicit soliton on a particular almost Abelian $7$-manifold., Comment: Minor corrections, proof's Lemma 4.1 modified