Search

Scientific machine learning and the advent of the Physics-Informed Neural Network (PINN) have shown high potential in their ability to solve complex differential equations. One example is the use of PINNs to solve the gravity field modeling problem -- learning convenient representations of the gravitational potential from position and acceleration data. These PINN gravity models, or PINN-GMs, have demonstrated advantages in model compactness, robustness to noise, and sample efficiency when compared to popular alternatives; however, further investigation has revealed various failure modes for these and other machine learning gravity models which this manuscript aims to address. Specifically, this paper introduces the third generation Physics-Informed Neural Network Gravity Model (PINN-GM-III) which includes design changes that solve the problems of feature divergence, bias towards low-altitude samples, numerical instability, and extrapolation error. Six evaluation metrics are proposed to expose these past pitfalls and illustrate the PINN-GM-III's robustness to them. This study concludes by evaluating the PINN-GM-III modeling accuracy on a heterogeneous density asteroid, and comparing its performance to other analytic and machine learning gravity models., Comment: 40 pages, 10 figures, submitted to The Journal of Astronautical Sciences

The Casas-Alvero conjecture predicts that every univariate polynomial $f$ over a field $K$ of characteristic zero having a common factor with each of its derivatives $H_i(f)$ is a power of a linear polynomial. Let $f=x^d+a_1x^{d-1}+\cdots+a_1x \in K[a_1,\ldots,a_{d-1}][x]$ and let $R_i = Res(f,H_i(f))\in K[a_1,\ldots,a_{d-1}]$ be the resultant of $f$ and $H_i(f)$, $i \in \{1,\ldots,d-1\}$. The Casas-Alvero Conjecture is equivalent to saying that $R_1,\ldots,R_{d-1}$ are ``independent'' in a certain sense, namely that the height $ht(R_1,\ldots,R_{d-1})=d-1$ in $K[a_1,\ldots,a_{d-1}]$. In this paper we prove a very partial result in this direction: if $i \in \{d-3,d-2,d-1\}$ then $R_i \notin \sqrt{(R_1,\ldots,\breve{R_i},\ldots,R_{d-1}}$.

Graph-based signal processing techniques have become essential for handling data in non-Euclidean spaces. However, there is a growing awareness that these graph models might need to be expanded into `higher-order' domains to effectively represent the complex relations found in high-dimensional data. Such higher-order domains are typically modeled either as hypergraphs, or as simplicial, cubical or other cell complexes. In this context, cell complexes are often seen as a subclass of hypergraphs with additional algebraic structure that can be exploited, e.g., to develop a spectral theory. In this article, we promote an alternative perspective. We argue that hypergraphs and cell complexes emphasize \emph{different} types of relations, which may have different utility depending on the application context. Whereas hypergraphs are effective in modeling set-type, multi-body relations between entities, cell complexes provide an effective means to model hierarchical, interior-to-boundary type relations. We discuss the relative advantages of these two choices and elaborate on the previously introduced concept of a combinatorial complex that enables co-existing set-type and hierarchical relations. Finally, we provide a brief numerical experiment to demonstrate that this modelling flexibility can be advantageous in learning tasks.

We use the numerical conformal bootstrap to study boundary quantum electrodynamics, the theory of a four dimensional photon in a half space coupled to charged conformal matter on the boundary. This system is believed to be a boundary conformal field theory with an exactly marginal coupling corresponding to the strength of the interaction between the photon and the matter degrees of freedom. In part one of this project, we present three results. We show how the Maxwell equations put severe constraints on boundary three-point functions involving two currents and a symmetric traceless tensor. We use semi-definite programming to show that any three dimensional conformal field theory with a global U(1) symmetry must have a spin two gap less than about 1.05. Finally, combining a numerical bound on an OPE coefficient and some Ward identities involving the current and the displacement operator, we bound the displacement operator two-point function above. This upper bound also constrains a boundary contribution to the anomaly in the trace of the stress tensor for these types of theories., Comment: 33 pages + appendices, 8 figures

The rich spectral information of the graph Laplacian has been instrumental in graph theory, machine learning, and graph signal processing for applications such as graph classification, clustering, or eigenmode analysis. Recently, the Hodge Laplacian has come into focus as a generalisation of the ordinary Laplacian for higher-order graph models such as simplicial and cellular complexes. Akin to the traditional analysis of graph Laplacians, many authors analyse the smallest eigenvalues of the Hodge Laplacian, which are connected to important topological properties such as homology. However, small eigenvalues of the Hodge Laplacian can carry different information depending on whether they are related to curl or gradient eigenmodes, and thus may not be comparable. We therefore introduce the notion of persistent eigenvector similarity and provide a method to track individual harmonic, curl, and gradient eigenvectors/-values through the so-called persistence filtration, leveraging the full information contained in the Hodge-Laplacian spectrum across all possible scales of a point cloud. Finally, we use our insights (a) to introduce a novel form of Hodge spectral clustering and (b) to classify edges and higher-order simplices based on their relationship to the smallest harmonic, curl, and gradient eigenvectors., Comment: 5 pages, 4 figures, comments welcome

The process of developing control functions for embedded systems is resource-, time-, and data-intensive, often resulting in sub-optimal cost and solutions approaches. Reinforcement Learning (RL) has great potential for autonomously training agents to perform complex control tasks with minimal human intervention. Due to costly data generation and safety constraints, however, its application is mostly limited to purely simulated domains. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. In this context, this work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation. For the use case of transient exhaust gas re-circulation control for an internal combustion engine, use of a computationally cheap Model-in-the-Loop (MiL) simulation is made to select a suitable algorithm, fine-tune hyperparameters, and finally train candidate agents for the transfer. These pre-trained RL agents are then fine-tuned in a Hardware-in-the-Loop (HiL) system via TL. The transfer revealed the need for adjusting the reward parameters when advancing to real hardware. Further, the comparison between a purely HiL-trained and a transferred agent showed a reduction of training time by a factor of 5.9. The results emphasize the necessity to train RL agents with real hardware, and demonstrate that the maturity of the transferred policies affects both training time and performance, highlighting the strong synergies between TL and XiL simulation.

Persistent Homology is a widely used topological data analysis tool that creates a concise description of the topological properties of a point cloud based on a specified filtration. Most filtrations used for persistent homology depend (implicitly) on a chosen metric, which is typically agnostically chosen as the standard Euclidean metric on $\mathbb{R}^n$. Recent work has tried to uncover the 'true' metric on the point cloud using distance-to-measure functions, in order to obtain more meaningful persistent homology results. Here we propose an alternative look at this problem: we posit that information on the point cloud is lost when restricting persistent homology to a single (correct) distance function. Instead, we show how by varying the distance function on the underlying space and analysing the corresponding shifts in the persistence diagrams, we can extract additional topological and geometrical information. Finally, we numerically show that non-isotropic persistent homology can extract information on orientation, orientational variance, and scaling of randomly generated point clouds with good accuracy and conduct some experiments on real-world data., Comment: 30 pages, 17 figures, comments welcome!

Comparing graphs by means of optimal transport has recently gained significant attention, as the distances induced by optimal transport provide both a principled metric between graphs as well as an interpretable description of the associated changes between graphs in terms of a transport plan. As the lack of symmetry introduces challenges in the typically considered formulations, optimal transport distances for graphs have mostly been developed for undirected graphs. Here, we propose two distance measures to compare directed graphs based on variants of optimal transport: (i) an earth movers distance (Wasserstein) and (ii) a Gromov-Wasserstein (GW) distance. We evaluate these two distances and discuss their relative performance for both simulated graph data and real-world directed cell-cell communication graphs, inferred from single-cell RNA-seq data., Comment: 5 pages, 1 figure

A solution to the MiniBooNE excess invoking rare three-body decays of the charged pions and kaons to new states in the MeV mass scale was recently proposed as a dark-sector explanation. This class of solution illuminates the fact that, while the charged pions were focused in the target-mode run, their decay products were isotropically suppressed in the beam-dump-mode run in which no excess was observed. This suggests a new physics solution correlated to the mesonic sector. We investigate an extended set of phenomenological models that can explain the MiniBooNE excess as a dark sector solution, utilizing long-lived particles that might be produced in the three-body decays of the charged mesons and the two-body anomalous decays of the neutral mesons. Over a broad set of interactions with the long-lived particles, we show that these scenarios can be compatible with constraints from LSND, KARMEN, and MicroBooNE, and evaluate the sensitivity of the ongoing and future data taken by the Coherent CAPTAIN Mills experiment (CCM) to a potential discovery in this parameter space. See addendum for updated predictions for future MicroBooNE sensitivity., Comment: Addendum (in preparation for PRD) appended with forecast for MicroBooNE sensitivity to the MiniBooNE excess. Added reference to open source code for simulation of the horn-focused charged mesons

Obtaining sparse, interpretable representations of observable data is crucial in many machine learning and signal processing tasks. For data representing flows along the edges of a graph, an intuitively interpretable way to obtain such representations is to lift the graph structure to a simplicial complex: The eigenvectors of the associated Hodge-Laplacian, respectively the incidence matrices of the corresponding simplicial complex then induce a Hodge decomposition, which can be used to represent the observed data in terms of gradient, curl, and harmonic flows. In this paper, we generalize this approach to cellular complexes and introduce the flow representation learning problem, i.e., the problem of augmenting the observed graph by a set of cells, such that the eigenvectors of the associated Hodge Laplacian provide a sparse, interpretable representation of the observed edge flows on the graph. We show that this problem is NP-hard and introduce an efficient approximation algorithm for its solution. Experiments on real-world and synthetic data demonstrate that our algorithm outperforms state-of-the-art methods with respect to approximation error, while being computationally efficient., Comment: 9 pages, 6 figures (plus appendix). For evaluation code, see https://github.com/josefhoppe/edge-flow-cell-complexes

The field of explainable artificial intelligence (XAI) aims to uncover the inner workings of complex deep neural models. While being crucial for safety-critical domains, XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We address this challenge by proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Our dataset allows performing semantically meaningful image interventions, e.g., removing individual object parts, which has three important implications. First, it enables analyzing explanations on a part level, which is closer to human comprehension than existing methods that evaluate on a pixel level. Second, by comparing the model output for inputs with removed parts, we can estimate ground-truth part importances that should be reflected in the explanations. Third, by mapping individual explanations into a common space of part importances, we can analyze a variety of different explanation types in a single common framework. Using our tools, we report results for 24 different combinations of neural models and XAI methods, demonstrating the strengths and weaknesses of the assessed methods in a fully automatic and systematic manner., Comment: Accepted at ICCV 2023. Code: https://github.com/visinf/funnybirds

Extensions of Answer Set Programming with language constructs from temporal logics, such as temporal equilibrium logic over finite traces (TELf), provide an expressive computational framework for modeling dynamic applications. In this paper, we study the so-called past-present syntactic subclass, which consists of a set of logic programming rules whose body references to the past and head to the present. Such restriction ensures that the past remains independent of the future, which is the case in most dynamic domains. We extend the definitions of completion and loop formulas to the case of past-present formulas, which allows capturing the temporal stable models of a set of past-present temporal programs by means of an LTLf expression.

We develop an approach called bounded combinatorial reconfiguration for solving combinatorial reconfiguration problems based on Answer Set Programming (ASP). The general task is to study the solution spaces of source combinatorial problems and to decide whether or not there are sequences of feasible solutions that have special properties. The resulting recongo solver covers all metrics of the solver track in the most recent international competition on combinatorial reconfiguration (CoRe Challenge 2022). recongo ranked first in the shortest metric of the single-engine solvers track. In this paper, we present the design and implementation of bounded combinatorial reconfiguration, and present an ASP encoding of the independent set reconfiguration problem that is one of the most studied combinatorial reconfiguration problems. Finally, we present empirical analysis considering all instances of CoRe Challenge 2022., Comment: 15 pages

The Casas-Alvero conjecture predicts that every univariate polynomial over a field of characteristic zero having a common factor with each of its derivatives $H_i(f)$ is a power of a linear polynomial. One approach to proving the conjecture is to first prove it for polynomials of some small degree $d$, compile a list of bad primes for that degree (namely, those primes $p$ for which the conjecture fails in degree $d$ and characteristic $p$) and then deduce the conjecture for all degrees of the form $dp^\ell$, $\ell\in\mathbb{N}$, where $p$ is a good prime for $d$. In this paper we calculate certain distinguished monomials appearing in the resultant $R(f,H_i(f))$ and obtain a (non-exhaustive) list of bad primes for every degree $d\in\mathbb{N}\setminus\{0\}$.

Graph Neural Network (GNN) research has highlighted a relationship between high homophily (i.e., the tendency of nodes of the same class to connect) and strong predictive performance in node classification. However, recent work has found the relationship to be more nuanced, demonstrating that simple GNNs can learn in certain heterophilous settings. To resolve these conflicting findings and align closer to real-world datasets, we go beyond the assumption of a global graph homophily level and study the performance of GNNs when the local homophily level of a node deviates from the global homophily level. Through theoretical and empirical analysis, we systematically demonstrate how shifts in local homophily can introduce performance degradation, leading to performance discrepancies across local homophily levels. We ground the practical implications of this work through granular analysis on five real-world datasets with varying global homophily levels, demonstrating that (a) GNNs can fail to generalize to test nodes that deviate from the global homophily of a graph, and (b) high local homophily does not necessarily confer high performance for a node. We further show that GNNs designed for globally heterophilous graphs can alleviate performance discrepancy by improving performance across local homophily levels, offering a new perspective on how these GNNs achieve stronger global performance., Comment: 30 pages

Dynamical systems on hypergraphs can display a rich set of behaviours not observable for systems with pairwise interactions. Given a distributed dynamical system with a putative hypergraph structure, an interesting question is thus how much of this hypergraph structure is actually necessary to faithfully replicate the observed dynamical behaviour. To answer this question, we propose a method to determine the minimum order of a hypergraph necessary to approximate the corresponding dynamics accurately. Specifically, we develop an analytical framework that allows us to determine this order when the type of dynamics is known. We utilize these ideas in conjunction with a hypergraph neural network to directly learn the dynamics itself and the resulting order of the hypergraph from both synthetic and real data sets consisting of observed system trajectories.

Similar to community detection, partitioning the nodes of a network according to their structural roles aims to identify fundamental building blocks of a network. The found partitions can be used, e.g., to simplify descriptions of the network connectivity, to derive reduced order models for dynamical processes unfolding on processes, or as ingredients for various graph mining tasks. In this work, we offer a fresh look on the problem of role extraction and its differences to community detection and present a definition of node roles related to graph-isomorphism tests, the Weisfeiler-Leman algorithm and equitable partitions. We study two associated optimization problems (cost functions) grounded in ideas from graph isomorphism testing, and present theoretical guarantees associated to the solutions of these problems. Finally, we validate our approach via a novel "role-infused partition benchmark", a network model from which we can sample networks in which nodes are endowed with different roles in a stochastic way.

Many convolutional neural networks (CNNs) rely on progressive downsampling of their feature maps to increase the network's receptive field and decrease computational cost. However, this comes at the price of losing granularity in the feature maps, limiting the ability to correctly understand images or recover fine detail in dense prediction tasks. To address this, common practice is to replace the last few downsampling operations in a CNN with dilated convolutions, allowing to retain the feature map resolution without reducing the receptive field, albeit increasing the computational cost. This allows to trade off predictive performance against cost, depending on the output feature resolution. By either regularly downsampling or not downsampling the entire feature map, existing work implicitly treats all regions of the input image and subsequent feature maps as equally important, which generally does not hold. We propose an adaptive downsampling scheme that generalizes the above idea by allowing to process informative regions at a higher resolution than less informative ones. In a variety of experiments, we demonstrate the versatility of our adaptive downsampling strategy and empirically show that it improves the cost-accuracy trade-off of various established CNNs., Comment: Accepted at CVPR 2023 Workshop on Efficient Deep Learning for Computer Vision (ECV). Code: https://github.com/visinf/cad