Search

This paper proposes a hierarchical Bayesian multitask learning model that is applicable to the general multi-task binary classification learning problem where the model assumes a shared sparsity structure across different tasks. We derive a computationally efficient inference algorithm based on variational inference to approximate the posterior distribution. We demonstrate the potential of the new approach on various synthetic datasets and for predicting human health status based on microbiome profile. Our analysis incorporates data pooled from multiple microbiome studies, along with a comprehensive comparison with other benchmark methods. Results in synthetic datasets show that the proposed approach has superior support recovery property when the underlying regression coefficients share a common sparsity structure across different tasks. Our experiments on microbiome classification demonstrate the utility of the method in extracting informative taxa while providing well-calibrated predictions with uncertainty quantification and achieving competitive performance in terms of prediction metrics. Notably, despite the heterogeneity of the pooled datasets (e.g., different experimental objectives, laboratory setups, sequencing equipment, patient demographics), our method delivers robust results.

We develop a representation theory of categories as a means to explore characteristic structures in algebra. Characteristic structures play a critical role in isomorphism testing of groups and algebras, and their construction and description often rely on specific knowledge of the parent object and its automorphisms. In many cases, questions of reproducibility and comparison arise. Here we present a categorical framework that addresses these questions. We prove that every characteristic structure is the image of a functor equipped with a natural transformation. This shifts the local description in the parent object to a global one in the ambient category. Through constructions in representation theory, such as tensor products, we can combine characteristic structure across multiple categories. Our results are constructive, stated in the language of a constructive type theory, which facilitates implementations in theorem checkers., Comment: 47 pages

Active matter systems, composed of self-propelled agents that convert energy into directed motion, exhibit a wide range of emergent behaviors, such as motility-induced phase separation, flocking, and swarming. These phenomena, observed across natural and engineered systems, hold immense potential for applications in programmable materials, directed assembly, and micro-robotics. However, precisely controlling their macroscopic continuum fields, e.g., density or flux, remains a significant challenge due to the complexity of multibody interactions and correlated particle dynamics. To address this challenge, we present a framework that combines physics-informed machine learning with Model Predictive Control. Our approach learns a closure model for complex particle interactions while incorporating known physical principles, resulting in an accurate predictive model suitable for real-time control. By integrating this model into a Model Predictive Control framework, we enable systematic optimization of control actions that can guide the system toward desired macroscopic behaviors. Through two illustrative examples, we showcase the versatility of the framework. First, we control the spatial distribution of particles by splitting them into two groups and dynamically juggling their densities. Second, we simultaneously control both the number density and the mean flux, guiding the latter to follow a prescribed sinusoidal profile. These results highlight the framework's potential to systematically control complex dynamics in active matter systems and provide a foundation for broader applications in programmable and adaptive materials.

Quantum computing (QC) is expected to solve incredibly difficult problems, including finding optimal solutions to combinatorial optimization problems. However, to date, QC alone is still far to demonstrate this capability except on small-sized problems. Hybrid approaches where QC and classical computing work together have shown the most potential for solving real-world scale problems. This work aims to show that we can enhance a classical optimization algorithm with QC so that it can overcome this limitation. We present a new hybrid quantum-classical tabu search (HQTS) algorithm to solve the capacitated vehicle routing problem (CVRP). Based on our prior work, HQTS leverages QC for routing within a classical tabu search framework. The quantum component formulates the traveling salesman problem (TSP) for each route as a QUBO, solved using D-Wave's Advantage system. Experiments investigate the impact of quantum routing frequency and starting solution methods. While different starting solution methods, including quantum-based and classical heuristics methods, it shows minimal overall impact. HQTS achieved optimal or near-optimal solutions for several CVRP problems, outperforming other hybrid CVRP algorithms and significantly reducing the optimality gap compared to preliminary research. The experimental results demonstrate that more frequent quantum routing improves solution quality and runtime. The findings highlight the potential of integrating QC within meta-heuristic frameworks for complex optimization in vehicle routing problems.

We demonstrate that scalable neuromorphic hardware can implement the finite element method, which is a critical numerical method for engineering and scientific discovery. Our approach maps the sparse interactions between neighboring finite elements to small populations of neurons that dynamically update according to the governing physics of a desired problem description. We show that for the Poisson equation, which describes many physical systems such as gravitational and electrostatic fields, this cortical-inspired neural circuit can achieve comparable levels of numerical accuracy and scaling while enabling the use of inherently parallel and energy-efficient neuromorphic hardware. We demonstrate that this approach can be used on the Intel Loihi 2 platform and illustrate how this approach can be extended to nontrivial mesh geometries and dynamics., Comment: Pre-publication submission

We introduce a novel framework for developing fully-automated trading model algorithms. Unlike the traditional approach, which is grounded in analytical complexity favored by most quantitative analysts, we propose a paradigm shift that embraces real-world complexity. This approach leverages key concepts relating to self-organization, emergence, complex systems theory, scaling laws, and utilizes an event-based reframing of time. In closing, we describe an example algorithm that incorporates the outlined elements, called the Delta Engine., Comment: 6 pages

Federated learning is a computing paradigm that enhances privacy by enabling multiple parties to collaboratively train a machine learning model without revealing personal data. However, current research indicates that traditional federated learning platforms are unable to ensure privacy due to privacy leaks caused by the interchange of gradients. To achieve privacy-preserving federated learning, integrating secure aggregation mechanisms is essential. Unfortunately, existing solutions are vulnerable to recently demonstrated inference attacks such as the disaggregation attack. This paper proposes TAPFed, an approach for achieving privacy-preserving federated learning in the context of multiple decentralized aggregators with malicious actors. TAPFed uses a proposed threshold functional encryption scheme and allows for a certain number of malicious aggregators while maintaining security and privacy. We provide formal security and privacy analyses of TAPFed and compare it to various baselines through experimental evaluation. Our results show that TAPFed offers equivalent performance in terms of model quality compared to state-of-the-art approaches while reducing transmission overhead by 29%-45% across different model training scenarios. Most importantly, TAPFed can defend against recently demonstrated inference attacks caused by curious aggregators, which the majority of existing approaches are susceptible to., Comment: The paper has been published in IEEE TDSC

We present the first general stability results for nonlinear offset-free model predictive control (MPC). Despite over twenty years of active research, the offset-free MPC literature has not shaken the assumption of closed-loop stability for establishing offset-free performance. In this paper, we present a nonlinear offset-free MPC design that is robustly stable with respect to the tracking errors, and thus achieves offset-free performance, despite plant-model mismatch and persistent disturbances. Key features and assumptions of this design include quadratic costs, differentiability of the plant and model functions, constraint backoffs at steady state, and a robustly stable state and disturbance estimator. We first establish nominal stability and offset-free performance. Then, robustness to state and disturbance estimate errors and setpoint and disturbance changes is demonstrated. Finally, the results are extended to sufficiently small plant-model mismatch. The results are illustrated by numerical examples., Comment: 55 pages, 4 figures

We consider the asymptotic stability of MPC under plant-model mismatch, considering primarily models where the origin remains a steady state despite mismatch. Our results differ from prior results on the inherent robustness of MPC, which guarantee only convergence to a neighborhood of the origin, the size of which scales with the magnitude of the mismatch. For MPC with quadratic costs, continuous differentiability of the system dynamics is sufficient to demonstrate exponential stability of the closed-loop system despite mismatch. For MPC with general costs, a joint comparison function bound and scaling condition guarantee asymptotic stability despite mismatch. The results are illustrated in both algebraic and engineering examples. The tools developed to establish these results can address the stability of offset-free MPC, an open and interesting question in the MPC research literature., Comment: 42 pages, 9 figures

We present a series of equations that track the total realized and unrealized profits and losses at any time, incorporating the spread. The resulting formalism is ideally suited to evaluate the performance of trading model algorithms.

We explore the potential of gravitational waves (GWs) to probe the pre-BBN era of the early universe, focusing on the effects of energy injection. Specifically, we examine a hidden sector alongside the Standard Model that undergoes a strong first-order phase transition (FOPT), producing a GW signal. Once the phase transition has completed, energy injection initiates reheating in the hidden sector, which positions the hidden sector field so that additional phase transitions can occur. This can result in a total of three distinct phase transitions with a unique three-peak GW spectrum. Among these transitions, the first and third are of the standard type, while the intermediate second transition is inverted, moving from a broken to an unbroken phase. Using polynomial potentials as a framework, we derive analytical relations among the phase transition parameters and the resulting GW spectrum. Our results indicate that the second and third transitions generate GWs with higher amplitudes than the first, with a peak frequency ratio differing by up to an order of magnitude. This three-peak GW spectrum is detectable by upcoming facilities such as LISA, BBO, and UDECIGO. Notably, the phenomenon is robust across various potentials and model parameters, suggesting that hidden sector GWs provide a powerful tool for exploring new physics scenarios in the pre-BBN era.

Novel devices and novel computing paradigms are key for energy efficient, performant future computing systems. However, designing devices for new applications is often time consuming and tedious. Here, we investigate the design and optimization of spin orbit torque and spin transfer torque magnetic tunnel junction models as the probabilistic devices for true random number generation. We leverage reinforcement learning and evolutionary optimization to vary key device and material properties of the various device models for stochastic operation. Our AI guided codesign methods generated different candidate devices capable of generating stochastic samples for a desired probability distribution, while also minimizing energy usage for the devices.

In this study, we propose a novel approach to uncover subgroup-specific and subgroup-common latent factors addressing the challenges posed by the heterogeneity of neurological and mental disorders, which hinder disease understanding, treatment development, and outcome prediction. The proposed approach, sparse Group Factor Analysis (GFA) with regularised horseshoe priors, was implemented with probabilistic programming and can uncover associations (or latent factors) among multiple data modalities differentially expressed in sample subgroups. Synthetic data experiments showed the robustness of our sparse GFA by correctly inferring latent factors and model parameters. When applied to the Genetic Frontotemporal Dementia Initiative (GENFI) dataset, which comprises patients with frontotemporal dementia (FTD) with genetically defined subgroups, the sparse GFA identified latent disease factors differentially expressed across the subgroups, distinguishing between "subgroup-specific" latent factors within homogeneous groups and "subgroup common" latent factors shared across subgroups. The latent disease factors captured associations between brain structure and non-imaging variables (i.e., questionnaires assessing behaviour and disease severity) across the different genetic subgroups, offering insights into disease profiles. Importantly, two latent factors were more pronounced in the two more homogeneous FTD patient subgroups (progranulin (GRN) and microtubule-associated protein tau (MAPT) mutation), showcasing the method's ability to reveal subgroup-specific characteristics. These findings underscore the potential of sparse GFA for integrating multiple data modalities and identifying interpretable latent disease factors that can improve the characterization and stratification of patients with neurological and mental health disorders., Comment: 38 pages, 14 figures

We consider neural networks (NNs) where the final layer is down-scaled by a fixed hyperparameter $\gamma$. Recent work has identified $\gamma$ as controlling the strength of feature learning. As $\gamma$ increases, network evolution changes from "lazy" kernel dynamics to "rich" feature-learning dynamics, with a host of associated benefits including improved performance on common tasks. In this work, we conduct a thorough empirical investigation of the effect of scaling $\gamma$ across a variety of models and datasets in the online training setting. We first examine the interaction of $\gamma$ with the learning rate $\eta$, identifying several scaling regimes in the $\gamma$-$\eta$ plane which we explain theoretically using a simple model. We find that the optimal learning rate $\eta^*$ scales non-trivially with $\gamma$. In particular, $\eta^* \propto \gamma^2$ when $\gamma \ll 1$ and $\eta^* \propto \gamma^{2/L}$ when $\gamma \gg 1$ for a feed-forward network of depth $L$. Using this optimal learning rate scaling, we proceed with an empirical study of the under-explored "ultra-rich" $\gamma \gg 1$ regime. We find that networks in this regime display characteristic loss curves, starting with a long plateau followed by a drop-off, sometimes followed by one or more additional staircase steps. We find networks of different large $\gamma$ values optimize along similar trajectories up to a reparameterization of time. We further find that optimal online performance is often found at large $\gamma$ and could be missed if this hyperparameter is not tuned. Our findings indicate that analytical study of the large-$\gamma$ limit may yield useful insights into the dynamics of representation learning in performant models., Comment: 33 Pages, 38 figures, preprint text corrected

We prove a strong form of the hot spots conjecture for a class of domains in $\mathbb{R}^d$ which are a natural generalization of the lip domains of Atar and Burdzy [J. Amer. Math. Soc. 17 (2004), 243-265] in dimension two, as well as for a class of symmetric domains in $\mathbb{R}^d$ generalizing the domains studied by Jerison and Nadirashvili [J. Amer. Math. Soc. 13 (2000), 741-772]. Our method of proof is based on studying a vector-valued Laplace operator whose spectrum contains the spectrum of the Neumann Laplacian. This proof is essentially variational and does not require tools from stochastic analysis, nor does it use deformation arguments. In particular, it contains a new proof of the main result of Jerison and Nadirashvili.

A tensor consists of data, $t$, equipped with a multilinear product $\langle t|u_1,\ldots, u_{\ell}\rangle$, called a tensor contraction. Each vector $u_a$ comes from a space $U_a$ called an axis (or mode), the output $\langle t|u_1,\ldots, u_{\ell}\rangle$ lies in a base space $U_0$. In applications, axes often function as linear models of input parameters to a multiparameter process. The tensor then models the complex ways these parameters interact to produce a vector in the base which models outcomes. Thus some linear methods become accessible on a nonlinear problem. Many applications of tensors use decompositions $U_a=X_{a,1}\oplus \cdots \oplus X_{a,k_a}$ with a prescribed subset $\Delta$ of coordinates $(i_0,i_1,\ldots, i_{\ell})$, $1\leq i_a\leq k_a$, where \[ (i_0,i_1,\ldots, i_{\ell})\not\in \Delta \qquad \Longrightarrow \qquad \langle t|X_{1,i_1},\ldots, X_{\ell,i_{\ell}}\rangle \leq \sum_{j\neq i_0} X_{0,j}. \] We call $\Delta$ a cluster pattern and observe that these generalize familiar structures such as echelon forms, block diagonalization, orthogonal, eigen, and singular value decompositions, and clustering more broadly. This article introduces a class of efficiently computable cluster patterns that emerge from an underlying Lie theory of multilinear maps. The new family generalizes known cluster patterns such as Tucker decompositions and block diagonal decompositions but also includes decompositions that approximate curves and surfaces. The latter patterns can capture properties of data categories that do not decompose into the types of disjoint clustering for which existing methods are designed., Comment: 17 pages, 8 figures

The purpose of this note is to summarize the arguments required to derive the results appearing in robust minmax control of linear dynamical systems using a quadratic stage cost. The main results required in robust minmax control are Corollary 19 and Proposition 20. Moreover, the solution to the trust-region problem given in Proposition 15 and Lemma 16 may be of more general interest.

The concept of time mostly plays a subordinate role in finance and economics. The assumption is that time flows continuously and that time series data should be analyzed at regular, equidistant intervals. Nonetheless, already nearly 60 years ago, the concept of an event-based measure of time was first introduced. This paper expands on this theme by discussing the paradigm of intrinsic time, its origins, history, and modern applications. Departing from traditional, continuous measures of time, intrinsic time proposes an event-based, algorithmic framework that captures the dynamic and fluctuating nature of real-world phenomena more accurately. Unsuspected implications arise in general for complex systems and specifically for financial markets. For instance, novel structures and regularities are revealed, otherwise obscured by any analysis utilizing equidistant time intervals. Of particular interest is the emergence of a multiplicity of scaling laws, a hallmark signature of an underlying organizational principle in complex systems. Moreover, a central insight from this novel paradigm is the realization that universal time does not exist; instead, time is observer-dependent, shaped by the intrinsic activity unfolding within complex systems. This research opens up new avenues for economic modeling and forecasting, paving the way for a deeper understanding of the invisible forces that guide the evolution and emergence of market dynamics and financial systems. An exciting and rich landscape of possibilities emerges within the paradigm of intrinsic time., Comment: 12 pages, 1 figure

This paper addresses the identification of models for offset-free model predictive control (MPC), where LTI models are augmented with (fictitious) uncontrollable integrating modes, called integrating disturbances. The states and disturbances are typically estimated with a Kalman filter. The disturbance estimates effectively provide integral control, so the quality of the disturbance model (and resulting filter) directly influences the control performance. We implement eigenvalue constraints to protect against undesirable filter behavior (unstable or marginally stable modes, high-frequency oscillations). Specifically, we consider the class of linear matrix inequality (LMI) regions for eigenvalue constraints. These LMI regions are open sets by default, so we introduce a barrier function method to create tightened, but closed, eigenvalue constraints. To solve the resulting nonlinear semidefinite program, we approximate it as a nonlinear program using a Cholesky factorization method that exploits known sparsity structures of semidefinite optimization variables and matrix inequalities. The algorithm is applied to real-world data taken from two physical systems: first, a low-cost benchmark temperature microcontroller suitable for classroom laboratories, and second, an industrial-scale chemical reactor at Eastman Chemical's plant in Kingsport, TN., Comment: 21 pages, 14 figures

Feedback driven massive outflows play a crucial role in galaxy evolution by regulating star formation and influencing the dynamics of surrounding media. Extracting outflow properties from spectral lines is a notoriously difficult process for a number of reasons, including the possibility that a substantial fraction of the outflow is carried by dense gas in a very narrow range in velocity. This gas can hide in spectra with insufficient resolution. Empirically motivated analysis based on the Apparent Optical Depth method, commonly used in the literature, neglects the contribution of this gas, and may therefore underestimate the true gas column density. More complex semi-analytical line transfer (e.g., SALT) models, on the other hand, allow for the presence of this gas by modeling the radial density and velocity of the outflows as power laws. Here we compare the two approaches to quantify the uncertainties in the inferences of outflow properties based on 1-D "down-the-barrel" using the UV spectra of the CLASSY galaxy sample. We find that empirical modeling may significantly underestimate the column densities relative to SALT analysis, particularly in the optically thick regime. We use simulations to show that the main reason for this discrepancy is the presence of large amount of dense material at low velocities, which can be hidden by the finite spectral resolution of the data. The SALT models in turn could over-estimate the column densities if the assumed power laws of the density profiles strong are not a property of actual outflows., Comment: Accepted to the Astrophysical Journal

Superradiance provides a unique opportunity for investigating dark sectors as well as primordial black holes, which themselves are candidates for dark matter over a wide mass range. Using axion-like particles as an example, we show that line signals emerging from a superradiated axion cloud combined with black hole Hawking radiation in extragalactic and galactic halos, along with microlensing observations lead to complementary constraints on parameter space combinations including the axion-photon coupling, axion mass, black hole mass, and its dark matter fraction, $f_{\rm PBH}$. For the asteroid mass range $\sim10^{16}-10^{22}~{\rm g}$, where primordial black holes can provide the totality of dark matter, we demonstrate that ongoing and upcoming observations such as SXI, JWST, and AMEGO-X will be sensitive to possible line and continuum signals, respectively, providing probes of previously inaccessible regions of $f_{\rm PBH}$ parameter space. Further complementarity from a stochastic gravitational-wave background emerging from the black hole formation mechanism is also considered., Comment: 6+3 pages, 6 figures; v2: matches published version

Non-trivial spatial topology of the Universe may give rise to potentially measurable signatures in the cosmic microwave background. We explore different machine learning approaches to classify harmonic-space realizations of the microwave background in the test case of Euclidean $E_1$ topology (the 3-torus) with a cubic fundamental domain of a size scale significantly smaller than the diameter of the last scattering surface. This is the first step toward developing a machine learning approach to classification of cosmic topology and likelihood-free inference of topological parameters. Different machine learning approaches are capable of classifying the harmonic-space realizations with accuracy greater than 99% if the topology scale is half of the diameter of the last-scattering surface and orientation of the topology is known. For distinguishing random rotations of these sky realizations from realizations of the covering space, the extreme gradient boosting classifier algorithm performs best with an accuracy of 88%. Slightly lower accuracies of 83% to 87% are obtained with the random forest classifier along with one- and two-dimensional convolutional neural networks. The techniques presented here can also accurately classify non-rotated cubic $E_1$ topology realizations with a topology scale slightly larger than the diameter of the last-scattering surface, if enough training data are provided. While information compressing methods like most machine learning approaches cannot exceed the statistical power of a likelihood-based approach that captures all available information, they potentially offer a computationally cheaper alternative. A principle challenge appears to be accounting for arbitrary orientations of a given topology, although this is also a significant hurdle for likelihood-based approaches., Comment: 34 pages, 12 figures, 4 tables

Users often struggle with decision-making between two options (A vs B), as it usually requires time-consuming research across multiple web pages. We propose STRUM-LLM that addresses this challenge by generating attributed, structured, and helpful contrastive summaries that highlight key differences between the two options. STRUM-LLM identifies helpful contrast: the specific attributes along which the two options differ significantly and which are most likely to influence the user's decision. Our technique is domain-agnostic, and does not require any human-labeled data or fixed attribute list as supervision. STRUM-LLM attributes all extractions back to the input sources along with textual evidence, and it does not have a limit on the length of input sources that it can process. STRUM-LLM Distilled has 100x more throughput than the models with comparable performance while being 10x smaller. In this paper, we provide extensive evaluations for our method and lay out future directions for our currently deployed system.

We examine diagonal combs, a recently identified class of infinite metric graphs whose properties depend on one parameter. These graphs exhibit a fascinating regime where they possess infinite volume while maintaining purely discrete spectrum for the Neumann Laplacian. In this regime, we establish polynomial upper and lower bounds on the $k$-th eigenvalue, revealing that the eigenvalues grow at a rate strictly slower than quadratic. However, once the diagonal combs transition to finite volume, their growth accelerates to a quadratic rate. Our methodology involves employing spectral geometric principles tailored for metric graphs, complemented by deriving estimates for the $k$-th eigenvalue on compact metric graphs., Comment: 10 pages, 1 figure

The passive retroreflector arrays placed on the moon by Apollo 11, 14 and 15 astronauts continue to produce valuable Earth-Moon range measurements that enable high-precision tests of gravitational physics, as well as studies of geo- and selenophysics. The optical throughput of these retroreflectors has declined since their deployment, with an additional signal loss at full moon when the reflectors experience direct solar illumination. We show that the loss in return rate can be attributed to the accumulation of a thin layer of lunar dust on the surfaces of the corner cube retroreflectors. First, a careful analysis of the optical link budget for the Apache Point Observatory Lunar Laser-ranging Operation (APOLLO) experiment reveals that the lunar return rate is 15--20 times smaller than predicted, a deficit that can be explained by a reflector dust covering fraction of ${\sim} 50$\%. Second, range measurements taken during a lunar eclipse indicate that the solar illumination of the retroreflectors degrades their throughput by an additional factor of ${\sim}15$. Finally, a numerical simulation of heat transfer in dust-coated reflectors is able to model the resulting thermal lensing effect, in which thermal gradients in the retroreflectors degrade their far-field diffraction pattern. A comparison of this simulation to eclipse measurements finds a dust coverage fraction of ${\sim}50$%. Taken together, the link analysis, eclipse observations and thermal modeling support the claim that the retroreflectors are obscured by lunar dust, with both link budget and simulation independently finding the dust fraction to be $\sim$50%.

We report measurements of the transverse diffusion of electrons in P-10 gas (90% Ar, 10% CH4) in a laboratory-scale time projection chamber (TPC) utilizing a novel pixelated signal capture and digitization technique known as Q-Pix. The Q-Pix method incorporates a precision switched integrating transimpedance amplifier whose output is compared to a threshold voltage. Upon reaching the threshold, a comparator sends a 'reset' signal, initiating a discharge of the integrating capacitor. The time difference between successive resets is inversely proportional to the average current at the pixel in that time interval, and the number of resets is directly proportional to the total collected charge. We developed a 16-channel Q-Pix prototype fabricated from commercial off-the-shelf components and coupled them to 16 concentric annular anode electrodes to measure the spatial extent of the electron swarm that reaches the anode after drifting through the uniform field of the TPC. The swarm is produced at a gold photocathode using pulsed UV light. The measured transverse diffusion agrees with simulations in PyBoltz across a range of operating pressures (200-1500 Torr). These results demonstrate that a Q-Pix readout can successfully reconstruct the ionization topology in a TPC., Comment: 18 pages, 9 figures

Characterising the representation of sensory stimuli in the brain is a fundamental scientific endeavor, which can illuminate principles of information coding. Most characterizations reduce the dimensionality of neural data by converting spike trains to firing rates or binned spike counts, applying explicitly named methods of ``dimensionality reduction'', or collapsing trial-to-trial variability. Characterisation of the full-dimensional geometry of timing-based representations may provide unexpected insights into how complex high-dimensional information is encoded. Recent research shows that the distribution of representations elicited over trials of a single stimulus can be geometrically characterized without the application of dimensionality reduction, maintaining the temporal spiking information of individual neurons in a cell assembly and illuminating rich geometric structure. We extend these results, showing that precise spike time patterns for larger cell assemblies are time-warped (i.e. stretched or compressed) on each trial. Moreover, by geometrically characterizing distributions of large spike time patterns, our analysis supports the hypothesis that the degree to which a spike time pattern is time-warped depends on the cortical area's background activity level on a single trial. Finally, we suggest that the proliferation of large electrophysiology datasets and the increasing concentration of ``neural geometrists'', creates ideal conditions for characterization of full-dimensional spike time representations, in complement to dimensionality reduction approaches., Comment: Accepted as an extended abstract at the NeurReps workshop at NeurIPS 2023. The workshop doesn't publish extended abstracts so submitting here

We study the problem of minimizing or maximizing the fundamental spectral gap of Schr\"odinger operators on metric graphs with either a convex potential or a ``single-well'' potential on an appropriate specified subset. (In the case of metric trees, such a subset can be the entire graph.) In the convex case we find that the minimizing and maximizing potentials are piecewise linear with only a finite number of points of non-smoothness, but give examples showing that the optimal potentials need not be constant. This is a significant departure from the usual scenarios on intervals and domains where the constant potential is typically minimizing. In the single-well case we show that the optimal potentials are piecewise constant with a finite number of jumps, and in both cases give an explicit estimate on the number of points of non-smoothness, respectively jumps, the minimizing potential can have. Furthermore, we show that, unlike on domains, it is not generally possible to find nontrivial bounds on the fundamental gap in terms of the diameter of the graph alone, within the given classes.

Observers for PDEs are themselves PDEs. Therefore, producing real time estimates with such observers is computationally burdensome. For both finite-dimensional and ODE systems, moving-horizon estimators (MHE) are operators whose output is the state estimate, while their inputs are the initial state estimate at the beginning of the horizon as well as the measured output and input signals over the moving time horizon. In this paper we introduce MHEs for PDEs which remove the need for a numerical solution of an observer PDE in real time. We accomplish this using the PDE backstepping method which, for certain classes of both hyperbolic and parabolic PDEs, produces moving-horizon state estimates explicitly. Precisely, to explicitly produce the state estimates, we employ a backstepping transformation of a hard-to-solve observer PDE into a target observer PDE, which is explicitly solvable. The MHEs we propose are not new observer designs but simply the explicit MHE realizations, over a moving horizon of arbitrary length, of the existing backstepping observers. Our PDE MHEs lack the optimality of the MHEs that arose as duals of MPC, but they are given explicitly, even for PDEs. In the paper we provide explicit formulae for MHEs for both hyperbolic and parabolic PDEs, as well as simulation results that illustrate theoretically guaranteed convergence of the MHEs., Comment: 6 pages, 1 figure. ACC 2024

Co-design is a prominent topic presently in computing, speaking to the mutual benefit of coordinating design choices of several layers in the technology stack. For example, this may be designing algorithms which can most efficiently take advantage of the acceleration properties of a given architecture, while simultaneously designing the hardware to support the structural needs of a class of computation. The implications of these design decisions are influential enough to be deemed a lottery, enabling an idea to win out over others irrespective of the individual merits. Coordination is a well studied topic in the mathematics of game theory, where in many cases without a coordination mechanism the outcome is sub-optimal. Here we consider what insights game theoretic analysis can offer for computer architecture co-design. In particular, we consider the interplay between algorithm and architecture advances in the field of neuromorphic computing. Analyzing developments of spiking neural network algorithms and neuromorphic hardware as a co-design game we use the Stag Hunt model to illustrate challenges for spiking algorithms or architectures to advance the field independently and advocate for a strategic pursuit to advance neuromorphic computing., Comment: 8 pages, 2 figures, accepted to First Workshop on Machine Learning with New Compute Paradigms at NeurIPS 2023

We introduce a natural notion of mean (or average) distance in the context of compact metric graphs, and study its relation to geometric properties of the graph. We show that it exhibits a striking number of parallels to the reciprocal of the spectral gap of the graph Laplacian with standard vertex conditions: it is maximised among all graphs of fixed length by the path graph (interval), or by the loop in the restricted class of doubly connected graphs, and it is minimised among all graphs of fixed length and number of edges by the equilateral flower graph. We also establish bounds for the correctly scaled product of the spectral gap and the square of the mean distance which depend only on combinatorial, and not metric, features of the graph. This raises the open question whether this product admits absolute upper and lower bounds valid on all compact metric graphs.

We demonstrate for the first time that Galactic cosmic rays with energies as high as 1e10 eV can trigger a cascade of low-energy (< 20 eV) secondary electrons that could be a significant contributor to the interstellar synthesis of prebiotic molecules whose delivery by comets, meteorites, and interplanetary dust particles may have kick-started life on Earth. We explore the relative importance of low-energy (< 20 eV) secondary electrons--agents of radiation chemistry--and low-energy (< 10 eV), non-ionizing photons--instigators of photochemistry. Our calculations indicate fluxes of 100 electrons/cm2/s for low-energy secondary electrons produced within interstellar ices due to incident attenuated Galactic cosmic-ray (CR) protons. Consequently, in certain star-forming regions where internal high-energy radiation sources produce ionization rates that are observed to be a thousand times greater than the typical interstellar Galactic ionization rate, the flux of low-energy secondary electrons should far exceed that of non-ionizing photons. Because reaction cross-sections can be several orders of magnitude larger for electrons than for photons, even in the absence of such enhancement our calculations indicate that secondary low-energy electrons are at least as significant as low-energy (< 10 eV) non-ionizing photons in the interstellar synthesis of prebiotic molecules. Most importantly, our results demonstrate the pressing need for explicitly incorporating low-energy electrons in current and future astrochemical simulations of cosmic ices. Such models are critically important for interpreting James Webb Space Telescope infrared measurements, which are currently being used to probe the origins of life by studying complex organic molecules found in ices near star-forming regions., Comment: 14 pages, 6 figures

In our era of enormous neural networks, empirical progress has been driven by the philosophy that more is better. Recent deep learning practice has found repeatedly that larger model size, more data, and more computation (resulting in lower training loss) improves performance. In this paper, we give theoretical backing to these empirical observations by showing that these three properties hold in random feature (RF) regression, a class of models equivalent to shallow networks with only the last layer trained. Concretely, we first show that the test risk of RF regression decreases monotonically with both the number of features and the number of samples, provided the ridge penalty is tuned optimally. In particular, this implies that infinite width RF architectures are preferable to those of any finite width. We then proceed to demonstrate that, for a large class of tasks characterized by powerlaw eigenstructure, training to near-zero training loss is obligatory: near-optimal performance can only be achieved when the training error is much smaller than the test error. Grounding our theory in real-world data, we find empirically that standard computer vision tasks with convolutional neural tangent kernels clearly fall into this class. Taken together, our results tell a simple, testable story of the benefits of overparameterization, overfitting, and more data in random feature models., Comment: Appeared in ICLR 2024

Probabilistic artificial neural networks offer intriguing prospects for enabling the uncertainty of artificial intelligence methods to be described explicitly in their function; however, the development of techniques that quantify uncertainty by well-understood methods such as Monte Carlo sampling has been limited by the high costs of stochastic sampling on deterministic computing hardware. Emerging computing systems that are amenable to hardware-level probabilistic computing, such as those that leverage stochastic devices, may make probabilistic neural networks more feasible in the not-too-distant future. This paper describes the scANN technique -- \textit{sampling (by coinflips) artificial neural networks} -- which enables neural networks to be sampled directly by treating the weights as Bernoulli coin flips. This method is natively well suited for probabilistic computing techniques that focus on tunable stochastic devices, nearly matches fully deterministic performance while also describing the uncertainty of correct and incorrect neural network outputs., Comment: 9 pages, accepted to 2023 IEEE International Conference on Rebooting Computing

Using Commercial Off-The-Shelf (COTS) Operational Amplifiers (OpAmps) and Complementary Metal-Oxide Semiconductor (CMOS) transistors, we present a demonstration of the Q-Pix front-end architecture, a novel readout solution for kiloton-scale Liquid Argon Time Projection Chamber (LArTPC) detectors. The Q-Pix scheme employs a Charge-Integrate/Reset process based on the Least Action principle, enabling pixel-scale self-triggering charge collection and processing, minimizing energy consumption, and maximizing data compression. We examine the architecture's sensitivity, linearity, noise, and other features at the circuit board level and draw comparisons to SPICE simulations. Furthermore, we highlight the resemblance between the Q-Pix front-end and Sigma-Delta modulator, emphasizing that digital data processing techniques for Sigma-Delta can be directly applied to Q-Pix, resulting in enhanced signal-to-noise performance. These insights will inform the development of Q-Pix front-end designs in integrated circuits (IC) and guide data collection and processing for future large-scale LArTPC detectors in neutrino physics and other high-energy physics experiments.

The push to train ever larger neural networks has motivated the study of initialization and training at large network width. A key challenge is to scale training so that a network's internal representations evolve nontrivially at all widths, a process known as feature learning. Here, we show that feature learning is achieved by scaling the spectral norm of weight matrices and their updates like $\sqrt{\texttt{fan-out}/\texttt{fan-in}}$, in contrast to widely used but heuristic scalings based on Frobenius norm and entry size. Our spectral scaling analysis also leads to an elementary derivation of \emph{maximal update parametrization}. All in all, we aim to provide the reader with a solid conceptual understanding of feature learning in neural networks.

Calysto Scheme is written in Scheme in Continuation-Passing Style, and converted through a series of correctness-preserving program transformations into Python. It has support for standard Scheme functionality, including call/cc, as well as syntactic extensions, a nondeterministic operator for automatic backtracking, and many extensions to allow Python interoperation. Because of its Python foundation, it can take advantage of modern Python libraries, including those for machine learning and other pedagogical contexts. Although Calysto Scheme was developed with educational purposes in mind, it has proven to be generally useful due to its simplicity and ease of installation. It has been integrated into the Jupyter Notebook ecosystem and used in the classroom to teach introductory Programming Languages with some interesting and unique twists., Comment: Presented at The 2023 Scheme and Functional Programming Workshop (arXiv:cs/0101200)

We introduce and study a natural multispecies variant of the inhomogeneous PushTASEP with site-dependent rates on the finite ring. We show that the stationary distribution of this process is proportional to the ASEP polynomials at $q = 1$ and $t = 0$. This is done by constructing a multiline process which projects to the multispecies PushTASEP, and identifying its stationary distribution using time-reversal arguments. We also study symmetry properties of the process under interchange of the rates associated to the sites. These results hold not just for events depending on the configuration at a single time in equilibrium, but also for systems out of equilibrium and for events depending on the path of the process over time. Lastly, we give explicit formulas for nearest-neighbour two-point correlations in terms of Schur functions., Comment: 31 pages, 7 figures, minor improvements

Lifelong learning - an agent's ability to learn throughout its lifetime - is a hallmark of biological learning systems and a central challenge for artificial intelligence (AI). The development of lifelong learning algorithms could lead to a range of novel AI applications, but this will also require the development of appropriate hardware accelerators, particularly if the models are to be deployed on edge platforms, which have strict size, weight, and power constraints. Here, we explore the design of lifelong learning AI accelerators that are intended for deployment in untethered environments. We identify key desirable capabilities for lifelong learning accelerators and highlight metrics to evaluate such accelerators. We then discuss current edge AI accelerators and explore the future design of lifelong learning accelerators, considering the role that different emerging technologies could play.

We study partition problems based on two ostensibly different kinds of energy functionals defined on $k$-partitions of metric graphs: Cheeger-type functionals whose minimisers are the $k$-Cheeger cuts of the graph, and the corresponding values are the $k$-Cheeger constants of the graph; and functionals built using the first eigenvalue of the Laplacian with positive, i.e. absorbing, Robin (delta) vertex conditions at the boundary of the partition elements. We prove existence of minimising $k$-partitions, $k \geq 2$, for both these functionals. We also show that, for each $k \geq 2$, as the Robin parameter $\alpha \to 0$, up to a renormalisation the spectral minimal Robin energy converges to the $k$-Cheeger constant. Moreover, up to a subsequence, the Robin spectral minimal $k$-partitions converge in a natural sense to a $k$-Cheeger cut of the graph. Finally, we show that as $\alpha \to \infty$ there is convergence in a similar sense to the corresponding Dirichlet minimal energy and partitions. It is strongly expected that similar results hold on general (smooth, bounded) Euclidean domains and manifolds., Comment: Revised version, accepted for publication in Journal d'Analyse Math\'ematique

These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training., Comment: These are notes from lectures delivered by Yasaman Bahri and Boris Hanin at the 2022 Les Houches Summer School on Statistics Physics and Machine Learning and a first version of them were transcribed by Antonin Brossollet, Vittorio Erba, Christian Keup, Rosalba Pacelli, James B. Simon

We present a new optimization-based method for aggregating preferences in settings where each voter expresses preferences over pairs of alternatives. Our approach to identifying a consensus partial order is motivated by the observation that collections of votes that form a cycle can be treated as collective ties. Our approach then removes unions of cycles of votes, or circulations, from the vote graph and determines aggregate preferences from the remainder. Specifically, we study the removal of maximal circulations attained by any union of cycles the removal of which leaves an acyclic graph. We introduce the strong maximum circulation, the removal of which guarantees a unique outcome in terms of the induced partial order, called the strong partial order. The strong maximum circulation also satisfies strong complementary slackness conditions, and is shown to be solved efficiently as a network flow problem. We further establish the relationship between the dual of the maximum circulation problem and Kemeny's method, a popular optimization-based approach for preference aggregation. We also show that identifying a minimum maximal circulation -- i.e., a maximal circulation containing the smallest number of votes -- is an NP-hard problem. Further an instance of the minimum maximal circulation may have multiple optimal solutions whose removal results in conflicting partial orders., Comment: INFORMS Journal on Optimization, Forthcoming

Inspired by a recent result of Funano's, we provide a sharp quantitative comparison result between the first nontrivial eigenvalues of the Neumann Laplacian on bounded convex domains $\Omega_{1} \subset \Omega_{2}$ in any dimension $d$ greater than or equal to two, recovering domain monotonicity up to an explicit multiplicative factor. We provide upper and lower bounds for such multiplicative factors for higher-order eigenvalues, and study their behaviour with respect to the dimension and order. We further consider different scenarios where convexity is no longer imposed. In a final section we formulate some related open problems.

Axions and axion-like pseudoscalar particles with dimension-5 couplings to photons exhibit coherent Primakoff scattering with ordered crystals at keV energy scales, making for a natural detection technique in searches for solar axions. We find that there are large suppressive corrections, potentially greater than a factor of $\mathcal{O}(10^3)$, to the coherent enhancement when taking into account absorption of the final state photon. This effect has already been accounted for in light-shining-through-wall experiments through the language of Darwin classical diffraction, but is missing from the literature in the context of solar axion searches that use a matrix element approach. We extend the treatment of the event rate with a heuristic description of absorption effects to bridge the gap between these two languages. Furthermore, we explore the Borrmann effect of anomalous absorption in lifting some of the event rate suppression by increasing the coherence length of the conversion. We study this phenomenon in Ge, NaI, and CsI crystal experiments and its impact on the the projected sensitivities of SuperCDMS, LEGEND, and SABRE to the solar axion parameter space. Lastly, we comment on the reach of multi-tonne scale crystal detectors and strategies to maximize the discovery potential of experimental efforts in this vein., Comment: 15 pages, 9 figures (including appendix); updated references

A satisfactory understanding of information processing in spiking neural networks requires appropriate computational abstractions of neural activity. Traditionally, the neural population state vector has been the most common abstraction applied to spiking neural networks, but this requires artificially partitioning time into bins that are not obviously relevant to the network itself. We introduce a distinct set of techniques for analyzing spiking neural networks that decomposes neural activity into multiple, disjoint, parallel threads of activity. We construct these threads by estimating the degree of causal relatedness between pairs of spikes, then use these estimates to construct a directed acyclic graph that traces how the network activity evolves through individual spikes. We find that this graph of spiking activity naturally decomposes into disjoint connected components that overlap in space and time, which we call Graphical Neural Activity Threads (GNATs). We provide an efficient algorithm for finding analogous threads that reoccur in large spiking datasets, revealing that seemingly distinct spike trains are composed of similar underlying threads of activity, a hallmark of compositionality. The picture of spiking neural networks provided by our GNAT analysis points to new abstractions for spiking neural computation that are naturally adapted to the spatiotemporally distributed dynamics of spiking neural networks.

If the Universe has non-trivial spatial topology, observables depend on both the parameters of the spatial manifold and the position and orientation of the observer. In infinite Euclidean space, most cosmological observables arise from the amplitudes of Fourier modes of primordial scalar curvature perturbations. Topological boundary conditions replace the full set of Fourier modes with specific linear combinations of selected Fourier modes as the eigenmodes of the scalar Laplacian. We present formulas for eigenmodes in orientable Euclidean manifolds with the topologies $E_{1}-E_{6}$, $E_{11}$, $E_{12}$, $E_{16}$, and $E_{18}$ that encompass the full range of manifold parameters and observer positions, generalizing previous treatments. Under the assumption that the amplitudes of primordial scalar curvature eigenmodes are independent random variables, for each topology we obtain the correlation matrices of Fourier-mode amplitudes (of scalar fields linearly related to the scalar curvature) and the correlation matrices of spherical-harmonic coefficients of such fields sampled on a sphere, such as the temperature of the cosmic microwave background (CMB). We evaluate the detectability of these correlations given the cosmic variance of the observed CMB sky. We find that topologies where the distance to our nearest clone is less than about 1.2 times the diameter of the last scattering surface of the CMB give a correlation signal that is larger than cosmic variance noise in the CMB. This implies that if cosmic topology is the explanation of large-angle anomalies in the CMB, then the distance to our nearest clone is not much larger than the diameter of the last scattering surface. We argue that the topological information is likely to be better preserved in three-dimensional data, such as will eventually be available from large-scale structure surveys., Comment: 79 pages, 9 figures. v3: version published in JCAP

Modern heterogeneous supercomputing systems are comprised of CPUs, GPUs, and high-speed network interconnects. Communication libraries supporting efficient data transfers involving memory buffers from the GPU memory typically require the CPU to orchestrate the data transfer operations. A new offload-friendly communication strategy, stream-triggered (ST) communication, was explored to allow offloading the synchronization and data movement operations from the CPU to the GPU. A Message Passing Interface (MPI) one-sided active target synchronization based implementation was used as an exemplar to illustrate the proposed strategy. A latency-sensitive nearest neighbor microbenchmark was used to explore the various performance aspects of the implementation. The offloaded implementation shows significant on-node performance advantages over standard MPI active RMA (36%) and point-to-point (61%) communication. The current multi-node improvement is less (23% faster than standard active RMA but 11% slower than point-to-point), but plans are in progress to purse further improvements., Comment: 12 pages, 17 figures

We study the cost of overfitting in noisy kernel ridge regression (KRR), which we define as the ratio between the test error of the interpolating ridgeless model and the test error of the optimally-tuned model. We take an "agnostic" view in the following sense: we consider the cost as a function of sample size for any target function, even if the sample size is not large enough for consistency or the target is outside the RKHS. We analyze the cost of overfitting under a Gaussian universality ansatz using recently derived (non-rigorous) risk estimates in terms of the task eigenstructure. Our analysis provides a more refined characterization of benign, tempered and catastrophic overfitting (cf. Mallinar et al. 2022)., Comment: This is the ICLR CR version

Hyperparameter tuning of deep learning models can lead to order-of-magnitude performance gains for the same amount of compute. Despite this, systematic tuning is uncommon, particularly for large models, which are expensive to evaluate and tend to have many hyperparameters, necessitating difficult judgment calls about tradeoffs, budgets, and search bounds. To address these issues and propose a practical method for robustly tuning large models, we present Cost-Aware Pareto Region Bayesian Search (CARBS), a Bayesian optimization algorithm that performs local search around the performance-cost Pareto frontier. CARBS does well even in unbounded search spaces with many hyperparameters, learns scaling relationships so that it can tune models even as they are scaled up, and automates much of the "black magic" of tuning. Among our results, we effectively solve the entire ProcGen benchmark just by tuning a simple baseline (PPO, as provided in the original ProcGen paper). We also reproduce the model size vs. training tokens scaling result from the Chinchilla project (Hoffmann et al. 2022), while simultaneously discovering scaling laws for every other hyperparameter, via an easy automated process that uses significantly less compute and is applicable to any deep learning problem (not just language models).