Search

We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks. With manifold-to-manifold hidden layers and an arbitrary last layer, they can model manifold- and scalar-valued functions, as well as vector fields. We target data inherently supported on manifolds, which is too complex for shallow Gaussian processes thereon. For example, while the latter perform well on high-altitude wind data, they struggle with the more intricate, nonstationary patterns at low altitudes. Our models significantly improve performance in these settings, enhancing prediction quality and uncertainty calibration, and remain robust to overfitting, reverting to shallow models when additional complexity is unneeded. We further showcase our models on Bayesian optimisation problems on manifolds, using stylised examples motivated by robotics, and obtain substantial improvements in later stages of the optimisation process. Finally, we show our models to have potential for speeding up inference for non-manifold data, when, and if, it can be mapped to a proxy manifold well enough.

Robust modeling of non-linear scales is critical for accurate cosmological inference in Stage IV surveys. For weak lensing analyses in particular, a key challenge arises from the incomplete understanding of how non-gravitational processes, such as supernovae and active galactic nuclei - collectively known as baryonic feedback - affect the matter distribution. Several existing methods for modeling baryonic feedback treat it independently from the underlying cosmology, an assumption which has been found to be inaccurate by hydrodynamical simulations. In this work, we examine the impact of this coupling between baryonic feedback and cosmology on parameter inference at LSST Y1 precision. We build mock 3$\times$2pt data vectors using the Magneticum suite of hydrodynamical simulations, which span a wide range of cosmologies while keeping subgrid parameters fixed. We perform simulated likelihood analyses for two baryon mitigation techniques: (i) the Principal Component Analysis (PCA) method which identifies eigenmodes for capturing the effect baryonic feedback on the data vector and (ii) HMCode2020 (Mead et al. 2021) which analytically models the modification in the matter distribution using a halo model approach. Our results show that the PCA method is robust to the coupling between cosmology and baryonic feedback, whereas, when using HMCode2020 there can be up to $0.5\sigma$ bias in $\Omega_\text{m}$-$S_8$. For HMCode2020, the bias also correlates with the input cosmology while for PCA we find no such correlation., Comment: correction to acknowledgements

Precision cardiology based on cardiac digital twins requires accurate simulations of cardiac arrhythmias. However, detailed models, such as the monodomain model, are computationally costly and have limited applicability in practice. Thus, it desirable to have fast models that can still represent the main physiological features presented during cardiac arrhythmias. The eikonal model is an approximation of the monodomain model that is widely used to describe the arrival times of the electrical wave. However, the standard eikonal model does not generalize to the complex re-entrant dynamics that characterize the cardiac arrhythmias. In this work, we propose an eikonal model that includes the tissue re-excitability, which allows to describe re-entries. The re-excitability properties are inferred from the monodomain model. Our eikonal model also handles the tissue anisotropy and heterogeneity. We compare the eikonal model to the monodomain model in various numerical experiments in the atria and the ventricles. The eikonal model is qualitatively accurate in the simulation of re-entries and can be potentially ran in real-time, opening the door to its clinical applicability., Comment: 18 pages, 11 figures

We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including Variational AutoEncoders (VAEs), Generative Adversarial Networks (GANs), Normalizing Flows, Diffusion models, and models based on Conditional Flow Matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in 1-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space., Comment: 204 pages, 100+ figures, 30+ tables

Statistical fluctuations of local tensorial fields beyond the mean are relevant to predict localized failure or overall behavior of the inelastic composites. The expression for second moments of the local fields can be established using Hill-Mandel condition. Complete estimation of statistical fluctuations via second moments is usually ignored despite its significance. In Eshelby-based mean-field approaches, the second moments are evaluated through derivatives of Hill's Polarization tensor $ \left ( \mathbb{P}_{\mathrm{o}} \right )$ using a singular approximation. Typically, semi-analytical procedures using numerical integration are used to evaluate the derivatives of the polarization tensor $\mathbb{P}_{\mathrm{o}}$. Here, new analytically derived explicit expressions are presented for calculating the derivatives, specifically for unidirectional fibrous composites with isotropic phases. Full-field homogenization using finite element is used to compute the statistical distribution of local fields (exact solution) for the class of random fibrous microstructures. The mean-field estimates are validated with the exact solution across different fiber volume fractions and aspect ratios. The results indicate that the fiber volume fraction significantly influences the fluctuation of stress tensor invariants, whereas the aspect ratio has minimal effect., Comment: 37 pages, 8 figures, 1 table

We introduce MODEL SELECTOR, a framework for label-efficient selection of pretrained classifiers. Given a pool of unlabeled target data, MODEL SELECTOR samples a small subset of highly informative examples for labeling, in order to efficiently identify the best pretrained model for deployment on this target dataset. Through extensive experiments, we demonstrate that MODEL SELECTOR drastically reduces the need for labeled data while consistently picking the best or near-best performing model. Across 18 model collections on 16 different datasets, comprising over 1,500 pretrained models, MODEL SELECTOR reduces the labeling cost by up to 94.15% to identify the best model compared to the cost of the strongest baseline. Our results further highlight the robustness of MODEL SELECTOR in model selection, as it reduces the labeling cost by up to 72.41% when selecting a near-best model, whose accuracy is only within 1% of the best model.

Reinforcement learning (RL) is ubiquitous in the development of modern AI systems. However, state-of-the-art RL agents require extensive, and potentially unsafe, interactions with their environments to learn effectively. These limitations confine RL agents to simulated environments, hindering their ability to learn directly in real-world settings. In this work, we present ActSafe, a novel model-based RL algorithm for safe and efficient exploration. ActSafe learns a well-calibrated probabilistic model of the system and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics, while enforcing pessimism w.r.t. the safety constraints. Under regularity assumptions on the constraints and dynamics, we show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time. In addition, we propose a practical variant of ActSafe that builds on latest model-based RL advancements and enables safe exploration even in high-dimensional settings such as visual control. We empirically show that ActSafe obtains state-of-the-art performance in difficult exploration tasks on standard safe deep RL benchmarks while ensuring safety during learning.

Recent efforts in fine-tuning language models often rely on automatic data selection, commonly using Nearest Neighbors retrieval from large datasets. However, we theoretically show that this approach tends to select redundant data, limiting its effectiveness or even hurting performance. To address this, we introduce SIFT, a data selection algorithm designed to reduce uncertainty about the model's response given a prompt, which unifies ideas from retrieval and active learning. Whereas Nearest Neighbor retrieval typically fails in the presence of information duplication, SIFT accounts for information duplication and optimizes the overall information gain of the selected examples. We focus our evaluations on fine-tuning at test-time for prompt-specific language modeling on the Pile dataset, and show that SIFT consistently outperforms Nearest Neighbor retrieval, with minimal computational overhead. Moreover, we show that our uncertainty estimates can predict the performance gain of test-time fine-tuning, and use this to develop an adaptive algorithm that invests test-time compute proportional to realized performance gains. We provide the $\texttt{activeft}$ (Active Fine-Tuning) library which can be used as a drop-in replacement for Nearest Neighbor retrieval.

SHAP values are a popular local feature-attribution method widely used in interpretable and explainable AI. We tackle the problem of efficiently computing these values. We cover both the model-agnostic (black-box) setting, where one only has query access to the model and also the case of (ensembles of) trees where one has access to the structure of the tree. For both the black-box and the tree setting we propose a two-stage approach for estimating SHAP values. Our algorithm's first step harnesses recent results showing that many real-world predictors have a spectral bias that allows us to either exactly represent (in the case of ensembles of decision trees), or efficiently approximate them (in the case of neural networks) using a compact Fourier representation. In the second step of the algorithm, we use the Fourier representation to exactly compute SHAP values. The second step is computationally very cheap because firstly, the representation is compact and secondly, we prove that there exists a closed-form expression for SHAP values for the Fourier basis functions. Furthermore, the expression we derive effectively linearizes the computation into a simple summation and is amenable to parallelization on multiple cores or a GPU. Since the function approximation (first step) is only done once, it allows us to produce Shapley values in an amortized way. We show speedups compared to relevant baseline methods equal levels of accuracy for both the tree and black-box settings. Moreover, this approach introduces a reliable and fine-grained continuous trade-off between computation and accuracy through the sparsity of the Fourier approximation, a feature previously unavailable in all black-box methods., Comment: Submitted as a conference paper to ICLR 2025

Pre-trained generalist policies are rapidly gaining relevance in robot learning due to their promise of fast adaptation to novel, in-domain tasks. This adaptation often relies on collecting new demonstrations for a specific task of interest and applying imitation learning algorithms, such as behavioral cloning. However, as soon as several tasks need to be learned, we must decide which tasks should be demonstrated and how often? We study this multi-task problem and explore an interactive framework in which the agent adaptively selects the tasks to be demonstrated. We propose AMF (Active Multi-task Fine-tuning), an algorithm to maximize multi-task policy performance under a limited demonstration budget by collecting demonstrations yielding the largest information gain on the expert policy. We derive performance guarantees for AMF under regularity assumptions and demonstrate its empirical effectiveness to efficiently fine-tune neural policies in complex and high-dimensional environments.

In this note, we prove that $K_{2i} (\mathbb{Z}/p^n) \neq 0$ if and only if $p-1$ divides $i$ and $0 \leq i \leq (p-1) p^{n-2}$, refining the even vanishing theorem of Antieau, Nikolaus and the first author in this case. As a corollary of our proof, we determine that the nilpotence order of $v_1$ in $\pi_* K(\mathbb{Z}/p^n)/p$ is equal to $\frac{p^n-1}{p-1}$. Our proof combines the recent crystallinity result for reduced syntomic cohomology of Hahn, Levy and the second author with the explicit complex computing the syntomic cohomology of $\mathcal{O}_K /\varpi^n$ constructed by Antieau, Nikolaus and the first author., Comment: 10 pages

In recent years, there has been a renewed interest in preconditioning for multilevel Toeplitz systems, a research field that has been extensively explored over the past several decades. This work introduces novel preconditioning strategies using multilevel $\tau$ matrices for both symmetric and nonsymmetric multilevel Toeplitz systems. Our proposals constitute a general framework, as they are constructed solely based on the generating function of the multilevel Toeplitz coefficient matrix, when it can be defined. We begin with nonsymmetric systems, where we employ a symmetrization technique by permuting the coefficient matrix to produce a real symmetric multilevel Hankel structure. We propose a multilevel $\tau$ preconditioner tailored to the symmetrized system and prove that the eigenvalues of the preconditioned matrix sequence cluster at $\pm 1$, leading to rapid convergence when using the preconditioned minimal residual method. The high effectiveness of this approach is demonstrated through its application in solving space fractional diffusion equations. Next, for symmetric systems we introduce another multilevel $\tau$ preconditioner and show that the preconditioned conjugate gradient method can achieve an optimal convergence rate, namely a rate that is independent of the matrix size, when employed for a class of ill-conditioned multilevel Toeplitz systems. Numerical examples are provided to critically assess the effectiveness of our proposed preconditioners compared to several leading existing preconditioned solvers, highlighting their superior performance.

Bayesian optimization (BO) is a powerful framework to optimize black-box expensive-to-evaluate functions via sequential interactions. In several important problems (e.g. drug discovery, circuit design, neural architecture search, etc.), though, such functions are defined over large $\textit{combinatorial and unstructured}$ spaces. This makes existing BO algorithms not feasible due to the intractable maximization of the acquisition function over these domains. To address this issue, we propose $\textbf{GameOpt}$, a novel game-theoretical approach to combinatorial BO. $\textbf{GameOpt}$ establishes a cooperative game between the different optimization variables, and selects points that are game $\textit{equilibria}$ of an upper confidence bound acquisition function. These are stable configurations from which no variable has an incentive to deviate$-$ analog to local optima in continuous domains. Crucially, this allows us to efficiently break down the complexity of the combinatorial domain into individual decision sets, making $\textbf{GameOpt}$ scalable to large combinatorial spaces. We demonstrate the application of $\textbf{GameOpt}$ to the challenging $\textit{protein design}$ problem and validate its performance on four real-world protein datasets. Each protein can take up to $20^{X}$ possible configurations, where $X$ is the length of a protein, making standard BO methods infeasible. Instead, our approach iteratively selects informative protein configurations and very quickly discovers highly active protein variants compared to other baselines.

Self-enrichment is one of the leading explanations for chemical anomalies in globular clusters. In this scenario, various candidate polluter stars have been proposed to eject gas with altered chemical composition during the self-enrichment process. Most of the proposed polluters will also eject radioactive $^{26}$Al into the surroundings. Hence, any detection of $^{26}$Al in young massive star clusters (YMCs) would support the self-enrichment scenario if YMCs were indeed the progenitors of globular clusters. Observations of gamma-ray data from COMPTEL and INTEGRAL, as well as detections of $^{26}$AlF molecules by the Atacama Large Millimeter-submillimeter Array (ALMA), indicate the maturing of $^{26}$Al detection methods. Detection possibilities will be enhanced in the short- to mid-term by the upcoming launch of the Compton Spectrometer and Imager (COSI). The Square Kilometer Array (SKA) could in principle also detect radio recombination lines of the positronium formed from the decay products of $^{26}$Al. Here, we show for a sample of YMCs in the nearby Universe, where self-enrichment could plausibly take place. For some nearby galaxies, this could enhance $^{26}$Al by an order of one magnitude. Detecting $^{26}$AlF with ALMA appears feasible for many candidate self-enrichment clusters, although significant challenges remain with other detection methods. The Large Magellanic Cloud, with its YMC R136, stands out as the most promising candidate. Detecting a 1.8~MeV radioactive decay line of $^{26}$Al here would require at least 15 months of targeted observation with COSI, assuming ongoing self-enrichment in R136., Comment: 13 pages, 5 figures

We present age estimates for over 8100 sub-giant branch (SGB) stars in Omega Centauri ($\omega$ Cen) to study its star formation history. Our large data set, which combines multi-wavelength HST photometry with MUSE metallicities, provides an unprecedented opportunity to measure individual stellar ages. We do this by fitting each star's photometry and metallicity with theoretical isochrones, that are embedded with an empirical [C+N+O]-[Fe/H] relation specifically for $\omega$ Cen. The bulk of the stars have ages between 13 and 10 Gyr, with the mean stellar age being 12.08$\pm$0.01 Gyrs and the median age uncertainty being 0.68 Gyrs. From these ages we construct the most complete age-metallicity relation (AMR) for $\omega$ Cen to date. We find that the mean age of stars decreases with increasing metallicity and find two distinct streams in the age-metallicity plane, hinting at different star formation pathways. We derive an intrinsic spread in the ages of 0.75$\pm$0.01 Gyr for the whole cluster, with the age spread showing a clear increase with metallicity. We verify the robustness of our age estimations by varying isochrone parameters and constraining our systematics. We find the C+N+O relation to be the most critical consideration for constraining the AMR. We also present the SGB chromosome map with age information. In the future, these stellar ages could be combined with chemical abundances to study age differences in subpopulations, and uncover the chemical evolution history of this massive nuclear star cluster., Comment: 23 pages, 11 figures

Variational models for image deblurring problems typically consist of a smooth term and a potentially non-smooth convex term. A common approach to solving these problems is using proximal gradient methods. To accelerate the convergence of these first-order iterative algorithms, strategies such as variable metric methods have been introduced in the literature. In this paper, we prove that, for image deblurring problems, the variable metric strategy can be reinterpreted as a right preconditioning method. Consequently, we explore an inexact left-preconditioned version of the same proximal gradient method. We prove the convergence of the new iteration to the minimum of a variational model where the norm of the data fidelity term depends on the preconditioner. The numerical results show that left and right preconditioning are comparable in terms of the number of iterations required to reach a prescribed tolerance, but left preconditioning needs much less CPU time, as it involves fewer evaluations of the preconditioner matrix compared to right preconditioning. The quality of the computed solutions with left and right preconditioning are comparable. Finally, we propose some non-stationary sequences of preconditioners that allow for fast and stable convergence to the solution of the variational problem with the classical $\ell^2$--norm on the fidelity term.

Theories of localised pattern formation are important to understand a broad range of natural patterns, but are less well-understood than more established mechanisms of domain-filling pattern formation. Here, we extend recent work on pattern localisation via slow spatial heterogeneity in reaction-diffusion systems to the Swift-Hohenberg equation. We use a WKB asymptotic approach to show that, in the limit of a large domain and slowly varying heterogeneity, conditions for Turing-type linear instability localise in a simple way, with the spatial variable playing the role of a parameter. For nonlinearities locally corresponding to supercritical bifurcations in the spatially homogeneous system, this analysis asymptotically predicts regions where patterned states are confined, which we confirm numerically. We resolve the inner region of this asymptotic approach, finding excellent agreement with the tails of these confined pattern regions. In the locally subcritical case, however, this theory is insufficient to fully predict such confined regions, and so we propose an approach based on numerical continuation of a local homogeneous analog system. Pattern localisation in the heterogeneous system can then be determined based on the Maxwell point of this system, with the spatial variable parameterizing this point. We compare this theory of localisation via spatial heterogeneity to localised patterns arising from homoclinic snaking, and suggest a way to distinguish between different localisation mechanisms in natural systems based on how these structures decay to the background state (i.e. how their tails decay). We also explore cases where both of these local theories of pattern formation fail to capture the interaction between spatial heterogeneity and underlying pattern-forming mechanisms, suggesting that more work needs to be done to fully disentangle exogenous and intrinsic heterogeneity., Comment: 36 pages, 17 figures

Through exposing items to users, implicit feedback recommender systems influence the logged interactions, and, ultimately, their own recommendations. This effect is called exposure bias and it can lead to issues such as filter bubbles and echo chambers. Previous research employed the multinomial logit model (MNL) with exposure information to reduce exposure bias on synthetic data. This extended abstract summarizes our previous study in which we investigated whether (i) these findings hold for human-generated choices, (ii) other discrete choice models mitigate bias better, and (iii) an item's estimated relevance can depend on the relevances of the other items that were presented with it. We collected a data set of biased and unbiased choices in a controlled online user study and measured the effects of overexposure and competition. We found that (i) the discrete choice models effectively mitigated exposure bias on human-generated choice data, (ii) there were no significant differences in robustness among the different discrete choice models, and (iii) only multivariate discrete choice models were robust to competition between items. We conclude that discrete choice models mitigate exposure bias effectively because they consider item-co-exposure. Moreover, exposing items alongside more or less popular items can bias future recommendations significantly and item exposure must be tracked for overcoming exposure bias. We consider our work vital for understanding what exposure bias it, how it forms, and how it can be mitigated., Comment: Accepted at the CONSEQUENCES '24 workshop, co-located with ACM RecSys '24

Statistical properties of LSS serve as powerful tools to constrain the cosmological properties of our Universe. Tracing the gas pressure, the tSZ effect is a biased probe of mass distribution and can be used to test the physics of feedback or cosmological models. Therefore, it is crucial to develop robust modeling of hot gas pressure for applications to tSZ surveys. Since gas collapses into bound structures, it is expected that most of the tSZ signal is within halos produced by cosmic accretion shocks. Hence, simple empirical halo models can be used to predict the tSZ power spectra. In this study, we employed the HMx halo model to compare the tSZ power spectra with those of several hydrodynamical simulations: the Horizon suite and the Magneticum simulation. We examined various contributions to the tSZ power spectrum across different redshifts, including the one- and two-halo term decomposition, the amount of bound gas, the importance of different masses and the electron pressure profiles. Our comparison of the tSZ power spectrum reveals discrepancies that increase with redshift. We find a 20% to 50% difference between the measured and predicted tSZ angular power spectrum over the multipole range $\ell=10^3-10^4$. Our analysis reveals that these differences are driven by the excess of power in the predicted two-halo term at low k and in the one-halo term at high k. At higher redshifts (z~3), simulations indicate that more power comes from outside the virial radius than from inside suggesting a limitation in the applicability of the halo model. We observe differences in the pressure profiles, despite the fair level of agreement on the tSZ power spectrum at low redshift with the default calibration of the halo model. In conclusion, our study suggests that the properties of the halo model need to be carefully controlled against real or mock data to be proven useful for cosmological purposes., Comment: 15 pages, 9 figures

We show that on a $\sigma$-finite measure preserving system $X = (X,\nu, T)$, the non-conventional ergodic averages $$ \mathbb{E}_{n \in [N]} \Lambda(n) f(T^n x) g(T^{P(n)} x)$$ converge pointwise almost everywhere for $f \in L^{p_1}(X)$, $g \in L^{p_2}(X)$, and $1/p_1 + 1/p_2 \leq 1$, where $P$ is a polynomial with integer coefficients of degree at least $2$. This had previously been established with the von Mangoldt weight $\Lambda$ replaced by the constant weight $1$ by the first and third authors with Mirek, and by the M\"obius weight $\mu$ by the fourth author. The proof is based on combining tools from both of these papers, together with several Gowers norm and polynomial averaging operator estimates on approximants to the von Mangoldt function of ''Cram\'er'' and ''Heath-Brown'' type., Comment: 37 pages

Learning uncertain dynamics models using Gaussian process~(GP) regression has been demonstrated to enable high-performance and safety-aware control strategies for challenging real-world applications. Yet, for computational tractability, most approaches for Gaussian process-based model predictive control (GP-MPC) are based on approximations of the reachable set that are either overly conservative or impede the controller's safety guarantees. To address these challenges, we propose a robust GP-MPC formulation that guarantees constraint satisfaction with high probability. For its tractable implementation, we propose a sampling-based GP-MPC approach that iteratively generates consistent dynamics samples from the GP within a sequential quadratic programming framework. We highlight the improved reachable set approximation compared to existing methods, as well as real-time feasible computation times, using two numerical examples., Comment: to be published in 63rd IEEE Conference on Decision and Control (CDC 2024)

We present the first detection of a cluster lensing signal with `Kinematic Lensing' (KL), a novel weak lensing method that combines photometry, spectroscopy, and the Tully-Fisher relation to enable shear measurements with individual source galaxies. This is the second paper in a two-part series aimed at measuring a KL signal from data. The first paper, arXiv:2209.11811, describes the inference pipeline, which jointly forward models galaxy imaging and spectroscopy, and demonstrates unbiased shear inference with simulated data. This paper presents measurements of the lensing signal from the galaxy cluster Abell 2261. We obtain spectroscopic observations of background disk galaxies in the cluster field selected from the CLASH Subaru catalog. The final sample consists of three source galaxies while the remaining are rejected due to insufficient signal-to-noise, spectroscopic failures, and inadequately sampled rotation curves. We apply the KL inference pipeline to the three sources and find the shear estimates to be in broad agreement with traditional weak lensing measurements. The typical shear measurement uncertainty for our sources is $\sigma(g_+)\approx 0.026$, which represents approximately a ten-fold improvement over the weak lensing shape noise. We identify target selection and observing strategy as the key avenues of improvement for future KL programs., Comment: v1: To be submitted to MNRAS. Comments are welcome!

Interfacial ferroelectricity, prevalent in various parallel-stacked layered materials, allows switching of out-of-plane ferroelectric order by in-plane sliding of adjacent layers. Its resilience against doping potentially enables next-generation storage and logic devices. However, studies have been limited to indirect sensing or visualization of ferroelectricity. For transition metal dichalcogenides, there is little knowledge about the influence of ferroelectric order on their intrinsic valley and excitonic properties. Here, we report direct probing of ferroelectricity in few-layer 3R-MoS$_2$ using reflectance contrast spectroscopy. Contrary to a simple electrostatic perception, layer-hybridized excitons with out-of-plane electric dipole moment remain decoupled from ferroelectric ordering, while intralayer excitons with in-plane dipole orientation are sensitive to it. Ab initio calculations identify stacking-specific interlayer hybridization leading to this asymmetric response. Exploiting this sensitivity, we demonstrate optical readout and control of multi-state polarization with hysteretic switching in a field-effect device. Time-resolved Kerr ellipticity reveals a direct correspondence between spin-valley dynamics and stacking order.

General conditions are established under which reaction-cross-diffusion systems can undergo spatiotemporal pattern-forming instabilities. Recent work has focused on designing systems theoretically and experimentally to exhibit patterns with specific features, but the case of non-diagonal diffusion matrices has yet to be analysed. Here, a framework is presented for the design of general $n$-component reaction-cross-diffusion systems that exhibit Turing and wave instabilities of a given wavelength. For a fixed set of reaction kinetics, it is shown how to choose diffusion matrices that produce each instability; conversely, for a given diffusion tensor, how to choose linearised kinetics. The theory is applied to several examples including a hyperbolic reaction-diffusion system, two different 3-component models, and a spatio-temporal version of the Ross-Macdonald model for the spread of malaria., Comment: 49 pages, 14 figures

We introduce a two-level trust-region method (TLTR) for solving unconstrained nonlinear optimization problems. Our method uses a composite iteration step, which is based on two distinct search directions. The first search direction is obtained through minimization in the full/high-resolution space, ensuring global convergence to a critical point. The second search direction is obtained through minimization in the randomly generated subspace, which, in turn, allows for convergence acceleration. The efficiency of the proposed TLTR method is demonstrated through numerical experiments in the field of machine learning

In landscape-aware algorithm selection problem, the effectiveness of feature-based predictive models strongly depends on the representativeness of training data for practical applications. In this work, we investigate the potential of randomly generated functions (RGF) for the model training, which cover a much more diverse set of optimization problem classes compared to the widely-used black-box optimization benchmarking (BBOB) suite. Correspondingly, we focus on automated algorithm configuration (AAC), that is, selecting the best suited algorithm and fine-tuning its hyperparameters based on the landscape features of problem instances. Precisely, we analyze the performance of dense neural network (NN) models in handling the multi-output mixed regression and classification tasks using different training data sets, such as RGF and many-affine BBOB (MA-BBOB) functions. Based on our results on the BBOB functions in 5d and 20d, near optimal configurations can be identified using the proposed approach, which can most of the time outperform the off-the-shelf default configuration considered by practitioners with limited knowledge about AAC. Furthermore, the predicted configurations are competitive against the single best solver in many cases. Overall, configurations with better performance can be best identified by using NN models trained on a combination of RGF and MA-BBOB functions.

Geological applications of phase-field methods for fracture are notably scarce. This work conducts a numerical examination of the applicability of standard phase-field models in reproducing jointing within sedimentary layers. We explore how the volumetric-deviatoric split alongside the AT1 and AT2 phase-field formulations have several advantages in simulating jointing, but also have intrinsic limitations that prevent a reliable quantitative analysis of rock fracture. The formulations qualitatively reproduce the process of joint saturation, along with the negative correlation between joint spacing and the height of the sedimentary layer. However, in quantitative comparison to alternative numerical methods and outcrop observations, the phase-field method overestimates joint spacings by a factor of 2 and induces unrealistic compressive fractures in the AT1 model, alongside premature shearing at layer interfaces for the AT2 model. The causes are identified to be intrinsic to the phase-field lengthscale and the unsuitable strength envelope arising from the Volumetric-Deviatoric split. Finally, our analysis elucidates on the phase-field lengthscale's distortion of the stress field around dilating fractures, causing the sedimentary layer to reach joint saturation prematurely, thereby stopping the nucleation of new fractures and leading to larger joint spacings than in natural examples. Decreasing the lengthscale results in gradual improvement but becomes prohibitively computationally expensive for very small lengthscales such that the limit of ``natural'' behavior was not reached in this study. Overall, our results motivate the development of constitutive phase-field models that are more suitable for geological applications and their benchmarking against geological observations., Comment: 24 pages

Decreasing costs and new technologies have led to an increase in the amount of data available to plant breeding programs. High-throughput phenotyping (HTP) platforms routinely generate high-dimensional datasets of secondary features that may be used to improve genomic prediction accuracy. However, integration of this data comes with challenges such as multicollinearity, parameter estimation in $p > n$ settings, and the computational complexity of many standard approaches. Several methods have emerged to analyze such data, but interpretation of model parameters often remains challenging. We propose genetic factor best linear unbiased prediction (gfBLUP), a seven-step prediction pipeline that reduces the dimensionality of the original secondary HTP data using generative factor analysis. In short, gfBLUP uses redundancy filtered and regularized genetic and residual correlation matrices to fit a maximum likelihood factor model and estimate genetic latent factor scores. These latent factors are subsequently used in multi-trait genomic prediction. Our approach performs on par or better than alternatives in extensive simulations and a real-world application, while producing easily interpretable and biologically relevant parameters. We discuss several possible extensions and highlight gfBLUP as the basis for a flexible and modular multi-trait genomic prediction framework.

Linear temporal logic (LTL) is a powerful language for task specification in reinforcement learning, as it allows describing objectives beyond the expressivity of conventional discounted return formulations. Nonetheless, recent works have shown that LTL formulas can be translated into a variable rewarding and discounting scheme, whose optimization produces a policy maximizing a lower bound on the probability of formula satisfaction. However, the synthesized reward signal remains fundamentally sparse, making exploration challenging. We aim to overcome this limitation, which can prevent current algorithms from scaling beyond low-dimensional, short-horizon problems. We show how better exploration can be achieved by further leveraging the LTL specification and casting its corresponding Limit Deterministic B\"uchi Automaton (LDBA) as a Markov reward process, thus enabling a form of high-level value estimation. By taking a Bayesian perspective over LDBA dynamics and proposing a suitable prior distribution, we show that the values estimated through this procedure can be treated as a shaping potential and mapped to informative intrinsic rewards. Empirically, we demonstrate applications of our method from tabular settings to high-dimensional continuous systems, which have so far represented a significant challenge for LTL-based reinforcement learning algorithms.

Fruit preparations are used in various forms in the food industry. For example, they are used as an ingredient in dairy products such as yogurt with added fruit. The dispersed fruit pieces can be described as soft particles with viscoelastic material behavior. The continuous phase is represented by fluids with complex flow behavior depending on the formulation. Characterization has shown that this can be described by the Herschel-Bulkley model. Since damage to fruit pieces is undesirable in industrial transport processes, the potential for damage to fruit pieces during transport of pipes in cross-sectional constrictions is analyzed. The analysis is performed numerically using the homogenized lattice Boltzmann method and validated by an experiment on industrial fruit preparations at pilot plant scale. The results show a strong dependence of the damage potential on the (local) Metzner-Reed Reynolds number.

The almost unlimited possibilities to customize the logic in an FPGA are one of the main reasons for the versatility of these devices. Partial reconfiguration exploits this capability even further by allowing to replace logic in predefined FPGA regions at runtime. This is especially relevant in heterogeneous SoCs, combining FPGA fabric with conventional processors on a single die. Tight integration and supporting frameworks like the FPGA subsystem in Linux facilitate use, for example, to dynamically load custom hardware accelerators. Although this example is one of the most common use cases for partial reconfiguration, the possible applications go far beyond. We propose to use partial reconfiguration in combination with the AXI C2C cross-chip bus to extend the resources of heterogeneous MPSoC and RFSoC devices by connecting peripheral FPGAs. With AXI C2C it is easily possible to link the programmable logic of the individual devices, but partial reconfiguration on peripheral FPGAs utilising the same channel is not officially supported. By using an AXI ICAP controller in combination with custom Linux drivers, we show that it is possible to enable the PS of the heterogeneous SoC to perform partial reconfiguration on peripheral FPGAs, and thus to seamlessly access and manage the entire multi-device system. As a result, software and FPGA firmware updates can be applied to the entire system at runtime, and peripheral FPGAs can be added and removed during operation., Comment: 8 pages, double-column, 9 figures. Paper submitted as proceeding for the 24rd IEEE Real Time Conference (2024)

Developers usually use diagrams and source code to jointly discuss and plan software architecture changes. With this poster, we present our on-going work on a novel approach that enables developers to collaboratively use software city visualization to design and plan software architecture changes., Comment: submitted to VISSOFT 2024, see https://vissoft.info/2024/program.html

Software visualization approaches for code reviews are often implemented as standalone applications, which use static code analysis. The goal is to visualize the structural changes introduced by a pull / merge request to facilitate the review process. In this way, for example, structural changes that hinder code evolution can be more easily identified, but understanding the changed program behavior is still mainly done by reading the code. For software visualization to be successful in code review, tools must be provided that go beyond an alternative representation of code changes and integrate well into the developers' daily workflow. In this paper, we report on the novel and in-progress design and implementation of a web-based approach capable of combining static and dynamic analysis data in software city visualizations. Our architectural tool design incorporates modern web technologies such as the integration into common Git hosting services. As a result, code reviewers can explore how the modified software evolves and execute its use cases, which is especially helpful for distributed software systems. In this context, developers can be directly linked from the Git hosting service's issue tracking system to the corresponding software city visualization. This approach eliminates the recurring action of manual data collection and setup. We implement our design by extending the web-based software visualization tool ExplorViz. We invite other researchers to extend our open source software and jointly research this approach. Video URL: https://youtu.be/DYxijdCEdrY, Comment: submitted to VISSOFT 2024, see https://vissoft.info/2024/program.html

Homogeneous planar superfluids exhibit a range of low-energy excitations that also appear in highly excited states like superfluid turbulence. In dilute gas Bose-Einstein condensates, the Jones- Roberts soliton family includes vortex dipoles and rarefaction pulses in the low and high velocity regimes, respectively. These excitations carry both energy and linear momentum, making their decay characteristics crucial for understanding superfluid dynamics. In this work, we develop the theory of planar soliton decay due to thermal effects, as described by the stochastic projected Gross-Pitaevskii theory of reservoir interactions. We analyze two distinct damping terms involving transfer between the condensate and the non-condensate reservoir: particle transfer that also involves energy and usually drives condensate growth, and number-conserving energy transfer. We provide analytical treatments for both the low and high velocity regimes and identify conditions under which either mechanism dominates. Our findings indicate that energy damping prevails at high phase space density. These theoretical results are supported by numerical studies covering the entire velocity range from vortex dipole to rarefaction pulse. We use interaction energy to characterize rarefaction pulses, analogous to the distance between vortices in vortex dipoles, offering an experimentally accessible test for finite temperature theory in Bose-Einstein condensates., Comment: 13 pages + appendices, 15 figures. Comments welcome!

It is shown that a quiver is left noetherian if and only if the category of quiver representations in any locally noetherian abelian category is again locally noetherian. Here, locally noetherian means that any object is the directed union of its noetherian subobjects. For a quiver to be left noetherian means that the left ideals of paths starting at any fixed vertex satisfy the ascending chain condition. The proof generalises to representations of any small category that admits a Gr\"obner enrichment., Comment: 10 pages

The projected sensitivity of the effective electron neutrino-mass measurement with the KATRIN experiment is below 0.3 eV (90 % CL) after five years of data acquisition. The sensitivity is affected by the increased rate of the background electrons from KATRIN's main spectrometer. A special shifted-analysing-plane (SAP) configuration was developed to reduce this background by a factor of two. The complex layout of electromagnetic fields in the SAP configuration requires a robust method of estimating these fields. We present in this paper a dedicated calibration measurement of the fields using conversion electrons of gaseous $^\mathrm{83m}$Kr, which enables the neutrino-mass measurements in the SAP configuration., Comment: 19 pages, 11 figures

Characterization of the redshift distribution of ensembles of galaxies is pivotal for large scale structure cosmological studies. In this work, we focus on improving the Self-Organizing Map (SOM) methodology for photometric redshift estimation (SOMPZ), specifically in anticipation of the Dark Energy Survey Year 6 (DES Y6) data. This data set, featuring deeper and fainter galaxies than DES Year 3 (DES Y3), demands adapted techniques to ensure accurate recovery of the underlying redshift distribution. We investigate three strategies for enhancing the existing SOM-based approach used in DES Y3: 1) Replacing the Y3 SOM algorithm with one tailored for redshift estimation challenges; 2) Incorporating $\textit{g}$-band flux information to refine redshift estimates (i.e. using $\textit{griz}$ fluxes as opposed to only $\textit{riz}$); 3) Augmenting redshift data for galaxies where available. These methods are applied to DES Y3 data, and results are compared to the Y3 fiducial ones. Our analysis indicates significant improvements with the first two strategies, notably reducing the overlap between redshift bins. By combining strategies 1 and 2, we have successfully managed to reduce redshift bin overlap in DES Y3 by up to 66$\%$. Conversely, the third strategy, involving the addition of redshift data for selected galaxies as an additional feature in the method, yields inferior results and is abandoned. Our findings contribute to the advancement of weak lensing redshift characterization and lay the groundwork for better redshift characterization in DES Year 6 and future stage IV surveys, like the Rubin Observatory.

Gravitational interactions were studied by measuring the influence of small external field masses on a microwave resonator. It consisted of two spherical mirrors, which acted as independent pendulumsindividually suspended by strings. Two identical field masses weremoved along the axis of the resonator symmetrically and periodically betweena near and a far position. Their gravitational interaction altered the distance between the mirrors, changing the resonance frequency, which was measured and found consistent with Newton's law of gravity. The acceleration of a single mirror caused by the two field masses at the closest position varied from $5.4 \cdot 10^{-12} m/s^2$ to $259 \cdot 10^{-12}\ m/s^2$., Comment: 19 pages, 8 figures

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

We explore the science prospects of a 14,000 deg$^2$ Kinematic Lensing (KL) survey with the Dark Energy Spectroscopic Instrument (DESI) and overlapping imaging surveys. KL infers the cosmic shear signal by jointly forward modeling the observed photometric image and velocity field of a disk galaxy. The latter can be constrained by placing multiple DESI fibers along the galaxy's major and minor axis, a concept similar to the DESI Peculiar Velocity Survey. We study multiple subset galaxy samples of the DESI Legacy Survey Data Release 9 catalog and quantify the residual shape noise, $\sigma_\epsilon$, of each sample as a function of cuts in $r$-band magnitude using mock observations. We conduct simulated likelihood analyses for these galaxy samples and find that a DESI-KL program can place highly interesting constraints on structure formation at very low redshifts, i.e. $\sigma_8(z<0.15)$. We conclude that if the $S_8$ tension consolidates as a phenomenon, a DESI-KL survey can provide unique insights into this phenomenon in the very late-time Universe. Given the different footprints of DESI and Rubin Observatory's Legacy Survey of Space and Time (LSST), lensing results from both surveys are highly complementary and can be combined into a joint lensing survey. We further note that DESI-KL benefits multiple additional science cases, e.g. studies of modified gravity models when combined with peculiar velocity surveys, and dark matter studies that are based on galaxy-galaxy lensing of dwarf galaxies.

The Single Instruction Multiple Data (SIMD) parallel paradigm is a well-established and heavily-used hardware-driven technique to increase the single-thread performance in different system domains such as database or machine learning. Depending on the hardware vendor and the specific processor generation/version, SIMD capabilities come in different flavors concerning the register size and the supported SIMD instructions. Due to this heterogeneity and the lack of standardized calling conventions, building high-performance and portable systems is a challenging task. To address this challenge, academia and industry have invested a remarkable effort into creating SIMD abstraction libraries that provide unified access to different SIMD hardware capabilities. However, those one-size-fits-all library approaches are inherently complex, which hampers maintainability and extensibility. Furthermore, they assume similar SIMD hardware designs, which may be invalidated through ARM SVE's emergence. Additionally, while existing SIMD abstraction libraries do a great job of hiding away the specifics of the underlying hardware, their lack of expressiveness impedes crucial algorithm design decisions for system developers. To overcome these limitations, we present TSLGen, a novel end-to-end framework approach for generating an SIMD abstraction library in this paper. We have implemented our TSLGen framework and used our generated Template SIMD Library (TSL) to program various system components from different domains. As we will show, the programming effort is comparable to existing libraries, and we achieve the same performance results. However, our framework is easy to maintain and to extend, which simultaneously supports disruptive changes to the interface by design and exposes valuable insights for assessing provided functionality., Comment: 20 pages

How can a scientist use a Reinforcement Learning (RL) algorithm to design experiments over a dynamical system's state space? In the case of finite and Markovian systems, an area called Active Exploration (AE) relaxes the optimization problem of experiments design into Convex RL, a generalization of RL admitting a wider notion of reward. Unfortunately, this framework is currently not scalable and the potential of AE is hindered by the vastness of experiment spaces typical of scientific discovery applications. However, these spaces are often endowed with natural geometries, e.g., permutation invariance in molecular design, that an agent could leverage to improve the statistical and computational efficiency of AE. To achieve this, we bridge AE and MDP homomorphisms, which offer a way to exploit known geometric structures via abstraction. Towards this goal, we make two fundamental contributions: we extend MDP homomorphisms formalism to Convex RL, and we present, to the best of our knowledge, the first analysis that formally captures the benefit of abstraction via homomorphisms on sample efficiency. Ultimately, we propose the Geometric Active Exploration (GAE) algorithm, which we analyse theoretically and experimentally in environments motivated by problems in scientific discovery., Comment: ICML 2024

Large language model (LLM) chatbots are susceptible to biases and hallucinations, but current evaluations of mental wellness technologies lack comprehensive case studies to evaluate their practical applications. Here, we address this gap by introducing the MHealth-EVAL framework, a new role-play based interactive evaluation method designed specifically for evaluating the appropriateness, trustworthiness, and safety of mental wellness chatbots. We also introduce Psyfy, a new chatbot leveraging LLMs to facilitate transdiagnostic Cognitive Behavioral Therapy (CBT). We demonstrate the MHealth-EVAL framework's utility through a comparative study of two versions of Psyfy against standard baseline chatbots. Our results showed that Psyfy chatbots outperformed the baseline chatbots in delivering appropriate responses, engaging users, and avoiding untrustworthy responses. However, both Psyfy and the baseline chatbots exhibited some limitations, such as providing predominantly US-centric resources. While Psyfy chatbots were able to identify most unsafe situations and avoid giving unsafe responses, they sometimes struggled to recognize subtle harmful intentions when prompted in role play scenarios. Our study demonstrates a practical application of the MHealth-EVAL framework and showcases Psyfy's utility in harnessing LLMs to enhance user engagement and provide flexible and appropriate responses aligned with an evidence-based CBT approach.

We introduce the AutoGRAMS framework for programming multi-step interactions with language models. AutoGRAMS represents AI agents as a graph, where each node can execute either a language modeling instruction or traditional code. Likewise, transitions in the graph can be governed by either language modeling decisions or traditional branch logic. AutoGRAMS supports using variables as memory and allows nodes to call other AutoGRAMS graphs as functions. We show how AutoGRAMS can be used to design highly sophisticated agents, including self-referential agents that can modify their own graph. AutoGRAMS's graph-centric approach aids interpretability, controllability, and safety during the design, development, and deployment of AI agents. We provide our framework as open source at https://github.com/autograms/autograms .

In classic Reinforcement Learning (RL), the agent maximizes an additive objective of the visited states, e.g., a value function. Unfortunately, objectives of this type cannot model many real-world applications such as experiment design, exploration, imitation learning, and risk-averse RL to name a few. This is due to the fact that additive objectives disregard interactions between states that are crucial for certain tasks. To tackle this problem, we introduce Global RL (GRL), where rewards are globally defined over trajectories instead of locally over states. Global rewards can capture negative interactions among states, e.g., in exploration, via submodularity, positive interactions, e.g., synergetic effects, via supermodularity, while mixed interactions via combinations of them. By exploiting ideas from submodular optimization, we propose a novel algorithmic scheme that converts any GRL problem to a sequence of classic RL problems and solves it efficiently with curvature-dependent approximation guarantees. We also provide hardness of approximation results and empirically demonstrate the effectiveness of our method on several GRL instances., Comment: ICML 2024

Commonsense reasoning is a difficult task for a computer, but a critical skill for an artificial intelligence (AI). It can enhance the explainability of AI models by enabling them to provide intuitive and human-like explanations for their decisions. This is necessary in many areas especially in question answering (QA), which is one of the most important tasks of natural language processing (NLP). Over time, a multitude of methods have emerged for solving commonsense reasoning problems such as knowledge-based approaches using formal logic or linguistic analysis. In this paper, we investigate the effectiveness of large language models (LLMs) on different QA tasks with a focus on their abilities in reasoning and explainability. We study three LLMs: GPT-3.5, Gemma and Llama 3. We further evaluate the LLM results by means of a questionnaire. We demonstrate the ability of LLMs to reason with commonsense as the models outperform humans on different datasets. While GPT-3.5's accuracy ranges from 56% to 93% on various QA benchmarks, Llama 3 achieved a mean accuracy of 90% on all eleven datasets. Thereby Llama 3 is outperforming humans on all datasets with an average 21% higher accuracy over ten datasets. Furthermore, we can appraise that, in the sense of explainable artificial intelligence (XAI), GPT-3.5 provides good explanations for its decisions. Our questionnaire revealed that 66% of participants rated GPT-3.5's explanations as either "good" or "excellent". Taken together, these findings enrich our understanding of current LLMs and pave the way for future investigations of reasoning and explainability., Comment: 19 pages

This paper critically evaluates the attempts to align Artificial Intelligence (AI) systems, especially Large Language Models (LLMs), with human values and intentions through Reinforcement Learning from Feedback (RLxF) methods, involving either human feedback (RLHF) or AI feedback (RLAIF). Specifically, we show the shortcomings of the broadly pursued alignment goals of honesty, harmlessness, and helpfulness. Through a multidisciplinary sociotechnical critique, we examine both the theoretical underpinnings and practical implementations of RLxF techniques, revealing significant limitations in their approach to capturing the complexities of human ethics and contributing to AI safety. We highlight tensions and contradictions inherent in the goals of RLxF. In addition, we discuss ethically-relevant issues that tend to be neglected in discussions about alignment and RLxF, among which the trade-offs between user-friendliness and deception, flexibility and interpretability, and system safety. We conclude by urging researchers and practitioners alike to critically assess the sociotechnical ramifications of RLxF, advocating for a more nuanced and reflective approach to its application in AI development., Comment: 12 pages, 1 table, to be submitted

Bandits with preference feedback present a powerful tool for optimizing unknown target functions when only pairwise comparisons are allowed instead of direct value queries. This model allows for incorporating human feedback into online inference and optimization and has been employed in systems for fine-tuning large language models. The problem is well understood in simplified settings with linear target functions or over finite small domains that limit practical interest. Taking the next step, we consider infinite domains and nonlinear (kernelized) rewards. In this setting, selecting a pair of actions is quite challenging and requires balancing exploration and exploitation at two levels: within the pair, and along the iterations of the algorithm. We propose MAXMINLCB, which emulates this trade-off as a zero-sum Stackelberg game, and chooses action pairs that are informative and yield favorable rewards. MAXMINLCB consistently outperforms existing algorithms and satisfies an anytime-valid rate-optimal regret guarantee. This is due to our novel preference-based confidence sequences for kernelized logistic estimators., Comment: In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS), 30 pages, 8 figures

The fact that neutrinos carry a non-vanishing rest mass is evidence of physics beyond the Standard Model of elementary particles. Their absolute mass bears important relevance from particle physics to cosmology. In this work, we report on the search for the effective electron antineutrino mass with the KATRIN experiment. KATRIN performs precision spectroscopy of the tritium $\beta$-decay close to the kinematic endpoint. Based on the first five neutrino-mass measurement campaigns, we derive a best-fit value of $m_\nu^{2} = {-0.14^{+0.13}_{-0.15}}~\mathrm{eV^2}$, resulting in an upper limit of $m_\nu < {0.45}~\mathrm{eV}$ at 90 % confidence level. With six times the statistics of previous data sets, amounting to 36 million electrons collected in 259 measurement days, a substantial reduction of the background level and improved systematic uncertainties, this result tightens KATRIN's previous bound by a factor of almost two., Comment: 61 pages, 20 figures, 2 tables