Search

With the increase in the number of parameters in large language models, the process of pre-training and fine-tuning increasingly demands larger volumes of GPU memory. A significant portion of this memory is typically consumed by the optimizer state. To overcome this challenge, recent approaches such as low-rank adaptation (LoRA (Hu et al., 2021)), low-rank gradient projection (GaLore (Zhao et al., 2024)), and blockwise optimization (BAdam (Luo et al., 2024)) have been proposed. However, in all these algorithms, the $\textit{effective rank of the weight updates remains low-rank}$, which can lead to a substantial loss of information from the gradient. This loss can be critically important, especially during the pre-training stage. In this paper, we introduce $\texttt{FRUGAL}$ ($\textbf{F}$ull-$\textbf{R}$ank $\textbf{U}$pdates with $\textbf{G}$r$\textbf{A}$dient sp$\textbf{L}$itting), a new memory-efficient optimization framework. $\texttt{FRUGAL}$ leverages gradient splitting to perform low-dimensional updates using advanced algorithms (such as Adam), while updates along the remaining directions are executed via state-free methods like SGD or signSGD (Bernstein et al., 2018). Our framework can be integrated with various low-rank update selection techniques, including GaLore and BAdam. We provide theoretical convergence guarantees for our framework when using SGDM for low-dimensional updates and SGD for state-free updates. Additionally, our method consistently outperforms concurrent approaches across various fixed memory budgets, achieving state-of-the-art results in pre-training and fine-tuning tasks while balancing memory efficiency and performance metrics.

We present a novel 4.5B parameter small language model that can handle multiple input and output modalities, including text, images, videos, and audio. Despite its small size, the model achieves near state-of-the-art performance on a variety of tasks, demonstrating the potential of multi-modal models to tackle complex real-world problems. Our approach leverages recent advancements in language modeling and multi-task learning to create a versatile and high-performing model that can even be deployed for edge inference. Experimental results show the model's strong performance across multiple benchmarks, paving the way for further progress in multi-modal artificial intelligence.

The measurement and description of the charge-carrier lifetime (tauc) is crucial for the wide-ranging applications of lead-halide perovskites. We present time-resolved microwave-detected photoconductivity decay (TRMCD) measurements and a detailed analysis of the possible recombination mechanisms including trap-assisted, radiative, and Auger recombination. We prove that performing injection-dependent measurement is crucial in identifying the recombination mechanism. We present temperature and injection level dependent measurements in CsPbBr_3, which is an inorganic lead-halide perovskite. In this material, we observe the dominance of charge-carrier trapping, which results in ultra-long charge-carrier lifetimes. Although charge trapping can limit the effectiveness of materials in photovoltaic applications, it also offers significant advantages for various alternative uses, including delayed and persistent photodetection, charge-trap memory, afterglow light-emitting diodes, quantum information storage, and photocatalytic activity.

Stable diffusion models have ushered in a new era of advancements in image generation, currently reigning as the state-of-the-art approach, exhibiting unparalleled performance. The process of diffusion, accompanied by denoising through iterative convolutional or transformer network steps, stands at the core of their implementation. Neural networks operating in continuous time naturally embrace the concept of diffusion, this way they could enable more accurate and energy efficient implementation. Within the confines of this paper, my focus delves into an exploration and demonstration of the potential of celllular neural networks in image generation. I will demonstrate their superiority in performance, showcasing their adeptness in producing higher quality images and achieving quicker training times in comparison to their discrete-time counterparts on the commonly cited MNIST dataset.

Binary population synthesis (BPS) is an essential tool for extracting information about massive binary evolution from gravitational-wave (GW) detections of compact object mergers. It has been successfully used to constrain the most likely permutations of evolution models among hundreds of alternatives, while initial condition models, in contrast, have not yet received the same level of attention. Here, we introduce BOSSA, a detailed initial sampling code including a set of 192 initial condition permutations for BPS that capture both "invariant" and "varying" models, the latter accounting for a possible metallicity- and star formation rate (SFR)-dependence of the initial mass function (IMF); as well as correlations between the initial primary mass, orbital period, mass ratio and eccentricity of binaries. We include 24 metallicity-specific cosmic star formation history (cSFH) models and propose two alternate models for the mass-dependent binary fraction. We build a detailed pipeline for time-evolving BPS, such that each binary has well-defined initial conditions, and we are able to distinguish the contributions from populations of different ages. We discuss the meaning of the IMF for binaries and introduce a refined initial sampling procedure for component masses. We also discuss the treatment of higher-order multiple systems when normalizing a binary sample. In particular, we argue for how a consistent interpretation of the IMF implies that this is not the distribution from which any set of component masses should be independently drawn, and show how the individual IMF of primaries and companions is expected to deviate from the full IMF., Comment: 23 pages, 16 figures, 2 tables. Submitted to MNRAS. Repository (under construction) at https://github.com/lmdesa/BOSSA

Compact stars in the Kaluza-Klein space-time are investigated, with multiple additional compactified spatial dimensions ($d$). Within the extended phenomenological model, a static, spherically symmetric solution is considered, with the equation of state provided by a zero temperature, interacting multi-dimensional Fermi gas. The maximal masses of compact stars are calculated for different model parameters. We investigated the effect of the existence of multiple extra compactified dimensions within the Kaluza-Klein compact star structure. We found that the number of extra dimensions plays a similar role, and to a similar order, as the excitation number: increasing their number, $d$, reduces the maximal mass by a few percent., Comment: 12 pages, 4 figures, submitted to Zim\'anyi School 2023

Distribution Regression (DR) on stochastic processes describes the learning task of regression on collections of time series. Path signatures, a technique prevalent in stochastic analysis, have been used to solve the DR problem. Recent works have demonstrated the ability of such solutions to leverage the information encoded in paths via signature-based features. However, current state of the art DR solutions are memory intensive and incur a high computation cost. This leads to a trade-off between path length and the number of paths considered. This computational bottleneck limits the application to small sample sizes which consequently introduces estimation uncertainty. In this paper, we present a methodology for addressing the above issues; resolving estimation uncertainties whilst also proposing a pipeline that enables us to use DR for a wide variety of learning tasks. Integral to our approach is our novel distance approximator. This allows us to seamlessly apply our methodology across different application domains, sampling rates, and stochastic process dimensions. We show that our model performs well in applications related to estimation theory, quantitative finance, and physical sciences. We demonstrate that our model generalises well, not only to unseen data within a given distribution, but also under unseen regimes (unseen classes of stochastic models)., Comment: 24 pages, 4 figures

Federated learning is increasingly popular as it enables multiple parties with limited datasets and resources to train a high-performing machine learning model collaboratively. However, similarly to other collaborative systems, federated learning is vulnerable to free-riders -- participants who do not contribute to the training but still benefit from the shared model. Free-riders not only compromise the integrity of the learning process but also slow down the convergence of the global model, resulting in increased costs for the honest participants. To address this challenge, we propose FRIDA: free-rider detection using privacy attacks, a framework that leverages inference attacks to detect free-riders. Unlike traditional methods that only capture the implicit effects of free-riding, FRIDA directly infers details of the underlying training datasets, revealing characteristics that indicate free-rider behaviour. Through extensive experiments, we demonstrate that membership and property inference attacks are effective for this purpose. Our evaluation shows that FRIDA outperforms state-of-the-art methods, especially in non-IID settings.

Non-iid data is prevalent in real-world federated learning problems. Data heterogeneity can come in different types in terms of distribution shifts. In this work, we are interested in the heterogeneity that comes from concept shifts, i.e., shifts in the prediction across clients. In particular, we consider multi-task learning, where we want the model to adapt to the task of the client. We propose a parameter-efficient framework to tackle this issue, where each client learns to mix between parameter-efficient adaptors according to its task. We use Low-Rank Adaptors (LoRAs) as the backbone and extend its concept to other types of layers. We call our framework Federated Low-Rank Adaptive Learning (FLoRAL). This framework is not an algorithm but rather a model parameterization for a multi-task learning objective, so it can work on top of any algorithm that optimizes this objective, which includes many algorithms from the literature. FLoRAL is memory-efficient, and clients are personalized with small states (e.g., one number per adaptor) as the adaptors themselves are federated. Hence, personalization is--in this sense--federated as well. Even though clients can personalize more freely by training an adaptor locally, we show that collaborative and efficient training of adaptors is possible and performs better. We also show that FLoRAL can outperform an ensemble of full models with optimal cluster assignment, which demonstrates the benefits of federated personalization and the robustness of FLoRAL to overfitting. We show promising experimental results on synthetic datasets, real-world federated multi-task problems such as MNIST, CIFAR-10, and CIFAR-100. We also provide a theoretical analysis of local SGD on a relaxed objective and discuss the effects of aggregation mismatch on convergence., Comment: 36 pages, 10 figures

Statistical data heterogeneity is a significant barrier to convergence in federated learning (FL). While prior work has advanced heterogeneous FL through better optimization objectives, these methods fall short when there is extreme data heterogeneity among collaborating participants. We hypothesize that convergence under extreme data heterogeneity is primarily hindered due to the aggregation of conflicting updates from the participants in the initial collaboration rounds. To overcome this problem, we propose a warmup phase where each participant learns a personalized mask and updates only a subnetwork of the full model. This personalized warmup allows the participants to focus initially on learning specific subnetworks tailored to the heterogeneity of their data. After the warmup phase, the participants revert to standard federated optimization, where all parameters are communicated. We empirically demonstrate that the proposed personalized warmup via subnetworks (FedPeWS) approach improves accuracy and convergence speed over standard federated optimization methods.

We perform a first study of the impact of varying two components of the initial conditions in binary population synthesis of compact binary mergers - the initial mass function, which is made metallicity- and star formation rate-dependent, and the orbital parameter (orbital period, mass ratio and eccentricity) distributions, which are assumed to be correlated - within a larger grid of initial condition models also including alternatives for the primary mass-dependent binary fraction and the metallicity-specific cosmic star formation history. We generate the initial populations with the sampling code BOSSA and evolve them with the rapid population synthesis code COMPAS. We find strong suggestions that the main role of initial conditions models is to set the relative weights of key features defined by the evolution models. In the two models we compare, black hole-black hole (BHBH) mergers are the most strongly affected, which we connect to a shift from the common envelope to the stable Roche lobe overflow formation channels with decreasing redshift. We also characterize variations in the black hole-neutron star (BHNS) and neutron star-neutron star (NSNS) final parameter distributions. We obtain the merger rate evolution for BHBH, BHNS and NSNS mergers up to $z=10$, and find a variation by a factor of $\sim50-60$ in the local BHBH and BHNS merger rates, suggesting a more important contribution from initial conditions than previously thought, and calling for a complete exploration of the initial conditions model permutations., Comment: 28 pages, 14 figures, 3 tables. Accepted for publication in MNRAS. Code/data to be made publicly available

Considering potentials defined by Bessel kernel with Bessel convolution a Kerman-Sawyer type characterization of trace inequality is given. As an application an estimate on the least eigenvalue of Schr\"odinger-Bessel operators is derived.

Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most popular assumptions of this type is $(L_0,L_1)$-smoothness (Zhang et al., 2020). In this paper, we focus on the class of (strongly) convex $(L_0,L_1)$-smooth functions and derive new convergence guarantees for several existing methods. In particular, we derive improved convergence rates for Gradient Descent with (Smoothed) Gradient Clipping and for Gradient Descent with Polyak Stepsizes. In contrast to the existing results, our rates do not rely on the standard smoothness assumption and do not suffer from the exponential dependency from the initial distance to the solution. We also extend these results to the stochastic case under the over-parameterization assumption, propose a new accelerated method for convex $(L_0,L_1)$-smooth optimization, and derive new convergence rates for Adaptive Gradient Descent (Malitsky and Mishchenko, 2020)., Comment: 51 pages, 1 figure

A theoretical framework for calculating the mass-radius curve of compact stars in the Kaluza-Klein space-time is introduced, with one additional compact spatial dimension. Static, spherically symmetric solutions are considered, with the equation of state provided by a zero temperature, interacting multidimensional Fermi gas. To model the strong force between baryons, a repulsive potential is introduced, which is linear in the particle number density. The maximal mass of compact stars is calculated for different model parameters, and with a physical parameter choice, it satisfies observational data, meaning that it is possible to model simple, realistic objects within this framework. Based on this comparison, a limiting size for the observational regime of extra dimensions in compact stars is provided, with $r_c \gtrsim 0.2$~fm., Comment: 11 pages, 8 figures

We present a long-period radio transient (GLEAM-X J0704-37) discovered to have an optical counterpart, consistent with a cool main sequence star of spectral type M3. The radio pulsations occur at the longest period yet found, 2.9 hours, and were discovered in archival low-frequency data from the Murchison Widefield Array (MWA). High time resolution observations from MeerKAT show that pulsations from the source display complex microstructure and high linear polarisation, suggesting a pulsar-like emission mechanism occurring due to strong, ordered magnetic fields. The timing residuals, measured over more than a decade, show tentative evidence of a ~6-yr modulation. The high Galactic latitude of the system and the M-dwarf star excludes the magnetar interpretation, suggesting a more likely M-dwarf / white dwarf binary scenario for this system., Comment: 11 pages, 4 figures, submitted to ApJL

We study the effects of integrability breaking on the relaxation dynamics of the (double) sine-Gordon model. Compared to previous studies, we apply an alternative viewpoint motivated by open-system physics by separating the phase field into homogeneous and inhomogeneous parts, describing a quantum pendulum (subsystem) and an interacting phononic bath (environment). To study the relaxation dynamics in the model, we perform quantum quenches using the mini-superspace-based truncated Hamiltonian approach developed recently and simulate the real-time evolution of various entanglement measures and the energy transfer between the subsystem and its environment. Our findings demonstrate that in the presence of integrability-breaking perturbations, the relaxation dynamics is substantially faster, signalled by the increase of entanglement and energy transfer between the quantum pendulum and the phonon bath., Comment: 9 pages, 28 figures

CRISPR (clustered regularly interspaced short palindromic repeat)- based diagnostics are at the forefront of rapid detection platforms of infectious diseases. The integration of reverse transcription-loop-mediated isothermal amplification (RT-LAMP) with CRISPR-Cas protein systems has led to the creation of advanced one-pot assays. The sensitivity of these assays has been bolstered by the utilization of a thermophilic Cas12 protein, BrCas12b, and its engineered variant, which exhibits enhanced thermal stability and allows for broader operation temperatures of the assay. Here, we perform all-atom molecular dynamics (MD) simulations on wild-type and mutant BrCas12b to reveal the mechanism of stabilization conferred by the mutation. High-temperature simulations reveal a small structural change along with greater flexibility in the PAM-interacting domain of the mutant BrCas12b, with marginal structural and flexibility changes in the other mutated domains. Comparative essential dynamics analysis between the wild-type and mutant BrCas12b at both ambient and elevated temperatures provides insights into the stabilizing effects of the mutations. Our findings not only offer a comprehensive insight into the dynamic alterations induced by mutations but reveal important motions in BrCas12b, important for the rational design of diagnostic and therapeutic platforms of Cas12 proteins.

std::string view is a reference-like data structure in the C++ Standard Template Library (STL) that enables fast and cheap processing of read-only strings. Due to its wide applicability and performance enhancing power, std::string view has been very popular since its introduction in the C++17 standard. However, its careless use can lead to serious memory management bugs. As the lifetime of a std::string view is not tied to the lifetime of the referenced string in any way, it is the user's responsibility to ensure that the view is only used while the viewed string is live and its buffer is not reallocated. This paper describes a static analysis tool that finds programming errors caused by the incorrect use of std::string view. Our work included modeling std::string view operations in the analysis, defining steps to detect lifetime errors, constructing user-friendly diagnostic messages, and performing an evaluation of the checker.

Static analysis is a method of analyzing source code without executing it. It is widely used to find bugs and code smells in industrial software. Besides other methods, the most important techniques are those based on the abstract syntax tree and those performing symbolic execution. Both of these methods found their role in modern software development as they have different advantages and limitations. In this tutorial, we present two problems from the C++ programming language: the elimination of redundant pointers, and the reporting of dangling pointers originating from incorrect use of the std::string class. These two issues have different theoretical backgrounds and finding them requires different implementation techniques. We will provide a step-by-step guide to implement the checkers (software to identify the aforementioned problems) - one based on the abstract syntax analysis method, the other exploring the possibilities of symbolic execution. The methods are explained in great detail and supported by code examples. The intended audience for this tutorial are both architects of static analysis tools and developers who want to understand the advantages and constraints of the different methods.

Results are presented for the measurement of large-scale anisotropies in the arrival directions of ultra-high-energy cosmic rays detected at the Pierre Auger Observatory during 19 years of operation, prior to AugerPrime, the upgrade of the Observatory. The 3D dipole amplitude and direction are reconstructed above $4\,$EeV in four energy bins. Besides the established dipolar anisotropy in right ascension above $8\,$EeV, the Fourier amplitude of the $8$ to $16\,$EeV energy bin is now also above the $5\sigma$ discovery level. No time variation of the dipole moment above $8\,$EeV is found, setting an upper limit to the rate of change of such variations of $0.3\%$ per year at the $95\%$ confidence level. Additionally, the results for the angular power spectrum are shown, demonstrating no other statistically significant multipoles. The results for the equatorial dipole component down to $0.03\,$EeV are presented, using for the first time a data set obtained with a trigger that has been optimized for lower energies. Finally, model predictions are discussed and compared with observations, based on two source emission scenarios obtained in the combined fit of spectrum and composition above $0.6\,$EeV., Comment: Minor modifications, accepted for publication in Astrophysical Journal

CodeChecker is an open source project that integrates different static analysis tools such as the Clang Static Analyzer and Clang-Tidy into the build systems, continuous integration loops, and development workflows of C++ programmers. It has a powerful issue management system to make it easier to evaluate the reports of the static analysis tools. This document was handed out as supportive material for a code analysis lecture at the 2018 3COWS conference in Kosice, Slovakia.

Static analysis is the analysis of a program without executing it, usually carried out by an automated tool. Symbolic execution is a popular static analysis technique used both in program verification and in bug detection software. It works by interpreting the code, introducing a symbol for each value unknown at compile time (e.g. user-given inputs), and carrying out calculations symbolically. The analysis engine strives to explore multiple execution paths simultaneously, although checking all paths is an intractable problem, due to the vast number of possibilities. We focus on an error finding framework called the Clang Static Analyzer, and the infrastructure built around it named CodeChecker. The emphasis is on achieving end-to-end scalability. This includes the run time and memory consumption of the analysis, bug presentation to the users, automatic false positive suppression, incremental analysis, pattern discovery in the results, and usage in continuous integration loops. We also outline future directions and open problems concerning these tools. While a rich literature exists on program verification software, error finding tools normally need to settle for survey papers on individual techniques. In this paper, we not only discuss individual methods, but also how these decisions interact and reinforce each other, creating a system that is greater than the sum of its parts. Although the Clang Static Analyzer can only handle C-family languages, the techniques introduced in this paper are mostly language-independent and applicable to other similar static analysis tools.

The neutron-rich unbound fluorine isotope $^{30}$F$_{21}$ has been observed for the first time by measuring its neutron decay at the SAMURAI spectrometer (RIBF, RIKEN) in the quasi-free proton knockout reaction of $^{31}$Ne nuclei at 235 MeV/nucleon. The mass and thus one-neutron-separation energy of $^{30}$F has been determined to be $S_n = -472\pm 58 \mathrm{(stat.)} \pm 33 \mathrm{(sys.)}$ keV from the measurement of its invariant-mass spectrum. The absence of a sharp drop in $S_n$($^{30}$F) shows that the ``magic'' $N=20$ shell gap is not restored close to $^{28}$O, which is in agreement with our shell-model calculations that predict a near degeneracy between the neutron $d$ and $fp$ orbitals, with the $1p_{3/2}$ and $1p_{1/2}$ orbitals becoming more bound than the $0f_{7/2}$ one. This degeneracy and reordering of orbitals has two potential consequences: $^{28}$O behaves like a strongly superfluid nucleus with neutron pairs scattering across shells, and both $^{29,31}$F appear to be good two-neutron halo-nucleus candidates., Comment: 9 pages, 2 figures, accepted for publication in Physical Review Letters

For a subgroup $H$ of a finite group $G$, the Frobenius graph $\Gamma(G, H)$ records the constituents of the restrictions to $H$ of the irreducible characters of $G$. We investigate when this graph has diameter 3., Comment: 27 pages, 7 tables, 3 figures

Ultra-high-energy cosmic rays are known to be mainly of extragalactic origin, and their propagation is limited by energy losses, so their arrival directions are expected to correlate with the large-scale structure of the local Universe. In this work, we investigate the possible presence of intermediate-scale excesses in the flux of the most energetic cosmic rays from the direction of the supergalactic plane region using events with energies above 20 EeV recorded with the surface detector array of the Pierre Auger Observatory up to 31 December 2022, with a total exposure of 135,000 km^2 sr yr. The strongest indication for an excess that we find, with a post-trial significance of 3.1{\sigma}, is in the Centaurus region, as in our previous reports, and it extends down to lower energies than previously studied. We do not find any strong hints of excesses from any other region of the supergalactic plane at the same angular scale. In particular, our results do not confirm the reports by the Telescope Array collaboration of excesses from two regions in the Northern Hemisphere at the edge of the field of view of the Pierre Auger Observatory. With a comparable exposure, our results in those regions are in good agreement with the expectations from an isotropic distribution., Comment: submitted to ApJ

Semiconductor-superconductor hybrid nanocircuits are of high interest due to their potential applications in quantum computing. Semiconductors with a strong spin-orbit coupling and large $g$-factor are particularly attractive since they are the basic building blocks of novel qubit architectures. However, for the engineering of these complex circuits, the building blocks must be characterized in detail. We have investigated a Josephson junction where the weak link is a two-dimensional electron gas (2DEG) hosted in an InAs/InGaAs heterostructure grown on a GaAs substrate. We employed the in-situ epitaxially grown Al layer as superconducting contacts to form an rf SQUID, and also to create a microwave resonator for sensing the Josephson inductance. We determined the gate-dependent current-phase relation, and observed supercurrent interference in out-of-plane magnetic fields. With the application of an in-plane magnetic field, we induced asymmetry in the interference pattern, which was found to be anisotropic in the device plane.

In response to the negative environmental, social, and economic externalities of the dominant food model, a consensus is emerging on the need to develop alternative food systems capable of facilitating a transition to a more sustainable model. Among these alternatives, short food supply chains (SFSCs) are identified as potential crucibles of alternativity. Defined as distribution channels with a maximum of one intermediary between producer and consumer, SFSCs can contribute to the reterritorialization of food flows and the creation of relational and geographical proximities among food chain actors. This study examines how logistical engagement translates into the choice of SFSC points of sale by consumers and farmers. It evaluates this engagement through the travel times farmers and consumers are willing to endure to sell or buy at an SFSC point of sale, and the prices they are willing to accept or pay for SFSC products. A discrete choice experiment conducted in 2022 with 154 farm managers and 1022 consumers analyzed preferences regarding types of points of sale, sales prices, travel times, and relational proximity. The results show that consumers engaged in SFSCs favour alternative points of sale such as farms and associations, even for longer travel times. Farmers also value these points of sale, with logistical engagement more pronounced among those already marketing through SFSCs. Supermarkets appear less attractive to alternative profiles but serve as entry points for new participants in SFSCs. Relational proximity between farmers and consumers is important, with a preference for direct exchanges during sales. This study highlights the importance of actor engagement in SFSCs and offers perspectives for developing alternative logistics by valuing relational proximity and using conventional points of sale as entry points for new actors in the SFSCs., Comment: in French language

We present a new approach based on the Personalized Federated Learning algorithm MeritFed that can be applied to Natural Language Tasks with heterogeneous data. We evaluate it on the Low-Resource Machine Translation task, using the dataset from the Large-Scale Multilingual Machine Translation Shared Task (Small Track #2) and the subset of Sami languages from the multilingual benchmark for Finno-Ugric languages. In addition to its effectiveness, MeritFed is also highly interpretable, as it can be applied to track the impact of each language used for training. Our analysis reveals that target dataset size affects weight distribution across auxiliary languages, that unrelated languages do not interfere with the training, and auxiliary optimizer parameters have minimal impact. Our approach is easy to apply with a few lines of code, and we provide scripts for reproducing the experiments at https://github.com/VityaVitalich/MeritFed, Comment: 18 pages, 7 figures

Process mining is concerned with deriving formal models capable of reproducing the behaviour of a given organisational process by analysing observed executions collected in an event log. The elements of an event log are finite sequences (i.e., traces or words) of actions. Many effective algorithms have been introduced which issue a control flow model (commonly in Petri net form) aimed at reproducing, as precisely as possible, the language of the considered event log. However, given that identical executions can be observed several times, traces of an event log are associated with a frequency and, hence, an event log inherently yields also a stochastic language. By exploiting the trace frequencies contained in the event log, the stochastic extension of process mining, therefore, consists in deriving stochastic (Petri nets) models capable of reproducing the likelihood of the observed executions. In this paper, we introduce a novel stochastic process mining approach. Starting from a "standard" Petri net model mined through classical mining algorithms, we employ optimization to identify optimal weights for the transitions of the mined net so that the stochastic language issued by the stochastic interpretation of the mined net closely resembles that of the event log. The optimization is either based on the maximum likelihood principle or on the earth moving distance. Experiments on some popular real system logs show an improved accuracy w.r.t. to alternative approaches.

The Pierre Auger Observatory is the most sensitive instrument to detect photons with energies above $10^{17}$ eV. It measures extensive air showers generated by ultra high energy cosmic rays using a hybrid technique that exploits the combination of a fluorescence detector with a ground array of particle detectors. The signatures of a photon-induced air shower are a larger atmospheric depth of the shower maximum ($X_{max}$) and a steeper lateral distribution function, along with a lower number of muons with respect to the bulk of hadron-induced cascades. In this work, a new analysis technique in the energy interval between 1 and 30 EeV (1 EeV = $10^{18}$ eV) has been developed by combining the fluorescence detector-based measurement of $X_{max}$ with the specific features of the surface detector signal through a parameter related to the air shower muon content, derived from the universality of the air shower development. No evidence of a statistically significant signal due to photon primaries was found using data collected in about 12 years of operation. Thus, upper bounds to the integral photon flux have been set using a detailed calculation of the detector exposure, in combination with a data-driven background estimation. The derived 95% confidence level upper limits are 0.0403, 0.01113, 0.0035, 0.0023, and 0.0021 km$^{-2}$ sr$^{-1}$ yr$^{-1}$ above 1, 2, 3, 5, and 10 EeV, respectively, leading to the most stringent upper limits on the photon flux in the EeV range. Compared with past results, the upper limits were improved by about 40% for the lowest energy threshold and by a factor 3 above 3 EeV, where no candidates were found and the expected background is negligible. The presented limits can be used to probe the assumptions on chemical composition of ultra-high energy cosmic rays and allow for the constraint of the mass and lifetime phase space of super-heavy dark matter particles., Comment: 19 pages, 22 figures

This work tackles the challenges of data heterogeneity and communication limitations in decentralized federated learning. We focus on creating a collaboration graph that guides each client in selecting suitable collaborators for training personalized models that leverage their local data effectively. Our approach addresses these issues through a novel, communication-efficient strategy that enhances resource efficiency. Unlike traditional methods, our formulation identifies collaborators at a granular level by considering combinatorial relations of clients, enhancing personalization while minimizing communication overhead. We achieve this through a bi-level optimization framework that employs a constrained greedy algorithm, resulting in a resource-efficient collaboration graph for personalized learning. Extensive evaluation against various baselines across diverse datasets demonstrates the superiority of our method, named DPFL. DPFL consistently outperforms other approaches, showcasing its effectiveness in handling real-world data heterogeneity, minimizing communication overhead, enhancing resource efficiency, and building personalized models in decentralized federated learning scenarios.

Long-distance quantum communication using quantum repeaters is an enabling technology for secure communication, distributed quantum computing and quantum-enhanced sensing and metrology. As a building block of quantum repeaters, spin-photon entanglement has been demonstrated with both atomic and solid-state qubits. However, previously demonstrated qubits with long spin coherence do not directly emit photons into the low-loss telecom band that is needed for long-distance communication. Here, we demonstrate spin-photon entanglement using a single Er$^{3+}$ ion in a solid-state crystal, integrated into a silicon nanophotonic circuit. Direct emission into the telecom band enables an entanglement rate of 1.48 Hz over 15.6 km of optical fiber, with a fidelity of 73(3)$\%$. This opens the door to large-scale quantum networks based on scalable nanophotonic devices and many spectrally multiplexed Er$^{3+}$ ions., Comment: Added references

We report an investigation of the mass composition of cosmic rays with energies from 3 to 100 EeV (1 EeV=$10^{18}$ eV) using the distributions of the depth of shower maximum $X_\mathrm{max}$. The analysis relies on ${\sim}50,000$ events recorded by the Surface Detector of the Pierre Auger Observatory and a deep-learning-based reconstruction algorithm. Above energies of 5 EeV, the data set offers a 10-fold increase in statistics with respect to fluorescence measurements at the Observatory. After cross-calibration using the Fluorescence Detector, this enables the first measurement of the evolution of the mean and the standard deviation of the $X_\mathrm{max}$ distributions up to 100 EeV. Our findings are threefold: (1.) The evolution of the mean logarithmic mass towards a heavier composition with increasing energy can be confirmed and is extended to 100 EeV. (2.) The evolution of the fluctuations of $X_\mathrm{max}$ towards a heavier and purer composition with increasing energy can be confirmed with high statistics. We report a rather heavy composition and small fluctuations in $X_\mathrm{max}$ at the highest energies. (3.) We find indications for a characteristic structure beyond a constant change in the mean logarithmic mass, featuring three breaks that are observed in proximity to the ankle, instep, and suppression features in the energy spectrum., Comment: submitted to Phys. Rev. D, 28 pages, 18 figures, 5 tables

We present measurements of the atmospheric depth of the shower maximum $X_\mathrm{max}$, inferred for the first time on an event-by-event level using the Surface Detector of the Pierre Auger Observatory. Using deep learning, we were able to extend measurements of the $X_\mathrm{max}$ distributions up to energies of 100 EeV ($10^{20}$ eV), not yet revealed by current measurements, providing new insights into the mass composition of cosmic rays at extreme energies. Gaining a 10-fold increase in statistics compared to the Fluorescence Detector data, we find evidence that the rate of change of the average $X_\mathrm{max}$ with the logarithm of energy features three breaks at $6.5\pm0.6~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, $11\pm 2~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, and $31\pm5~(\mathrm{stat})\pm3~(\mathrm{sys})$ EeV, in the vicinity to the three prominent features (ankle, instep, suppression) of the cosmic-ray flux. The energy evolution of the mean and standard deviation of the measured $X_\mathrm{max}$ distributions indicates that the mass composition becomes increasingly heavier and purer, thus being incompatible with a large fraction of light nuclei between 50 EeV and 100 EeV., Comment: submitted to Phys. Rev. Lett., 10 pages, 3 figures, 1 table

Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the stochastic gradients is heavy-tailed for the later ones. Gradient clipping provably helps to achieve good high-probability convergence for such noises. However, despite the similarity between AdaGrad/Adam and Clip-SGD, the high-probability convergence of AdaGrad/Adam has not been studied in this case. In this work, we prove that AdaGrad (and its delayed version) can have provably bad high-probability convergence if the noise is heavy-tailed. To fix this issue, we propose a new version of AdaGrad called Clip-RAdaGradD (Clipped Reweighted AdaGrad with Delay) and prove its high-probability convergence bounds with polylogarithmic dependence on the confidence level for smooth convex/non-convex stochastic optimization with heavy-tailed noise. Our empirical evaluations, including NLP model fine-tuning, highlight the superiority of clipped versions of AdaGrad/Adam in handling the heavy-tailed noise., Comment: 37 pages, 8 figures

We propose MoE-F -- a formalised mechanism for combining $N$ pre-trained expert Large Language Models (LLMs) in online time-series prediction tasks by adaptively forecasting the best weighting of LLM predictions at every time step. Our mechanism leverages the conditional information in each expert's running performance to forecast the best combination of LLMs for predicting the time series in its next step. Diverging from static (learned) Mixture of Experts (MoE) methods, MoE-F employs time-adaptive stochastic filtering techniques to combine experts. By framing the expert selection problem as a finite state-space, continuous-time Hidden Markov model (HMM), we can leverage the Wohman-Shiryaev filter. Our approach first constructs $N$ parallel filters corresponding to each of the $N$ individual LLMs. Each filter proposes its best combination of LLMs, given the information that they have access to. Subsequently, the $N$ filter outputs are aggregated to optimize a lower bound for the loss of the aggregated LLMs, which can be optimized in closed-form, thus generating our ensemble predictor. Our contributions here are: (I) the MoE-F algorithm -- deployable as a plug-and-play filtering harness, (II) theoretical optimality guarantees of the proposed filtering-based gating algorithm, and (III) empirical evaluation and ablative results using state of the art foundational and MoE LLMs on a real-world Financial Market Movement task where MoE-F attains a remarkable 17% absolute and 48.5% relative F1 measure improvement over the next best performing individual LLM expert., Comment: 29 pages, 5 Appendix sections

Highly excited Rydberg states and their interactions play an important role in quantum computing and simulation. These properties can be predicted accurately for alkali atoms with simple Rydberg level structures. However, an extension of these methods to more complex atoms such as alkaline-earth atoms has not been demonstrated or experimentally validated. Here, we present multichannel quantum defect (MQDT) models for highly excited $^{174}$Yb and $^{171}$Yb Rydberg states with $L \leq 2$. The models are developed using a combination of existing literature data and new, high-precision laser and microwave spectroscopy in an atomic beam, and validated by detailed comparison with experimentally measured Stark shifts and magnetic moments. We then use these models to compute interaction potentials between two Yb atoms, and find excellent agreement with direct measurements in an optical tweezer array. From the computed interaction potential, we identify an anomalous F\"orster resonance that likely degraded the fidelity of previous entangling gates in $^{171}$Yb using $F=3/2$ Rydberg states. We then identify a more suitable $F=1/2$ state, and achieve a state-of-the-art controlled-Z gate fidelity of $F=0.994(1)$, with the remaining error fully explained by known sources. This work establishes a solid foundation for the continued development of quantum computing, simulation and entanglement-enhanced metrology with Yb neutral atom arrays., Comment: 85 pages, 23 figures, 43 tables

Federated learning (FL) has emerged as a pivotal approach in machine learning, enabling multiple participants to collaboratively train a global model without sharing raw data. While FL finds applications in various domains such as healthcare and finance, it is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. To overcome this challenge, principled mechanisms are required to evaluate the contributions made by individual participants in the FL setting. Existing solutions for contribution assessment rely on general accuracy evaluation, often failing to capture nuanced dynamics and class-specific influences. This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL. Our approach uses Shapley values from cooperative game theory to provide a granular understanding of class-specific influences. Based on ShapFed, we introduce a weighted aggregation method called ShapFed-WA, which outperforms conventional federated averaging, especially in class-imbalanced scenarios. Personalizing participant updates based on their contributions further enhances collaborative fairness by delivering differentiated models commensurate with the participant contributions. Experiments on CIFAR-10, Chest X-Ray, and Fed-ISIC2019 datasets demonstrate the effectiveness of our approach in improving utility, efficiency, and fairness in FL systems. The code can be found at https://github.com/tnurbek/shapfed., Comment: Accepted by IJCAI 2024

One of the inherent challenges in deploying transformers on time series is that \emph{reality only happens once}; namely, one typically only has access to a single trajectory of the data-generating process comprised of non-i.i.d. observations. We derive non-asymptotic statistical guarantees in this setting through bounds on the \textit{generalization} of a transformer network at a future-time $t$, given that it has been trained using $N\le t$ observations from a single perturbed trajectory of a Markov process. Under the assumption that the Markov process satisfies a log-Sobolev inequality, we obtain a generalization bound which effectively converges at the rate of ${O}(1/\sqrt{N})$. Our bound depends explicitly on the activation function ($\operatorname{Swish}$, $\operatorname{GeLU}$, or $\tanh$ are considered), the number of self-attention heads, depth, width, and norm-bounds defining the transformer architecture. Our bound consists of three components: (I) The first quantifies the gap between the stationary distribution of the data-generating Markov process and its distribution at time $t$, this term converges exponentially to $0$. (II) The next term encodes the complexity of the transformer model and, given enough time, eventually converges to $0$ at the rate ${O}(\log(N)^r/\sqrt{N})$ for any $r>0$. (III) The third term guarantees that the bound holds with probability at least $1$-$\delta$, and converges at a rate of ${O}(\sqrt{\log(1/\delta)}/\sqrt{N})$., Comment: 11 pages (+30 appendix), 3 figures, 6 tables

In this paper, we study a dynamic pickup and delivery problem with docking constraints. There is a homogeneous fleet of vehicles to serve pickup-and-delivery requests at given locations. The vehicles can be loaded up to their capacity, while unloading has to follow the last-in-first-out (LIFO) rule. The locations have a limited number of docking ports for loading and unloading, which may force the vehicles to wait. The problem is dynamic since the transportation requests arrive real-time, over the day. Accordingly, the routes of the vehicles are to be determined dynamically. The goal is to satisfy all the requests such that a combination of tardiness penalties and traveling costs is minimized. We propose a cost function approximation based solution method. In each decision epoch, we solve the respective optimization problem with a perturbed objective function to ensure the solutions remain adaptable to accommodate new requests. We penalize waiting times and idle vehicles. We propose a variable neighborhood search based method for solving the optimization problems, and we apply two existing local search operators, and we also introduce a new one. We evaluate our method using a widely adopted benchmark dataset, and the results demonstrate that our approach significantly surpasses the current state-of-the-art methods.

In this paper, we develop two families of sequential monitoring procedure to (timely) detect changes in a GARCH(1,1) model. Whilst our methodologies can be applied for the general analysis of changepoints in GARCH(1,1) sequences, they are in particular designed to detect changes from stationarity to explosivity or vice versa, thus allowing to check for volatility bubbles. Our statistics can be applied irrespective of whether the historical sample is stationary or not, and indeed without prior knowledge of the regime of the observations before and after the break. In particular, we construct our detectors as the CUSUM process of the quasi-Fisher scores of the log likelihood function. In order to ensure timely detection, we then construct our boundary function (exceeding which would indicate a break) by including a weighting sequence which is designed to shorten the detection delay in the presence of a changepoint. We consider two types of weights: a lighter set of weights, which ensures timely detection in the presence of changes occurring early, but not too early after the end of the historical sample; and a heavier set of weights, called Renyi weights which is designed to ensure timely detection in the presence of changepoints occurring very early in the monitoring horizon. In both cases, we derive the limiting distribution of the detection delays, indicating the expected delay for each set of weights. Our theoretical results are validated via a comprehensive set of simulations, and an empirical application to daily returns of individual stocks.

We compute the renormalized Dirac spectral density in $N_f = 2+1$ QCD at physical quark masses, temperature $T = 230$ MeV and system size $L_s = 3.4$ fm. To that end, we perform a point-wise continuum limit of the staggered density in lattice QCD with staggered quarks. We find, for the first time, that a clear infrared structure (IR peak) emerges in the density of Dirac operator describing dynamical quarks. We also provide numerical evidence that a component of this peak, which becomes dominant in the thermodynamic limit, is due to a non-trivial accumulation of near-zero modes. Features of this structure are consistent with those previously attributed to the recently-proposed IR phase of thermal QCD. Our results (i) provide the only complete first-principles evidence that these IR features exist and are physical; (ii) improve the upper bound for IR-phase transition temperature $T_{\mathrm{IR}}$ so that the new window is $200 < T_{\mathrm{IR}} < 230\,$MeV; (iii) are consistent with non-restoration of anomalous U$_{\mathrm A}$(1) symmetry (chiral limit) below $T = 230$ MeV., Comment: 7 pages, 6 figures; v2: 8 pages, 7 figures, added results on the topological nature of the peak, other results and conclusions unchanged, version to appear in PRD

Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other RL algorithms, its practical application is often hindered by sensitivity to hyper-parameters, particularly the step-size. In this paper, we introduce the integration of the Polyak step-size in RL, which automatically adjusts the step-size without prior knowledge. To adapt this method to RL settings, we address several issues, including unknown f* in the Polyak step-size. Additionally, we showcase the performance of the Polyak step-size in RL through experiments, demonstrating faster convergence and the attainment of more stable policies.

The flux of ultra-high energy cosmic rays reaching Earth above the ankle energy (5 EeV) can be described as a mixture of nuclei injected by extragalactic sources with very hard spectra and a low rigidity cutoff. Extragalactic magnetic fields existing between the Earth and the closest sources can affect the observed CR spectrum by reducing the flux of low-rigidity particles reaching Earth. We perform a combined fit of the spectrum and distributions of depth of shower maximum measured with the Pierre Auger Observatory including the effect of this magnetic horizon in the propagation of UHECRs in the intergalactic space. We find that, within a specific range of the various experimental and phenomenological systematics, the magnetic horizon effect can be relevant for turbulent magnetic field strengths in the local neighbourhood of order $B_{\rm rms}\simeq (50-100)\,{\rm nG}\,(20\rm{Mpc}/{d_{\rm s})( 100\,\rm{kpc}/L_{\rm coh}})^{1/2}$, with $d_{\rm s}$ the typical intersource separation and $L_{\rm coh}$ the magnetic field coherence length. When this is the case, the inferred slope of the source spectrum becomes softer and can be closer to the expectations of diffusive shock acceleration, i.e., $\propto E^{-2}$. An additional cosmic-ray population with higher source density and softer spectra, presumably also extragalactic and dominating the cosmic-ray flux at EeV energies, is also required to reproduce the overall spectrum and composition results for all energies down to 0.6~EeV., Comment: Matches published version

The smart grid domain requires bolstering the capabilities of existing energy management systems; Federated Learning (FL) aligns with this goal as it demonstrates a remarkable ability to train models on heterogeneous datasets while maintaining data privacy, making it suitable for smart grid applications, which often involve disparate data distributions and interdependencies among features that hinder the suitability of linear models. This paper introduces a framework that combines FL with a Trust Region Policy Optimization (FL TRPO) aiming to reduce energy-associated emissions and costs. Our approach reveals latent interconnections and employs personalized encoding methods to capture unique insights, understanding the relationships between features and optimal strategies, allowing our model to generalize to previously unseen data. Experimental results validate the robustness of our approach, affirming its proficiency in effectively learning policy models for smart grid challenges., Comment: ICLR 2024 Workshop: Tackling Climate Change with Machine Learning

We formulate an XAI-based model improvement approach for Graph Neural Networks (GNNs) for node classification, called Explanation Enhanced Graph Learning (EEGL). The goal is to improve predictive performance of GNN using explanations. EEGL is an iterative self-improving algorithm, which starts with a learned "vanilla" GNN, and repeatedly uses frequent subgraph mining to find relevant patterns in explanation subgraphs. These patterns are then filtered further to obtain application-dependent features corresponding to the presence of certain subgraphs in the node neighborhoods. Giving an application-dependent algorithm for such a subgraph-based extension of the Weisfeiler-Leman (1-WL) algorithm has previously been posed as an open problem. We present experimental evidence, with synthetic and real-world data, which show that EEGL outperforms related approaches in predictive performance and that it has a node-distinguishing power beyond that of vanilla GNNs. We also analyze EEGL's training dynamics.