Search

Federated learning is increasingly popular as it enables multiple parties with limited datasets and resources to train a high-performing machine learning model collaboratively. However, similarly to other collaborative systems, federated learning is vulnerable to free-riders -- participants who do not contribute to the training but still benefit from the shared model. Free-riders not only compromise the integrity of the learning process but also slow down the convergence of the global model, resulting in increased costs for the honest participants. To address this challenge, we propose FRIDA: free-rider detection using privacy attacks, a framework that leverages inference attacks to detect free-riders. Unlike traditional methods that only capture the implicit effects of free-riding, FRIDA directly infers details of the underlying training datasets, revealing characteristics that indicate free-rider behaviour. Through extensive experiments, we demonstrate that membership and property inference attacks are effective for this purpose. Our evaluation shows that FRIDA outperforms state-of-the-art methods, especially in non-IID settings.

Non-iid data is prevalent in real-world federated learning problems. Data heterogeneity can come in different types in terms of distribution shifts. In this work, we are interested in the heterogeneity that comes from concept shifts, i.e., shifts in the prediction across clients. In particular, we consider multi-task learning, where we want the model to adapt to the task of the client. We propose a parameter-efficient framework to tackle this issue, where each client learns to mix between parameter-efficient adaptors according to its task. We use Low-Rank Adaptors (LoRAs) as the backbone and extend its concept to other types of layers. We call our framework Federated Low-Rank Adaptive Learning (FLoRAL). This framework is not an algorithm but rather a model parameterization for a multi-task learning objective, so it can work on top of any algorithm that optimizes this objective, which includes many algorithms from the literature. FLoRAL is memory-efficient, and clients are personalized with small states (e.g., one number per adaptor) as the adaptors themselves are federated. Hence, personalization is--in this sense--federated as well. Even though clients can personalize more freely by training an adaptor locally, we show that collaborative and efficient training of adaptors is possible and performs better. We also show that FLoRAL can outperform an ensemble of full models with optimal cluster assignment, which demonstrates the benefits of federated personalization and the robustness of FLoRAL to overfitting. We show promising experimental results on synthetic datasets, real-world federated multi-task problems such as MNIST, CIFAR-10, and CIFAR-100. We also provide a theoretical analysis of local SGD on a relaxed objective and discuss the effects of aggregation mismatch on convergence., Comment: 36 pages, 10 figures

Statistical data heterogeneity is a significant barrier to convergence in federated learning (FL). While prior work has advanced heterogeneous FL through better optimization objectives, these methods fall short when there is extreme data heterogeneity among collaborating participants. We hypothesize that convergence under extreme data heterogeneity is primarily hindered due to the aggregation of conflicting updates from the participants in the initial collaboration rounds. To overcome this problem, we propose a warmup phase where each participant learns a personalized mask and updates only a subnetwork of the full model. This personalized warmup allows the participants to focus initially on learning specific subnetworks tailored to the heterogeneity of their data. After the warmup phase, the participants revert to standard federated optimization, where all parameters are communicated. We empirically demonstrate that the proposed personalized warmup via subnetworks (FedPeWS) approach improves accuracy and convergence speed over standard federated optimization methods.

We perform a first study of the impact of varying two components of the initial conditions in binary population synthesis of compact binary mergers - the initial mass function, which is made metallicity- and star formation rate-dependent, and the orbital parameter (orbital period, mass ratio and eccentricity) distributions, which are assumed to be correlated - within a larger grid of initial condition models also including alternatives for the primary mass-dependent binary fraction and the metallicity-specific cosmic star formation history. We generate the initial populations with the sampling code BOSSA and evolve them with the rapid population synthesis code COMPAS. We find strong suggestions that the main role of initial conditions models is to set the relative weights of key features defined by the evolution models. In the two models we compare, black hole-black hole (BHBH) mergers are the most strongly affected, which we connect to a shift from the common envelope to the stable Roche lobe overflow formation channels with decreasing redshift. We also characterize variations in the black hole-neutron star (BHNS) and neutron star-neutron star (NSNS) final parameter distributions. We obtain the merger rate evolution for BHBH, BHNS and NSNS mergers up to $z=10$, and find a variation by a factor of $\sim50-60$ in the local BHBH and BHNS merger rates, suggesting a more important contribution from initial conditions than previously thought, and calling for a complete exploration of the initial conditions model permutations., Comment: 28 pages, 14 figures, 3 tables. Accepted for publication in MNRAS. Code/data to be made publicly available

Considering potentials defined by Bessel kernel with Bessel convolution a Kerman-Sawyer type characterization of trace inequality is given. As an application an estimate on the least eigenvalue of Schr\"odinger-Bessel operators is derived.

Due to the non-smoothness of optimization problems in Machine Learning, generalized smoothness assumptions have been gaining a lot of attention in recent years. One of the most popular assumptions of this type is $(L_0,L_1)$-smoothness (Zhang et al., 2020). In this paper, we focus on the class of (strongly) convex $(L_0,L_1)$-smooth functions and derive new convergence guarantees for several existing methods. In particular, we derive improved convergence rates for Gradient Descent with (Smoothed) Gradient Clipping and for Gradient Descent with Polyak Stepsizes. In contrast to the existing results, our rates do not rely on the standard smoothness assumption and do not suffer from the exponential dependency from the initial distance to the solution. We also extend these results to the stochastic case under the over-parameterization assumption, propose a new accelerated method for convex $(L_0,L_1)$-smooth optimization, and derive new convergence rates for Adaptive Gradient Descent (Malitsky and Mishchenko, 2020)., Comment: 51 pages, 1 figure

A theoretical framework for calculating the mass-radius curve of compact stars in the Kaluza-Klein space-time is introduced, with one additional compact spatial dimension. Static, spherically symmetric solutions are considered, with the equation of state provided by a zero temperature, interacting multidimensional Fermi gas. To model the strong force between baryons, a repulsive potential is introduced, which is linear in the particle number density. The maximal mass of compact stars is calculated for different model parameters, and with a physical parameter choice, it satisfies observational data, meaning that it is possible to model simple, realistic objects within this framework. Based on this comparison, a limiting size for the observational regime of extra dimensions in compact stars is provided, with $r_c \gtrsim 0.2$~fm., Comment: 11 pages, 8 figures

We present a long-period radio transient (GLEAM-X J0704-37) discovered to have an optical counterpart, consistent with a cool main sequence star of spectral type M3. The radio pulsations occur at the longest period yet found, 2.9 hours, and were discovered in archival low-frequency data from the Murchison Widefield Array (MWA). High time resolution observations from MeerKAT show that pulsations from the source display complex microstructure and high linear polarisation, suggesting a pulsar-like emission mechanism occurring due to strong, ordered magnetic fields. The timing residuals, measured over more than a decade, show tentative evidence of a ~6-yr modulation. The high Galactic latitude of the system and the M-dwarf star excludes the magnetar interpretation, suggesting a more likely M-dwarf / white dwarf binary scenario for this system., Comment: 11 pages, 4 figures, submitted to ApJL

We study the effects of integrability breaking on the relaxation dynamics of the (double) sine-Gordon model. Compared to previous studies, we apply an alternative viewpoint motivated by open-system physics by separating the phase field into homogeneous and inhomogeneous parts, describing a quantum pendulum (subsystem) and an interacting phononic bath (environment). To study the relaxation dynamics in the model, we perform quantum quenches using the mini-superspace-based truncated Hamiltonian approach developed recently and simulate the real-time evolution of various entanglement measures and the energy transfer between the subsystem and its environment. Our findings demonstrate that in the presence of integrability-breaking perturbations, the relaxation dynamics is substantially faster, signalled by the increase of entanglement and energy transfer between the quantum pendulum and the phonon bath., Comment: 9 pages, 28 figures

CRISPR (clustered regularly interspaced short palindromic repeat)- based diagnostics are at the forefront of rapid detection platforms of infectious diseases. The integration of reverse transcription-loop-mediated isothermal amplification (RT-LAMP) with CRISPR-Cas protein systems has led to the creation of advanced one-pot assays. The sensitivity of these assays has been bolstered by the utilization of a thermophilic Cas12 protein, BrCas12b, and its engineered variant, which exhibits enhanced thermal stability and allows for broader operation temperatures of the assay. Here, we perform all-atom molecular dynamics (MD) simulations on wild-type and mutant BrCas12b to reveal the mechanism of stabilization conferred by the mutation. High-temperature simulations reveal a small structural change along with greater flexibility in the PAM-interacting domain of the mutant BrCas12b, with marginal structural and flexibility changes in the other mutated domains. Comparative essential dynamics analysis between the wild-type and mutant BrCas12b at both ambient and elevated temperatures provides insights into the stabilizing effects of the mutations. Our findings not only offer a comprehensive insight into the dynamic alterations induced by mutations but reveal important motions in BrCas12b, important for the rational design of diagnostic and therapeutic platforms of Cas12 proteins.

std::string view is a reference-like data structure in the C++ Standard Template Library (STL) that enables fast and cheap processing of read-only strings. Due to its wide applicability and performance enhancing power, std::string view has been very popular since its introduction in the C++17 standard. However, its careless use can lead to serious memory management bugs. As the lifetime of a std::string view is not tied to the lifetime of the referenced string in any way, it is the user's responsibility to ensure that the view is only used while the viewed string is live and its buffer is not reallocated. This paper describes a static analysis tool that finds programming errors caused by the incorrect use of std::string view. Our work included modeling std::string view operations in the analysis, defining steps to detect lifetime errors, constructing user-friendly diagnostic messages, and performing an evaluation of the checker.

Static analysis is a method of analyzing source code without executing it. It is widely used to find bugs and code smells in industrial software. Besides other methods, the most important techniques are those based on the abstract syntax tree and those performing symbolic execution. Both of these methods found their role in modern software development as they have different advantages and limitations. In this tutorial, we present two problems from the C++ programming language: the elimination of redundant pointers, and the reporting of dangling pointers originating from incorrect use of the std::string class. These two issues have different theoretical backgrounds and finding them requires different implementation techniques. We will provide a step-by-step guide to implement the checkers (software to identify the aforementioned problems) - one based on the abstract syntax analysis method, the other exploring the possibilities of symbolic execution. The methods are explained in great detail and supported by code examples. The intended audience for this tutorial are both architects of static analysis tools and developers who want to understand the advantages and constraints of the different methods.

Results are presented for the measurement of large-scale anisotropies in the arrival directions of ultra-high-energy cosmic rays detected at the Pierre Auger Observatory during 19 years of operation, prior to AugerPrime, the upgrade of the Observatory. The 3D dipole amplitude and direction are reconstructed above $4\,$EeV in four energy bins. Besides the established dipolar anisotropy in right ascension above $8\,$EeV, the Fourier amplitude of the $8$ to $16\,$EeV energy bin is now also above the $5\sigma$ discovery level. No time variation of the dipole moment above $8\,$EeV is found, setting an upper limit to the rate of change of such variations of $0.3\%$ per year at the $95\%$ confidence level. Additionally, the results for the angular power spectrum are shown, demonstrating no other statistically significant multipoles. The results for the equatorial dipole component down to $0.03\,$EeV are presented, using for the first time a data set obtained with a trigger that has been optimized for lower energies. Finally, model predictions are discussed and compared with observations, based on two source emission scenarios obtained in the combined fit of spectrum and composition above $0.6\,$EeV., Comment: Minor modifications, accepted for publication in Astrophysical Journal

CodeChecker is an open source project that integrates different static analysis tools such as the Clang Static Analyzer and Clang-Tidy into the build systems, continuous integration loops, and development workflows of C++ programmers. It has a powerful issue management system to make it easier to evaluate the reports of the static analysis tools. This document was handed out as supportive material for a code analysis lecture at the 2018 3COWS conference in Kosice, Slovakia.

Static analysis is the analysis of a program without executing it, usually carried out by an automated tool. Symbolic execution is a popular static analysis technique used both in program verification and in bug detection software. It works by interpreting the code, introducing a symbol for each value unknown at compile time (e.g. user-given inputs), and carrying out calculations symbolically. The analysis engine strives to explore multiple execution paths simultaneously, although checking all paths is an intractable problem, due to the vast number of possibilities. We focus on an error finding framework called the Clang Static Analyzer, and the infrastructure built around it named CodeChecker. The emphasis is on achieving end-to-end scalability. This includes the run time and memory consumption of the analysis, bug presentation to the users, automatic false positive suppression, incremental analysis, pattern discovery in the results, and usage in continuous integration loops. We also outline future directions and open problems concerning these tools. While a rich literature exists on program verification software, error finding tools normally need to settle for survey papers on individual techniques. In this paper, we not only discuss individual methods, but also how these decisions interact and reinforce each other, creating a system that is greater than the sum of its parts. Although the Clang Static Analyzer can only handle C-family languages, the techniques introduced in this paper are mostly language-independent and applicable to other similar static analysis tools.

The neutron-rich unbound fluorine isotope $^{30}$F$_{21}$ has been observed for the first time by measuring its neutron decay at the SAMURAI spectrometer (RIBF, RIKEN) in the quasi-free proton knockout reaction of $^{31}$Ne nuclei at 235 MeV/nucleon. The mass and thus one-neutron-separation energy of $^{30}$F has been determined to be $S_n = -472\pm 58 \mathrm{(stat.)} \pm 33 \mathrm{(sys.)}$ keV from the measurement of its invariant-mass spectrum. The absence of a sharp drop in $S_n$($^{30}$F) shows that the ``magic'' $N=20$ shell gap is not restored close to $^{28}$O, which is in agreement with our shell-model calculations that predict a near degeneracy between the neutron $d$ and $fp$ orbitals, with the $1p_{3/2}$ and $1p_{1/2}$ orbitals becoming more bound than the $0f_{7/2}$ one. This degeneracy and reordering of orbitals has two potential consequences: $^{28}$O behaves like a strongly superfluid nucleus with neutron pairs scattering across shells, and both $^{29,31}$F appear to be good two-neutron halo-nucleus candidates., Comment: 9 pages, 2 figures, accepted for publication in Physical Review Letters

For a subgroup $H$ of a finite group $G$, the Frobenius graph $\Gamma(G, H)$ records the constituents of the restrictions to $H$ of the irreducible characters of $G$. We investigate when this graph has diameter 3., Comment: 27 pages, 7 tables, 3 figures

Ultra-high-energy cosmic rays are known to be mainly of extragalactic origin, and their propagation is limited by energy losses, so their arrival directions are expected to correlate with the large-scale structure of the local Universe. In this work, we investigate the possible presence of intermediate-scale excesses in the flux of the most energetic cosmic rays from the direction of the supergalactic plane region using events with energies above 20 EeV recorded with the surface detector array of the Pierre Auger Observatory up to 31 December 2022, with a total exposure of 135,000 km^2 sr yr. The strongest indication for an excess that we find, with a post-trial significance of 3.1{\sigma}, is in the Centaurus region, as in our previous reports, and it extends down to lower energies than previously studied. We do not find any strong hints of excesses from any other region of the supergalactic plane at the same angular scale. In particular, our results do not confirm the reports by the Telescope Array collaboration of excesses from two regions in the Northern Hemisphere at the edge of the field of view of the Pierre Auger Observatory. With a comparable exposure, our results in those regions are in good agreement with the expectations from an isotropic distribution., Comment: submitted to ApJ

Semiconductor-superconductor hybrid nanocircuits are of high interest due to their potential applications in quantum computing. Semiconductors with a strong spin-orbit coupling and large $g$-factor are particularly attractive since they are the basic building blocks of novel qubit architectures. However, for the engineering of these complex circuits, the building blocks must be characterized in detail. We have investigated a Josephson junction where the weak link is a two-dimensional electron gas (2DEG) hosted in an InAs/InGaAs heterostructure grown on a GaAs substrate. We employed the in-situ epitaxially grown Al layer as superconducting contacts to form an rf SQUID, and also to create a microwave resonator for sensing the Josephson inductance. We determined the gate-dependent current-phase relation, and observed supercurrent interference in out-of-plane magnetic fields. With the application of an in-plane magnetic field, we induced asymmetry in the interference pattern, which was found to be anisotropic in the device plane.

In response to the negative environmental, social, and economic externalities of the dominant food model, a consensus is emerging on the need to develop alternative food systems capable of facilitating a transition to a more sustainable model. Among these alternatives, short food supply chains (SFSCs) are identified as potential crucibles of alternativity. Defined as distribution channels with a maximum of one intermediary between producer and consumer, SFSCs can contribute to the reterritorialization of food flows and the creation of relational and geographical proximities among food chain actors. This study examines how logistical engagement translates into the choice of SFSC points of sale by consumers and farmers. It evaluates this engagement through the travel times farmers and consumers are willing to endure to sell or buy at an SFSC point of sale, and the prices they are willing to accept or pay for SFSC products. A discrete choice experiment conducted in 2022 with 154 farm managers and 1022 consumers analyzed preferences regarding types of points of sale, sales prices, travel times, and relational proximity. The results show that consumers engaged in SFSCs favour alternative points of sale such as farms and associations, even for longer travel times. Farmers also value these points of sale, with logistical engagement more pronounced among those already marketing through SFSCs. Supermarkets appear less attractive to alternative profiles but serve as entry points for new participants in SFSCs. Relational proximity between farmers and consumers is important, with a preference for direct exchanges during sales. This study highlights the importance of actor engagement in SFSCs and offers perspectives for developing alternative logistics by valuing relational proximity and using conventional points of sale as entry points for new actors in the SFSCs., Comment: in French language

We present a new approach based on the Personalized Federated Learning algorithm MeritFed that can be applied to Natural Language Tasks with heterogeneous data. We evaluate it on the Low-Resource Machine Translation task, using the dataset from the Large-Scale Multilingual Machine Translation Shared Task (Small Track #2) and the subset of Sami languages from the multilingual benchmark for Finno-Ugric languages. In addition to its effectiveness, MeritFed is also highly interpretable, as it can be applied to track the impact of each language used for training. Our analysis reveals that target dataset size affects weight distribution across auxiliary languages, that unrelated languages do not interfere with the training, and auxiliary optimizer parameters have minimal impact. Our approach is easy to apply with a few lines of code, and we provide scripts for reproducing the experiments at https://github.com/VityaVitalich/MeritFed, Comment: 18 pages, 7 figures

Process mining is concerned with deriving formal models capable of reproducing the behaviour of a given organisational process by analysing observed executions collected in an event log. The elements of an event log are finite sequences (i.e., traces or words) of actions. Many effective algorithms have been introduced which issue a control flow model (commonly in Petri net form) aimed at reproducing, as precisely as possible, the language of the considered event log. However, given that identical executions can be observed several times, traces of an event log are associated with a frequency and, hence, an event log inherently yields also a stochastic language. By exploiting the trace frequencies contained in the event log, the stochastic extension of process mining, therefore, consists in deriving stochastic (Petri nets) models capable of reproducing the likelihood of the observed executions. In this paper, we introduce a novel stochastic process mining approach. Starting from a "standard" Petri net model mined through classical mining algorithms, we employ optimization to identify optimal weights for the transitions of the mined net so that the stochastic language issued by the stochastic interpretation of the mined net closely resembles that of the event log. The optimization is either based on the maximum likelihood principle or on the earth moving distance. Experiments on some popular real system logs show an improved accuracy w.r.t. to alternative approaches.

The Pierre Auger Observatory is the most sensitive instrument to detect photons with energies above $10^{17}$ eV. It measures extensive air showers generated by ultra high energy cosmic rays using a hybrid technique that exploits the combination of a fluorescence detector with a ground array of particle detectors. The signatures of a photon-induced air shower are a larger atmospheric depth of the shower maximum ($X_{max}$) and a steeper lateral distribution function, along with a lower number of muons with respect to the bulk of hadron-induced cascades. In this work, a new analysis technique in the energy interval between 1 and 30 EeV (1 EeV = $10^{18}$ eV) has been developed by combining the fluorescence detector-based measurement of $X_{max}$ with the specific features of the surface detector signal through a parameter related to the air shower muon content, derived from the universality of the air shower development. No evidence of a statistically significant signal due to photon primaries was found using data collected in about 12 years of operation. Thus, upper bounds to the integral photon flux have been set using a detailed calculation of the detector exposure, in combination with a data-driven background estimation. The derived 95% confidence level upper limits are 0.0403, 0.01113, 0.0035, 0.0023, and 0.0021 km$^{-2}$ sr$^{-1}$ yr$^{-1}$ above 1, 2, 3, 5, and 10 EeV, respectively, leading to the most stringent upper limits on the photon flux in the EeV range. Compared with past results, the upper limits were improved by about 40% for the lowest energy threshold and by a factor 3 above 3 EeV, where no candidates were found and the expected background is negligible. The presented limits can be used to probe the assumptions on chemical composition of ultra-high energy cosmic rays and allow for the constraint of the mass and lifetime phase space of super-heavy dark matter particles., Comment: 19 pages, 22 figures

This work tackles the challenges of data heterogeneity and communication limitations in decentralized federated learning. We focus on creating a collaboration graph that guides each client in selecting suitable collaborators for training personalized models that leverage their local data effectively. Our approach addresses these issues through a novel, communication-efficient strategy that enhances resource efficiency. Unlike traditional methods, our formulation identifies collaborators at a granular level by considering combinatorial relations of clients, enhancing personalization while minimizing communication overhead. We achieve this through a bi-level optimization framework that employs a constrained greedy algorithm, resulting in a resource-efficient collaboration graph for personalized learning. Extensive evaluation against various baselines across diverse datasets demonstrates the superiority of our method, named DPFL. DPFL consistently outperforms other approaches, showcasing its effectiveness in handling real-world data heterogeneity, minimizing communication overhead, enhancing resource efficiency, and building personalized models in decentralized federated learning scenarios.

Long-distance quantum communication using quantum repeaters is an enabling technology for secure communication, distributed quantum computing and quantum-enhanced sensing and metrology. As a building block of quantum repeaters, spin-photon entanglement has been demonstrated with both atomic and solid-state qubits. However, previously demonstrated qubits with long spin coherence do not directly emit photons into the low-loss telecom band that is needed for long-distance communication. Here, we demonstrate spin-photon entanglement using a single Er$^{3+}$ ion in a solid-state crystal, integrated into a silicon nanophotonic circuit. Direct emission into the telecom band enables an entanglement rate of 1.48 Hz over 15.6 km of optical fiber, with a fidelity of 73(3)$\%$. This opens the door to large-scale quantum networks based on scalable nanophotonic devices and many spectrally multiplexed Er$^{3+}$ ions., Comment: Added references

We report an investigation of the mass composition of cosmic rays with energies from 3 to 100 EeV (1 EeV=$10^{18}$ eV) using the distributions of the depth of shower maximum $X_\mathrm{max}$. The analysis relies on ${\sim}50,000$ events recorded by the Surface Detector of the Pierre Auger Observatory and a deep-learning-based reconstruction algorithm. Above energies of 5 EeV, the data set offers a 10-fold increase in statistics with respect to fluorescence measurements at the Observatory. After cross-calibration using the Fluorescence Detector, this enables the first measurement of the evolution of the mean and the standard deviation of the $X_\mathrm{max}$ distributions up to 100 EeV. Our findings are threefold: (1.) The evolution of the mean logarithmic mass towards a heavier composition with increasing energy can be confirmed and is extended to 100 EeV. (2.) The evolution of the fluctuations of $X_\mathrm{max}$ towards a heavier and purer composition with increasing energy can be confirmed with high statistics. We report a rather heavy composition and small fluctuations in $X_\mathrm{max}$ at the highest energies. (3.) We find indications for a characteristic structure beyond a constant change in the mean logarithmic mass, featuring three breaks that are observed in proximity to the ankle, instep, and suppression features in the energy spectrum., Comment: submitted to Phys. Rev. D, 28 pages, 18 figures, 5 tables

We present measurements of the atmospheric depth of the shower maximum $X_\mathrm{max}$, inferred for the first time on an event-by-event level using the Surface Detector of the Pierre Auger Observatory. Using deep learning, we were able to extend measurements of the $X_\mathrm{max}$ distributions up to energies of 100 EeV ($10^{20}$ eV), not yet revealed by current measurements, providing new insights into the mass composition of cosmic rays at extreme energies. Gaining a 10-fold increase in statistics compared to the Fluorescence Detector data, we find evidence that the rate of change of the average $X_\mathrm{max}$ with the logarithm of energy features three breaks at $6.5\pm0.6~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, $11\pm 2~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, and $31\pm5~(\mathrm{stat})\pm3~(\mathrm{sys})$ EeV, in the vicinity to the three prominent features (ankle, instep, suppression) of the cosmic-ray flux. The energy evolution of the mean and standard deviation of the measured $X_\mathrm{max}$ distributions indicates that the mass composition becomes increasingly heavier and purer, thus being incompatible with a large fraction of light nuclei between 50 EeV and 100 EeV., Comment: submitted to Phys. Rev. Lett., 10 pages, 3 figures, 1 table

Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the stochastic gradients is heavy-tailed for the later ones. Gradient clipping provably helps to achieve good high-probability convergence for such noises. However, despite the similarity between AdaGrad/Adam and Clip-SGD, the high-probability convergence of AdaGrad/Adam has not been studied in this case. In this work, we prove that AdaGrad (and its delayed version) can have provably bad high-probability convergence if the noise is heavy-tailed. To fix this issue, we propose a new version of AdaGrad called Clip-RAdaGradD (Clipped Reweighted AdaGrad with Delay) and prove its high-probability convergence bounds with polylogarithmic dependence on the confidence level for smooth convex/non-convex stochastic optimization with heavy-tailed noise. Our empirical evaluations, including NLP model fine-tuning, highlight the superiority of clipped versions of AdaGrad/Adam in handling the heavy-tailed noise., Comment: 37 pages, 8 figures

We propose MoE-F -- a formalised mechanism for combining $N$ pre-trained expert Large Language Models (LLMs) in online time-series prediction tasks by adaptively forecasting the best weighting of LLM predictions at every time step. Our mechanism leverages the conditional information in each expert's running performance to forecast the best combination of LLMs for predicting the time series in its next step. Diverging from static (learned) Mixture of Experts (MoE) methods, MoE-F employs time-adaptive stochastic filtering techniques to combine experts. By framing the expert selection problem as a finite state-space, continuous-time Hidden Markov model (HMM), we can leverage the Wohman-Shiryaev filter. Our approach first constructs $N$ parallel filters corresponding to each of the $N$ individual LLMs. Each filter proposes its best combination of LLMs, given the information that they have access to. Subsequently, the $N$ filter outputs are aggregated to optimize a lower bound for the loss of the aggregated LLMs, which can be optimized in closed-form, thus generating our ensemble predictor. Our contributions here are: (I) the MoE-F algorithm -- deployable as a plug-and-play filtering harness, (II) theoretical optimality guarantees of the proposed filtering-based gating algorithm, and (III) empirical evaluation and ablative results using state of the art foundational and MoE LLMs on a real-world Financial Market Movement task where MoE-F attains a remarkable 17% absolute and 48.5% relative F1 measure improvement over the next best performing individual LLM expert., Comment: 29 pages, 5 Appendix sections

We present multichannel quantum defect (MQDT) models for highly excited $^{174}$Yb and $^{171}$Yb Rydberg states with $L \leq 2$. The models are developed using a combination of existing literature data and new, high-precision laser and microwave spectroscopy in an atomic beam, and validated by detailed comparison with experimentally measured Stark shifts and magnetic moments. We then use these models to compute interaction potentials between two Yb atoms, and find excellent agreement with direct measurements in an optical tweezer array. From the computed interaction potential, we identify an anomalous F\"orster resonance that likely degraded the fidelity of previous entangling gates in $^{171}$Yb using $F=3/2$ Rydberg states. We then identify a more suitable $F=1/2$ state, and achieve a state-of-the-art controlled-Z gate fidelity of $\mathcal{F}=0.994(1)$, with the remaining error fully explained by known sources. This work establishes a solid foundation for the continued development quantum computing, simulation and entanglement-enhanced metrology with Yb neutral atom arrays.

Federated learning (FL) has emerged as a pivotal approach in machine learning, enabling multiple participants to collaboratively train a global model without sharing raw data. While FL finds applications in various domains such as healthcare and finance, it is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. To overcome this challenge, principled mechanisms are required to evaluate the contributions made by individual participants in the FL setting. Existing solutions for contribution assessment rely on general accuracy evaluation, often failing to capture nuanced dynamics and class-specific influences. This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL. Our approach uses Shapley values from cooperative game theory to provide a granular understanding of class-specific influences. Based on ShapFed, we introduce a weighted aggregation method called ShapFed-WA, which outperforms conventional federated averaging, especially in class-imbalanced scenarios. Personalizing participant updates based on their contributions further enhances collaborative fairness by delivering differentiated models commensurate with the participant contributions. Experiments on CIFAR-10, Chest X-Ray, and Fed-ISIC2019 datasets demonstrate the effectiveness of our approach in improving utility, efficiency, and fairness in FL systems. The code can be found at https://github.com/tnurbek/shapfed., Comment: Accepted by IJCAI 2024

One of the inherent challenges in deploying transformers on time series is that \emph{reality only happens once}; namely, one typically only has access to a single trajectory of the data-generating process comprised of non-i.i.d. observations. We derive non-asymptotic statistical guarantees in this setting through bounds on the \textit{generalization} of a transformer network at a future-time $t$, given that it has been trained using $N\le t$ observations from a single perturbed trajectory of a Markov process. Under the assumption that the Markov process satisfies a log-Sobolev inequality, we obtain a generalization bound which effectively converges at the rate of ${O}(1/\sqrt{N})$. Our bound depends explicitly on the activation function ($\operatorname{Swish}$, $\operatorname{GeLU}$, or $\tanh$ are considered), the number of self-attention heads, depth, width, and norm-bounds defining the transformer architecture. Our bound consists of three components: (I) The first quantifies the gap between the stationary distribution of the data-generating Markov process and its distribution at time $t$, this term converges exponentially to $0$. (II) The next term encodes the complexity of the transformer model and, given enough time, eventually converges to $0$ at the rate ${O}(\log(N)^r/\sqrt{N})$ for any $r>0$. (III) The third term guarantees that the bound holds with probability at least $1$-$\delta$, and converges at a rate of ${O}(\sqrt{\log(1/\delta)}/\sqrt{N})$., Comment: 11 pages (+30 appendix), 3 figures, 6 tables

In this paper, we study a dynamic pickup and delivery problem with docking constraints. There is a homogeneous fleet of vehicles to serve pickup-and-delivery requests at given locations. The vehicles can be loaded up to their capacity, while unloading has to follow the last-in-first-out (LIFO) rule. The locations have a limited number of docking ports for loading and unloading, which may force the vehicles to wait. The problem is dynamic since the transportation requests arrive real-time, over the day. Accordingly, the routes of the vehicles are to be determined dynamically. The goal is to satisfy all the requests such that a combination of tardiness penalties and traveling costs is minimized. We propose a cost function approximation based solution method. In each decision epoch, we solve the respective optimization problem with a perturbed objective function to ensure the solutions remain adaptable to accommodate new requests. We penalize waiting times and idle vehicles. We propose a variable neighborhood search based method for solving the optimization problems, and we apply two existing local search operators, and we also introduce a new one. We evaluate our method using a widely adopted benchmark dataset, and the results demonstrate that our approach significantly surpasses the current state-of-the-art methods.

In this paper, we develop two families of sequential monitoring procedure to (timely) detect changes in a GARCH(1,1) model. Whilst our methodologies can be applied for the general analysis of changepoints in GARCH(1,1) sequences, they are in particular designed to detect changes from stationarity to explosivity or vice versa, thus allowing to check for volatility bubbles. Our statistics can be applied irrespective of whether the historical sample is stationary or not, and indeed without prior knowledge of the regime of the observations before and after the break. In particular, we construct our detectors as the CUSUM process of the quasi-Fisher scores of the log likelihood function. In order to ensure timely detection, we then construct our boundary function (exceeding which would indicate a break) by including a weighting sequence which is designed to shorten the detection delay in the presence of a changepoint. We consider two types of weights: a lighter set of weights, which ensures timely detection in the presence of changes occurring early, but not too early after the end of the historical sample; and a heavier set of weights, called Renyi weights which is designed to ensure timely detection in the presence of changepoints occurring very early in the monitoring horizon. In both cases, we derive the limiting distribution of the detection delays, indicating the expected delay for each set of weights. Our theoretical results are validated via a comprehensive set of simulations, and an empirical application to daily returns of individual stocks.

We compute the renormalized Dirac spectral density in $N_f = 2+1$ QCD at physical quark masses, temperature $T = 230$ MeV and system size $L_s = 3.4$ fm. To that end, we perform a point-wise continuum limit of the staggered density in lattice QCD with staggered quarks. We find, for the first time, that a clear infrared structure (IR peak) emerges in the density of Dirac operator describing dynamical quarks. Features of this structure are consistent with those previously attributed to the recently-proposed IR phase of thermal QCD. Our results (i) provide solid evidence that these IR features are stable and physical; (ii) improve the upper bound for IR-phase transition temperature $T_{\mathrm{IR}}$ so that the new window is $200 < T_{\mathrm{IR}} < 230\,$MeV; (iii) support non-restoration of anomalous U$_{\mathrm A}$(1) symmetry (chiral limit) below $T \!=\! 230\,$MeV., Comment: 7 pages, 6 figures

Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other RL algorithms, its practical application is often hindered by sensitivity to hyper-parameters, particularly the step-size. In this paper, we introduce the integration of the Polyak step-size in RL, which automatically adjusts the step-size without prior knowledge. To adapt this method to RL settings, we address several issues, including unknown f* in the Polyak step-size. Additionally, we showcase the performance of the Polyak step-size in RL through experiments, demonstrating faster convergence and the attainment of more stable policies.

The flux of ultra-high energy cosmic rays reaching Earth above the ankle energy (5 EeV) can be described as a mixture of nuclei injected by extragalactic sources with very hard spectra and a low rigidity cutoff. Extragalactic magnetic fields existing between the Earth and the closest sources can affect the observed CR spectrum by reducing the flux of low-rigidity particles reaching Earth. We perform a combined fit of the spectrum and distributions of depth of shower maximum measured with the Pierre Auger Observatory including the effect of this magnetic horizon in the propagation of UHECRs in the intergalactic space. We find that, within a specific range of the various experimental and phenomenological systematics, the magnetic horizon effect can be relevant for turbulent magnetic field strengths in the local neighbourhood of order $B_{\rm rms}\simeq (50-100)\,{\rm nG}\,(20\rm{Mpc}/{d_{\rm s})( 100\,\rm{kpc}/L_{\rm coh}})^{1/2}$, with $d_{\rm s}$ the typical intersource separation and $L_{\rm coh}$ the magnetic field coherence length. When this is the case, the inferred slope of the source spectrum becomes softer and can be closer to the expectations of diffusive shock acceleration, i.e., $\propto E^{-2}$. An additional cosmic-ray population with higher source density and softer spectra, presumably also extragalactic and dominating the cosmic-ray flux at EeV energies, is also required to reproduce the overall spectrum and composition results for all energies down to 0.6~EeV., Comment: Matches published version

The smart grid domain requires bolstering the capabilities of existing energy management systems; Federated Learning (FL) aligns with this goal as it demonstrates a remarkable ability to train models on heterogeneous datasets while maintaining data privacy, making it suitable for smart grid applications, which often involve disparate data distributions and interdependencies among features that hinder the suitability of linear models. This paper introduces a framework that combines FL with a Trust Region Policy Optimization (FL TRPO) aiming to reduce energy-associated emissions and costs. Our approach reveals latent interconnections and employs personalized encoding methods to capture unique insights, understanding the relationships between features and optimal strategies, allowing our model to generalize to previously unseen data. Experimental results validate the robustness of our approach, affirming its proficiency in effectively learning policy models for smart grid challenges., Comment: ICLR 2024 Workshop: Tackling Climate Change with Machine Learning

We formulate an XAI-based model improvement approach for Graph Neural Networks (GNNs) for node classification, called Explanation Enhanced Graph Learning (EEGL). The goal is to improve predictive performance of GNN using explanations. EEGL is an iterative self-improving algorithm, which starts with a learned "vanilla" GNN, and repeatedly uses frequent subgraph mining to find relevant patterns in explanation subgraphs. These patterns are then filtered further to obtain application-dependent features corresponding to the presence of certain subgraphs in the node neighborhoods. Giving an application-dependent algorithm for such a subgraph-based extension of the Weisfeiler-Leman (1-WL) algorithm has previously been posed as an open problem. We present experimental evidence, with synthetic and real-world data, which show that EEGL outperforms related approaches in predictive performance and that it has a node-distinguishing power beyond that of vanilla GNNs. We also analyze EEGL's training dynamics.

We experimentally study effort provision and network formation in the linear-quadratic game characterized by positive externality and complementarity of effort choices among network neighbors. We compare experimental outcomes to the equilibrium and efficient allocations and study the impact of group size and linking costs. We find that individuals overprovide effort relative to the equilibrium level on the network they form. However, their payoffs are lower than the equilibrium payoffs because they create fewer links than it is optimal which limits the beneficial spillovers of effort provision. Reducing the linking costs does not significantly increase the connectedness of the network and the welfare loss is higher in larger groups. Individuals connect to the highest effort providers in the group and ignore links to relative low effort providers, even if those links would be beneficial to form. This effect explains the lack of links in the network.

We study whether competition for social status induces higher effort provision and efficiency when individuals collaborate with their network neighbors. We consider a laboratory experiment in which individuals choose a costly collaborative effort and their network neighbors. They benefit from their neighbors' effort and effort choices of direct neighbors are strategic complements. We introduce two types of social status in a 2x2 factorial design: 1) individuals receive monetary benefits for incoming links representing popularity; 2) they receive feedback on their relative payoff ranking within the group. We find that link benefits induce higher effort provision and strengthen the collaborative ties relative to the Baseline treatment without social status. In contrast, the ranking information induces lower effort as individuals start competing for higher ranking. Overall, we find that social status has no significant impact on the number of links in the network and the efficiency of collaboration in the group.

Monochromatic gamma-ray signals constitute a potential smoking gun signature for annihilating or decaying dark matter particles that could relatively easily be distinguished from astrophysical or instrumental backgrounds. We provide an updated assessment of the sensitivity of the Cherenkov Telescope Array (CTA) to such signals, based on observations of the Galactic centre region as well as of selected dwarf spheroidal galaxies. We find that current limits and detection prospects for dark matter masses above 300 GeV will be significantly improved, by up to an order of magnitude in the multi-TeV range. This demonstrates that CTA will set a new standard for gamma-ray astronomy also in this respect, as the world's largest and most sensitive high-energy gamma-ray observatory, in particular due to its exquisite energy resolution at TeV energies and the adopted observational strategy focussing on regions with large dark matter densities. Throughout our analysis, we use up-to-date instrument response functions, and we thoroughly model the effect of instrumental systematic uncertainties in our statistical treatment. We further present results for other potential signatures with sharp spectral features, e.g.~box-shaped spectra, that would likewise very clearly point to a particle dark matter origin., Comment: 44 pages JCAP style (excluding author list and references), 19 figures; minor changes to match published version

Adaptive methods are extremely popular in machine learning as they make learning rate tuning less expensive. This paper introduces a novel optimization algorithm named KATE, which presents a scale-invariant adaptation of the well-known AdaGrad algorithm. We prove the scale-invariance of KATE for the case of Generalized Linear Models. Moreover, for general smooth non-convex problems, we establish a convergence rate of $O \left(\frac{\log T}{\sqrt{T}} \right)$ for KATE, matching the best-known ones for AdaGrad and Adam. We also compare KATE to other state-of-the-art adaptive algorithms Adam and AdaGrad in numerical experiments with different problems, including complex machine learning tasks like image classification and text classification on real data. The results indicate that KATE consistently outperforms AdaGrad and matches/surpasses the performance of Adam in all considered scenarios., Comment: 27 pages, 12 figures

We address in this work the nature and evolution of the long-period compact star sources, which has recently added several unexpected members. The central hypothesis is that particle winds drive their evolution, being an important factor for these relatively old sources. We show the consistency of this picture and remark some unsolved problems and caveats within it., Comment: 22 pp., 2 Figs, to appear in the special volume "Pulsar Astronomy with China's Facilities", Eds. R.X.Xu, Z. Gao, J.E. Horvath anbd C.A.Z. Vasconcellos (World Scientifica, Singapore, 2024)

In Federated Learning (FL), forgetting, or the loss of knowledge across rounds, hampers algorithm convergence, particularly in the presence of severe data heterogeneity among clients. This study explores the nuances of this issue, emphasizing the critical role of forgetting in FL's inefficient learning within heterogeneous data contexts. Knowledge loss occurs in both client-local updates and server-side aggregation steps; addressing one without the other fails to mitigate forgetting. We introduce a metric to measure forgetting granularly, ensuring distinct recognition amid new knowledge acquisition. Leveraging these insights, we propose Flashback, an FL algorithm with a dynamic distillation approach that is used to regularize the local models, and effectively aggregate their knowledge. Across different benchmarks, Flashback outperforms other methods, mitigates forgetting, and achieves faster round-to-target-accuracy, by converging in 6 to 16 rounds.

In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. While collaboration among clients can significantly enhance the learning process, not all collaborations are beneficial; some may even be detrimental. In this study, we introduce a novel algorithm that assigns adaptive aggregation weights to clients participating in FL training, identifying those with data distributions most conducive to a specific learning objective. We demonstrate that our aggregation method converges no worse than the method that aggregates only the updates received from clients with the same data distribution. Furthermore, empirical evaluations consistently reveal that collaborations guided by our algorithm outperform traditional FL approaches. This underscores the critical role of judicious client selection and lays the foundation for more streamlined and effective FL implementations in the coming years.

A central problem of geometry is the tiling of space with simple structures. The classical solutions, such as triangles, squares, and hexagons in the plane and cubes and other polyhedra in three-dimensional space are built with sharp corners and flat faces. However, many tilings in Nature are characterized by shapes with curved edges, non-flat faces, and few, if any, sharp corners. An important question is then to relate prototypical sharp tilings to softer natural shapes. Here, we solve this problem by introducing a new class of shapes, the \textit{soft cells}, minimizing the number of sharp corners and filling space as \emph{soft tilings}. We prove that an infinite class of polyhedral tilings can be smoothly deformed into soft tilings and we construct the soft versions of all Dirichlet-Voronoi cells associated with point lattices in two and three dimensions. Remarkably, these ideal soft shapes, born out of geometry, are found abundantly in nature, from cells to shells., Comment: 18 pages, 7 figures

Recent studies suggest that with sufficiently wide models, most SGD solutions can, up to permutation, converge into the same basin. This phenomenon, known as the model re-basin regime, has significant implications for model averaging by ensuring the linear mode connectivity. However, current re-basin strategies are ineffective in many scenarios due to a lack of comprehensive understanding of underlying mechanisms. Addressing this gap, this paper provides novel insights into understanding and improving the standard practice. Firstly, we decompose re-normalization into rescaling and reshift, uncovering that rescaling plays a crucial role in re-normalization while re-basin performance is sensitive to shifts in model activation. The finding calls for a more nuanced handling of the activation shift. Secondly, we identify that the merged model suffers from the issue of activation collapse and magnitude collapse. Varying the learning rate, weight decay, and initialization method can mitigate the issues and improve model performance. Lastly, we propose a new perspective to unify the re-basin and pruning, under which a lightweight yet effective post-pruning technique is derived, which can significantly improve the model performance after pruning. Our implementation is available at https://github.com/XingyuQu/rethink-re-basin., Comment: 39 pages