Search

Federated learning enables multiple actors to collaboratively train models without sharing private data. This unlocks the potential for scaling machine learning to diverse applications. Existing algorithms for this task are well-justified when clients and the intended target domain share the same distribution of features and labels, but this assumption is often violated in real-world scenarios. One common violation is label shift, where the label distributions differ across clients or between clients and the target domain, which can significantly degrade model performance. To address this problem, we propose FedPALS, a novel model aggregation scheme that adapts to label shifts by leveraging knowledge of the target label distribution at the central server. Our approach ensures unbiased updates under stochastic gradient descent, ensuring robust generalization across clients with diverse, label-shifted data. Extensive experiments on image classification demonstrate that FedPALS consistently outperforms standard baselines by aligning model aggregation with the target domain. Our findings reveal that conventional federated learning methods suffer severely in cases of extreme client sparsity, highlighting the critical need for target-aware aggregation. FedPALS offers a principled and practical solution to mitigate label distribution mismatch, ensuring models trained in federated settings can generalize effectively to label-shifted target domains.

In the near future, the Jefferson Lab $b_1$ experiment will provide the second measurement of tensor polarized asymmetries in inclusive DIS on the deuteron. In this asymmetry, 4 independent tensor polarized structure functions contribute. This necessitates systematic approximations in the extraction of the leading twist structure function $b_1$ from a single tensor asymmetry measurement. Contamination from higher twist structure functions and kinematic effects is discussed here. Using a deuteron convolution model, we quantify the systematic errors from these approximations for two different choices for the target polarization direction (momentum transfer, electron beam direction). For Jefferson Lab 12 GeV kinematics, the systematic error turns out to be comparable between the two polarization options, while at higher $Q^2$ values the momentum transfer direction is preferred., Comment: 11 pages, 11 figures, submitted as part of the EPJA Topical Collection on Tensor Spin Observables

Decentralized Learning (DL) enables privacy-preserving collaboration among organizations or users to enhance the performance of local deep learning models. However, model aggregation becomes challenging when client data is heterogeneous, and identifying compatible collaborators without direct data exchange remains a pressing issue. In this paper, we investigate the effectiveness of various similarity metrics in DL for identifying peers for model merging, conducting an empirical analysis across multiple datasets with distribution shifts. Our research provides insights into the performance of these metrics, examining their role in facilitating effective collaboration. By exploring the strengths and limitations of these metrics, we contribute to the development of robust DL methods.

Land cover classification and change detection are two important applications of remote sensing and Earth observation (EO) that have benefited greatly from the advances of deep learning. Convolutional and transformer-based U-net models are the state-of-the-art architectures for these tasks, and their performances have been boosted by an increased availability of large-scale annotated EO datasets. However, the influence of different visual characteristics of the input EO data on a model's predictions is not well understood. In this work we systematically examine model sensitivities with respect to several color- and texture-based distortions on the input EO data during inference, given models that have been trained without such distortions. We conduct experiments with multiple state-of-the-art segmentation networks for land cover classification and show that they are in general more sensitive to texture than to color distortions. Beyond revealing intriguing characteristics of widely used land cover classification models, our results can also be used to guide the development of more robust models within the EO domain.

We report a high precision measurement of electron beam polarization using Compton polarimetry. The measurement was made in experimental Hall A at Jefferson Lab during the CREX experiment in 2020. A total uncertainty of dP/P=0.36% was achieved detecting the back-scattered photons from the Compton scattering process. This is the highest accuracy in a measurement of electron beam polarization using Compton scattering ever reported, surpassing the ground-breaking measurement from the SLAC Large Detector (SLD) Compton polarimeter. Such uncertainty reaches the level required for the future flagship measurements to be made by the MOLLER and SoLID experiments.

We propose to conduct a measurement of the Virtual Compton Scattering reaction in Hall C that will allow the precise extraction of the two scalar Generalized Polarizabilities (GPs) of the proton in the region of $Q^2=0.05~(GeV/c)^2$ to $Q^2=0.50~(GeV/c)^2$. The Generalized Polarizabilities are fundamental properties of the proton, that characterize the system's response to an external electromagnetic (EM) field. They describe how easily the charge and magnetization distributions inside the system are distorted by the EM field, mapping out the resulting deformation of the densities in the proton. As such, they reveal unique information regarding the underlying system dynamics and provide a key for decoding the proton structure in terms of the theory of the strong interaction that binds its elementary quark and gluon constituents together. Recent measurements of the proton GPs have challenged the theoretical predictions, particularly in regard to the electric polarizability. The magnetic GP, on the other hand, can provide valuable insight to the competing paramagnetic and diamagnetic contributions in the proton, but it is poorly known within the region where the interplay of these processes is very dynamic and rapidly changing.The unique capabilities of Hall C, namely the high resolution of the spectrometers combined with the ability to place the spectrometers in small angles, will allow to pin down the dynamic signature of the GPs through high precision measurements combined with a fine mapping as a function of $Q^2$. The experimental setup utilizes standard Hall C equipment, as was previously employed in the VCS-I (E12-15-001) experiment, namely the HMS and SHMS spectrometers and a 10 cm liquid hydrogen target. A total of 59 days of unpolarized 75 $\mu A$ electron beam with energy of 1100 MeV (6 days) and 2200 MeV (53 days) is requested for this experiment.

The signed double Roman domination problem is a combinatorial optimization problem on a graph asking to assign a label from $\{\pm{}1,2,3\}$ to each vertex feasibly, such that the total sum of assigned labels is minimized. Here feasibility is given whenever (i) vertices labeled $\pm{}1$ have at least one neighbor with label in $\{2,3\}$; (ii) each vertex labeled $-1$ has one $3$-labeled neighbor or at least two $2$-labeled neighbors; and (iii) the sum of labels over the closed neighborhood of any vertex is positive. The cumulative weight of an optimal labeling is called signed double Roman domination number (SDRDN). In this work, we first consider the problem on general cubic graphs of order $n$ for which we present a sharp $n/2+\Theta(1)$ lower bound for the SDRDN by means of the discharging method. Moreover, we derive a new best upper bound. Observing that we are often able to minimize the SDRDN over the class of cubic graphs of a fixed order, we then study in this context generalized Petersen graphs for independent interest, for which we propose a constraint programming guided proof. We then use these insights to determine the SDRDNs of subcubic $2\times m$ grid graphs, among other results.

Decentralized deep learning requires dealing with non-iid data across clients, which may also change over time due to temporal shifts. While non-iid data has been extensively studied in distributed settings, temporal shifts have received no attention. To the best of our knowledge, we are first with tackling the novel and challenging problem of decentralized learning with non-iid and dynamic data. We propose a novel algorithm that can automatically discover and adapt to the evolving concepts in the network, without any prior knowledge or estimation of the number of concepts. We evaluate our algorithm on standard benchmark datasets and demonstrate that it outperforms previous methods for decentralized learning., Comment: 4 pages, 2 figures

The grammatical gender of Swedish nouns is a mystery. While there are few rules that can indicate the gender with some certainty, it does in general not depend on either meaning or the structure of the word. In this paper we demonstrate the surprising fact that grammatical gender for Swedish nouns can be predicted with high accuracy using a recurrent neural network (RNN) working on the raw character sequence of the word, without using any contextual information.

Personalized decentralized learning is a promising paradigm for distributed learning, enabling each node to train a local model on its own data and collaborate with other nodes to improve without sharing any data. However, this approach poses significant privacy risks, as nodes may inadvertently disclose sensitive information about their data or preferences through their collaboration choices. In this paper, we propose Private Personalized Decentralized Learning (PPDL), a novel approach that combines secure aggregation and correlated adversarial multi-armed bandit optimization to protect node privacy while facilitating efficient node selection. By leveraging dependencies between different arms, represented by potential collaborators, we demonstrate that PPDL can effectively identify suitable collaborators solely based on aggregated models. Additionally, we show that PPDL surpasses previous non-private methods in model performance on standard benchmarks under label and covariate shift scenarios.

Generative adversarial networks have proven to be a powerful tool for learning complex and high-dimensional data distributions, but issues such as mode collapse have been shown to make it difficult to train them. This is an even harder problem when the data is decentralized over several clients in a federated learning setup, as problems such as client drift and non-iid data make it hard for federated averaging to converge. In this work, we study the task of how to learn a data distribution when training data is heterogeneously decentralized over clients and cannot be shared. Our goal is to sample from this distribution centrally, while the data never leaves the clients. We show using standard benchmark image datasets that existing approaches fail in this setting, experiencing so-called client drift when the local number of epochs becomes to large. We thus propose a novel approach we call EFFGAN: Ensembles of fine-tuned federated GANs. Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clients and mitigate client drift. It is able to train with a large number of local epochs, making it more communication efficient than previous works.

We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail.

Federated Learning (FL) is a promising framework for distributed learning when data is private and sensitive. However, the state-of-the-art solutions in this framework are not optimal when data is heterogeneous and non-Independent and Identically Distributed (non-IID). We propose a practical and robust approach to personalization in FL that adjusts to heterogeneous and non-IID data by balancing exploration and exploitation of several global models. To achieve our aim of personalization, we use a Mixture of Experts (MoE) that learns to group clients that are similar to each other, while using the global models more efficiently. We show that our approach achieves an accuracy up to 29.78 % and up to 4.38 % better compared to a local model in a pathological non-IID setting, even though we tune our approach in the IID setting., Comment: To appear in International Workshop on Trustworthy Federated Learning in Conjunction with IJCAI 2022 (FL-IJCAI'22). 8 pages

We report a precise measurement of the parity-violating asymmetry $A_{\rm PV}$ in the elastic scattering of longitudinally polarized electrons from $^{48}{\rm Ca}$. We measure $A_{\rm PV} =2668\pm 106\ {\rm (stat)}\pm 40\ {\rm (syst)}$ parts per billion, leading to an extraction of the neutral weak form factor $F_{\rm W} (q=0.8733$ fm$^{-1}) = 0.1304 \pm 0.0052 \ {\rm (stat)}\pm 0.0020\ {\rm (syst)}$ and the charge minus the weak form factor $F_{\rm ch} - F_{\rm W} = 0.0277\pm 0.0055$. The resulting neutron skin thickness $R_n-R_p=0.121 \pm 0.026\ {\rm (exp)} \pm 0.024\ {\rm (model)}$~fm is relatively thin yet consistent with many model calculations. The combined CREX and PREX results will have implications for future energy density functional calculations and on the density dependence of the symmetry energy of nuclear matter., Comment: 8 pages, 5 figures Replace 6-16-22: included ancillary files, corrected errors in references, author affiliations. Small text changes for clarity

This paper considers the following three Roman domination graph invariants on Kneser graphs: Roman domination, total Roman domination, and signed Roman domination. For Kneser graph $K_{n,k}$, we present exact values for Roman domination number $\gamma_{R}(K_{n,k})$ and total Roman domination number $\gamma_{tR}(K_{n,k})$ proving that for $n\geqslant k(k+1)$, $\gamma_{R}(K_{n,k}) =\gamma_{tR}(K_{n,k}) = 2(k+1)$. For signed Roman domination number $\gamma_{sR}(K_{n,k})$, the new lower and upper bounds for $K_{n,2}$ are provided: we prove that for $n\geqslant 12$, the lower bound is equal to 2, while the upper bound depends on the parity of $n$ and is equal to 3 if $n$ is odd, and equal to $5$ if $n$ is even. For graphs of smaller dimensions, exact values are found by applying exact methods from literature.

We report precision determinations of the beam normal single spin asymmetries ($A_n$) in the elastic scattering of 0.95 and 2.18~GeV electrons off $^{12}$C, $^{40}$Ca, $^{48}$Ca, and $^{208}$Pb at very forward angles where the most detailed theoretical calculations have been performed. The first measurements of $A_n$ for $^{40}$Ca and $^{48}$Ca are found to be similar to that of $^{12}$C, consistent with expectations thus demonstrating the validity of theoretical calculations for nuclei with Z~$\leq20$. We also report $A_n$ for $^{208}$Pb at two new momentum transfers (Q$^2$) extending the previous measurement. Our new data confirm the surprising result previously reported, with all three data points showing significant disagreement with the results from the $Z\leq 20$ nuclei. These data confirm our basic understanding of the underlying dynamics that govern $A_n$ for nuclei containing $\lesssim 50$ nucleons, but point to the need for further investigation to understand the unusual $A_n$ behaviour discovered for scattering off $^{208}$Pb., Comment: 7 pages, 2 figures

We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other's data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines., Comment: 7 pages, 2 figures

In this paper we deal with the calculation of the signed (total) Roman domination numbers, $\gamma_{sR}$ and $\gamma_{stR}$ respectively, on a few classes of planar graphs from the literature. We give proofs for the exact values of the numbers $\gamma_{sR}(A_n)$ and $\gamma_{sR}(R_n)$ as well as the numbers $\gamma_{stR}(S_n)$ and $\gamma_{stR}(T_n)$. For some other classes of planar graphs, such as $Q_n$, %$S_n"$ and $T_n"$, lower and upper bounds on $\gamma_{sR}$ are calculated and proved. %We give some open problems on the exact values of $\gamma_{sR}$ and $\gamma_{stR}$ for some classes of planar graphs.

We report a precision measurement of the parity-violating asymmetry $A_{PV}$ in the elastic scattering of longitudinally polarized electrons from $^{208}$Pb. We measure $A_{PV}=550\pm 16 {\rm (stat)}\pm 8\ {\rm (syst)}$ parts per billion, leading to an extraction of the neutral weak form factor $F_W(Q^2 = 0.00616\ {\rm GeV}^2) = 0.368 \pm 0.013$. Combined with our previous measurement, the extracted neutron skin thickness is $R_n-R_p=0.283 \pm 0.071$~fm. The result also yields the first significant direct measurement of the interior weak density of $^{208}$Pb: $\rho^0_W = -0.0796\pm0.0036\ {\rm (exp.)}\pm0.0013\ {\rm (theo.)}\ {\rm fm}^{-3}$ leading to the interior baryon density $\rho^0_b = 0.1480\pm0.0036\ {\rm (exp.)}\pm0.0013\ {\rm (theo.)}\ {\rm fm}^{-3}$. The measurement accurately constrains the density dependence of the symmetry energy of nuclear matter near saturation density, with implications for the size and composition of neutron stars.

Federated learning (FL) is a promising approach to distributed compute, as well as distributed data, and provides a level of privacy and compliance to legal frameworks. This makes FL attractive for both consumer and healthcare applications. While the area is actively being explored, few studies have examined FL in the context of larger language models and there is a lack of comprehensive reviews of robustness across tasks, architectures, numbers of clients, and other relevant factors. In this paper, we explore the fine-tuning of Transformer-based language models in a federated learning setting. We evaluate three popular BERT-variants of different sizes (BERT, ALBERT, and DistilBERT) on a number of text classification tasks such as sentiment analysis and author identification. We perform an extensive sweep over the number of clients, ranging up to 32, to evaluate the impact of distributed compute on task performance in the federated averaging setting. While our findings suggest that the large sizes of the evaluated models are not generally prohibitive to federated training, we found that the different models handle federated averaging to a varying degree. Most notably, DistilBERT converges significantly slower with larger numbers of clients, and under some circumstances, even collapses to chance level performance. Investigating this issue presents an interesting perspective for future research.

In federated learning, clients share a global model that has been trained on decentralized local client data. Although federated learning shows significant promise as a key approach when data cannot be shared or centralized, current methods show limited privacy properties and have shortcomings when applied to common real-world scenarios, especially when client data is heterogeneous. In this paper, we propose an alternative method to learn a personalized model for each client in a federated setting, with greater generalization abilities than previous methods. To achieve this personalization we propose a federated learning framework using a mixture of experts to combine the specialist nature of a locally trained model with the generalist knowledge of a global model. We evaluate our method on a variety of datasets with different levels of data heterogeneity, and our results show that the mixture of experts model is better suited as a personalized model for devices in these settings, outperforming both fine-tuned global models and local specialists., Comment: 8 pages, 6 figures

As more and more data is collected in various settings across organizations, companies, and countries, there has been an increase in the demand of user privacy. Developing privacy preserving methods for data analytics is thus an important area of research. In this work we present a model based on generative adversarial networks (GANs) that learns to obfuscate specific sensitive attributes in speech data. We train a model that learns to hide sensitive information in the data, while preserving the meaning in the utterance. The model is trained in two steps: first to filter sensitive information in the spectrogram domain, and then to generate new and private information independent of the filtered one. The model is based on a U-Net CNN that takes mel-spectrograms as input. A MelGAN is used to invert the spectrograms back to raw audio waveforms. We show that it is possible to hide sensitive information such as gender by generating new data, trained adversarially to maintain utility and realism., Comment: Submitted to ICML 2020 Workshop on Self-supervision in Audio and Speech (SAS)

Data privacy is an increasingly important aspect of many real-world Data sources that contain sensitive information may have immense potential which could be unlocked using the right privacy enhancing transformations, but current methods often fail to produce convincing output. Furthermore, finding the right balance between privacy and utility is often a tricky trade-off. In this work, we propose a novel approach for data privatization, which involves two steps: in the first step, it removes the sensitive information, and in the second step, it replaces this information with an independent random sample. Our method builds on adversarial representation learning which ensures strong privacy by training the model to fool an increasingly strong adversary. While previous methods only aim at obfuscating the sensitive information, we find that adding new random information in its place strengthens the provided privacy and provides better utility at any given level of privacy. The result is an approach that can provide stronger privatization on image data, and yet be preserving both the domain and the utility of the inputs, entirely independent of the downstream task.