Search

Decentralized learning (DL) is an emerging technique that allows nodes on the web to collaboratively train machine learning models without sharing raw data. Dealing with stragglers, i.e., nodes with slower compute or communication than others, is a key challenge in DL. We present DivShare, a novel asynchronous DL algorithm that achieves fast model convergence in the presence of communication stragglers. DivShare achieves this by having nodes fragment their models into parameter subsets and send, in parallel to computation, each subset to a random sample of other nodes instead of sequentially exchanging full models. The transfer of smaller fragments allows more efficient usage of the collective bandwidth and enables nodes with slow network links to quickly contribute with at least some of their model parameters. By theoretically proving the convergence of DivShare, we provide, to the best of our knowledge, the first formal proof of convergence for a DL algorithm that accounts for the effects of asynchronous communication with delays. We experimentally evaluate DivShare against two state-of-the-art DL baselines, AD-PSGD and Swift, and with two standard datasets, CIFAR-10 and MovieLens. We find that DivShare with communication stragglers lowers time-to-accuracy by up to 3.9x compared to AD-PSGD on the CIFAR-10 dataset. Compared to baselines, DivShare also achieves up to 19.4% better accuracy and 9.5% lower test loss on the CIFAR-10 and MovieLens datasets, respectively.

Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of heterogeneity in the training data's feature space. Such feature heterogeneity lowers model utility and negatively impacts fairness, particularly for nodes with under-represented training data. In this paper, we introduce \textsc{Facade}, a clustering-based DL algorithm specifically designed for fair model training when the training data exhibits several distinct features. The challenge of \textsc{Facade} is to assign nodes to clusters, one for each feature, based on the similarity in the features of their local data, without requiring individual nodes to know apriori which cluster they belong to. \textsc{Facade} (1) dynamically assigns nodes to their appropriate clusters over time, and (2) enables nodes to collaboratively train a specialized model for each cluster in a fully decentralized manner. We theoretically prove the convergence of \textsc{Facade}, implement our algorithm, and compare it against three state-of-the-art baselines. Our experimental results on three datasets demonstrate the superiority of our approach in terms of model accuracy and fairness compared to all three competitors. Compared to the best-performing baseline, \textsc{Facade} on the CIFAR-10 dataset also reduces communication costs by 32.3\% to reach a target accuracy when cluster sizes are imbalanced.

We study the effect of a varying pion mass on the quantum chromodynamics (QCD) phase diagram in the presence of an external magnetic field, aiming to understand it, for the first time, using Nambu\textendash Jona-Lasinio like effective models. We compare results from both its local and nonlocal versions. In both cases, we find that the inverse magnetic catalysis (IMC) near the crossover is eliminated with increasing pion mass, while the decreasing trend of crossover temperature with increasing magnetic field persists for pion mass values at least up to $440$ MeV. Thus, the models are capable of capturing qualitatively the results found by lattice QCD (LQCD) for heavy (unphysical) pions. The key feature in the models is the incorporation of the effect of a reduction in the coupling constant with increasing energy. Along with reproducing the IMC effect, it enables models to describe the effects of heavier current quark masses without introducing additional parameters. For the local NJL model, this agreement depends on how the parameters of the model are fit at the physical point. In this respect, the nonlocal version, which, due to its formulation, automatically exhibits the IMC effect around the crossover region, captures the physics more naturally. We further use the nonlocal framework to determine the pion mass beyond which the IMC effect around the transition region does not exist anymore., Comment: 22 pages, 15 captioned figures, pion mass value beyond which the IMC effect disappears calculated, published version

Decentralized learning (DL) offers a powerful framework where nodes collaboratively train models without sharing raw data and without the coordination of a central server. In the iterative rounds of DL, models are trained locally, shared with neighbors in the topology, and aggregated with other models received from neighbors. Sharing and merging models contribute to convergence towards a consensus model that generalizes better across the collective data captured at training time. In addition, the energy consumption while sharing and merging model parameters is negligible compared to the energy spent during the training phase. Leveraging this fact, we present SkipTrain, a novel DL algorithm, which minimizes energy consumption in decentralized learning by strategically skipping some training rounds and substituting them with synchronization rounds. These training-silent periods, besides saving energy, also allow models to better mix and finally produce models with superior accuracy than typical DL algorithms that train at every round. Our empirical evaluations with 256 nodes demonstrate that SkipTrain reduces energy consumption by 50% and increases model accuracy by up to 12% compared to D-PSGD, the conventional DL algorithm.

Decentralized learning (DL) faces increased vulnerability to privacy breaches due to sophisticated attacks on machine learning (ML) models. Secure aggregation is a computationally efficient cryptographic technique that enables multiple parties to compute an aggregate of their private data while keeping their individual inputs concealed from each other and from any central aggregator. To enhance communication efficiency in DL, sparsification techniques are used, selectively sharing only the most crucial parameters or gradients in a model, thereby maintaining efficiency without notably compromising accuracy. However, applying secure aggregation to sparsified models in DL is challenging due to the transmission of disjoint parameter sets by distinct nodes, which can prevent masks from canceling out effectively. This paper introduces CESAR, a novel secure aggregation protocol for DL designed to be compatible with existing sparsification mechanisms. CESAR provably defends against honest-but-curious adversaries and can be formally adapted to counteract collusion between them. We provide a foundational understanding of the interaction between the sparsification carried out by the nodes and the proportion of the parameters shared under CESAR in both colluding and non-colluding environments, offering analytical insight into the working and applicability of the protocol. Experiments on a network with 48 nodes in a 3-regular topology show that with random subsampling, CESAR is always within 0.5% accuracy of decentralized parallel stochastic gradient descent (D-PSGD), while adding only 11% of data overhead. Moreover, it surpasses the accuracy on TopK by up to 0.3% on independent and identically distributed (IID) data.

Decentralized learning (DL) enables collaborative learning without a server and without training data leaving the users' devices. However, the models shared in DL can still be used to infer training data. Conventional defenses such as differential privacy and secure aggregation fall short in effectively safeguarding user privacy in DL, either sacrificing model utility or efficiency. We introduce Shatter, a novel DL approach in which nodes create virtual nodes (VNs) to disseminate chunks of their full model on their behalf. This enhances privacy by (i) preventing attackers from collecting full models from other nodes, and (ii) hiding the identity of the original node that produced a given model chunk. We theoretically prove the convergence of Shatter and provide a formal analysis demonstrating how Shatter reduces the efficacy of attacks compared to when exchanging full models between nodes. We evaluate the convergence and attack resilience of Shatter with existing DL algorithms, with heterogeneous datasets, and against three standard privacy attacks. Our evaluation shows that Shatter not only renders these privacy attacks infeasible when each node operates 16 VNs but also exhibits a positive impact on model utility compared to standard DL. In summary, Shatter enhances the privacy of DL while maintaining the utility and efficiency of the model., Comment: Accepted at PETS 2025

This paper introduces ZIP-DL, a novel privacy-aware decentralized learning (DL) algorithm that exploits correlated noise to provide strong privacy protection against a local adversary while yielding efficient convergence guarantees for a low communication cost. The progressive neutralization of the added noise during the distributed aggregation process results in ZIP-DL fostering a high model accuracy under privacy guarantees. ZIP-DL further uses a single communication round between each gradient descent, thus minimizing communication overhead. We provide theoretical guarantees for both convergence speed and privacy guarantees, thereby making ZIP-DL applicable to practical scenarios. Our extensive experimental study shows that ZIP-DL significantly outperforms the state-of-the-art in terms of vulnerability/accuracy trade-off. In particular, ZIP-DL (i) reduces the efficacy of linkability attacks by up to 52 percentage points compared to baseline DL, (ii) improves accuracy by up to 37 percent w.r.t. the state-of-the-art privacy-preserving mechanism operating under the same threat model as ours, when configured to provide the same protection against membership inference attacks, and (iii) reduces communication by up to 10.5x against the same competitor for the same level of protection.

We present Epidemic Learning (EL), a simple yet powerful decentralized learning (DL) algorithm that leverages changing communication topologies to achieve faster model convergence compared to conventional DL approaches. At each round of EL, each node sends its model updates to a random sample of $s$ other nodes (in a system of $n$ nodes). We provide an extensive theoretical analysis of EL, demonstrating that its changing topology culminates in superior convergence properties compared to the state-of-the-art (static and dynamic) topologies. Considering smooth non-convex loss functions, the number of transient iterations for EL, i.e., the rounds required to achieve asymptotic linear speedup, is in $O(n^3/s^2)$ which outperforms the best-known bound $O(n^3)$ by a factor of $s^2$, indicating the benefit of randomized communication for DL. We empirically evaluate EL in a 96-node network and compare its performance with state-of-the-art DL approaches. Our results illustrate that EL converges up to $ 1.7\times$ quicker than baseline DL algorithms and attains $2.2 $\% higher accuracy for the same communication volume., Comment: Accepted paper at NeurIPS 2023

Decentralized learning (DL) systems have been gaining popularity because they avoid raw data sharing by communicating only model parameters, hence preserving data confidentiality. However, the large size of deep neural networks poses a significant challenge for decentralized training, since each node needs to exchange gigabytes of data, overloading the network. In this paper, we address this challenge with JWINS, a communication-efficient and fully decentralized learning system that shares only a subset of parameters through sparsification. JWINS uses wavelet transform to limit the information loss due to sparsification and a randomized communication cut-off that reduces communication usage without damaging the performance of trained models. We demonstrate empirically with 96 DL nodes on non-IID datasets that JWINS can achieve similar accuracies to full-sharing DL while sending up to 64% fewer bytes. Additionally, on low communication budgets, JWINS outperforms the state-of-the-art communication-efficient DL algorithm CHOCO-SGD by up to 4x in terms of network savings and time.

We perform a comprehensive analysis of medium modifications on ungroomed jet angularities, $\tau_a$, within the framework of Soft-Collinear Effective Theory with Glauber gluons (SCET$_{\rm G}$). Angularities are a one-parameter family of jet substructure observables with angularity exponent $a<2$ for infrared safety. Variation of the angularity exponent allows to modify the relative weighting of the collinear-to-soft radiations in the jet. In this article, we focus on angularity exponents $a<1$ and provide detailed results for $a=-1,0$, and $0.5$. Within SCET$_{\rm G}$, the interaction between jet and medium constituents is comprehended by off-shell Glauber gluons generated from the color gauge fields in the medium. While the medium modifications are incorporated in the jet function via the use of in-medium splitting functions, the soft function remains unmodified for $a<1$. For all values of $a$, we find that in the medium, the distributions are narrower and have a steeper fall compared to the vacuum ones. This redistribution of the ungroomed angularity spectrum is more apparent for a jet with a larger cone size. We also present results for the medium sensitivity towards $p_T$ of the jet and for a jet initiated in a less central event ($10-30\%$ centrality). Finally, we provide the ratios of nucleus-nucleus and proton-proton differential angularity distributions for different angularity exponents and for two values of the jet radius parameter., Comment: 38 pages, 5 figures

Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently iterative process of machine learning (ML) training. In addition, these nodes are connected in complex and potentially dynamic topologies. Assessing the intricate dynamics of such networks is clearly not an easy task. Often in literature, researchers resort to simulated environments that do not scale and fail to capture practical and crucial behaviors, including the ones associated to parallelism, data transfer, network delays, and wall-clock time. In this paper, we propose DecentralizePy, a distributed framework for decentralized ML, which allows for the emulation of large-scale learning networks in arbitrary topologies. We demonstrate the capabilities of DecentralizePy by deploying techniques such as sparsification and secure aggregation on top of several topologies, including dynamic networks with more than one thousand nodes.

Decentralized learning (DL) leverages edge devices for collaborative model training while avoiding coordination by a central server. Due to privacy concerns, DL has become an attractive alternative to centralized learning schemes since training data never leaves the device. In a round of DL, all nodes participate in model training and exchange their model with some other nodes. Performing DL in large-scale heterogeneous networks results in high communication costs and prolonged round durations due to slow nodes, effectively inflating the total training time. Furthermore, current DL algorithms also assume all nodes are available for training and aggregation at all times, diminishing the practicality of DL. This paper presents Plexus, an efficient, scalable, and practical DL system. Plexus (1) avoids network-wide participation by introducing a decentralized peer sampler that selects small subsets of available nodes that train the model each round and, (2) aggregates the trained models produced by nodes every round. Plexus is designed to handle joining and leaving nodes (churn). We extensively evaluate Plexus by incorporating realistic traces for compute speed, pairwise latency, network capacity, and availability of edge devices in our experiments. Our experiments on four common learning tasks empirically show that Plexus reduces time-to-accuracy by 1.2-8.3x, communication volume by 2.4-15.3x and training resources needed for convergence by 6.4-370x compared to baseline DL algorithms.

In this paper, we critically examine hierarchies between energy scales that determine quarkonium dynamics in the quark gluon plasma. A particularly important role is played by the ratio of the binding energy of species ($E_b$) and the medium scales; temperature ($T$) and Debye mass ($m_D$). It is well known that if these ratios are much larger than one then the dominant process governing quarkonium evolution is dissociation by thermal gluons (gluo-dissociation). On the other hand, if this ratio is much smaller than one then quarkonium dynamics is dominated by screening and Landau damping of the exchanged gluons. Here we show that over most of the evolution, the scale hierarchies do not fall in either limit and one needs to use the full structure of the gluonic spectral function to follow the dynamics of the $Q\bar{Q}$ pair. This has a significant bearing when we follow the quantum dynamics of quarkonia in the medium. The inverse medium relaxation time is also $\sim T$ and if $E_b$ is comparable (or larger) in magnitude to $T$, the quantum evolution of $Q\bar{Q}$ is non-local in time within the Brownian approximation., Comment: 15 pages, 10 figures

With the advent of multi-core systems, GPUs and FPGAs, loop parallelization has become a promising way to speed-up program execution. In order to stay up with time, various performance-oriented programming languages provide a multitude of constructs to allow programmers to write parallelizable loops. Correspondingly, researchers have developed techniques to automatically parallelize loops that do not carry dependences across iterations, and/or call pure functions. However, in managed languages with platform-independent runtimes such as Java, it is practically infeasible to perform complex dependence analysis during JIT compilation. In this paper, we propose AutoTornado, a first of its kind static+JIT loop parallelizer for Java programs that parallelizes loops for heterogeneous architectures using TornadoVM (a Graal-based VM that supports insertion of @Parallel constructs for loop parallelization). AutoTornado performs sophisticated dependence and purity analysis of Java programs statically, in the Soot framework, to generate constraints encoding conditions under which a given loop can be parallelized. The generated constraints are then fed to the Z3 theorem prover (which we have integrated with Soot) to annotate canonical for loops that can be parallelized using the @Parallel construct. We have also added runtime support in TornadoVM to use static analysis results for loop parallelization. Our evaluation over several standard parallelization kernels shows that AutoTornado correctly parallelizes 61.3% of manually parallelizable loops, with an efficient static analysis and a near-zero runtime overhead. To the best of our knowledge, AutoTornado is not only the first tool that performs program-analysis based parallelization for a real-world JVM, but also the first to integrate Z3 with Soot for loop parallelization.

We construct a measure of transverse momentum loss of jets in nuclear collisions at LHC directly using measurements of jet cross sections in PbPb and pp collisions. The proposal is shown to be equivalent to R_AA and is equally straightforward to construct. Using data from the ATLAS collaboration at two different collision energies, we show that the proposed measure has small statistical uncertainties. We argue that systematic errors can be easily improved over our estimates by the experimental collaboration to such an extent that it directly probes whether the jet-medium interaction is due to a strongly interacting medium or a weakly interacting plasma. We argue that the current data may marginally favour a strongly interacting medium. On the other hand, assuming that the medium is weakly interacting, we are able to provide estimates of the jet quenching parameter qhat which are in rough agreement with previously reported estimates.

Federated learning (FL) enables a set of client devices to collaboratively train a model without sharing raw data. This process, though, operates under the constrained computation and communication resources of edge devices. These constraints combined with systems heterogeneity force some participating clients to perform fewer local updates than expected by the server, thus slowing down convergence. Exhaustive tuning of hyperparameters in FL, furthermore, can be resource-intensive, without which the convergence is adversely affected. In this work, we propose GeL, the guess and learn algorithm. GeL enables constrained edge devices to perform additional learning through guessed updates on top of gradient-based steps. These guesses are gradientless, i.e., participating clients leverage them for free. Our generic guessing algorithm (i) can be flexibly combined with several state-of-the-art algorithms including FedProx, FedNova or FedYogi; and (ii) achieves significantly improved performance when the learning rates are not best tuned. We conduct extensive experiments and show that GeL can boost empirical convergence by up to 40% in resource-constrained networks while relieving the need for exhaustive learning rate tuning., Comment: 14 pages, 9 figures

The capsule network is a distinct and promising segment of the neural network family that drew attention due to its unique ability to maintain the equivariance property by preserving the spatial relationship amongst the features. The capsule network has attained unprecedented success over image classification tasks with datasets such as MNIST and affNIST by encoding the characteristic features into the capsules and building the parse-tree structure. However, on the datasets involving complex foreground and background regions such as CIFAR-10, the performance of the capsule network is sub-optimal due to its naive data routing policy and incompetence towards extracting complex features. This paper proposes a new design strategy for capsule network architecture for efficiently dealing with complex images. The proposed method incorporates wide bottleneck residual modules and the Squeeze and Excitation attention blocks upheld by the modified FM routing algorithm to address the defined problem. A wide bottleneck residual module facilitates extracting complex features followed by the squeeze and excitation attention block to enable channel-wise attention by suppressing the trivial features. This setup allows channel inter-dependencies at almost no computational cost, thereby enhancing the representation ability of capsules on complex images. We extensively evaluate the performance of the proposed model on three publicly available datasets, namely CIFAR-10, Fashion MNIST, and SVHN, to outperform the top-5 performance on CIFAR-10 and Fashion MNIST with highly competitive performance on the SVHN dataset., Comment: 13 pages, 5 figures

In presence of magnetic field we have done a systematic analysis to constrain the strength of 't Hooft determinant term using LQCD data within a non-local version of two flavour NJL model. Topological susceptibility, being related to the axial symmetry, have also been calculated and compared with LQCD results to further validate our calculation., Comment: Proceedings of the contributory talk at XXIV DAE-BNRS High Energy Physics Symposium 2020 held at NISER, Bhubaneswar, Odisha

The charged and neutral pion mass difference can be attributed to both the QED and QCD contributions. The current quark mass difference ($\Delta m$) is the source of the QCD contribution. Here, in a two-flavour non-local NJL model, we try to estimate the QCD contribution. Interestingly, we find that the strength of the $U(1)_A$ symmetry-breaking parameter $c$ plays a crucial role in obtaining the pion mass difference while intertwined with the current quark mass difference. To obtain the QCD contribution for the pion mass difference, we scan the parameter space in $\{\Delta m,\; c\}$, and by comparing this with the existing results, we constrained the parameter space. Further, using a fitted value of $c$, we determine the allowed range for the $\Delta m$ in the model. The model estimated $\Delta m$ ranges enable us to extract the chiral perturbation theory low-energy constant, $l_7$ and verify the dependence of the pion mass difference on $\Delta m$. We also find out its dependence on $c$ \textemdash\, it increases with the decreasing value of $c$, i.e., toward an axial anomaly restored phase., Comment: 12 pages, 6 captioned figures, discussion on l7 has been added, accepted in JPG

In relativistic heavy ion collisions at RHIC and the LHC, a quark gluon plasma (QGP) is created for a short duration of about 10fm/c. Quarkonia (bound states of $c\bar{c}$ and $b\bar{b}$) are sensitive probes of this phase on length scales comparable to the size of the bound states which are less than 1fm. Observations of quarkonia in these collisions provide us with a lot of information about how the presence of a QGP affects various quarkonium states. This has motivated the development of the theory of heavy quarks and their bound states in a thermal medium, and its application to the phenomenology of quarkonia in heavy ion collisions. We review some of these developments here., Comment: 34 pages, 7 figures, Invited review article for EPJ ST

In our world the standard model of particle physics contains within it the fairly intractable theory called QCD. A toy version with two colours is often studied as a model confining and chiral symmetry breaking field theory. Here we investigate the cascade of changes at various distance scales if we make this change within the standard model. It is possible to limit the changes at the hadronic scale. However, the minor changes that occur actually cascade down to the far infrared, into nuclear and atomic physics, and chemistry. Through this it also possibly affects the evolution of stars and galaxies. We remark on this unexpected sensitivity of the universe to physics at the scale of quarks., Comment: Submitted to Indian Journal of Physics memorial issue for Pushan Majumdar