Search

Continual Federated Learning (CFL) is essential for enabling real-world applications where multiple decentralized clients adaptively learn from continuous data streams. A significant challenge in CFL is mitigating catastrophic forgetting, where models lose previously acquired knowledge when learning new information. Existing approaches often face difficulties due to the constraints of device storage capacities and the heterogeneous nature of data distributions among clients. While some CFL algorithms have addressed these challenges, they frequently rely on unrealistic assumptions about the availability of task boundaries (i.e., knowing when new tasks begin). To address these limitations, we introduce Fed-A-GEM, a federated adaptation of the A-GEM method (Chaudhry et al., 2019), which employs a buffer-based gradient projection approach. Fed-A-GEM alleviates catastrophic forgetting by leveraging local buffer samples and aggregated buffer gradients, thus preserving knowledge across multiple clients. Our method is combined with existing CFL techniques, enhancing their performance in the CFL context. Our experiments on standard benchmarks show consistent performance improvements across diverse scenarios. For example, in a task-incremental learning scenario using the CIFAR-100 dataset, our method can increase the accuracy by up to 27%. Our code is available at https://github.com/shenghongdai/Fed-A-GEM., Comment: A preliminary version of this work was presented at the Federated Learning Systems (FLSys) Workshop @ Sixth Conference on Machine Learning and Systems, June 2023

Training Graph Neural Networks (GNNs) on large graphs presents unique challenges due to the large memory and computing requirements. Distributed GNN training, where the graph is partitioned across multiple machines, is a common approach to training GNNs on large graphs. However, as the graph cannot generally be decomposed into small non-interacting components, data communication between the training machines quickly limits training speeds. Compressing the communicated node activations by a fixed amount improves the training speeds, but lowers the accuracy of the trained GNN. In this paper, we introduce a variable compression scheme for reducing the communication volume in distributed GNN training without compromising the accuracy of the learned model. Based on our theoretical analysis, we derive a variable compression method that converges to a solution equivalent to the full communication case, for all graph partitioning schemes. Our empirical results show that our method attains a comparable performance to the one obtained with full communication. We outperform full communication at any fixed compression ratio for any communication budget.

Adversarial machine learning, focused on studying various attacks and defenses on machine learning (ML) models, is rapidly gaining importance as ML is increasingly being adopted for optimizing wireless systems such as Open Radio Access Networks (O-RAN). A comprehensive modeling of the security threats and the demonstration of adversarial attacks and defenses on practical AI based O-RAN systems is still in its nascent stages. We begin by conducting threat modeling to pinpoint attack surfaces in O-RAN using an ML-based Connection management application (xApp) as an example. The xApp uses a Graph Neural Network trained using Deep Reinforcement Learning and achieves on average 54% improvement in the coverage rate measured as the 5th percentile user data rates. We then formulate and demonstrate evasion attacks that degrade the coverage rates by as much as 50% through injecting bounded noise at different threat surfaces including the open wireless medium itself. Crucially, we also compare and contrast the effectiveness of such attacks on the ML-based xApp and a non-ML based heuristic. We finally develop and demonstrate robust training-based defenses against the challenging physical/jamming-based attacks and show a 15% improvement in the coverage rates when compared to employing no defense over a range of noise budgets, Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Beyond their impressive sampling capabilities, score-based diffusion models offer a powerful analysis tool in the form of unbiased density estimation of a query sample under the training data distribution. In this work, we investigate the robustness of density estimation using the probability flow (PF) neural ordinary differential equation (ODE) model against gradient-based likelihood maximization attacks and the relation to sample complexity, where the compressed size of a sample is used as a measure of its complexity. We introduce and evaluate six gradient-based log-likelihood maximization attacks, including a novel reverse integration attack. Our experimental evaluations on CIFAR-10 show that density estimation using the PF ODE is robust against high-complexity, high-likelihood attacks, and that in some cases adversarial samples are semantically meaningful, as expected from a robust estimator.

In conventional federated hyperdimensional computing (HDC), training larger models usually results in higher predictive performance but also requires more computational, communication, and energy resources. If the system resources are limited, one may have to sacrifice the predictive performance by reducing the size of the HDC model. The proposed resource-efficient federated hyperdimensional computing (RE-FHDC) framework alleviates such constraints by training multiple smaller independent HDC sub-models and refining the concatenated HDC model using the proposed dropout-inspired procedure. Our numerical comparison demonstrates that the proposed framework achieves a comparable or higher predictive performance while consuming less computational and wireless resources than the baseline federated HDC implementation., Comment: Accepted to Federated Learning Systems (FLSys) workshop, in Conjunction with the 6th MLSys Conference (MLSys 2023)

Federated systems enable collaborative training on highly heterogeneous data through model personalization, which can be facilitated by employing multi-task learning algorithms. However, significant variation in device computing capabilities may result in substantial degradation in the convergence rate of training. To accelerate the learning procedure for diverse participants in a multi-task federated setting, more efficient and robust methods need to be developed. In this paper, we design an efficient iterative distributed method based on the alternating direction method of multipliers (ADMM) for support vector machines (SVMs), which tackles federated classification and regression. The proposed method utilizes efficient computations and model exchange in a network of heterogeneous nodes and allows personalization of the learning model in the presence of non-i.i.d. data. To further enhance privacy, we introduce a random mask procedure that helps avoid data inversion. Finally, we analyze the impact of the proposed privacy mechanisms and participant hardware and data heterogeneity on the system performance., Comment: 14 pages, 12 figures, 4 tables, 1 algorithm; Added algorithm for iterative solution, updated the abstract, fixed typos

Hyperdimensional computing (HDC) is a paradigm for data representation and learning originating in computational neuroscience. HDC represents data as high-dimensional, low-precision vectors which can be used for a variety of information processing tasks like learning or recall. The mapping to high-dimensional space is a fundamental problem in HDC, and existing methods encounter scalability issues when the input data itself is high-dimensional. In this work, we explore a family of streaming encoding techniques based on hashing. We show formally that these methods enjoy comparable guarantees on performance for learning applications while being substantially more efficient than existing alternatives. We validate these results experimentally on a popular high-dimensional classification problem and show that our approach easily scales to very large data sets.

Named Data Networking (NDN) offers promising advantages in deploying next-generation service applications over distributed computing networks. We consider the problem of dynamic orchestration over a NDN-based computing network, in which nodes can be equipped with communication, computation, and data producing resources. Given a set of services with function-chaining structures, we address the design of distributed online algorithm that controls each node to make adaptive decisions on flowing service requests, committing function implementations, and/or producing data. We design a Service Discovery Assisted Dynamic Orchestration (SDADO) algorithm that reduces the end-to-end (E2E) delay of delivering the services, while providing optimal throughput performance. The proposed algorithm hybrids queuing-based flexibility and topology-based discipline, where the topological information is not pre-available but obtained through our proposed service discovery mechanism. We provide throughput-optimality analysis for SDADO, and then provide numerical results that confirm our analysis and demonstrates reduced round-trip E2E delay.

Today, various machine learning (ML) applications offer continuous data processing and real-time data analytics at the edge of a wireless network. Distributed real-time ML solutions are highly sensitive to the so-called straggler effect caused by resource heterogeneity and alleviated by various computation offloading mechanisms that seriously challenge the communication efficiency, especially in large-scale scenarios. To decrease the communication overhead, we rely on device-to-device (D2D) connectivity that improves spectrum utilization and allows efficient data exchange between devices in proximity. In particular, we design a novel D2D-aided coded federated learning method (D2D-CFL) for efficient load balancing across devices. The proposed solution captures system dynamics, including data (time-dependent learning model, varied intensity of data arrivals), device (diverse computational resources and volume of training data), and deployment (varied locations and D2D graph connectivity). To minimize the number of communication rounds, we derive an optimal compression rate for achieving minimum processing time and establish its connection with the convergence time. The resulting optimization problem provides suboptimal compression parameters, which improve the total training time. Our proposed method is beneficial for real-time collaborative applications, where the users continuously generate training data resulting in the model drift., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

The confluence of 5G and AI is transforming wireless networks to deliver diverse services at the Edge, driving towards a vision of pervasive distributed intelligence. Future 6G networks will need to deliver quality of experience through seamless integration of communication, computation and AI. Therefore, networks must become intelligent, distributed, scalable, and programmable platforms across the continuum of data delivery to address the ever-increasing service requirements and deployment complexity. We present novel results across three research directions that are expected to be integral to 6G systems and also discuss newer 6G metrics.

Federated learning enables training a global model from data located at the client nodes, without data sharing and moving client data to a centralized server. Performance of federated learning in a multi-access edge computing (MEC) network suffers from slow convergence due to heterogeneity and stochastic fluctuations in compute power and communication link qualities across clients. We propose a novel coded computing framework, CodedFedL, that injects structured coding redundancy into federated learning for mitigating stragglers and speeding up the training procedure. CodedFedL enables coded computing for non-linear federated learning by efficiently exploiting distributed kernel embedding via random Fourier features that transforms the training task into computationally favourable distributed linear regression. Furthermore, clients generate local parity datasets by coding over their local datasets, while the server combines them to obtain the global parity dataset. Gradient from the global parity dataset compensates for straggling gradients during training, and thereby speeds up convergence. For minimizing the epoch deadline time at the MEC server, we provide a tractable approach for finding the amount of coding redundancy and the number of local data points that a client processes during training, by exploiting the statistical properties of compute as well as communication delays. We also characterize the leakage in data privacy when clients share their local parity datasets with the server. We analyze the convergence rate and iteration complexity of CodedFedL under simplifying assumptions, by treating CodedFedL as a stochastic gradient descent algorithm. Furthermore, we conduct numerical experiments using practical network parameters and benchmark datasets, where CodedFedL speeds up the overall training time by up to $15\times$ in comparison to the benchmark schemes., Comment: Final version to appear in the first issue of the IEEE JSAC Series on Machine Learning for Communications and Networks

Federated Learning (FL) is an exciting new paradigm that enables training a global model from data generated locally at the client nodes, without moving client data to a centralized server. Performance of FL in a multi-access edge computing (MEC) network suffers from slow convergence due to heterogeneity and stochastic fluctuations in compute power and communication link qualities across clients. A recent work, Coded Federated Learning (CFL), proposes to mitigate stragglers and speed up training for linear regression tasks by assigning redundant computations at the MEC server. Coding redundancy in CFL is computed by exploiting statistical properties of compute and communication delays. We develop CodedFedL that addresses the difficult task of extending CFL to distributed non-linear regression and classification problems with multioutput labels. The key innovation of our work is to exploit distributed kernel embedding using random Fourier features that transforms the training task into distributed linear regression. We provide an analytical solution for load allocation, and demonstrate significant performance gains for CodedFedL through experiments over benchmark datasets using practical network parameters., Comment: Work accepted for presentation at the International Workshop on Federated Learning for User Privacy and Data Confidentiality, in Conjunction with ICML 2020 (FL-ICML'20). This work was part of Saurav Prakash's internship projects at Intel. arXiv admin note: text overlap with arXiv:2011.06223

Unmanned aerial vehicles (UAVs) are increasingly employed for numerous public and civil applications, such as goods delivery, medicine, surveillance, and telecommunications. For the latter, UAVs with onboard communication equipment may help temporarily offload traffic onto the neighboring cells in fifth-generation networks and beyond (5G+). In this paper, we propose and evaluate the use of UAVs traveling over the area of interest to relieve congestion in 5G+ systems under spontaneous traffic fluctuations. To this end, we assess two inherently different offloading schemes, named routed and controlled UAV `bridging'. Using the tools of renewal theory and stochastic geometry, we analytically characterize these schemes in terms of the fraction of traffic demand that can be offloaded onto the UAV `bridge' as our parameter of interest. This framework accounts for the unique features of millimeter-wave (mmWave) radio propagation and city deployment types with potential line-of-sight (LoS) link blockage by buildings. We also introduce enhancements to the proposed schemes that significantly improve the offloading gains. Our findings offer evidence that the UAV `bridges' may be used for efficient traffic offloading in various urban scenarios., Comment: This work has been accepted for publication in the IEEE Transactions on Vehicular Technology

Federated learning is a method of training a global model from decentralized data distributed across client devices. Here, model parameters are computed locally by each client device and exchanged with a central server, which aggregates the local models for a global view, without requiring sharing of training data. The convergence performance of federated learning is severely impacted in heterogeneous computing platforms such as those at the wireless edge, where straggling computations and communication links can significantly limit timely model parameter updates. This paper develops a novel coded computing technique for federated learning to mitigate the impact of stragglers. In the proposed Coded Federated Learning (CFL) scheme, each client device privately generates parity training data and shares it with the central server only once at the start of the training phase. The central server can then preemptively perform redundant gradient computations on the composite parity data to compensate for the erased or delayed parameter updates. Our results show that CFL allows the global model to converge nearly four times faster when compared to an uncoded approach, Comment: Presented at the Wireless Edge Intelligence Workshop, IEEE GLOBECOM 2019

The use of unmanned aerial vehicle (UAV)-based communication in millimeter-wave (mmWave) frequencies to provide on-demand radio access is a promising approach to improve capacity and coverage in beyond-5G (B5G) systems. There are several design aspects to be addressed when optimizing for the deployment of such UAV base stations. As traffic demand of mobile users varies across time and space, dynamic algorithms that correspondingly adjust the UAV locations are essential to maximize performance. In addition to careful tracking of spatio-temporal user/traffic activity, such optimization needs to account for realistic backhaul constraints. In this work, we first review the latest 3GPP activities behind integrated access and backhaul system design, support for UAV base stations, and mmWave radio relaying functionality. We then compare static and mobile UAV-based communication options under practical assumptions on the mmWave system layout, mobility and clusterization of users, antenna array geometry, and dynamic backhauling. We demonstrate that leveraging the UAV mobility to serve moving users may improve the overall system performance even in the presence of backhaul capacity limitations., Comment: 7 pages, 5 figures. This work has been accepted to IEEE Communications Magazine, 2019

We consider the problem of allocating radio resources over wireless communication links to control a series of independent wireless control systems. Low-latency transmissions are necessary in enabling time-sensitive control systems to operate over wireless links with high reliability. Achieving fast data rates over wireless links thus comes at the cost of reliability in the form of high packet error rates compared to wired links due to channel noise and interference. However, the effect of the communication link errors on the control system performance depends dynamically on the control system state. We propose a novel control-communication co-design approach to the low-latency resource allocation problem. We incorporate control and channel state information to make scheduling decisions over time on frequency, bandwidth and data rates across the next-generation Wi-Fi based wireless communication links that close the control loops. Control systems that are closer to instability or further from a desired range in a given control cycle are given higher packet delivery rate targets to meet. Rather than a simple priority ranking, we derive precise packet error rate targets for each system needed to satisfy stability targets and make scheduling decisions to meet such targets while reducing total transmission time. The resulting Control-Aware Low Latency Scheduling (CALLS) method is tested in numerous simulation experiments that demonstrate its effectiveness in meeting control-based goals under tight latency constraints relative to control-agnostic scheduling.

Millimeter-wave (mmWave) propagation is known to be severely affected by the blockage of the line-of-sight (LoS) path. In contrast to microwave systems, at shorter mmWave wavelengths such blockage can be caused by human bodies, where their mobility within environment makes wireless channel alternate between the blocked and non-blocked LoS states. Following the recent 3GPP requirements on modeling the dynamic blockage as well as the temporal consistency of the channel at mmWave frequencies, in this paper a new model for predicting the state of a user in the presence of mobile blockers for representative 3GPP scenarios is developed: urban micro cell (UMi) street canyon and park/stadium/square. It is demonstrated that the blockage effects produce an alternating renewal process with exponentially distributed non-blocked intervals, and blocked durations that follow the general distribution. The following metrics are derived (i) the mean and the fraction of time spent in blocked/non-blocked state, (ii) the residual blocked/non-blocked time, and (iii) the time-dependent conditional probability of having blockage/no blockage at time t1 given that there was blockage/no blockage at time t0. The latter is a function of the arrival rate (intensity), width, and height of moving blockers, distance to the mmWave access point (AP), as well as the heights of the AP and the user device. The proposed model can be used for system-level characterization of mmWave cellular communication systems. For example, the optimal height and the maximum coverage radius of the mmWave APs are derived, while satisfying the required mean data rate constraint. The system-level simulations corroborate that the use of the proposed method considerably reduces the modeling complexity., Comment: Accepted, IEEE Transactions on Vehicular Technology

The use of extremely high frequency (EHF) or millimeter-wave (mmWave) band has attracted significant attention for the next generation wireless access networks. As demonstrated by recent measurements, mmWave frequencies render themselves quite sensitive to "blocking" caused by obstacles like foliage, humans, vehicles, etc. However, there is a dearth of analytical models for characterizing such blocking and the consequent effect on the signal reliability. In this paper, we propose a novel, general, and tractable model for characterizing the blocking caused by humans (assuming them to be randomly located in the environment) to mmWave propagation as a function of system parameters like transmitter-receiver locations and dimensions, as well as density and dimensions of humans. Moreover, the proposed model is validated using a ray-launcher tool. Utilizing the proposed model, the blockage probability is shown to increase with human density and separation between the transmitter-receiver pair. Furthermore, the developed analysis is shown to demonstrate the existence of a transmitter antenna height that maximizes the received signal strength, which in turn is a function of the transmitter-receiver distance and their dimensions.

Traffic load balancing and radio resource management is key to harness the dense and increasingly heterogeneous deployment of next generation "$5$G" wireless infrastructure. Strategies for aggregating user traffic from across multiple radio access technologies (RATs) and/or access points (APs) would be crucial in such heterogeneous networks (HetNets), but are not well investigated. In this paper, we develop a low complexity solution for maximizing an $\alpha$-optimal network utility leveraging the multi-link aggregation (simultaneous connectivity to multiple RATs/APs) capability of users in the network. The network utility maximization formulation has maximization of sum rate ($\alpha=0$), maximization of minimum rate ($\alpha \to \infty$), and proportional fair ($\alpha=1$) as its special cases. A closed form is also developed for the special case where a user aggregates traffic from at most two APs/RATs, and hence can be applied to practical scenarios like LTE-WLAN aggregation (LWA) and LTE dual-connectivity solutions. It is shown that the required objective may also be realized through a decentralized implementation requiring a series of message exchanges between the users and network. Using comprehensive system level simulations, it is shown that optimal leveraging of multi-link aggregation leads to substantial throughput gains over single RAT/AP selection techniques.

Traffic load balancing and resource allocation is set to play a crucial role in leveraging the dense and increasingly heterogeneous deployment of multi-radio wireless networks. Traffic aggregation across different access points (APs)/radio access technologies (RATs) has become an important feature of recently introduced cellular standards on LTE dual connectivity and LTE-WLAN aggregation (LWA). Low complexity traffic splitting solutions for scenarios where the APs are not necessarily collocated are of great interest for operators. In this paper, we consider a scenario, where traffic for each user may be split across macrocell and an LTE or WiFi small cells connected by non-ideal backhaul links, and develop a closed form solution for optimal aggregation accounting for the backhaul delay. The optimal solution lends itself to a "water-filling" based interpretation, where the fraction of user's traffic sent over macrocell is proportional to ratio of user's peak capacity on that macrocell and its throughput on the small cell. Using comprehensive system level simulations, the developed optimal solution is shown to provide substantial edge and median throughput gain over algorithms representative of current 3GPP-WLAN interworking solutions. The achievable performance benefits hold promise for operators expecting to introduce aggregation solutions with their existing WLAN deployments.